JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT
2016, VOL. 36, NOS. 2–3, 210–253
http://dx.doi.org/10.1080/01608061.2016.1201036

Tactics to Ensure Durability of Behavior Change Following
the Removal of an Intervention Specialist: A Review of
Temporal Generality Within Organizational Behavior
Management
Anna L. Conard, Douglas A. Johnson, James D. Morrison, and Kyle DitzianWestern Michigan University, Kalamazoo, Michigan, USA
ABSTRACTKEYWORDS

The endurance of behavior change over time, including thetime period after a behavior specialist has ceased to offer
direct services, is an important consideration for both organizational behavior management and behavior analysis in general. This article considers a dozen strategies labelled temporal
generality tactics that may foster the maintenance and institutionalization of intervention efforts in organizational settings.
The use of such tactics was examined by conducting a comprehensive review of articles published between 1977 and2014 in the Journal of Organizational Behavior Management.
The results of this review suggest that although follow-up
measures are frequently collected, explicit and proactive strategies to systematically increase the success of long-term
behavior change are rarely used or researched.

Follow-up data;
generalization;
institutionalization;
maintenance; temporal
generality

For the average person, the value of science most likely does not come fromdemonstrations of functional relationships, verifications of hypotheses,
achievements of statistical significance, the cohesion of theoretical explanations, or other similar scientific activities and outcomes. The general public is
more interested in technology derived from the application of science andhow such applications can improve their lives in lasting ways. This has
relevance not just for application of science in general but for applied
behavior analysis as well. As Baer, Wolf, and Risley (1968) pointed out intheir seminal article delineating the scope of applied behavior analysis, thefield needs to achieve practical improvements with the behaviors that areconsidered to be socially important. In that same article, they also outlined
the attributes of applied behavior analysis, one of which was that any
application of behavioral science should show many forms of generality.
The generality of behavior can include the extension of behavior change
across settings, responses, and time (Foxx, 2013). Generality across settings
CONTACT Anna Conard
anna.l.conard@wmich.edu
Western Michigan University Department of
Psychology, 3700 Wood Hall, Kalamazoo, MI 49008-5439, USA.
Color versions of one or more of the figures in the article can be found online at www.tandfonline.com/WORG.
© 2016 Taylor & Francis

JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT

211

will be shown when behavior established under one set of conditions also
appears under a novel set of conditions. Within organizational behavior
management (OBM), it is common to try to foster this type of generality
when conducting training, wherein it is hoped that the behaviors developedin training situations will generalize to on-the-job situations. For example, a
trainer may conduct training in an environment that shares features with theon-the-job environment, such as a realistic simulation or the on-the-job
environment itself during off hours, in the hope that the new behaviors
will transfer to normal working conditions (Brethower & Smalley, 1998).
Generality across responses is shown when, in the process of modifying atargeted behavior, other nontargeted behaviors begin to vary in a similarmanner (Austin & Wilson, 2001; Ludwig, 2001). For example, an intervention to increase one class of targeted safety behaviors (e.g., wearing earplugs)
may also increase other nontargeted safety behaviors (e.g., wearing safety
glasses). If properly capitalized on, such generality across settings and behaviors may represent an efficient means of promoting new behavioral relations
with a minimal investment of intervention.
Although these types of generality are critical issues within OBM, thecurrent article concerns itself with the generality of behavior change acrosstime. In particular, this article explores the durability of intervention
effects after the intervention specialist (i.e., an OBM consultant or
researcher in charge of performance change) has been removed from theworkplace environment and direct consultation. As mentioned previously,
the value of applied behavioral science comes from not just creating
behavior change but creating a lasting change in behavior. With organizational clients, it is important not just that performance improvements
occur in the presence of an OBM consultant or researcher but that theorganization continues to benefit after that individual leaves and theorganization must manage the target performance on its own. There islittle practical value for an organization if performance improvements
disappear as soon as the consultant or researcher leaves the project, and
it is debatable as to whether such temporary improvements should even beconsidered evidence of effectiveness (Foxx, 2013). The endurance of OBM
successes over time is potentially how consultants can build notable
reputations for their services and how researchers might demonstrate
their value and the value of this science to society at large. The longterm continuation of OBM intervention outcomes may be one of the mostimportant aspects of OBM, and other empirically grounded consultants
could benefit from research on the durability of interventions. Thus, it isimportant to understand how to best maximize the durability of behavior
change over time, including the time period after the consultant or
researcher has withdrawn his or her direct services.

212

A. L. CONARD ET AL.

Some OBM scholars have argued for differentiating between types ofgenerality over time by distinguishing between maintenance and institutionalization (Boyce & Geller, 2001). Maintenance of behavior change in behavior analytic research has been defined in many different ways. Some havedefined maintenance as resistance to extinction (Kazdin, 1994), whereas
others have referred to maintenance as a generalization of behavior changeover time (Stokes & Baer, 1977). For the purposes of this article, maintenanceis described as a type of durability across time in which the target behavior
continues to occur after the intervention implemented by a consultant or
researcher has been removed from the setting. In contrast, institutionalization refers to the partial or full continuation of an intervention by theorganization after the consultant or researcher has departed the setting.
Whereas maintenance must be programmed into the experimental designof an intervention, institutionalization is up to the discretion of the organization for which the intervention is implemented, thus making it naturally
more difficult to directly manipulate. For example, a consultant could implement a series of training sessions to help salespeople identify behaviors thatthey could engage in to improve their rate of successfully closing a sale. If
those salespeople continued to engage in these new behaviors following the
cessation of the training sessions, then this would be an example of maintenance, because the direct contingencies of the intervention have been
removed but the behavior change persists. In another example, a consultant
may develop a new feedback and incentive system to improve sales. If
management decides to continue the practices of this new system after the
consultant leaves, then this would be considered institutionalization, because
the intervention contingencies are still being implemented even though theconsultant is no longer present. For the remainder of this article, the terms
maintenance and institutionalization are used in this manner.
Although the distinction between maintenance and institutionalization is
logical and defensible, it does raise some problems. As Fleming (2001)
pointed out, any emphasis on maintenance will imply that the goal is always
to remove an intervention, which is not always appropriate for certain typesof interventions, including some of the most common interventions within
OBM, such as feedback, goal setting, and incentives. Furthermore, for longterm maintenance to be successful, the work environment requires theavailability of natural reinforcers to maintain the newly acquired behaviors
in strength. Unfortunately, many environments are unlikely to offer sufficient rates of reinforcement for the continuation of a newly acquired behavior, even if the consultant took a low rate of reinforcement into accountwhen designing the intervention (Foxx, 2013; Stokes & Baer, 1977). For
example, some have referred to sales as an extinction business (Feeney,
Staelin, O’Brien, & Dickinson, 1982), in the sense that within certain industries, the majority of sales attempts will likely end in rejection for the

JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT

213

salesperson even when he or she is performing correctly. Although some may
persist given a learning history that has established the response products
associated with sales attempts as reinforcing irrespective of the client reaction(i.e., automatic reinforcement), such repertoires may prove too idiosyncratic
to uniformly build a successful salesforce. For most individuals in theabsence of a contrived contingency, it is unlikely that appropriate sales
behavior will persist in the presence of repeated natural failures. As Malott(2001) pointed out, there is the “myth of perpetual-behavior intervention” (p.
100), which refers to the notion that a behavioral intervention will besustained even in the absence of future consequences so long as the original
intervention has properly thinned out reinforcement. Malott argued that thisis a naïve error and that behavior analysts should presume that the normal
work environment will need to be modified on a long-term basis to provide
artificial consequences if sustained change is desired.
It is worth pointing out that there may be situations in which maintenancecan be assumed even in the absence of institutionalization—namely, when
there are natural sources of reinforcement but some knowledge or skill deficit
prevents the individual from coming into contact with these reinforcers(McSween & Matthews, 2001). When an intervention leaves the client witha new repertoire that allows for a self-sustaining behavioral relation to
emerge naturally between that person’s behavior and the work environment,
one can expect maintenance without further contrived endeavors.
Unfortunately, this largely restricts maintenance to issues related to acquisition (i.e., “can’t do” problems) and therefore neglects motivational issues (i.e.,“won’t do” problems) that require the continuation of an active intervention
for successful generality across time. Even within the realm of behavior
acquisition, it is often the case that newly acquired forms of behaviors will
not be sustained without some form of regularly programmed consequences.
Furthermore, in the process of simply altering an individual’s repertoire, thenatural environment itself is being changed because a new set of relations
between behavioral tendencies and environmental conditions has been established. This has led many authors to propose that the distinction between
maintenance and institutionalization is false or, at best, unclear (Boyce &Roman, 2002; Kessler, 2001).
These considerations suggest that the durability of behavior change overtime should be viewed not as the passive outcome resulting from an intervention but as part of the intervention itself (Stokes & Baer, 1977). If
maintenance is limited to knowledge and skill acquisition within contexts
in which natural reinforcement is readily available, this means that OBM
consultants and researchers cannot rely on a momentary intervention to
produce the lasting changes valued by clients. Rather, as Baer et al. (1968)
stated, “generalization should be programmed, rather than expected orlamented” (p. 97). Given the intertwined nature of maintenance and214

A. L. CONARD ET AL.

institutionalization and the importance of developing lasting change for bothacquisition and motivational issues, this article looks at the topic of durability
across time broadly by subsuming both maintenance and institutionalization
under the more inclusive label of temporal generality. Such generality could
be assessed in the form of follow-up data and probes collected after a delay ofthe consultant or researcher’s absence, data collected immediately regarding
the success of on-site workers and managers in self-managing some intervention program, or some combination of these forms of data.
Reports on the absence of follow-up data collected in OBM interventions can be traced back as early as 1986. To determine the typical focus of
OBM technology, Redmon and Lockwood (1986) examined the standard
consultation methodology in terms of 10 categories (pre-entry into theorganization, organization entry, goal setting, procedure selection, role
definition, implementation, evaluation, maintenance, withdrawal, and follow-up). The authors found that all studies considered the middle phasesof intervention implementation and evaluation, but very few considered
the beginning phases or the last few phases, including maintenance andfollow-up.
In many ways, the need for a better understanding of temporal generality
echoes the call for a better understanding of behavioral systems analysis in
OBM (Diener, McGee, & Miguel, 2009; D. A. Johnson, Casella, McGee, &Lee, 2014). With both areas, there is an emphasis on looking beyond theimmediate change in behavior or the middle stage of consultation andinstead looking more extensively at the broader context of relevant organizational factors. With behavioral systems analysis, the emphasis centers onunderstanding these factors prior to an intervention (i.e., pre-entry andentry), whereas with temporal generality, the emphasis centers on understanding these factors subsequent to the intervention (i.e., maintenance,
withdrawal, and follow-up). Furthermore, effectiveness with generality may
require an understanding of organizational systems and processes (Redmon,
1991). For example, an intervention that boosts productivity in one divisionof the organization may also divert resources from another division or place
undue pressure on other divisions. If an incentive program developed for themarketing department comes at the cost of resources for the research anddevelopment department, this may create hostility between these two departments. Likewise, if an improvement in productivity by a sales division places
an unsustainable demand on the production division, the production division may attempt to counteract the intervention. At an individual level, if anincrease in supervisor monitoring improves the quality of manufactured
parts but also creates additional work for that supervisor with no additional
benefit, this intervention is unlikely to be maintained by the supervisor. In
these cases, a failure to consider the overall functioning of the entire organization may create elements within the organization that will work inJOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT

215

opposition to the long-term maintenance of the behavior change as the
disadvantaged parties lobby to repeal the recent innovations.
Even without opposition, there may be a need to conduct more systemic analyses to increase the probability of an intervention being
retained. For example, in order for a safety intervention to be adopted
for an extended period of time, it may be necessary to demonstrate to thedecision-making executives that reduced injuries are not only good for thephysical health of the workers but also good for the financial health of thebusiness (e.g., reduced compensation, reduced fines, reduced insurance
rates). Without such a demonstration, the executives may decide thatinjuries are simply the cost of doing business. This is important because
most interventions will produce behavioral side effects beyond the targetedbehaviors, and these side effects, even if short term in duration, mayinterfere with the long-term success of the intervention (Boyce & Geller,
2001). Success in temporal generality may require a broad understandingof many organizational factors and an identification of why behavior
change would be beneficial to the key players within that organization
over the long run.
The literature has proposed many suggestions to improve the durability of
behavior change beyond the original intervention. In their discussion ofgeneralization, Stokes and Baer (1977) described several strategies for promoting the generalization of behavioral interventions. The authors classified
these strategies into the following categories: (a) train and hope, (b) sequential modification, (c) introduce to natural maintaining contingencies, (d)train sufficient exemplars, (e) train loosely, (f) use of indiscriminable contingencies, (g) program common stimuli, (h) mediate generalization, and (i)train to generalize. Training sufficient exemplars, training loosely, programming common stimuli, and training to generalize are generalization methods
mostly targeted at generality across stimuli or behaviors, rather than time,
and thus are not discussed in further detail. The remaining five strategies
formulated by Stokes and Baer represent plans that could promote generality
across time and therefore are considered in greater detail here. For the sakeof clarity and consistency, the tactics listed by Stokes and Baer and subsequent strategies are commonly referred to as temporal generality tactics inthis article.
As seen with Table 1, the first temporal generality tactic is to intervene andhope (slightly renamed from “train and hope” to expand its scope beyond
training situations), which involves the delivery of an intervention with noprogrammed efforts to promote generality. Any instances of generalization
across settings, behaviors, or time are simply hoped for and documented.
This is perhaps the most commonly implemented training procedure,
although it may be one of the least effective. It may be slightly misleading
to label this as a tactic because any instances of generality are not the result of

216

A. L. CONARD ET AL.

Table 1. Summary of Temporal Generality Tactics.
TacticIntervene and hopeSequential modification

Introduce to natural
maintaining contingencies

Use of indiscriminable
contingencies

Mediate generalization

Behavioral systems analysis
Promotion of social validity

Use of instructional design
factors
Involvement in intervention
designTraining internal staff
Formal data collection system
Formal system of dispensing
consequences

Description
Involving the delivery of an intervention with no programmed efforts topromote generality. Any instances of generalization across settings,
behaviors, or time are simply hoped for and documented.
Involving the delivery of an intervention with no programmed efforts topromote generality. However, if generalization is not seen, additional
interventions are conducted across conditions. If generalization occurs,
the results of the intervention are simply noted.
Natural sources of reinforcement are identified in the individual’s
normal environment. The intervention is then developed to teach new
and appropriate forms of behavior that will be maintained by thenatural sources of reinforcement in that environment.
Involving contingencies in which reinforcement is delivered on anintermittent schedule, such that the instances in which reinforcement
will or will not be delivered appear undifferentiated to the recipient of
the reinforcers.
Mediation of behavior change is accomplished through the use ofverbal behavior and, more specifically, through the use of rules topromote behavior change.
Involving a comprehensive and multilevel analysis of the organization inadvance of intervention development.
Collecting information on social validity prior to development andimplementation of the intervention. Such assessments provide
information regarding conditions to promote or inhibit the temporal
generality of intervention effects within that organization.
Involving the manipulation some instructional feature, such as teachingor assessment techniques, to promote retention over time.
Internal workers (management/nonmanagement) help to develop theintervention and the features related to the intervention prior to
implementation.
Internal workers are explicitly trained to implement the intervention
themselves.
Internal workers are trained to collect data on performance measures.
Internal workers are tasked with delivering the consequences to otherworkers as part of the planned intervention.

intentional planning, but it is included for the sake of classification becauseof its common occurrence.
The second temporal generality tactic is labelled sequential modification.
This strategy is similar to intervene and hope efforts in that there is only avery minimal level of analysis dedicated to promoting generalization. Onceagain, an intervention is implemented and then any subsequent generalization is noted. However, sequential modification goes beyond passive
observation in that if generalization is not seen, additional interventions
are conducted across conditions. For example, if an intervention developed
a new behavior during a workshop session, but the behavior did not thengeneralize outside of the workshop, the intervention would then be repeatedunder additional conditions outside of the workshop. The intervention would
continue to be repeated until the behavior was reliably occurring across all
appropriate settings. In regard to temporal generality, the intervention could
be implemented and then the effects observed at a later date. If behavior

JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT

217

change was not observed at that later time, the intervention could be
implemented once again, and this pattern would continue until the behavior
change was stable over time (i.e., retraining or booster sessions). Again, thisis one of the least analytic strategies because it merely involves a simple
repetition of the intervention whenever generality fails to occur.
The third tactic is labelled introduce to natural maintaining contingencies.
In this plan, natural sources of reinforcement are identified in the individual’s normal environment. The intervention is then developed to teach new
and appropriate forms of behavior that will be maintained by the natural
sources of reinforcement in that environment. As discussed earlier, this plan
is limited to situations in which natural reinforcement exists to a sufficientdegree to reliably maintain behavior.
The fourth tactic involves the use of indiscriminable contingencies.
Indiscriminable contingencies are those in which reinforcement is deliveredon an intermittent schedule, such that the instances in which reinforcement
will or will not be delivered appear undifferentiated to the recipient of thereinforcers. Lottery systems and other types of incentive programs are among
those interventions in the OBM literature utilizing indiscriminable
contingencies.
The fifth temporal generality tactic, and the final one discussed by Stokes
and Baer (1977), is to mediate generalization. This mediation is typicallyaccomplished through the use of verbal behavior and, more specifically,
through the use of rules to promote behavior change. Rules and rule-governed behavior are commonly used as parts of multicomponent
interventions.
Additional temporal generality tactics beyond those formulated by Stokes
and Baer (1977) should be considered as well. As suggested earlier, attempting to identify and minimize sources of resistance to interventions as well asaligning the various parties and processes with the intervention may facilitate
the long-term success of intended behavior change. Such efforts are more
likely to be successful when a comprehensive analysis of the organization isconducted in advance. Thus, the use of behavioral systems analysis could beconsidered a sixth temporal generality tactic.
A common concern within OBM involves employee buy-in (Boyce &Roman, 2002). One consideration for improving buy-in is to ensure thatthe intervention is acceptable to clients—that is, the intervention has social
validity. The use of social validity assessments has been an integral part ofdetermining the impact of various interventions in the OBM literature.
Initially proposed by Kazdin (1977) and Wolf (1978), the assessment ofsocial validity in applied behavior analysis has been an important focus
since its original inception. However, current perspectives on social validity
have focused on simply asking how satisfactory various aspects of an intervention are to the individuals involved (Kennedy, 2002). Thus, the most

218

A. L. CONARD ET AL.

common use of social validity typically does not include information related
to the durability of behavior change. However, if social validity measures areincluded, they may provide information regarding conditions to promote or
inhibit the temporal generality of intervention effects within that organization. In other words, social contexts may not always support newly learned
behavior or provide an occasion for these behaviors to occur; thus, thisinformation may be used to determine the environments and social contexts
that will most likely foster these new behaviors. Thus, the promotion of social
validity is being included as a seventh temporal generality tactic.
In regard to newly learned materials, a number of strategies within behavior-based instructional design have been proposed to not only make learning efficient and effective but promote retention over time (Markle, 1990;
Tiemann & Markle, 1990). That is, in the process of ensuring that learners
have fully mastered the contents of an instructional lesson, those learners arealso more likely to retain the newly acquired knowledge or skills. This
category would include techniques such as the use of appropriate rational
sets for conceptual stimulus control or training to fluency (D. A. Johnson,2014). This category could also include less systematic attempts at instructional design as well, such as repeated practice during training. The common
attribute within this category is that the trainers attempt to improve retention
by manipulating some instructional feature, such as teaching or assessment
techniques. Thus, the use of instructional design factors will be considered as
an eighth temporal generality tactic.
Sigurdsson and Austin (2006) conducted a review of institutionalization,and their classification system proposed four strategies aimed at promoting
institutionalization, which can be included in the current article’s proposed
temporal generality tactics. As suggested by Sigurdsson and Austin, a ninth
temporal generality tactic could be considered involvement in intervention
design. With this plan, the internal workers help develop the intervention andthe features related to the intervention. There are several reasons why thisplan might promote generality, such as an increase in social validity, thepotential for internal workers to select more realistic goals, and the probability that selected incentives are more likely to be effective.
A 10th tactic involves the proposal of training internal staff. With thisstrategy the internal workers are explicitly trained to implement the intervention themselves. Another tactic to increase temporal generality is thedevelopment of a formal data collection system. As part of this plan, internal
workers are the ones who collect data. With the 12th temporal generality
tactic, formal system of dispensing consequences, the internal workers aretasked with delivering the consequences to other workers as part of theplanned intervention. With all three of these tactics, the long-term promotion of an intervention is facilitated by getting the internal workers involved

JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT

219

in the active implementation of the various aspects of the intervention before
the services of the consultant or researcher are fully withdrawn.
Each of the 12 temporal generality tactics outlined here have been used tosome extent in OBM interventions, although to varying degrees and withvarying levels of success. However, many of these categories were largely
derived from applied behavior analysis as a whole and were not examinedwithin the specific context of OBM-related situations. Although these tactics
are logical extensions that are likely to result in generality, such extensions
have not been empirically proven. Thus, there remains a need for a comprehensive review of tactics that promote temporal generality within OBM
settings.
One of the earliest studies to attempt this was conducted by Redmon and
Lockwood (1986). Although the researchers concluded that very little data
existed on the practices of maintenance, this study was never formally
published and only a minimal summary of the data can be found in a later
publication by Redmon (1991). A more formal review was later conducted byBoyce and Geller (2001), who reviewed maintenance in the area of behaviorbased safety. Boyce and Geller found that the incentive programs that weremost effective in promoting maintenance were relatively short (3–5 weeks)
and implemented in larger organizations. In addition, the most successfulprograms were those that offered workers the choice to participate in the
program and utilized on-site workers to deliver the incentives/rewards tofellow employees rather than having a researcher deliver them. According toBoyce and Geller, feedback interventions that produced maintenance sharedcertain key characteristics: (a) They focused on several behaviors, (b) feedback was dependent on overall safe performance rather than individual
behaviors, (c) feedback emphasized safe performance and recognition forsuccess, and (d) feedback was delivered by on-site workers instead ofresearchers. The majority of the research that reported successful maintenance also included some sort of training component. The authors suggested
that through the delivery of feedback on overall safe performance, maintenance is promoted by the use of indiscriminable contingencies becausedelivery of feedback was not necessarily contingent on one specific behavior
but on a set of behaviors. Although these results were promising, the review
was too limited to allow for strong claims regarding temporal generality. The
Boyce and Geller review only included 23 studies and did not consider any
applications of OBM outside of safety interventions. Furthermore, the review
looked exclusively at maintenance effects (note that incentives and feedback
were withdrawn before the assessment of behavior change) and therefore
omitted studies that could be classified as institutionalization.
As mentioned earlier, Sigurdsson and Austin (2006) conducted one of themore comprehensive reviews of temporal generality within OBM, looking
specifically at the extent to which institutionalization of behavior change

220

A. L. CONARD ET AL.

processes has been programmed into OBM interventions. The authors
included studies published between 1991 and 2002. In order to be considereda maintenance phase, follow-up or maintenance data had to be collected forat least half of the dependent variables. As previously mentioned, in addition
to follow-up data, the authors used a classification system of institutionalization as suggested by McSween and Matthews (2001) and Grindle, Dickinson,
and Boettcher (2000). The authors found that a report of follow-up data
following the termination of experimenter involvement was collected in 16 of
the 31 studies reviewed, and the scope of the duration of follow-up or
maintenance periods stretched from 10 days to 4 years.
The review by Sigurdsson and Austin (2006) discovered that the incorporation of institutionalization factors was a strong determinant of thelong-term success of behavior change and further contributed to thefield’s understanding of how to promote temporal generality. However,there are some important limitations to the paper as well. Just as Boyceand Geller’s (2001) review was largely focused on the topic of maintenance, Sigurdsson and Austin’s review was largely focused on a consideration of factors that promote institutionalization. Sigurdsson and Austin
did include reports of maintenance, but they did not carefully analyze
factors that could promote temporal generality in the absence of institutionalization. Although Sigurdsson and Austin expanded their scope toinclude all areas of OBM, not just safety as was the case with Boyce andGeller, it was still restricted to only the time period between 1991 and2002. Thus, a potentially large number of studies from both before andafter that time period were neglected. To date, we are aware of no formal
comprehensive reviews of the temporal generality of OBM interventions
that incorporate factors that may influence both maintenance andinstitutionalization.
The Journal of Organizational Behavior Management (JOBM) is a professional journal dedicated to publishing OBM research in both applied and lab
settings. Beginning with an initial review conducted by Balcazar, Shupert,
Daniels, Mawhinney, and Hopkins in 1989, there have been reviews ofstudies published in JOBM for three decades, with each review measuring
the extent to which articles published in JOBM are meeting the specified
objectives of the journal. Although these reviews were not specifically examining the tactics that promote generality, they did include measures on thecollection of formal follow-up data. Specifically, Balcazar et al. reported thatthe durability of intervention effects is not reported in more than 30% of
research reports. A decade later, Nolan, Jarema, and Austin (1999) reported
that a much larger percentage (approximately 31%) of studies in the first
decade (1977–1987) provided follow-up data than in the second decade(approximately 20%). Furthermore, the authors reported that no studies
documented the collection of follow-up data in 1987, 1989, 1994, or 1995.

JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT

221

The third review of JOBM, conducted by VanStelle et al. (2012), reported
similar results for the third decade (1998–2009), in that an average of 24% ofstudies reported follow-up data. The VanStelle et al. review demonstrated
that OBM interventions have repeatedly been shown to be effective in a wide
variety of settings and employee populations. Most of this published research
begins with the collection of baseline data in a control condition in which thetreatment of interest is not in effect, followed by the collection of data withthe treatment implemented. Depending on a number of variables, the lengthof the intervention period will vary widely among interventions. In typical
OBM research, the researcher will terminate his or her involvement in the
study following a determined length of time. Given the mission of the journal
and the fact that many of the scholars who publish within the journal doformally withdraw their services to the organization at some point, it is awell-suited outlet for analyzing the durability of behavior change over timefor OBM interventions. Thus, the purpose of this article is to review tactics
that may promote temporal generality as reported in JOBM from 1977 to2014.
MethodInclusion criteria

All studies published in JOBM between 1977 and 2014 were selected for
initial review. The titles and abstracts of all articles published within this timeframe were reviewed in order to exclude nonexperimental studies from thisstudy. Thus, only those articles that included the manipulation of at least one
independent variable were evaluated for the inclusion of follow-up measures.
Not included in this review were conceptual papers, review articles, or
articles that did not include original presentations of data. As a result, 311
articles were carefully read to determine whether the author(s) reported thecollection of follow-up data, and only those articles reporting follow-up data
were selected for further review. As seen in Table 2, the final count of articles
included in this review was 53. Articles involving multiple experiments wereseparated and independently evaluated by the reviewers. Thus, an absolute
total of 63 studies were selected for analysis.
Interobserver agreement (IOA)

The first stage of calculating IOA involved selecting articles for inclusion inthis review; consensus was assessed for nine of the 34 volumes (approximately 27%). IOA for this preliminary stage of the review was 96.1%. The
second stage of calculating IOA involved the classification of included articles. Of the 63 studies included in this review, 18 (approximately 28%) wereAnderson et al. (1983)

Author(s)Amigo et al. (2008)

Temporalgenerality tactic
Intervention(s)
Task clarification Training internal
and manager
staff
verbal andgraphic feedback
Checklist,(a) Involvement
feedback, andin intervention
token economy design, (b)sequential
modification, (c)use ofindiscriminable
contingencies,and (d)promotion ofsocial validity(a) Employees
assisted in thedevelopment of thechecklist; (b)researchers adjusted
the checklist,
implemented a
prorated pointexchange system, andmodified the roomcheck procedures; (c)supervisors randomly
evaluated one or two
rooms per week; and(d) participants weresurveyed on
preference of theintervention

Components oftactic(s)Management wastrained to deliver
feedback

Table 2. Summary of Publications Involving Temporal Generality.

3 weeks

Weekly(randomchecks)45 days

Elapsed time
betweenexperimenter
Frequency of
Duration of
involvement and follow-up data follow-up data
follow-up
collection
collection
8 weeks
Five sessions
2 weeks(26
observations)
Survey forcontinuation of
the intervention:
85% approval

Form of social
validity andfindings
N/A(Continued )~90%–110%

Percentage of
behaviorchangegenerality
54%

222
A. L. CONARD ET AL.

Checklist,
feedback, andtoken economy
Prompts andtask clarification

Anderson et al. (1983)

Verbal prompt
andreinforcement
Coles & Blunden (1981) Activity period

Clayton et al. (2014)

Clayton & Blaskewicz(2012)

Intervention
Checklist,
feedback, andtoken economy

Author(s)Anderson et al. (1983)

Table 2. (Continued).

(a) Slight
modifications to theprogram based ontrials and interactions
with staff, and (b)staff trained to
conduct theintervention(a) Involvement
in intervention
design and (b)training internal
staffIntervene andhope

Signs remained
posted with noadditional steps topromote generality
No further steps topromote generality

Components oftactic(s)(a) Staff were invited
to share viewpoints
regarding themethods of the
project, and (b) staff
repeated practice tofluency
Checklist was tested
and revised

Intervene andhope

Sequential
modification

Temporalgenerality tactic(s)(a) Involvement
in intervention
design and (b)use ofinstructional
design factors

1 week2 months

4 months

1 weekTwice per
week

Five
observations

Daily

Weeklyaverage

6 months

1 week5 days3 weeks

Elapsed time
betweenexperimenter
Frequency of
Duration of
involvement and follow-up data follow-up data
follow-up
collection
collection
1 week
Weekly4 weeks
average

N/A

N/A

N/A

N/A

Form of social
validity andfindings
N/A(Continued )46% (highvariability)

15%

117% and72% (acrosssettings)~95%–110%

Percentage of
behaviorchangegenerality~100%–115%

JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT
223

Temporal
generality tactic
Author(s)Intervention(s)Coles & Blunden (1981) Altered theSequential
activity period to modification
include feedback
to staff, ahierarchicalreportingsystem, and amultidisciplinary
management
group
Craig & Leland (1983)
Environmental
Intervene andmanipulations
hope(more disposal
areas) andprompts
Culig et al. (2008)
Ergonomic
Use ofchanges,
instructional
information,
design factors
feedback, andpraiseDeluga & Andrews
Incentive system Intervene and(1985)and training
hope

Table 2. (Continued).

No further steps topromote generality

2 months

Disposal areas and7 weeks andposters remained
14 weeksavailable following
termination ofexperimenter
Participants were
4 months
required todemonstrate masteryof positions

Average of
1 week ofdata

Once permonth

Two weekly
averages

Once for four
participants
and 6 months
for threeparticipants
1 week2 weeks

Elapsed time
betweenexperimenter
Frequency of
Duration ofComponents ofinvolvement and follow-up data follow-up datatactic(s)follow-up
collection
collection
Modified the previous
30 daysDaily
5 daysintervention to
support maintenanceof behavior change

N/A

N/A

N/A

Form of social
validity andfindings
N/A(Continued )40%–105%(highvariability
acrossparticipants)
230%, 150%,
and 90%(three DVs)

77%–102%

Percentage of
behaviorchangegenerality
149%

224
A. L. CONARD ET AL.

Evans et al. (1988)

Elder et al. (1984)

Durgin et al. (2014)

Author(s)Doucette et al. (2012)

Table 2. (Continued).

Elapsed time
betweenTemporalexperimenter
Frequency of
Duration ofgenerality tactic
Components ofinvolvement and follow-up data follow-up data
Intervention(s)tactic(s)follow-up
collection
collection
Prompt (posted Mediate
Participants were told
6 weeks,
Three weekly
3 weeks
schedule)
generalization
by supervisors that
6 months, and averages
the completion of
1 yearcertain tasks was
required to receive
salary increases
Training and job (a) Training(a) Supervisors were Two types of(a) One or two One or two
aidinternal staff and trained to evaluate
follow-up: (a)sessions andsessions(b) use ofand provide feedback approximately(b) one or two
instructionalon trainer’s
1 week following sessions
design factors
performance, and (b) the independent
supervisors received supervision phase
immediate feedback and (b)and were required to immediately
master the calculation following theof scores
intervention
6 months
Weekly6 weeks
Training and(a) Use of(a) Repeated training
group meetings instructionaland modeling ofdesign factors
performance, and (b)and (b)staff assisted in thedevelopment of theinvolvement inproceduresintervention
designLottery incentive Intervene andNo further steps to
Immediate
Daily
2 weeks
systemhope
promote generality
N/A

N/A

High treatment
acceptability
amongsupervisors andemployees

Form of social
validity andfindings
Participants rated
the intervention
as highly accepted(Continued )51%

71% and200% (acrosssettings)

first follow-up:
104%, 106%,
and 110%;second followup: 136%,
141%, and
150%

Percentage of
behaviorchangegenerality
100%

JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT
225

Feedback

Incentives

Gaetani et al. (1983)

Geller et al. (1983)

Godbey & White (1993) Monitoring,objectives,
feedback, andpraiseGodat & Brigham (1999) Training

Intervention
Signs andprompts

Author(s)Fanslow et al. (1988)

Table 2. (Continued).

Sequential
modification

Intervene andhope
Use ofindiscriminable
contingencies
Intervene andhope

Temporalgenerality tactic(s)Intervene andhope

The intervention wasreinstated after awithdrawal but
modified slightly
based on previous
audit

No further steps topromote generality
following the training
sessionComponents oftactic(s)Placed signs andprompts indesignated areas withno additional steps topromote generality
No further steps topromote generality
Free dinners for thosewearing a seatbelt

6 months

Immediate(attendance
rates) and1 month(questionnaire)

3 months and6 months
2 weeksOnce

1 month ofdata and onequestionnaire

Daily

Daily

1 month ofdata

1 month

13 days

2 daysElapsed time
betweenexperimenter
Frequency of
Duration of
involvement and follow-up data follow-up data
follow-up
collection
collection
2.5 weeks
Three
1 day
observations

The training
program helped
to improve
performance
management andwork-related
problems
N/A

N/A

N/A

Form of social
validity andfindings
N/A(Continued )

108%

98%

63% and 37%

100%

Percentage of
behaviorchangegenerality
89%

226
A. L. CONARD ET AL.

Participative
management
process, lottery
system, andrewards

Training

Manual on time
management
and workshop

Haberlin et al. (2012)

Hanel et al. (1983)

Components oftactic(s)(a) Supervisors weretrained to conduct
observations andprovide feedback,and (b) supervisors
used the Performance
Diagnostic Checklist
to assess intervention
components(a) Promotion of Conducted asocial validity and preference(b) training
assessment andinternal staff
trained supervisors on
behavioral
procedures(a) Training(a) Management wasinternal staff and trained to deliver(b) use of
training, and (b)instructional
training included role
design factors
play
Promotion ofMultiple socialsocial validity
validity measures
were taken

Temporalgenerality tactic
Intervention(s)
Task clarification, (a) Training
feedback, andinternal staff andenvironmental(b) behavioral
manipulations
systems analysis

Green et al. (2008)

Author(s)Gravina et al. (2008)

Table 2. (Continued).

4 weeks

3 months

Weeklyaverage

One measure

12 weeks (first
Weeklyparticipant) and
13 weeks (second
participant)

1 week

1 day

One measure~102% and75% (acrossDVs)

100% and116% (acrossparticipants)

Percentage of
behaviorchangegenerality
66% and
83.3% (acrossgroups)(Continued )Participants found 90%, 53%, andthe manual to be 77% (threegroups)helpful andrelevant

Participants
reported thetraining as usefulParticipants weresatisfied with theintervention

Elapsed time
betweenexperimenter
Frequency of
Duration ofForm of social
involvement and follow-up data follow-up data
validity andfollow-up
collection
collection
findings
3 monthsOnce
One
Participants
observation favored graphic
feedback

JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT
227

Involvement inintervention
designUse ofinstructional
design factors

Intervene andhope

C. M. Johnson & Masotti Goal setting,(1990)
rewards, praise,and feedback

C. M. Johnson (1985)

Jamison et al. (2014)

Hantula et al. (1993)

Intervene andhope

Temporalgenerality tactic(s)Promotion ofsocial validity

Intervention
Manual on time
management
and workshop(no instructor
contact)
Prompts andmass media
campaign
Training andovert response
requirements(presence orabsence) on
computer-based
training
performance
Feedback andincentives

Author(s)Hanel et al. (1983)

Table 2. (Continued).

Staff helped todetermine
performance goals

No further steps topromote generality

Overt response
requirements andresponse feedback

No further steps topromote generality

Components oftactic(s)Multiple social
validity measures
were taken

3 weeks and
12 weeks

1 month

1 week

Weeklyaverages

Daily

One test

2 weeks, 3 weeks, Daily
and 2 months

N/A

N/A

2 weeks

N/A

N/A

7 days

One test

3-day periods

Elapsed time
betweenexperimenter
Frequency of
Duration ofForm of social
involvement and follow-up data follow-up data
validity andfollow-up
collection
collection
findings
3 weeks
Weekly1 weekParticipants foundaveragethe manual to be
helpful andrelevant(Continued )93%, 103%,
108%, and108% (acrosssettings)
89% and 64%;
42% and 44%;
25% and 50%(two follow-up
measuresacross threeDVs)~70% and~105% (acrossDVs)
96%, 92%,98%, and102% (acrossparticipants)

Percentage of
behaviorchangegenerality
96% and
102% (twogroups)228
A. L. CONARD ET AL.

Langeland et al. (1997) Training, goal
setting,
feedback, andpraiseAuthor(s)Intervention
Johnston & Hayes
Training, goal
(2005) (Experiment 2) setting, andfeedback
Jones et al. (1985)
Training andfeedback
package
Kopelman & Schneller Mixed(1981)
consequence(employee payprocedure)
Lamal & Benfield (1978) Daily selfrecording sheets(self-monitoring)

Table 2. (Continued).

No further steps topromote generality

Intervene andhope

Coworker informed
the subject that the
number of hours
spent working is very
important and that itis required by thecompany to work 8 hreach day(a) Promotion of (a) Follow-up meetingsocial validity and in which participants
discussed the(b) mediate
intervention and any
generalizationquestions/problems
with the intervention,and (b) job
descriptions wereused as rules of
performance

No further steps topromote generality

Intervene andhopeMediategeneralization

Components oftactic(s)No further steps topromote generality

Temporalgenerality tactic(s)Intervene and
hope

Four
measurements

2, 4, and6 weeks and
1, 2, and
4 years (sixmeasures)

2 weeks

Annualaverages

Monthly
averages

11 weeks

Immediate

1 weekEach was
1 week inlength

3–4 days

2.5 years

6 months

Elapsed time
betweenexperimenter
Frequency of
Duration of
involvement and follow-up data follow-up data
follow-up
collection
collection
Did not specify
300 trials
1 dayParticipants andparents were
extremely
satisfied with theintervention(Continued )98%, 102%,
and 102%(across DVs)

Percentage of
behaviorchangegenerality
90% and209% (acrossgroups)N/A
100%, 111%,
and 109%(across DVs)
53% of employees
338%
were satisfied
with the paychangeN/A
104% and96% (across
DVs)
Form of social
validity andfindings
N/A

JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT
229

Ludwig & Geller (2000) Group goal
setting andgroup feedback(included
individual
performancedata)

Intervene andhope

Ludwig & Geller (2000) Groupdiscussions,promise cards,
and prompts
Ludwig & Geller (2000) Group goal
setting andgroup feedback
Involvement inintervention
designInvolvement inintervention
designIntervene andhope

Temporalgenerality tactic
Intervention(s)Video modeling, (a) Use ofrole play, visual instructional
prompts, anddesign factors
feedbackand (b) training
internal staff

Ludwig & Geller (1999) Prompts

Author(s)Loughrey et al. (2013)

Table 2. (Continued).

One group ofparticipants wasinvolved in setting
performance goals
Employees voted on
the group
performance goal

5.5 weeks

5–6 weeks

Weeklyaverage

Weeklyaverage

8 weeks

9–10 weeks

N/A

N/A

Elapsed time
betweenexperimenter
Frequency of
Duration ofForm of social
Components ofinvolvement and follow-up data follow-up data
validity andtactic(s)follow-up
collection
collection
findings(a) Participants were
2 weeksOnce
One session The intervention
required to
was rated bydemonstrate mastery,
participants asand (b) supervisors
satisfactory andwere trained to
usefuldeliver feedback
No further steps toApproximately
At least nine
5 months
More than 50% of
promote generality
1 monthobservations(2,440participants
per dayobservations) reported that theywould use their
seatbelt with
program in placeNo further steps to
3–4 weeks
Weekly3 months
N/A
promote generality
average(Continued )65% and 61%(acrossgroups)88% and 72%(acrosssettings)

101% and59% (acrosssettings)

96%

Percentage of
behaviorchangegenerality
101% and93% (acrossparticipants)

230
A. L. CONARD ET AL.

McCuddy & Griggs(1984)

Temporalgenerality tactic(s)
Involvement inintervention
designIntervene andhope

Goal setting and Involvement infeedbackintervention
designLudwig & Geller (2000) Individual goal
setting andfeedback

Author(s)Intervention
Ludwig & Geller (2000) Rewards andfeedback

Table 2. (Continued).

Employees
participated in goal
setting, andmanagers conducted
all employee
meetings/reviews

Components oftactic(s)Participants
determined therewards
No further steps topromote generality

Did not specify

1 weekNot specified

Weeklyaverage

Approximately
1 year

4 weeks

Elapsed time
betweenexperimenter
Frequency of
Duration of
involvement and follow-up data follow-up data
follow-up
collection
collection
1 week
Weekly2–3 weeks
average

N/A

N/A

Form of social
validity andfindings
N/A(Continued )Percentage of
behaviorchangegenerality
91% and 67%(acrosssettings)
116%, 91%,
and 62%(acrossgroups)Informalreports frommanagement
suggestcontinued
success

JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT
231

Intervention
Training,
feedback, andsocialrecognition

Nordstrom et al. (1988) Group andindividual
incentives

Nordstrom et al. (1988) Goal setting,feedback, andincentives
Nordstrom et al. (1988) Feedback

Author(s)Myers et al. (2010)

Table 2. (Continued).

Intervene andhopeIntervene andhopeIntervene andhope

Temporalgenerality tactic(s)(a) Involvement
in intervention
design, (b)training internal
staff, (c) use ofindiscriminable
contingencies,and (d)sequential
modification

No further steps topromote generality

No further steps topromote generality

Components oftactic(s)(a/b) Certain
employees
participated incommittees in which
they worked with theconsultants todevelop theintervention andhelped train other
employees, (c)employees were toldthat they would beperiodically observedby coworkers andmanagement, and (d)the reward program
was altered following
the pilot
implementation
No further steps topromote generality

1 year

1 year

1 yearEvery 2 weeks

Four to sevenobservations
per day
Did not
specify

18 weeks

2 weeks

75
observations

Elapsed time
betweenexperimenter
Frequency of
Duration of
involvement and follow-up data follow-up data
follow-up
collection
collection
Immediate
AnnualApproximately
9 years

N/A

N/A

N/A

Form of social
validity andfindings
N/A(Continued )~40% and~60% (acrossgroups)26% and 53%(acrossgroups)77%

Percentage of
behaviorchangegenerality~115%–210%(somevariability)

232
A. L. CONARD ET AL.

Rose & Ludwig (2009)

Use ofinstructional
design factors

Involvement inintervention
designTemporalgenerality tactic(s)(a) Involvement
in intervention
design and (b)formal datacollection system

Staff helped todetermine
performance goals

Components oftactic(s)(a) Staff and client
families determined
the target behaviors,and (b) senior staff
conducted
observations andprovided feedback tocoworkers

Utilized various
instructional design
techniques (repeatedtesting, overt
response
requirements, andrepeated exposure tocorrect answer)(a) Management andTask clarification, (a) Training
self-monitoring, internal staff and employees weretrained on theand performance (b) behavioral
systems analysis checklist, and (b) use
feedbackof a PIC/NIC analysis

Author(s)Intervention
Porterfield et al. (1985) Various
procedures toincrease staff
organization,
staff–family
communication,and clients’recreational
activities
Ralis & O’Brien (1987)
Prompts, goal
setting,
feedback, andpraiseRohlman et al. (2004)
Training

Table 2. (Continued).

6 weeks

1 week and1 month

6 weeks

Daily

Two tests

Weeklyaverages

3 weeks

Two tests

2 months

N/A

N/A

N/A

(Continued )58%

85% (1 week)
and 79% (1month)

90% and 38%(across DVs)

Elapsed time
between
Percentage ofexperimenter
Frequency of
Duration ofForm of social
behavior
involvement and follow-up data follow-up data
validity andchangefollow-up
collection
collection
findings
generality
14 weeks
Daily
1 weekStaff were99%
satisfied with thechanges, andclients’ families
reported apositive change instaff behavior

JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT
233

Silva et al. (1981)

Feedback andpraiseIntervention
Feedback andreinforcement
system
Sasson & Austin (2005) Training,
feedback, andemployeeobservations

Author(s)Runnion et al. (1978)

Table 2. (Continued).

Training internal
staff(a) Use of
instructional
design factors
and (b) formal
data collection
systemTemporalgenerality tactic(s)Training internal
staff(a) Participants were
4 months
required todemonstrate masteryand meet criterion
levels, and (b)participants were
tasked with observing
and collecting dataon coworker
performance
Management was
Immediate
trained to plot
attendance and later
trained in the theory
and use of social
reinforcement

Components oftactic(s)Trained management
to provide feedback

Daily average

Three or four
sessions(dependingonparticipant)

12 daysThree or four
sessions

Elapsed time
betweenexperimenter
Frequency of
Duration of
involvement and follow-up data follow-up data
follow-up
collection
collection
4 weeks
Weekly4 weeks
averages

N/A

Participants weresatisfied with theintervention

Form of social
validity andfindings
N/A(Continued )

104%

~105%

Percentage of
behaviorchangegenerality
98%

234
A. L. CONARD ET AL.

Training, goal
setting, andfeedback

Group meetingand personalcommitment(promise card)

Author(s)Slowiak (2014)

Stephens & Ludwig(2005)

Streff et al. (1993)
Intervene andhope

Involvement inintervention
designTemporalgenerality tactic
Intervention(s)
Task clarification, Involvement ingoal setting,intervention
feedback, anddesignrewards

Table 2. (Continued).

No further steps topromote generality

Nurses participated insetting performancegoals

1 month and2 months

5 weeks

Elapsed time
betweenexperimenter
Components ofinvolvement andtactic(s)follow-up
Participants
Two types: (a)completed aimmediate andreinforcer assessment (b) 5 months

3 weeks

Percentage of
behaviorchangegenerality
Correct
greeting
components:(a) 94% and(b) 82%;friendly tone:(a) 91% and(b) 108%;
abandon rate:(a) 86% and(b) 135%~116%–155%(Continued )The nurses
reported highlevels of
satisfaction andthey reported thatenvironmental
factors may havebeen confounds
N/A
Two follow-up
Four
381 and 341(safety glasses)
periods: 87%
observations
observations
and 95% (first(1-monthDV) and 36%
follow-up) and and 442 and205 (seatbelt)
and 49%threeobservations(second DV)
observations(2-monthfollow-up)

Five to eightobservations
over 3 weeks

Frequency of
Duration ofForm of social
follow-up data follow-up data
validity andcollection
collection
findings(a) Once(a) 7 weeks This information
weekly andand (b) one was not available(b) onesessionto researchers
sessionJOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT
235

Intervention
Performance
scorecards,
training, teammeetings, andreinforcement

Monetary
incentives

Feedback andfeedback plus
praiseAuthor(s)Szabo et al. (2012)

Thurkow et al. (2000)

Wikoff et al. (1983)

Table 2. (Continued).

(Continued )Elapsed time
between
Percentage of
Temporalexperimenter
Frequency of
Duration ofForm of social
behavior
generality tactic
Components ofinvolvement and follow-up data follow-up data
validity andchange(s)tactic(s)follow-up
collection
collection
findings
generality
Training internal Supervisors were
3 months,
Monthly
Three probes (a) Supervisors
99%
staff
trained to implement 6 months, andwere satisfied
the intervention
12 months
with training, (b)componentsclients weresatisfied after 2
years of clinicalconsultation
services, and (c)stakeholders weresatisfiedIntervene andNo further steps to
2 months
Daily
2 days80% preferred
~110%
hope
promote generality
individual
incentives(a) Training(a) Supervisors were Immediate
Weekly8 weeks
N/A~90%–105%
internal staff and coached to contact(b) involvement and praise
in intervention
employees, and (b)designselected participants
were interviewed todevelop the rating
instrument

236
A. L. CONARD ET AL.

Intervention
Feedback andgoal setting

Training andfeedbackTraining, job
aids, praise, andfeedback(feedbackpackage)

Author(s)Wilk & Redmon (1997)

Wilson et al. (1997)

Wittkopp et al. (1991)

Table 2. (Continued).

Components oftactic(s)(a) An employee
assessment of
workplace problems
was used to develop
the intervention, (b)supervisors weretrained on behavioral
techniques, and (c)supervisors wererequired todemonstrate masteryof the material(a) Training(a) Supervisors were
6 weeks
internal staff and trained to provide(b) sequential
feedback to staff; andmodification(b) supervisors wereexperiencing practicalproblems withperformancemonitoring, so theintervention wasalteredTraining internal Trained supervisors to Immediate
staff
conduct audits

Temporalgenerality tactic(s)(a) Involvement
in intervention
design, (b)training internal
staff, and (c) use
of instructional
design factors

WeeklyDaily

10–15 weeks

3 weeks

N/A

N/A

Elapsed time
betweenexperimenter
Frequency of
Duration ofForm of social
involvement and follow-up data follow-up data
validity andfollow-up
collection
collection
findings
Approximately
Every
1 weekParticipants were
30 weeks
4–6 weekssatisfied with theintervention(Continued )~90%, ~102%,~101%, and~98% (acrossparticipants)

68%, 116%,
92%, and 98%(across
participants)

Percentage of
behaviorchangegenerality
112%, 115%,
97%, and105% (across
DVs)
JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT
237

Temporal
generality tactic(s)(a) Involvement
in intervention
design and (b)formal system ofdispensing
consequences

Elapsed time
betweenexperimenter
Frequency of
Duration ofComponents ofinvolvement and follow-up data follow-up datatactic(s)follow-up
collection
collection(a) Internal staff
Immediate andTwice per
12 weeksparticipated in the
6 monthsweek(immediate)
development of theand 4 weeks
token system, and (b)(6 month)
continued supervisor
praise and otherreinforcement
following terminationof token system

Note. N/A = not applicable (article did not report any form of social validity measures); DV = dependent variable.

Author(s)Intervention
Zohar & Fussfeld (1981) Token economy

Table 2. (Continued).

Form of social
validity andfindings
N/A

Percentage of
behaviorchangegenerality~100% (first
follow-up) and
51% (second
follow-up)

238
A. L. CONARD ET AL.

JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT

239

independently reviewed, and IOA was 94.6%. For both stages, IOA wascalculated by using the number of agreements divided by the number ofagreements plus disagreements. The disagreements were then analyzed anddiscussed between both reviewers until an agreement was reached.

Categories and definitions
Intervention
All intervention components were classified exactly as described by theoriginal article’s author. If the study involved a multicomponent intervention, then all components of the intervention were recorded.
Temporal generality tactic
Articles were classified as including one or more of the following tactics: (a)intervene and hope, (b) sequential modification, (c) introduce to natural
maintaining contingencies, (d) use of indiscriminable contingencies, (e)mediate generalization, (f) behavioral systems analysis, (g) promotion ofsocial validity, (h) use of instructional design factors, (i) involvement inintervention design, (j) training internal staff, (k) formal data collection
system, or (l) formal system of dispensing consequences. We coded articles
based on the descriptions provided by Stokes and Baer (1977) and Sigurdssonand Austin (2006) as well as the information provided in the introduction ofthis article. If a study incorporated more than one temporal generality tactic,
multiple tactics were recorded. Furthermore, only those tactics with a direct
influence were recorded as such. For example, if a study reported training
internal staff or incorporating social validity information, it was not recorded
as involvement in intervention design because this tactic specifically refers to
direct involvement in the intervention design.
Components of tactics
The specific component(s) of each generality tactic were also recorded. For
example, if the use of instructional design factors was listed as a tactic, thespecific components of that tactic were recorded (e.g., trained to mastery,
incorporation of fluency training, repeated testing).
Elapsed time between experimenter involvement and follow-up
The total amount of time that passed between the last data point involving
active intervention by the experimenter and the first data point of the followup phase was recorded. If follow-up data collection began immediately
following the experimenter’s termination, the elapsed time between experimenter involvement and follow-up was recorded as “immediate.” If theauthor(s) reported multiple follow-up probes, each time period was recorded.

240

A. L. CONARD ET AL.

Frequency of follow-up data collection
The frequency of follow-up data collection was recorded as the average rateof data collection throughout the follow-up period. For example, if follow-up
data were collected every Monday throughout the follow-up period, thefrequency of follow-up data collection was recorded as weekly.
Furthermore, if several dependent variables were recorded at different frequencies throughout the follow-up period, multiple frequencies wererecorded. For example, if two dependent variables were both measured
weekly, a weekly frequency of data collection was recorded for both.
However, if one dependent variable was collected daily and the other wascollected weekly, both daily and weekly frequencies were recorded.
Duration of follow-up data collection
The total amount of time that elapsed between the first data point and thelast data point of the follow-up period was also recorded.
Form of social validity and findings
Details on social validity measures were also documented for each of thestudies. If an article did not report any form of social validity measures, thecategory was labeled “N/A.” If a study included social validity measures, thefindings are reported exactly as described by the article’s author.
Percentage of behavior change generality
The total amount of behavior change that occurred between the intervention
phase and follow-up phase was recorded as a percentage. The reviewers
determined the percentage of change by comparing the average performancefor the intervention period(s) to the average performance for the follow-up
period. In studies that did not report statistical comparisons of follow-up
performance to intervention performance, the reviewing author relied ongraphical representations of the data as well as the written judgements of theauthor(s). In such cases, a tilde was used to indicate a judgement by theauthor(s). If an article reported a range of scores, the average of that range
was recorded as the total performance change. If multiple follow-up periods,
dependent variables, settings, groups, or participants were reported, eachpercentage was recorded separately.

Results
Generality tactics

As seen in Figure 1, the most common tactic for promoting generalization
among the 63 studies reviewed was to intervene and hope (21 studies).
Following this popular tactic was involvement in intervention design (17Percentage of Studies

JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT

241

40
35
30
25
20
15
10
5
0

Figure 1. Percentage of temporal generality tactics reported in the Journal of Organizational
Behavior Management.

studies) and training internal staff (16 studies). Lesser utilized generality
tactics were the use of instructional design factors (10 studies), sequential
modification (six studies), promotion of social validity (five studies), the useof indiscriminable contingencies (three studies), mediate generalization(three studies), behavioral systems analysis (two studies), formal data collection system (two studies), and formal system of dispensing consequences(one study). The introduce to natural maintaining contingencies tactic was
not utilized by any of the studies reviewed. Of the 63 studies reviewed, 45
included only one generality tactic, whereas 15 studies incorporated two
generality tactics. Far less common was the combination of three tactics(one study) or four (two studies).

Comparison of intervention types

In the 63 studies analyzed for follow-up data, intervention type was recorded
exactly as presented by the article’s author. Of the 63 studies, 33 (52.3%)
included feedback, either in isolation or in combination. Training was anintervention component for 19 (30.1%) of the 63 studies, whereas incentive
systems were an intervention component for 13 (20.6%) of the total studies.
Lesser utilized interventions were goal setting (19.0%), prompts (15.8%),
praise (12.6%), reinforcement/rewards (11.1%), task clarification (9.5%), job
aids/checklist (9.5%), group meetings (6.3%), environmental manipulations(6.3%), self-monitoring (3.1%), promise cards (3.1%), performance monitoring (3.1%), activity periods (3.1%), social recognition (1.5%), performance

242

A. L. CONARD ET AL.

scorecards (1.5%), multidisciplinary management group (1.5%), announcements/mass media campaign (1.5%), and various procedures (1.5%).
Duration of data collection

The average duration of follow-up data collection was 4.03 weeks (28.2 days)for 48 of the 63 studies. In 12 of the 63 studies, the duration of follow-up datacollection was not included in the average because the authors of these studies
did not report the follow-up period in days, weeks, months, or years. Those
durations omitted from the average were reports of three probes, one or two
sessions, one measure (six studies), 75 observations, two tests, three or four
sessions, 205–381 observations, approximately 1 year, approximately 9 years,
and 2.5 years. Furthermore, if the duration of follow-up data collection
exceeded 1 year, the duration was not included in the average. This wasdone because only a few studies exceeded 1 year, but those outliers had extreme
values that greatly skewed the average numbers. These outliers were from threestudies in which one author reported follow-up data collection for approximately 1 year, another for 2.5 years, and the most extreme for approximately
9 years. The most common durations of follow-up data collection were 1 weekand 2 weeks (four studies), and the range of follow-up data collection spanned
from 1 day to approximately 9 years.
Elapsed time between experimenter involvement and follow-up datacollection

The average elapsed time from the termination of the experimenter’s involvement to the follow-up period was approximately 5.1 weeks (36.1 days). In
16 of the 63 studies, the elapsed time was not included in the average because
the author did not report the time period in hours, days, weeks, months, or
years. Furthermore, if the author reported a time period greater than
30 weeks, the duration was not included in the average. Those omitted hadtime periods of immediately (10 studies), did not specify (two studies), 1 year
(three studies), and 30 weeks (one study). If the aforementioned studies areincluded in the average, the average elapsed time increases to 8.1 weeks(56.8 days). The most frequently reported time period was 1 week (sevenstudies), and the range of elapsed time was immediately to 2 years. It should
be noted that if a study included more than one follow-up period, only theelapsed time to the first follow-up period was included in the average.
Frequency of follow-up data collection

The most commonly reported frequency of follow-up data collection was
weekly, with 21 studies (33.3%) reporting this frequency. Daily data

JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT

243

collection (12 studies) and a single follow-up measure (six studies) were thenext most common frequencies of follow-up data collection. Less commonly
reported frequencies were monthly (four studies), annually (two studies),
twice per week (two studies), nine per day (one study), four to seven per day(one study), every 4–6 weeks (one study), and every 2 weeks (one study).
There were a total of 11 studies for which the author did not report thefrequency of data collection in specific units of time. Rather, the data werereported in terms of sessions, observations, trials, measurements, or tests
without any indication of the amount of time passing between data points.
Form of social validity and findings

In the 63 studies reviewed for follow-up data, 20 of the studies (approximately 31.7%) included some sort of social validity measure. The mostcommonly utilized social validity measure was a participant/staff survey,
with 15 of the 20 studies reporting this measure. The next most commonsocial validity measure was a customer/client survey (four studies) followed
by a supervisor survey (one study). All of the authors who reported usingsocial validity measures stated that the findings revealed satisfaction with theintervention.
Percentage of performance change

In the 63 studies reviewed for follow-up data, a total of 127 performancemeasurements were reported (across participants, settings, groups, or dependent variables). Of the 127 performance scores, 45 measures (35.4%) reported
a performance change of 101% or higher. With a total of 30 measures, thenext most frequently reported percentage of performance change was 61%–
90%, followed closely by 91%–100% with 29 measures. The next mostcommon percentage of performance change was 31%–60% with 18 measures.
Lastly, only two measures reported a performance change of 0%–30%.
Discussion
The results of this review suggest that temporal generalization was frequently
planned for or at least assessed in many of the studies reported in JOBM
between 1977 and 2014. Of the 311 articles reviewed, 53 included some formof follow-up data. Given that many laboratory studies were omitted because
institutionalization is not always possible outside the context of an organization (although generalization in the form of maintenance could still beassessed), the substantial portion of articles collecting follow-up data is
encouraging. The results of this study also indicate notable improvementsin the percentage of performance change for those studies reporting follow-

244

A. L. CONARD ET AL.

up data. It is important to note that the percentage of performance change is
one of the most important aspects of this review, given that such changes are
representative of longevity. These improvements suggest that those OBM
practitioners and researchers planning for generalization of behavior change
are successful in their goal of maintaining and increasing performance
improvements. Nevertheless, based on the findings of this review, it seems
warranted to recommend that OBM researchers and practitioners continue
to plan for generalization if the goal of the intervention is to attain lasting
performance improvements.
The majority of studies reporting the collection of follow-up data wereclassified as utilizing the generality tactic of intervene and hope. As previously mentioned, studies falling within this classification simply assessed
the levels of targeted performance following the withdrawal of direct consultant/researcher services without an explicit strategy to improve the durability of changes. Although it is laudable that follow-up measures are being
collected, there is clearly much potential for a more extensive approach tomaximizing the value of the average intervention. This absence of a comprehensive approach can also be shown by the infrequency of temporal generality tactic combinations (i.e., including more than one generality tactic). To be
fair, there is a possibility that the use of more than one generality tactic is not
worth the additional resources, and additional tactics are not necessary forsuccessful performance change. It is also possible that some generality tactics
may conflict with one another or that certain packages of generality tactics
may be redundant. Nevertheless, these are empirical questions that have notbeen answered. It is just as likely that the reason is simply neglect and that
the OBM community has not done enough as a field to truly understand theutility of generality tactics. Given this finding, further investigation of generality tactics and combinations thereof may be worthwhile in understanding
the most effective methods of promoting generalization.
On a related note, the results of this study indicate that certain generality
tactics have never been or are rarely assessed. For example, behavioral
systems analysis, formal data collection system, formal system of dispensing
consequences, and introduce to natural maintaining contingencies wererarely utilized in the published JOBM articles reviewed. Some possible reasons for the lack of some generality tactics relate to the cost and timerequired to use these tactics. This is of particular concern with the behavioral
systems analysis tactic, as this particular tactic is labor intensive and may not
be necessary for further behavior change. As for the formal data collection
system and formal system of dispensing consequences tactics, time andresources may also be a potential barrier to their use. Lastly, nearly all, if
not all, OBM interventions are used with verbally sophisticated adults, and as
a result it may not be necessary to gradually expose them to natural maintaining contingencies. Instead of slowly shaping the performer’s behavior

JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT

245

through direct contingencies, intervention specialists could simply communicate the consequences for the targeted behavior and then allow the contingencies underlying rule-governed behavior to immediately promote
behavior change.
Unfortunately, we currently do not have the research to fully explain
the reasoning behind the implementation of some tactics and theabsence of others. Until we conduct empirical research to better understand the effects of certain generality tactic combinations and the effectiveness of various tactics, we will continue to use these strategies
ignorant of their potential effects. In accordance with this statement,
the present review found no studies that used a control group tocompare the effects of different generality tactics or generality tactic
combinations. As with any treatment, it is necessary to conduct comparative research to discover the optimal use of intervention strategies,
but again there has never been a comparative study on temporal generality in OBM research. Until such empirical evidence is collected, the fieldof OBM will remain blind to the impact of certain generality tactics withcertain types of tasks or interventions. For example, it may be the casethat the training internal staff and involvement in intervention design
tactics is the most effective combination for maintaining performancewith feedback interventions. If this were the case, would incentive
packages then see the same amount of success with this combination?
At this point in time, most researchers are simply implementing anintervention with only one or two generality tactics and hoping for the
best. In order for OBM to maintain and further grow its reputation forgenerating lasting behavior change, it is important for the field toinvestigate the temporal generality tactics that will promote such behavior change over an extended period of time.
The findings of this review also indicate a clear deficiency of social validity
measures in the OBM literature. Because social validity measures are a meansof assessing intervention acceptability and significance of results, such measures are key for the lasting effects of any intervention. As previously mentioned, social validity measures may provide information regarding
conditions to promote the temporal generality of intervention effects within
an organization. In other words, this information may be used to determine
the environment and social context that will most likely foster these new
behaviors. If there is buy-in among those responsible for carrying out theintervention following the removal of the intervention specialist, there is amuch greater chance of success following the termination of the researcher
or consultant. Given this, information on social validity should be an integral
part of designing any intervention to maintain successful performancechange. Considering the lack of current understanding of temporal generality
tactics, combinations of tactics, effect on various dependent variables, and246

A. L. CONARD ET AL.

effect on social validity, there is too much that researchers do not know aboutsomething so important to the field. This is not a criticism of previous
research, nor is it invalidation of OBM interventions, but this is certainly
an area of research that warrants greater attention.
The short duration of follow-up periods frequently reported by authors
is also worth mentioning. Again, the average duration of the follow-up
period was approximately 4.03 weeks, with 1 and 2 weeks being the mostfrequently reported durations. Considering the significant amount of time
researchers spend collecting baseline and experimental data, follow-up
data collection seems to be lacking attention. Most researchers spend asignificant amount of time on baseline and experimental phases whileneglecting possibly one of the most important phases of the project.
Although the collection of follow-up data is an arduous task and competes
with the timely demands of graduation, tenure, supervision, and newclients, the durability of performance change remains an issue for organizations. Because an intervention’s lasting effect is likely a primaryconcern for those within any organization, there remains a potential
conflict of priorities between the intervention specialist and the organization. Given the high pressure for researchers, particularly those in academia, to publish frequently rather than spend excessive time collecting data
for a single publication, the primary goals of the researcher are often in
conflict with the goals of the organization.
This issue brings about the practical obstacle in determining how
researchers can arrange support for such time-intensive projects. Again,
many researchers are subject to publication demands that do not easily
allow for lengthy follow-up periods. Many graduate students are also
under pressure to meet graduation deadlines and therefore avoid thoseprojects that require a lengthy time demand or simply neglect follow-up
data collection altogether. Furthermore, once students leave the institution
where they are collecting research data and go on to get a job, there islittle incentive to follow up with the project. Although these obstacles will
likely remain challenges in the future, there are some possible ways toencourage more research that could demonstrate temporal generality. For
example, many consulting firms develop long-term relations with clientsthat may provide an opportunity to examine temporal generality strategies. Most consultants are not under strong pressures to publish frequently and therefore can afford a more extensive time period for datacollection, although this brings up the challenge of creating incentives for
consultants to publish at all. Perhaps collaborative efforts between academics and practitioners could surmount this challenge created by theneeds and opportunities related to the publication of time-extensive
research, because consultants are more motivated to foster long-termclient relations and academics are more motivated to publish accounts

JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT

247

of these relations. Alternatively, perhaps publication outlets could encourage the submission of secondary articles focused solely on follow-up.
That is, years after an original primary study is published, a secondary
study could be published examining whether the effects found in the
primary study were maintained or institutionalized by the organization.
Even if performance change did not endure across time, the secondary
study could be quite valuable if it detailed the reasons why the intervention failed in the long run. Such lessons learned would provide an
opportunity for the development or refinement of temporal generality
strategies in order to foster more effective interventions in the future.
One possible limitation of the current review is that it does not reportall of the possible mechanisms used to sustain changes, as we werelimited to publicly accessible resources. For example, some consulting
firms may have developed strategies for promoting generalization, andthere may be more extensive in-house knowledge on how to foster
temporal generality; however, by their nature as trade secrets they are
not publically known. This review may consequently undersample theknown knowledge on the topic. Furthermore, consultants are typicallybrought into an organization by top management, and thus it is management who supports, mandates, and typically follows up on implementation following the removal of the consultant. This approach includes
both formal and informal social and nonsocial consequences, so it does
not quite fit with any of the tactics described in this article. Although
this approach may be considered a part of the formal system of dispensing consequences tactic, it does not fully account for this strategy by
top management. Despite these limitations, it is important to note that
this was the most representative sample that we had access to, and JOBM
is the best outlet for the current review.
There may be additional strategies for promoting temporal generalitythat are not highlighted in this review. For example, the difficulty of theintervention will likely have an impact on whether maintenance of
behavior change is achieved. Furthermore, there may be internal workers
who champion the intervention and help maintain it, which is likely an
important factor in achieving temporal generality. Lastly, leader support
is often a very important factor in the long-term maintenance of interventions. None of the strategies outlined in this article addressed thesetactics.
The generality tactics outlined in this review may serve as a roadmap
for future research as well as provide inspiration for consultation. The
potential of the 12 tactics for improving the durability of performancechange over time has not been well investigated. This represents a richsource of research studies on the relative effectiveness of each of thesetactics, both in isolation and in various combinations. These tactics also

248

A. L. CONARD ET AL.

offer conceptually sound guidelines for consultants looking to extend thereach of the interventions and services they are already offering to clients.
Given the clear importance of maintaining performance change over time,those concerned with the growth of OBM and OBM interventions should
be interested in determining the most effective means of maintaining thisperformance change. This represents a potentially important advancementin building this science and selling its services to the world at large.
References
Asterisks indicate works included in the meta-analysis.
*Amigo, S., Smith, A., & Ludwig, T. (2008). Using task clarification, goal setting, and feedback
to decrease table busing times in a franchise pizza restaurant. Journal of Organizational
Behavior Management, 28(3), 176–187. doi:10.1080/01608060802251106*Anderson, D. C., Crowell, C. R., Sponsel, S. S., Clarke, M., & Brence, J. (1983). Behavior
management in the public accommodations industry. Journal of Organizational Behavior
Management, 4(1–2), 33–66. doi:10.1300/J075v04n01_02
Austin, J., & Wilson, K. G. (2001). Response-response relationships in organizational behavior management. Journal of Organizational Behavior Management, 21(4), 39–53.
doi:10.1300/J075v21n04_04
Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some current dimensions of applied
behavior analysis. Journal of Applied Behavior Analysis, 1, 91–97. doi:10.1901/
jaba.1968.1-91
Balcazar, F. E., Shupert, M. K., Daniels, A. C., Mawhinney, T. C., & Hopkins, B. L. (1989). An
objective review and analysis of ten years of publication in the Journal of Organizational
Behavior Management. Journal of Organizational Behavior Management, 10(1), 7–37.
doi:10.1300/J075v10n01_02
Boyce, T., & Geller, E. S. (2001). Applied behavior analysis and occupational safety. Journal of
Organizational Behavior Management, 21, 31–60. doi:10.1300/J075v21n01_03
Boyce, T. E., & Roman, H. R. (2002). Institutionalizing behavior-based safety: Theories,
concepts, and practical suggestions. The Behavior Analyst Today, 3(1), 76–82.
doi:10.1037/h0099962
Brethower, D., & Smalley, K. (1998). Performance-based instruction: Linking training tobusiness results. San Francisco, CA: Jossey-Bass/Pfeiffer.
*Clayton, M. C., & Blaskewicz, J. (2012). The use of visual prompts to increase the cleanlinessof restrooms on a college campus. Journal of Organizational Behavior Management, 32(4),329–337. doi:10.1080/01608061.2012.729393*Clayton, M. C., Blaskewicz Boron, J., & Mattila, L. (2014). Child safety in grocery stores:
The impact of verbal prompts and reinforcement on safety strap use in shopping carts.
Journal of Organizational Behavior Management, 34(1), 52–58. doi:10.1080/
01608061.2013.873380*Coles, E., & Blunden, R. (1981). Maintaining new procedures using feedback to staff, ahierarchical reporting system, and a multidisciplinary management group. Journal of
Organizational Behavior Management, 3(2), 19–33. doi:10.1300/J075v03n02_03*Craig, H. B., & Leland, L. S. (1983). Improving cafeteria patrons’ waste disposal. Journal of
Organizational Behavior Management, 5(2), 79–88. doi:10.1300/J075v05n02_05*Culig, K. M., Dickinson, A. M., Lindstrom-Hazel, D., & Austin, J. (2008). Combining
workstation design and performance management to increase ergonomically correct

JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT

249

computer typing postures. Journal of Organizational Behavior Management, 28(3), 146–
175. doi:10.1080/01608060802251064*Deluga, R. J., & Andrews, H. M. (1985). A case study investigating the effects of a low-costintervention to reduce three attendance behavior problems in a clerical training program. Journalof Organizational Behavior Management, 7(3–4), 115–124. doi:10.1300/J075v07n03_07
Diener, L. H., McGee, H. M., & Miguel, C. F. (2009). An integrated approach for conducting
a behavioral systems analysis. Journal of Organizational Behavior Management, 29(2), 108–135. doi:10.1080/01608060902874534*Doucette, S., DiGennaro Reed, F. D., Reed, D. D., Maguire, H., & Marquardt, H. (2012).
Implementation of a posted schedule to increase class-wide interobserver agreement
assessment. Journal of Organizational Behavior Management, 32(3), 263–269.
doi:10.1080/01608061.2012.698187*Durgin, A., Mahoney, A., Cox, C., Weetjens, B. J., & Poling, A. (2014). Using task clarification and feedback training to improve staff performance in an East African nongovernmental organization. Journal of Organizational Behavior Management, 34(2), 122–143.
doi:10.1080/01608061.2014.914007*Elder, J. P., Sundstrom, P., Brezinski, W., Waldeck, J. P., Calpin, J. P., & Boggs, S. R. (1984).
An organizational behavior management approach to quality assurance in a community
mental health center. Journal of Organizational Behavior Management, 5(3–4), 19–35.
doi:10.1300/J075v05n03_02*Evans, K. M., Kienast, P., & Mitchell, T. R. (1988). The effects of lottery incentive programson performance. Journal of Organizational Behavior Management, 9(2), 113–135.
doi:10.1300/J075v09n02_08*Fanslow, J. L., Leland, L. S., Craig, T., Hahn, H., Polonowita, R., & Teavae, S. (1988).
Reducing passive smoking by promoting self segregation of smokers using signs andthematic prompts. Journal of Organizational Behavior Management, 9(2), 23–34.
doi:10.1300/J075v09n02_03
Feeney, E. J., Staelin, J. R., O’Brien, R. M., & Dickinson, A. M. (1982). Increasing sales
performance among airline reservation personnel. In R. M. O’Brien, A. M. Dickinson, &M. P. Rosow (Eds.), Industrial behavior modification: A management handbook (pp.
141–158). New York, NY: Pergamon Press.
Fleming, R. K. (2001). Diverse problems call for diverse analyses and intervention in
behavioral safety: A commentary on Boyce and Geller. Journal of Organizational
Behavior Management, 21(1), 65–69. doi:10.1300/J075v21n01_05
Foxx, R. M. (2013). The maintenance of behavioral change: The case for long-term followups. American Psychologist, 68(8), 728–736. doi:10.1037/a0033713*Gaetani, J. J., Johnson, C. M., & Austin, J. T. (1983). Self management by an owner of a small
business. Journal of Organizational Behavior Management, 5(1), 31–39. doi:10.1300/
J075v05n01_04*Geller, E. S., Davis, L., & Spicer, K. (1983). Industry-based incentives for promoting seat belt
use. Journal of Organizational Behavior Management, 5(1), 17–29. doi:10.1300/
J075v05n01_03*Godat, L. M., & Brigham, T. A. (1999). The effect of a self-management training program on
employees of a mid-sized organization. Journal of Organizational Behavior Management,
19(1), 65–83. doi:10.1300/J075v19n01_06*Godbey, C. L., & White, A. G. (1993). Increasing the accuracy of computerized summaries ofcourt case activity. Journal of Organizational Behavior Management, 13(1), 113–127.
doi:10.1300/J075v13n01_07

250

A. L. CONARD ET AL.

*Gravina, N., VanWagner, M., & Austin, J. (2008). Increasing physical therapy equipment
preparation using task clarification, feedback and environmental manipulations. Journal of
Organizational Behavior Management, 28(2), 110–122. doi:10.1080/01608060802100931*Green, C. W., Reid, D. H., Passante, S., & Canipe, V. (2008). Changing less-preferred duties
to more-preferred: A potential strategy for improving supervisor work enjoyment. Journalof Organizational Behavior Management, 28(2), 90–109. doi:10.1080/01608060802100899
Grindle, A. C., Dickinson, A. M., & Boettcher, W. (2000). Behavioral safety research inmanufacturing settings: A review of the literature. Journal of Organizational Behavior
Management, 20(1), 29–68. doi:10.1300/J075v20n01_03*Haberlin, A. T., Beauchamp, K., Agnew, J., & O’Brien, F. (2012). A comparison of pyramidal
staff training and direct staff training in community-based day programs. Journal of
Organizational Behavior Management, 32(1), 65–74. doi:10.1080/01608061.2012.646848*Hanel, F., Martina, G. L., & Koop, S. (1983). Field testing of a self-instructional time
management manual with managerial staff in an institutional setting. Journal of
Organizational Behavior Management, 4(3–4), 81–96. doi:10.1300/J075v04n03_08*Hantula, D. A., Stillman, F. A., & Waranch, H. R. (1993). Can a mass-media campaign
modify tobacco smoking in a large organization? Evaluation of the Great American
Smokeout in an urban hospital. Journal of Organizational Behavior Management, 13(1),33–47. doi:10.1300/J075v13n01_04*Jamison, C. A., Kelley, D. P., Schmitt, C., Harvey, M. T., Harvey, A. C., & Meyer, E. (2014).
Impact of an overt response system on staff performance and maintenance of computerbased instruction. Journal of Organizational Behavior Management, 34(4), 279–290.
doi:10.1080/01608061.2014.973630*Johnson, C. M. (1985). Customer feedback to the main office. Journal of Organizational
Behavior Management, 7(1–2), 37–50. doi:10.1300/J075v07n01_04*Johnson, C. M., & Masotti, R. M. (1990). Suggestive selling by waitstaff in family-stylerestaurants. Journal of Organizational Behavior Management, 11(1), 35–54. doi:10.1300/
J075v11n01_04
Johnson, D. A. (2014). The need for an integration of technology, behavior-based instructional design, and contingency management: An opportunity for behavior analysis. Revista
Mexicana De Análisis De La Conducta, 40, 58–72.
Johnson, D. A., Casella, S. E., McGee, H., & Lee, S. C. (2014). The use and validation of preintervention diagnostic tools in organizational behavior management. Journal of
Organizational Behavior Management, 34, 104–121. doi:10.1080/01608061.2014.914009*Johnston, M. R., & Hayes, L. J. (2005). Use of a simulated work setting to study behaviorbased safety. Journal of Organizational Behavior Management, 25(1), 1–34. doi:10.1300/
J075v25n01_01*Jones, H. H., Morris, E. K., & Barnard, J. D. (1985). Increasing staff completion of civil
commitment forms through instructions and graphed group performance feedback.
Journal of Organizational Behavior Management, 7(3–4), 29–43. doi:10.1300/
J075v07n03_03
Kazdin, A. E. (1977). Assessing the clinical or applied significance of behavior change throughsocial validation. Behavior Modification, 1, 427–452. doi:10.1177/014544557714001
Kazdin, A. E. (1994). Behavior modification in applied settings. Pacific Grove, CA: Brooks/
Cole.
Kennedy, C. H. (2002). The maintenance of behavior change as an indicator of social validity.
Behavior Modification, 26(5), 594–604. doi:10.1177/014544502236652

JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT

251

Kessler, M. L. (2001). Transitioning from maintenance to institutionalization: Critical additions to the Boyce and Geller analysis. Journal of Organizational Behavior Management, 21(1), 71–73. doi:10.1300/J075v21n01_06*Kopelman, R. E., & Schneller, G. O. (1981). A mixed-consequence system for reducing
overtime and unscheduled absences. Journal of Organizational Behavior Management, 3(1),17–28. doi:10.1300/J075v03n01_02*Lamal, P. A., & Benfield, A. (1978). The effects of self-monitoring on job tardiness andpercentage of time spend working. Journal of Organizational Behavior Management, 1(2),142–149. doi:10.1300/J075v01n02_05*Langeland, K. L., Johnson, C. M., & Mawhinney, T. C. (1997). Improving staff performancein a community mental health setting. Journal of Organizational Behavior Management, 18(1), 21–43. doi:10.1300/J075v18n01_03*Loughrey, T. O., Marshall, G. K., Bellizzi, A., & Wilder, D. A. (2013). The use of video
modeling, prompting, and feedback to increase credit card promotion in a retail setting.
Journal of Organizational Behavior Management, 33(3), 200–208. doi:10.1080/
01608061.2013.815097
Ludwig, T. D. (2001). On the necessity of structure in an arbitrary world: Using concurrent
schedules of reinforcement to describe response generalization. Journal of Organizational
Behavior Management, 21(4), 13–38. doi:10.1300/J075v21n04_03*Ludwig, T. D., & Geller, E. S. (1999). Behavior change among agents of a community safety
program. Journal of Organizational Behavior Management, 19(2), 3–24. doi:10.1300/
J075v19n02_02*Ludwig, T. D., & Geller, E. S. (2000). Intervening to improve the safety of delivery drivers.
Journal of Organizational Behavior Management, 19(4), 1–124. doi:10.1300/J075v19n04_01
Malott, R. W. (2001). Occupational safety and response maintenance: An alternate view.
Journal of Organizational Behavior Management, 21(1), 85–102. doi:10.1300/
J075v21n01_08
Markle, S. M. (1990). Designs for instructional designers. Champaign, IL: Stipes.
*McCuddy, M. K., & Griggs, M. H. (1984). Goal setting and feedback in the management of aprofessional department. Journal of Organizational Behavior Management, 6(1), 53–64.
doi:10.1300/J075v06n01_04
McSween, T., & Matthews, G. A. (2001). Maintenance in safety behavior management.
Journal of Organizational Behavior Management, 21, 75–83. doi:10.1300/J075v21n01_07*Myers, W. V., McSween, T. E., Medina, R. E., Rost, K., & Alvero, A. M. (2010). The
implementation and maintenance of a behavioral safety process in a petroleum refinery.
Journal of Organizational Behavior Management, 30(4), 285–307. doi:10.1080/
01608061.2010.499027
Nolan, T. V., Jarema, K. A., & Austin, J. (1999). An objective review of the Journal of
Organizational Behavior Management: 1987-1997. Journal of Organizational Behavior
Management, 19(3), 83–115. doi:10.1300/J075v19n03_09*Nordstrom, R., Hall, R., Lorenzi, P., & Delquadri, J. (1988). Organizational behavior
modification in the public sector. Journal of Organizational Behavior Management, 9(2),91–112. doi:10.1300/J075v09n02_07*Porterfield, J., Evans, G., & Blunden, R. (1985). Involving families and staff in service
improvement. Journal of Organizational Behavior Management, 7(1–2), 117–134.
doi:10.1300/J075v07n01_08*Ralis, M. T., & O’Brien, R. M. (1987). Prompts, goal setting and feedback to increasesuggestive selling. Journal of Organizational Behavior Management, 8(1), 5–18.
doi:10.1300/J075v08n01_02

252

A. L. CONARD ET AL.

Redmon, W. K. (1991). Pinpointing the technological fault in applied behavior analysis.
Journal of Applied Behavior Analysis, 14(3), 441–444. doi:10.1901/jaba.1991.24-441
Redmon, W. K., & Lockwood, K. (1986, May). A content analysis of selected behavioral
consultation publications. Paper presented at the meeting of the Association for Behavior
Analysis, Milwaukee, WI.
*Rohlman, D. S., Eckerman, D. A., Ammerman, T. A., Fercho, H. L., Lundeen, C. A.,
Blomquist, C., & Anger, W. K. (2004). Quizzing and feedback in computer-based andbook-based training for workplace safety and health. Journal of Organizational Behavior
Management, 24(3), 1–26. doi:10.1300/J075v24n03_01*Rose, H. M. S., & Ludwig, T. D. (2009). Swimming pool hygiene: Self monitoring, task
clarification, and performance feedback increase lifeguard cleaning behaviors. Journal of
Organizational Behavior Management, 29(1), 69–79. doi:10.1080/01608060802660157*Runnion, A., Johnson, J. O., & McWhorter, J. (1978). Energy savings in interstate transportation through feedback and reinforcement. Journal of Organizational Behavior
Management, 1(3), 181–191. doi:10.1300/J075v01n03_02*Sasson, J. R., & Austin, J. (2005). The effects of training, feedback, and participant involvement in behavioral safety observations on office ergonomic behavior. Journal of
Organizational Behavior Management, 24(4), 1–30. doi:10.1300/J075v24n04_01
Sigurdsson, S. O., & Austin, J. (2006). Institutionalization and response maintenance inorganizational behavior management. Journal of Organizational Behavior Management,
26, 41–77. doi:10.1300/J075v26n04_03*Silva, D. B., Duncan, P. K., & Doudna, D. (1981). The effects of attendance contingent
feedback and praise on attendance and work efficiency. Journal of Organizational Behavior
Management, 3(2), 59–69. doi:10.1300/J075v03n02_07*Slowiak, J. M. (2014). “How may I help you?” Improving telephone customer service in a
medical clinic setting. Journal of Organizational Behavior Management, 34(1), 39–51.
doi:10.1080/01608061.2013.873382*Stephens, S. D., & Ludwig, T. D. (2005). Improving anesthesia nurse compliance with
universal precautions using group goals and public feedback. Journal of Organizational
Behavior Management, 25(2), 37–71. doi:10.1300/J075v25n02_02
Stokes, T. F., & Baer, D. M. (1977). An implicit technology of generalization. Journal ofApplied Behavior Analysis, 10, 349–367. doi:10.1901/jaba.1977.10-349*Streff, F. M., Kalsher, M. J., & Geller, E. S. (1993). Developing efficient workplace safety programs.
Journal of Organizational Behavior Management, 13(2), 3–14. doi:10.1300/J075v13n02_02*Szabo, T. G., Williams, W. L., Rafacz, S. D., Newsome, W., & Lydon, C. A. (2012).
Evaluation of the service review model with performance scorecards. Journal of
Organizational Behavior Management, 32(4), 274–296. doi:10.1080/01608061.2012.729408*Thurkow, N. M., Bailey, J. S., & Stamper, M. R. (2000). The effects of group and individual
monetary incentives on productivity of telephone interviewers. Journal of Organizational
Behavior Management, 20(2), 3–25. doi:10.1300/J075v20n02_02
Tiemann, P. W., & Markle, S. M. (1990). Analyzing instructional content: A guide to instruction and evaluation. Champaign, IL: Stipes.
VanStelle, S. E., Vicars, S. M., Harr, V., Miguel, C. F., Koerber, J. L., Kazbour, R., & Austin, J.
(2012). The publication history of the Journal of Organizational Behavior Management: An
objective review and analysis: 1998–2009. Journal of Organizational Behavior Management,
32, 93–123. doi:10.1080/01608061.2012.675864

JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT

253*Wikoff, M. B., Anderson, D. C., & Crowell, C. R. (1983). Behavior management in a factory
setting. Journal of Organizational Behavior Management, 4(1–2), 97–128. doi:10.1300/
J075v04n01_04*Wilk, L. A., & Redmon, W. K. (1997). The effects of feedback and goal setting on the
productivity and satisfaction of university admissions staff. Journal of Organizational
Behavior Management, 18(1), 45–68. doi:10.1300/J075v18n01_04*Wilson, C., Boni, N., & Hogg, A. (1997). The effectiveness of task clarification, positive
reinforcement and corrective feedback in changing courtesy among police staff. Journal of
Organizational Behavior Management, 17(1), 65–99. doi:10.1300/J075v17n01_04*Wittkopp, C. J., Rowan, J. F., & Poling, A. (1991). Use of a feedback package to reducemachine set-up time in a manufacturing setting. Journal of Organizational Behavior
Management, 11(2), 7–22. doi:10.1300/J075v11n02_02
Wolf, M. M. (1978). Social validity: The case for subjective measurement or how applied
behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11, 203–214.
doi:10.1901/jaba.1978.11-203*Zohar, D., & Fussfeld, N. (1981). Modifying earplug wearing behavior by behavior modification techniques. Journal of Organizational Behavior Management, 3(2), 41–52.
doi:10.1300/J075v03n02_0