# Presentation: 814541

## SAMHSA/CSAP Prevention Fellowship TrainingImplementation Workshop: SPF Step 4Day 2

- Carol Oliver, TA and Training ManagerCSAP’s Northeast Center for the Application of Prevention Technologies
- Wayne M. Harding, Ed.M., Ph.D.
- Director of Projects
- Social Science Research & Evaluation, Inc.

## Learning Objectives

- List at least four criteria that should be considered when selecting strategies.
- List the steps in implementation.
- Identify the role of cultural competency within each of the implementation steps.
- Describe the external and internal influences that can affect successful implementation.
- Understand the concepts of fidelity and adaptation and identify cultural considerations in adapting evidence-based practices.
- Explain why implementation is critical to program success and how it relates to strategy selection and outcomes.

## Section I:Overview of the Day

## Agenda

- Welcome and Review of Day 2
- Adaptation and Fidelity: A Closer Look
- Factors That Facilitate Implementation
- Putting It All Together: Implementation Panel

## Spirit of the Day

## Section II: Adaptation and Fidelity:  A Closer Look

- Wayne M. Harding, Ed.M., Ph.D.
- Director of Projects
- 	  Social Science Research & Evaluation, Inc.
- 	Burlington, MA 01803
- 781-273-4206  wharding@ssre.org

## Topics

**Topics**

- Define fidelity and adaptation
- Advantages and disadvantages of replication vs. adaptation
- Why is this an issue (why not just replicate)?
- Guidelines for adaptation
- Monitoring fidelity across multiple program sites (e.g., in a state)

**Notes:**
These are the major topics I plan to cover:
 I am going to define fidelity and adaptation.
 Discuss the advantages & disadvantages of  replication vs.  adaptation.  Think of replication as the the most rigorous or most stringent form of fidelity when you are implementing a program.
 Why adaptation is this an issue (Why not just avoid 	the possibility that adaptation may decrease a 	program’s effectiveness by replicate programs – by 	implementing them with high fidelity? 
  I will offering you some guidelines for 	adapting programs.  BUT, before you get all excited about this,  I should warn you that these guidelines are not all you or I would like.  During the last few 	years, there has been a lot of interest in research on  adaptation, but there is still a great deal more to  learn. 
 Most of the material is about the fidelity and adaptation of single prevention programs or interventions, I will close by introducing you to a fidelity assessment tool that was designed to monitor fidelity in multiple program sites.  For example, a state agency might use this tool to monitor implementation of the prevention programs it funds. 

## Focus

**Focus**

- Evidence-based programs to prevent substance abuse.
- A prevention activity is judged to be evidence-based if “good” research (research shown to be rigorous according to a set of defined criteria) demonstrates that:
- 1. The activity produced the expected positive       	  results, and
- 2. The results can be attributed to the activity	     	  rather than to other factors.

**Notes:**
		The state of the art concerning fidelity and adaptation varies depending on the type of program on which you are focusing.  For example the measurement of fidelity is more advanced for substance abuse treatment programs than for prevention  programs.  I am focusing on prevention programs to prevent substance abuse. 
		For the most part I am going to be talking about fidelity and adaptation as they apply to implementation of evidence-based interventions.  A prevention activity is usually judged to be science- based if good research shows that it produces the expected positive results and if the results can be clearly attributed to the activity rather than to other factors. As many of you know, those other factors are usually referred to as threats to internal validity or rival hypotheses. One example, perhaps the most important example,  is the possibility that people would have improved even if they had not been exposed to the prevention activity. 

## Why Is Understanding Fidelity Important?

**Why Is Understanding Fidelity Important?**

-  Information about fidelity is essential to 	interpreting outcomes.
-  Fidelity reveals information about 	program feasibility. (Dusenbury et al., 2003)

**Notes:**
		There are at least two main reasons that it is important to understand the fidelity with with a program is implemented.
   The first is that it helps explain why an intervention succeeds or fails. For example, if a program failed to produce the expected outcomes, this may be because it was not implemented as designed.  It may not have lasted as long as as it was supposed to, or it may have been delivered to a population it wasn’t designed for.  We will come back to the relationship between fidelity and outcomes in a few minutes.
		The second reason fidelity is important is that the extent to which a program can be delivered with high fidelity is an indication of how feasible it is to implement the program.  Fidelity can tells us something about how easy or difficult it is   to disseminate the program as it was designed.  Also, Information about fidelity collected across implementations may point to program features that may need to be modified because they are difficult to implement well.  For example, in the Massachusetts SIG (State Incentive Grant) we found that all the mentoring programs experienced a common problem recruiting mentors. 
  Now let’s look at the definitions of fidelity and adaptation.

## Fidelity: Did You Keep It the Same?

- 	“The degree of fit between the developer-defined components of a program and its actual implementation in a given organizational or community setting.”  (Backer, 2002)

**Notes:**
Fidelity is all about whether an intervention was implemented  as it was designed. 

There is no generally accepted definition of either fidelity or adaptation, but this definition of fidelity by Tom Backer is one of the most common.   “The degree of fit between the developer-defined components of a program and its actual implementation in a given organizational or community setting.” 

## Fidelity—A Complication

- There are two types:
- Fidelity to the original program
- Fidelity to the proposed/planned program
- (Cummins et al., 2002)

**Notes:**
Almost all discussions of fidelity in the literature concerns the extent to which the way the program that is implemented matches the original or a model program.  There is, however, another level of fidelity. 

If you start out to implement a program in an adapted form, then you can also ask if the way you actually implement it matches the way in which you proposed to deliver the now adapted program. 

As the next slide shows, these two types of fidelity have different implications for how you assess or track fidelity.

## Fidelity—A Complication (continued)

- With already adapted programs, document modifications to the original program prior to implementation, and then track adaptations relative to the modified program

**Original**

**Proposed**

**Implemented**

- With replications (or implementations of innovative programs), track fidelity to the original program

**Notes:**
  When you implement a model program (or when you implement an innovative program of your own design) you assess  or track fidelity to the original program.  This is shown by the top line on the diagram that runs from the implemented program back to the original program.

  When you implement a program that you have already adapted, things are a little more complicated, as shown by the blue lines on the bottom of the diagram.  You should document how the adapted program differs from the original (it was 9 sessions long instead of ten, as it was originally designed.).  You also need to track whether you implement the adapted program as you planned to.  Did you deliver 9 sessions as you had planned?  

## Adaptation— How Did You Change It? 

- “Deliberate or accidental modification of the program.” (Backer, 2002)
- Types: additions, deletions, or modifications to the content, delivery method, target population, setting, or delivery agent (Formica & Harding, 2001)

**Notes:**
  Adaptation is about whether and how you changed a program. There are two ideas on the slide I want to emphasize.
  The first is the idea that adaptations can be accidental (or unplanned) as well as planned.  You can start off planning to replicate a program and encounter unexpected circumstances that force an adaptation.  For example, you can start out planning to implement all 10 sessions of a school-based prevention program, and wind up delivering only 9 because one session is cancelled due to a snow storm, and the school isn’t willing to let you make up the missing session. 
  The second thing I want to emphasize is that adaptation is a complicated idea.  Adaptations can be additions, deletions or modifications to the program and they may apply to program content, the method of delivery, the target population, the setting in which the program is delivered, or the agent who delivers it.  I’ll come back to these categories in a minute.

## Adaptation—Complications

- Adaptation may continue.
-     (Formica and Harding, 2001)
- Adaptations to the evaluation are potentially as significant as adaptations to the program. (Formica and Harding, 2001)

**Notes:**
Just as there were some complications in defining fidelity, there are complications concerning adaptation. . . Some issues that are not addressed by the usual, standard definitions of adaptation.
 •Adaptation may continue.  Adaptation is not necessarily a one-time event. It may take place at the outset of the program -- I decide to translate a program into Spanish to make it more appropriate for my target audience. But, adaptations may continue as the program continues.   You may make on adaptation when the program begins, another half way though and so on.
 It appears from limited evidence, that multiple adaptations are likely in situations where there are multiple iterations of a program -- when, for example, a school-based program is delivered to two or more cohorts of students -- 7th graders this year, 7th graders next year...  Why does this happen?  Because people implementing the program learn from experience how it works and what might be changed to improve it in the next iteration with new participants.
		GO TO NEXT PAGE FOR MORE TEXT

## Some Key Features of Comprehensive Fidelity Assessment

- Assess fidelity to original or model program and to proposed (adapted) program
- Assess fidelity at intervals
- Assess all aspects of the program (e.g., content, delivery methods, target population, setting delivery agent)
- Assess adaptations to evaluation methods (in both original and proposed program)
- Assess rationale for adaptation

**Notes:**
   Before we discuss some of the controversy surrounding fidelity versus adaptation, I want to use what I have said about the definitions of these concepts to point out some of the issues you should track when assessing fidelity.
   Assess fidelity to original or model program and to proposed (adapted) program. 
   Since adaptations can occurs over time, it is important to assess fidelity at intervals.  You night do this  once every 6 months, or halfway through a program and again at the end.
   It is also important to assess all the dimensions of a program for fidelity to the original program: content, delivery methods, population served, etc.). We will come back to these. 
   You should also assess adaptations to evaluation methods (in both original and proposed program).
   And finally it is important to assess the rationale for an adaptation. It is important to learn for example whether the change was planned or unplanned.  Knowing why an adaptation was made can help forecast how necessary it may be to repeat it in the future. 

## Opposing Viewpoints

- 	“Fidelity is related to effectiveness and any bargaining away of fidelity will most likely decrease program effectiveness.  There is very little experimental evidence on the impact of local enhancements or modifications on the effectiveness of programs.” (Elliott & Mihalic, 2004)

**Notes:**
There are different views about the dangers and the value of adaptation.  This quote is a sample of the conservative position. 

“Fidelity is related to effectiveness and any bargaining away of fidelity will most likely decrease program effectiveness.  There is very little experimental evidence on the impact of local enhancements or modifications on the effectiveness of programs.”

The danger, being spoken to here, is that adaptation of a program may reduce its effectiveness. In fact the authors this this is likely.  At the same time, however, the authors acknowledge that we don’t know much about what will happen. 

## Opposing Viewpoints

- 	“Adaptation may render a program more responsive to a particular target population. Adaptation could increase a program’s cultural sensitivity and its fit within a new implementation setting . . . cultural adaptation, for example, has been found necessary to engage the interest of prevention program participants.” (Schinke, Brounstein, & Gardner, 2002)

**Notes:**
  Here is an example, of the more liberal position: 
“Adaptation may render a program more responsive to a particular target population.  Adaptation could increase a program’s cultural sensitivity and its fit within a new implementation setting... cultural adaptation, for example, has been found necessary to engage the interest of prevention program participants.”  
	As you can see this argument moves beyond the argument that adaptation may be needed, to suggest that, it some cases, it may enhance program effectiveness. 
  For example, Karol Kumpfer looked at five culturally-adapted versions of her Strengthening Families Program.  This included adaptations such as including culturally relevant examples, graphics, stories, and reduced reading level.  Although the original program had slightly better outcomes than the culturally-adapted versions, retention among attending families was 41% better for the culturally-adapted versions. So in this case, the gamble of adapting the program was probably worth it

## The Great Compromise

- 	“Attention to BOTH fidelity and adaptation is essential for successful implementation of science-based prevention programs.” (Backer, 2002)

**Notes:**
In the last few years, controversy about fidelity versus adaptation has given way to a compromise position.  The compromise position which you se on the slide,  essentially agues that fidelity to the original program on the one hand must be balanced with judicious or wise adaptation on the other hand.

Tom  Backer is one of the strongest advocates of balancing fidelity and adaptation.

This sound logical enough, the problem is knowing when to adapt and when not to.  Or, put another way, how to make adaptations without jeopardizing the effectiveness of the model or original program.  As we will see, there are few guideline about how to do this. 
.. 

## Why Is This an Issue?

- Replication can be hard.
- Limited supply of evidence-based programs makes adaptation likely.

**Notes:**
 Why is this such a big deal?  Why  not  avoid adaptation?  Why not simply replicate programs with high fidelity and thereby avoid the risk of compromising their effectiveness?
There are at least three reasons why this advice is hard to follow. 
The first is that, as you know,  it can be very hard to replicate.  You need a precise road map for implementation and one may not be available, or it may lack some important details.  Also,  you need to monitor implementation carefully to keep the program in line with the original, and you may need ready access to the developer/evaluator of the original program if problems develop. 
			GO ONTO NEXT PAGE FOR TEXT

## Adaptation Happens

- Between 23% and 81% of program activities may be omitted during implementation. (Durlak, 1998)
- Only 19% of schools implementing research-based curricula with fidelity. (Hallfors & Godette, 2002)
- Only about 75% of the students received 60% or more of the Life Skills Training Program. (Botvin et al., 1995)

**Notes:**
Although fidelity to the original program is the safer course of action, it appears that adaptation is commonplace. I say “appears to be commonplace” because the research evidence is limited.  I have listed three studies here as examples.
 1. The first statement comes from a from a review of research literature including 9 studies that reported data on fidelity.
2. The second citation is from a 1999 survey of 81 Safe and Drug Free School Coordinators in 11 states.  For science- based programs, the researchers asked questions of a sub-sample of coordinators (N=58) to about the extent to which they were implemented as designed  (whether teachers followed the curriculum, whether the program was delivered to the appropriate grade levels. Common problems included lack of teacher training, lack of required materials, lack of exposure to parts of the program. 
			GO TO TEXT ON NEXT PAGE

## Adaptation Happens Under Restricted Conditions: MassCALL

- The RFP, site visits, evaluation plans, and biannual reports all required information about adaptations.
- Approval for adaptation required by MassCALL.
- MassCALL adopted conservative position regarding adaptations.

**Notes:**
 Understanding the frequency with which adaptation takes place requires knowledge about the efforts in place to regulate it. This is rarely reported.
 The Massachusetts state Incentive Grant, called MassCALL, systematically collected information about adaptations made across local subrecipients through RFP requirements, site visits, evaluation plans required from each site, and in biannual reports.    
  Also, subrecipients were required to seek approval for adaptation was required by the funder. (And they did so almost all of the time.)
 In principle, and to a large degree in practice, adaptation was to be allowed only when clearly needed given the local situation, or when unexpected events occurred.  The system wasn’t perfect, but it was designed to constrain evaluation.  So what happened?

## Adaptation Happens: MassCALL Types and Frequency of Adaptations Among MassCALL Sub-Recipients (Formica & Harding, 2001) 

**Notes:**
First we found that all of the 21 sites made one or program adaptations.   
Second we found that adaptation occurred for all of the types of  programs. The 5 types (mentoring, family-based, etc) are listed in the left hand column.
Third we found that adaptation occurred for every major aspect of the programs.  We classified the adaptations into 5 types shown in the columns:  content, delivery methods, target population, setting, and the delivery agent. 
Fourth, we found that adaptation was common -- look along the bottom row and you will see (adding up the numbers)  that we identified 83 adaptations across the 21 programs
[I omitted data about environmental strategies, because they don’t tend to have standard curricula against which to judge fidelity.]

## Adaptation Happens: MassCALL

- All 21 program sites made one or more program adaptations.
- Adaptations occurred for all program types.
- The adaptations applied to all aspects of the programs (content, delivery methods, target population, setting, delivery agent).
- Conservatively, 83 adaptations were made.
-  (Formica & Harding, 2001)

**Notes:**
I gave you a slide summarizing the findings I just mentioned. 

##     Adaptation Happens: MassCALL

**Content: **Strengthening Families Program adapted for Cambodian population
**Delivery Methods:** Families in Action delivered to combined vs.separate groups of parents and children
**Target Population: **SMART Moves, recipients changed from 14–17 to 12–14
**Setting: **Life Skills Training implemented in community center after school versus in school
**Delivery Agent: **For Across Ages, age of mentors  changed from over 55 years to 25 and over

**Notes:**
This slide presents some examples of what is meant by the various types of adaptations, content delivery methods, target population, and so on.. .
Let’s look at two of these.
 Under setting the example given is Gill Botvin’s Life Skills program that was implemented in a community center after school, rather than in school.  This is an adaptation we approved because as schools have become more focused on generated acceptable scores on standardized tests, they have become less willing to give up instructional time to  prevention programs. We also knew the evaluator was highly skilled and could in fact provide the outcome data of interest.  The result was a failure. Anybody care to guess why? It appeared that low dosage due to inconsistent attendance was the problem.  Kids are not required to attend the community Center.   
			GO TO NEXT SLIDE

## Guideline 1: Select Programs 

**Guideline 1: ****Select Programs **
**with the Best Initial Fit**

- 	In order to minimize adaptation, select programs with the best initial fit to local needs and conditions. (Goddard and Harding, 2002)
- Resources (e.g., time, money)
- Target population (e.g.,does the target population have a positive history with the organization?)
- Organizational climate (e.g., is there buy-in of key staff?)
- Community climate (e.g., is the community willing to engage in another program?)
- Evaluability (e.g., does the practitioner have the tools and skills needed to evaluate the program?)
- Future sustainability (e.g., likelihood that the program can be sustained once “seed” money ends?)

**Notes:**
I am going to close with 7 guidelines about adaptation. 
Perhaps  the single most important step to take in order to minimize the need for adaptation is to select an intervention that best fits local needs and conditions.
This may seem obvious BUT data about the distribution of science-based programs suggests that  they may be selected more according to their familiarity (their brand name recognition)  than  according to whether they are the best fit with local conditions.   Out of dozens of science-based programs a handful account for the bulk of all the programs that are implemented.   
The list on the slide represents areas that are important to consider when screening potential programs for their suitability to local conditions. .   

## Distribution of Evidence-Based Programs

- Partial data from a 2003 national survey of 191 middle school prevention coordinators. The list includes only those programs defined as science-based by either USED or CSAP.

**Notes:**
I wanted to show you how disproportionate the distribution of evidence-based programs can be. In 2003 we did a survey of 191 middle school coordinators funded by USED to help prevent substance abuse.  We asked them to identify the prevention programs that were being used in their school districts.  

The results showed that a handful of programs dominated this marketplace. 48% of the programs implemented were Life Skills, 41% Project Alert, and 40% Second Step. The rest accounted for 16% or less of the programs.  These data suggest that school personnel are choosing those programs that have the highest brand recognition, rather that those that best fit their situation. 

## Guideline 2: Select Programs 

**Guideline 2: ****Select Programs **
**with the Largest Effect Size**

- 	In general, a program with a large effect size is less likely to have the relevant outcome reduced by an adaptation than a similar program with a small effect size.

**Notes:**
Effect size is a statistical measure of the amount of change a program produces.  It is especially useful for comparing outcomes across programs. One important aspect of effect size is that it is independent of  statistical significance...You can have a program with a highly significant result but a low effect size.  That is the amount of change it produced was small.   You can learn more about this from your friendly neighborhood evaluator. 
A program with a large effect sizes should,  on average stand up adaptation better than a similar  program with a small effect size. If there is a reduction in its effectiveness, its more likely that some residual effect will be left than when you start out with a small effect. 
		GO ON TO NEXT PAGE

## Guideline 3: 

**Guideline 3: **
**Change Capacity ****Before Program**

-  It may be easier to change the program,  but changing local capacity to deliver it as it was designed is a safer choice. (Goddard and Harding, 2002)

**Notes:**

I have seen many examples when the first response to a bad fit  is to think about how to change the program.  The first response should be to think about how to change your capacity or local conditions to better fir the program. 

For example, if a program requires the use of elderly mentors, think about improving your ability to recruit this type of mentor, before you decide to use younger mentors to solve your recruitment problem.

## Guideline 4: 

**Guideline 4: **
**Consult with the Program Developer**

-     Consult with the program developer to determine what experience and/or advice he or she has about adapting  the program to a particular setting and circumstances. (Backer, 2002)

**Notes:**
Some program developers track instances in which their program has been adapted and their impact on program outcomes.  Some developers conduct research about adaptation of their program.  So it makes send to ask them what they know about the possible consequences of an adaptation you are considering.  Karol Kumpfer, who I mentioned earlier, tracks adaptations made to her Strengthening Families program
If you can’t get guidance from the program developer, you might try and locate a preventionists who has implemented the program using the same adaptation you are considering. Single state agencies for substance abuse may help you locate others who have adapted the program.  You might also try various listserves.

## Guideline 5:  

**Guideline 5:  **
**Retain Core Components**

- There is a greater likelihood of effectiveness when a program retains the core component of the original intervention.
-  We still know little about the core components 	of specific programs.
-  Look to what developers identify as  core 	components or “implementation essentials.”

**Notes:**
 Retaining the core components of a program (those elements that contribute most to the programs’ outcomes) is good advice.  Unfortunately, a listing of core components is hard to come by. 
 I would, again, look to  the program developer for help. Ask them what evidence they have about core components for their specific program also,  I also refer you a CSAP publication: Science-Based Prevention Programs and Principles 2002 by Schinke, Brounstein and Gardner. For each model program it identifies "Implementation Essentials."  These are not core components in a formal sense, but they do identify parts of the program that the developer feels you should try to avoid adapting. You can order this resource on the SAMHSA website or view it online. It is listed in the bibliography for this presentation. The next slide shows an excerpt from the website.

## Excerpt for Across Ages from

**Excerpt for Across Ages from**
** ****SAMHSA Model Programs Web Site**

- “To replicate with fidelity, programs must:
-   Use all program components
-   Have mentors who are 55 years or older
-   Implement State or agency approved screening and training of  	mentors that includes 8 to 10 hours of preservice training and monthly inservice meetings
-  	Provide training and orientation for all participants
-  	Provide stipends or reimbursement to mentors
-  	Vigilantly monitor the mentor-youth matches
-  	Prepare written agreements between collaborating organizations
-   Staff the program adequately (i.e., a minimum of one full-time and  	one part-time staff person for 30 youth and 15 to 20 mentors).”

**Notes:**
 This is  a partial list of information that appears in a description of the Across Ages prevention program, developed by Andrea Taylor.

As you cab see, it provides guidance about which features of this mentoring program Andrea feels are essential.  These include paying a stipend to mentors, and  a minimum staff size of one full-time and one part-time person for every 30 youths.   

## Guideline 6: Be Consistent with Science-Based Principles

**Guideline 6: Be Consistent with Science-Based Principles**

-  There is a greater likelihood of success if an 	adaptation does not violate a science-based 	prevention *principle*. (Brounstein, Zweig, & Gardner, 	1998).
-  Example of prevention principle: Peer leaders 	are an effective prevention delivery method.

**Notes:**
An adaptation you make should be consistent with science-based principles.   For example, if the program that contains a peer leadership component, it is important to consider retaining this element as there is considerable evidence of the effectiveness of this delivery mechanism.  There is a resource for information about science-based principles in the bibliography. (“Science-Based Principles in Substance Abuse Prevention:  A Guide.” (Brounstein, Zweig, & Gardner, 1998).  

## Guideline 7:  

**Guideline 7:  **
**Add Rather Than Subtract**

- It is safer to add to a program than to modify or to subtract from it. (Brounstein, 2003)

**Notes:**
This is common sense.  If you subtract from a program you risk removing key elements that may account for its effectiveness. And, thinking back to the discussion of core components, the crucial elements may not be known or obvious.  

## Small-Group Activity Adaptation Guidelines 

## 	Summary of Guidelines

- 1.** **Select Programs with the Best Initial Fit
- 2. Select Programs with the Largest Effect Size
- 3. Change Capacity Before Program
- 4. Consult with the Program Developer
- 5. Retain Core Components
- 6. Be Consistent with Science-Based Principles
- 7. Add Rather Than Subtract

## 	Small-Group Questions

- State Level
- Which guidelines could the State assist local implementers in ensuring?
- How could the State ensure that the local guidelines were followed?

## Small-Group Questions

- Community Level
- Which of the guidelines do you feel would be most challenging to follow at the local level and why?
- What recommendations might you give to overcome these barriers?

## Tool Kit for Assessing

**Tool Kit for Assessing**
** ****Fidelity and Adaptation**

- Applies to “any” program
- Designed to assess fidelity across programs
- Treats adaptation as ongoing
- Distinguishes between fidelity to original 	program and fidelity to proposed (already adapted) program

**Notes:**
Your other handout is a Toolkit for Assessing program fidelity and Adaptations.  I am not going to review this. It was designed to be self-explanatory.  These last 2 slides list some of its key features. ,

## Tool Kit for Assessing

**Tool Kit for Assessing**
** ****Fidelity and Adaptation**

- Addresses range in types of adaptations (content, duration, delivery method, target population, setting, delivery agent, etc.)
- Tracks evaluation changes
- Collects data about reasons for adaptations
- Designed to assess fidelity across programs

**Notes:**

QUESTIONS... COMMENTS

## References

**References**

- Backer, T.E. (2002). *Finding the balance: Program fidelity and adaptation in substance abuse prevention: A state-of-the art review*. Rockville, MD: Substance Abuse and Mental Health Services Administration (SAMHSA), Center for Substance Abuse Prevention (CSAP).
- Botvin, G.J., Baker, E., Dusenbury, L., Botvin, E.M., and Diaz, T. (1995). Long-term follow-up results of a randomized drug abuse prevention trial in a white middle-class population. *Journal of the American Medical Association*, 273: 1106–1112.
- Brounstein, P. (2003). *The continuing adventures of fidelity and adaptation*. Presentation at “Implementing Science-based Prevention Programs Effectively: A forum on Fidelity and Adaptation,” satellite broadcast sponsored by the Northeast CAPT, Knoxville, TN, April 16.
- Brounstein, P., Zweig, and Gardner, S. (1998). *Science-based practices in substance abuse prevention: a guide*. Rockville, MD: Substance Abuse and Mental Health Services Administration (SAMHSA), Center for Substance Abuse Prevention (CSAP).
- Cummins, M., Goddard, C., Formica, S., Cohen, D., and Harding, W. (2002). *An approach to assessing the fidelity of prevention programs*. Paper presented at the 10th Annual Meeting of the Society for Prevention Research, Effectiveness and Dissemination in Prevention Research, Seattle, Washington, May 31.

## References

**References**

- Durlack, J.A. (1998). Why program implementation is important. *Journal of Prevention & Intervention in the Community*, 17(2): 5–18.
- Dusenbury, L., Brannigan, R., Falco, M., and Hansen, B. (2003). A review of research on fidelity of implementation: Implications for drug abuse prevention in school settings. *Health Education Research*, 18(2): 237–256.
- Elliot, D.S. and Mihalic, S. (2004). Issues in disseminating and replicating effective prevention programs. *Prevention Science*, 5 (1): 47–53.
- Formica, S. and Harding, W. M. (2001). Documentation of adaptations to science-based programs implemented by MassCALL sub-recipient communities. Massachusetts Collaboration for Action, Leadership, & Learning (MassCALL), Massachusetts Department of Public Health, Bureau of Substance Abuse Services, Boston, MA.
- Goddard, C. and Harding, W. (2002). *Selecting Appropriate Prevention Programs Through Feasibility Assessment, Training Manual*. Newton, MA: Education Development Center, Inc.
- Hallfors, D. and Godette, D. (2002). Will the “Principles of Effectiveness“ improve prevention practice? Early findings from a diffusion study. *Health Education Research*, 17(4): 461–470.

## References

**References**

- Kumpfer, K., Alvarado, R., Smith, P., and Bellamy, N. (2004). Cultural sensitivity and adaptation in family-based prevention interventions, *Prevention Science*, 3(3): 241–246.
- Schinke, S., Brounstein, P., and Gardner, S. (2002). *Science-based prevention programs and principles, 2002*. DHHS Pub. No. (SMA0 03-3764. Rockville, MD: Center for Substance Abuse Prevention, Substance Abuse and Mental Health Administration. (http://modelprograms.samhsa.gov)

## Section III: Factors That Facilitate Implementation

## Factors That Facilitate Implementation

- Factors that Facilitate Implementation refers to the most essential and indispensable components of an intervention practice or program
- Knowing these factors may:
- Allow for more efficient and cost-effective implementation
- Lead to decisions about what can be adapted to suit local conditions
- Assist prevention practitioner on basing his or her interactions with clients and stakeholder on research findings

## Small-Group Activity 

- As a group brainstorm potential Factors That Facilitate Implementation.
- Place the list in order of importance.

## Factors That Facilitate Implementation

- Staff selection
- Pre-service and in-service training
- Ongoing consultation and coaching
- Staff and program evaluation
- Facilitative administrative support
- Systems interventions

## Small-Group Activity

- Groups are given one of the six factors that facilitate implementation. Each group will answer the following questions:
- Has anyone in your group had experience in this area and what were his/her experiences?
- What is the importance of this component as it relates to implementation?
- What are the potential challenges of this component** **and what are potential solutions?
- Does the State have a role in facilitating assistance with this component?
- What is the role of cultural competency in that factor

## Cultural Competency Considerations

- Do staff represent the target population?
- Is pre-service and in-service training
- 	culturally appropriate and responsive to your target population?
- Is the design and delivery of consultation and coaching culturally appropriate and responsive?
- Who interprets the staff and program evaluation information and how is the information integrated back into the program?
- What mechanism are used to build leadership throughout the coalition and who is involved in decision making?
- What is the relationship between the coalition and or organization and external systems?

## Multilevel Influences on Successful Implementation

**Core Implementation**

## Organizational Factors

- Commitment of leadership
- Involvement of stakeholders
- Resources for extra costs
- Alignment of organizational structures
- Alignment of organizational structures to achieve horizontal and vertical integration
- Commitment of ongoing resources and support for providing time and scheduling

## Small Group Activity

- Looking at the organizational components answer the following questions:
*Are there any factors that you would add to the list? *
*Which of the factors do you feel can contribute most to potential problems that can develop in the implementation phase?*
*Should the State consider any of these factors when writing RFP’s and when making funding decisions?*

## Large-Group DiscussionRegarding Activity and Group Results

**Notes:**
Start with one group and process one of their organizational components
Ask if any other groups also used that components and have them process the answer.
Additional Questions:
Of the components listed are some more important than others?
We have talked about the ideal situation here, from your experience what are some of the barriers in leadership and building stakeholder buy-in.
What might be the of the State in developing RFP’s and disseminating funding regarding these components?

## Section IV:Implementation Panel

## Purpose

- 	To provide participants with real-life exposure to implementation, including successes, challenges, and lessons learned across different types of prevention strategies

## Panelists

- Kitty Bowman, Director of Revere CARES
- Sandra Del Sesto, Executive Director of Human Ecology Associates and Initiatives for Human Development

## 					Questions 					     and
					Evaluation 					    Forms

**	****	****	****	****	****Evaluation ****	****	****	****	****	****    Forms**