Enhancing Implementation Science - 16 -Department of Veteran Affairs

EIS-060712

**Department of Veterans Affairs**

**Enhancing Implementation Science**

**Intro Program Session 3: The Role and Selection of Theoretical Frameworks in Implementation Research**

**Laura Damschroder, MS, MPH**

**June 7, 2012**

Moderator:  I would like to introduce our presenter for today.  Presenting for us is Doctor Laura Damschroder.  I’m sorry, Laura Damschroder.  I apologize for adding the doctor.  She works for the Department of Veterans Affairs Ann Arbor and University of Michigan.  Laura Damschroder is Co-Implementation Research Coordinator for the Diabetes QUERI.  A group dedicated to translating evidence based practices in VA and has authored or co-authorized over 30 peer reviewed published papers or book chapters.  She has co-[inaudible] for a large [inaudible] trial for a weight management program in VA and has [inaudible] the study that evaluated the Move program and is currently the PI of a study evaluating a “Timeful My Style” coaching program and Performance majors MBA among other studies.  She has [inaudible] development of a implantation framework that consolidated a framework from implementation research AKA CFIR’s which is being used in studies around the world.  She’s currently and has been on several invited national expert panels related to implementation sustainability of evidence based practices.  She has over 11 years of experience using quantitative experimental designs and analyses methods and over 10 years of qualitative and mixed message experience on a broad range of health services research topics.  We are very pleased to have Laura Damschroder joining us today.  And before I turn it over to her, I do want to go ahead and launch a poll question for our audience members. 

I will ask that you please select one option of the polls.  The poll question is what is your experience in conducting and implementation research?  The options are I don’t know where to start; I worked on proposal using a hybrid implementation study; I’ve read a hybrid implementation study; I’ve worked on an hybrid implementation study; or other.  

And we are seeing a lot of responses streaming in.  Thank you to those respondents.  We have had about two-thirds of the audience vote thus far.  So I will leave the poll open for just a few more seconds and give everyone else who would like to respond a chance to do so.   We are up to 70% response rate and it looks like the responses are starting to slow down.  I’m going to go ahead and leave the poll up for about 10 more seconds as people are joining in the session and responding to this poll.  We do appreciate your responses.  It helps guide Laura’s presentation toward you all.  Okay and we’ve had about 80% respondents—response rate, I’m sorry.  So I’m going to go ahead and close the poll and share the results now.  Laura can you see those results on your screen and would you like to talk through them?  

Laura Damschroder:Yes I can.  It looks like about a quarter of listeners don’t know where to start.  That’s great.  I think you’ve come to the right place.  Hopefully you’ve listened to the other two sessions in this series and if you haven’t, I highly recommend that you go to the cyber seminar catalog and listen to those sessions.  I think those along with this and then the future sessions will help you tremendously.  About a quarter—about a third—well 29% had worked on a proposal using a hybrid or implementation studies.  So it sounds like your kind of getting into the—getting your feet wet in this in this whole domain which is wonderful.  And 13% have actually [inaudible] a hybrid implementation study.  Nineteen percent have worked on a hybrid implementation study and then 16% are other.  Could be any mix of things.  So at this point—

Moderator:  Thank you Laura.  Are you ready to share your screen with us? 

Laura Damschroder:Yes.

Moderator:  Okay, I’m going to turn it over to you now and I’ll let you know when we can see the slides.  Go ahead and click yes to the pop up.  

Laura Damschroder:Okay I’m not seeing the pop-up.

Moderator:  Okay.  One moment please stand by.  One second, we are getting started here.  Okay.  thank you all for your patience.  And while I am pulling this up, I would like to remind you that you did come in to today’s presentation muted.  So if you would like to submit any questions or comments, you will have to do so using the Q&A portion of your Goto webinar.

Laura Damschroder:Okay I see and I have said okay.  Can you see the slide yet?

Moderator:  Let me change it over to you one more time.  Okay here we go.  I’ll go ahead and accept it.

Laura Damschroder:Yep.

Moderator:  Excellent.  

Laura Damschroder:How’s that?  

Moderator:  Perfect.  

Laura Damschroder:Okay.  Well thank you everyone for calling and taking the time out of your busy schedules to listen to this presentation and I hope you find it helpful.  First of all I want to acknowledge the fact that this entire series including this session is funded by QUERI by the VA.  I also just want to acknowledge the invaluable collaboration over the years in kind of developing this material together especially by Brian Mittman, Cheryl Stetler, Teresa Damush as well as many other of my colleagues within the VA.  So, a general outline of the content of today is first of all to present a case of “Why Use” theory and then a case study showing an application of theory and a QUERI steps three study and then an approach for selecting and applying theory in your own work with an example case study that helps to illustrate the principles and then also to provide or suggest some resources and help.  

The objectives are that I hope you come out of this presentation with an increased understanding about the role and value of theory in implementation research and to equip you with an approach of a kind of an overarching approach for applying theory in your work that will hopefully help you design studies, conduct your studies and also in grant writing.  So, first of all a word on this term “theory” I’m using term loosely especially as far as the title of the presentation goes.  I’m calling it a kind of a generalized loose definition of theory of science will sometimes be used today.  I have a definition there; “A set of statements or principles devised to explain a group of facts or phenomenon and in this usage, theory may be imbedded in a framework, a model or a specific hypothesis.  I just want to note that the literature is rife with inconsistent use of terms in terms of frameworks, models, theories, and so on.  I provide a slide with my favorite working definitions for framework, theory and model.  I’m not going to spend time on those definitions, but I provide them here as a resource and part of the presentation slides which you have access to.  So, this is sort of a Dilbert cartoon kind of expressing the state of the literature and the state of the use of these terms.  It says that we’re going to use OMMI.  It’s a mode for developing a process to create a framework or it might be a process for creating a framework to make a model.  At this point, that really well illustrates I think this kind of the inconsistent use of those terms.  Then it goes on to say that there’s no budget for training so we’ll be relying guessing more usual.  So that to me really kind of captures the state of implementation all too often on the ground.  

Taking a step back, I just want to kind of remind or maybe just acknowledge that theory is used in clinical interventions to help guide development of clinical interventions like a new weight management approach or a coaching approach or behavior change approach with patients.  For those of you who have written proposals and have succeeded in getting proposals funded, it’s pretty much a given that you will have a theoretical framework of some sort that guides your proposal or guides the intervention that you are proposing to task.  Let’s say in a randomized control trial.  For example in our own weight loss intervention and trial, ASPIRE-VA, we draw on Social Cognitive Theory, Self-Regulation Theory, Problem-solving Theory, Small Change Theory and here is an underlying theoretical framework that we had in our proposal.  So in the same way we need to use or we’re advocating theory, use of theory in implementation work.  These theories may come from other literature including organizational climate theory, theory of conformity and other psychological & organizational theories that may help inform and underline framework or a conceptual model for implementation in the same way an underline theory maybe used to guide a clinical intervention.  

So in terms of putting these two together to just simply make the ploy that ultimately program outcomes is relies on not only effectiveness of the intervention, which has its own underlying theoretical model or theoretical framework, but also the effectiveness of implementation.  So to highlight the kind of type three errors in a sense where incorrect conclusions might made about effectiveness of clinical programs or clinical interventions that can result when there’s no treatment or too little treatment was actually provided or the wrong treatment was provided or treatment was non-standard, uncontrolled or various across setting or populations.  So in other words, the fidelity, the use, the implementation of the program is not complete.  So these type three errors can result from inadequate implementation.  

The objectives of implementations research really...and this is just kind of summarizing or maybe stating it in a little different way from the two earlier sessions.  Our first to replicate successful implantation in terms of this requires us to be able to define core components of implementation strategies and the rational for those components.  We want to be able to generalize knowledge about how to implement and sustain interventions.  We want to be able to navigate complex implementations and we to be improve prospects for longer term sustainability.  The premise that I have here is that theory driven implementation enables us to accomplish these objectives.  

So a quick review the literature.  Systematic reviews of implementation related intervention consistently show that some work all of the time; none work all of the time; and more research is needed to figure what works where and why.  We see this message over and over again.  For example recently, a Cochran review has had two reviews recently updated I think last year about opinion leaders and really finding mixed [inaudible] of opinion leaders.  Going on, the literature is largely atheoretical related to implementation and by the way all too often for interventions as well and really kind of both cases, but especially in implementation.  Theories often used only as a heuristic.  A theory maybe presented and described in the introduction or perhaps as an introduction into the methods, but then it’s dropped shortly thereafter.  Or it’s maybe brought into a paper to help organize discussion of findings.  There are a lot of case studies that actually derive new theories and so maybe adding new theories to the literature, but then those theories aren’t picked up and developed further.  

So, there are also inadequate descriptions of interventions.  A large majority of trials have no qualitative component and implementation studies suffer from small samples.  So an example from a very recent paper on team formation “Findings revealed limited information about the attributes of successful and unsuccessful team initiatives, barriers and facilitators to team initiatives, unique or combined contribution of selected interventions, or how to effectively establish these teams.” This leads wide open a large area of needed research into implementation and I really believe that use of theory can really help inform this.  So theory is a way forward.  An international panel convened by AHRQ recently stated that “Generalization through theory can potentially offers a more efficient and appropriate method of generalization than study replication in many possible setting.”  Theory can help us to build knowledge through a strengthened confidence in the usefulness of a theory.  It can help us identify factors that predict likelihood of success and also theory can guide adaptation of the intervention and tailoring of implementation by bring out or calling out a [inaudible] potential determinants or levers of change.  

So I want to go through a case example.  This is what we characterize as a Step 3 QUERI Study.  For those of you who were with us in the last presentation, well actually the two previous presentations, we talked about Step 3 work.  First of all to just to kind of very quickly orient you, step 1 is identifying a high priority disease and in this particular study, prevention was actually the topic.  And it is a high priority in the VA with new emphasis being put onto prevention and initiatives.  In particular, obesity prevalence is high among veterans.  And obesity is linked to many chronic conditions and so if we prevent obesity, we prevent many other obesity related conditions.  In step 2, the Move weight management program which is evidence based on published on evidence was dissimilated in January 2006.  In terms of Step 3A, in terms of identifying gaps in practice, we could see from the data from the administrative data that many facilities were not reporting MOVE workload which seems to indicate that they had not implemented the MOVE program.  Step 3D which is getting into a little more detail, but still this is another facet of Step 3 is to identify the barriers and facilitators of MOVE implementation.  So we really wanted to understand why are so many facilities so successful in implementing those and other facilities are not?  This was approximately a year and a half after initial dissemination of the MOVE program.  We did an embedded mix cross sectional study using a purposive sample of 5 sites and we conducted semi-structured interviews with 24 stakeholders across  these facilities over the phone.  We used theories to guide our exploration of areas and facilitators at these five facilities.  

I will explain the different types of theories, but quickly to say that we used two explanatory theories and we also focused on organizational level theories as opposed to—and again, I’ll get into what that means.  We used a consolidated framework for implantation research and this was one of the recommended readings for this presentation.  The CFIR guided our qualitative data collection and analysis.  Then we also used the model of implementation effectiveness and this guided our quantitative data collection and analysis because there were measures that were associated with this model that was published by Klein, Conn and Sorra which is the second paper that we recommended for reading for this session.  Then we also used qualitative data to confirm some of the construct in this particular model.  

So I’m going to give a very quick overview of the CFIR framework and I have the [inaudible] out there.  There’s the wiki that we have developed.  The paper’s published in Implementation Science in 2009.  It is a comprehensive framework to promote consist use of constructs, terminology and definitions.  The goal with the development of this framework was to consolidate the myriad existing models and frameworks that were published in the literature at that point.  We wanted this framework to be comprehensive in scope, kind of a one stop shopping kind of framework where you could go to a single framework and look at all of the kind of the world of potential constructs that are believed to be associated with successful implementation.  It is important to tailor the use of the framework to your particular study.  There are 39 different constructs.  It’s a large, as I said, comprehensive framework and we wouldn’t expect the entire framework to be used in every study.

There are 5 major domains in the CFIR framework.  First is the intervention and these are constructs that are primarily out of Roger’s Theory of Diffusion which describes attributes of the intervention including the complexity of the interventions and also other attributes including perception of the evidence strength and quality underlying the clinical intervention.  There’s the second domain is related to the outer setting.  These are influences or kind of triggers sometimes that are in the outer setting that are received by the inner settings.  The way that they are processed by the inner setting is important to understand.  Of course, for use being in health care, patient needs and resources, it’s really an important construct within the outer setting that is very important to internalize within the organization.  The third domain is the inner setting.  This is the single most flushed or complicated domain with 14 different constructs including things like the degree of leadership engagement into the implementation initiative, available resources and many other constructs.  The four domain are the individual involved.  Of course organizations are comprised of individuals and its very important to understand their knowledge about the intervention and the targeted intervention that we want to implement.  What their self-efficacy is.  What their level of confidence is and being able to use that intervention or to change their practice.  The fifth domain is process.  There are 8 constructs related to process and it includes things like planning, and engaging stakeholder, and championing.

Then on the next slide, I just show a excerpt of the list of constructs organized by the 5 domains.  This is just showing a few of the constructs from the first two domains.  I show you this just so you can get a feel for how the CFIR framework is taxonomy.  It does not hypothesize relationships between these variables.  In general these constructs could be regarded as independent variables.  Although that’s not always the case, but in general as independent variables in trying to predict or understand what differentiate successful implementation from sites with unsuccessful implementation.  

So as I’ve said earlier, we also had a model of implementation effectiveness which takes the broader range of constructs from this CFIR and really focuses in on a specific model.  So here you can see there are fewer constructs related and there’s a hypothesized relationship between these constructs and this is drawn out of our published model.  

In terms of quantitative results which align with that earlier slide that I just showed you, we had measures of management support whether people felt that they had sufficient time, available resources, communications about the MOVE program, and implementation climate, which really was a measure of relative priority of MOVE verses other initiatives going on at the time.  In every case all of these constructs facilities with high implementation of effectiveness had much higher scores on these dimensions than facilities with low implementation effectiveness.  Because our sample size was so small, really we started with five, but one of the sites was really a transition site.  So we were left with four sites so that we’re clearly either high or low implantation and that’s not a sufficient sample size of course to do statistical analysis.  But none the less, there’s a lot to be learned especially with adding qualitative data into those analysis.  

So in terms of qualitative results, we did find that the importance of champions, which was one of the model constructs, was mixed.  Every site had an appointed champion, but they were largely absent in one low and one high implementation site.  So you can see how the world of champions was really mixed.  It was very important in some sites and not so much in others.  

Innovation values fit, which is related to compatibility from the CFIR framework, we found was an important differentiator between high and low implementation sites.  This is just one quote where someone said “I would say that 99.99% of the providers recognize that obesity is in some way hindering their success in managing diabetes or managing blood pressures or managing hyperlipidemia.  So everyone is very receptive to refer the patients to MOVE.”  This is an example of compatibility with respect to values or innovation values fit.  We did not find this list as the case two low implementation sites. 

So again we’re coming back to our specific implementation model we found that there was support in our data for four of the constructs within this model.  We did not find support for the role of champion in this particular context in this study.  So then the question is was this theory useful?  As I said earlier, we found that the champion influence was mixed, but the other constructs there seemed to be good support for those constructs being playing an important role in implementation for the MOVE program.  We also had additional insights because we used the CFIR we actually explored other constructs that were not in the specific implementation model.  For example, we found that teamness mattered.  We called it teamness and that was the degree to which the MOVE teams the interdisciplinary MOVE teams coalesced and we also found out that in some other constructs or the importance of other constructs.  But we also found that not all of the CFIR constructs were salient in this particular study.  

So now we want to go into the types of theories.  There are multiple types of theories and often as this study as this step 3 relatively straight forward study of barriers and facilitators showed, we use two different—we used a framework and we used a model and they complemented one another.  Both of these were explanatory theories and some people referred to those as descriptive or impact theories.  These are theories where the hypothesis and assumptions about how implementation activities will facilitate a desire change as well as helping to identify potential facilitators and barriers for success.   

Process theories on the other hand or otherwise sometimes referred to as prescriptive theories or planned action theories.  These types of theories actually talk about how implementation should be planned organized or scheduled.  Then there are some theories that we just call mixed theories or you can call them hybrid theories and they really have elements of both.  They have an explanatory component and a prescriptive component.  So an example of a process model or process theory is what we call the “4 E’s” Translation Model published by Peter Pronovost and colleagues.  And this was a model that underlined studies related to the Keystone project in Michigan to reduce the number of infection in ICU and then it was expanded to a nationwide effort.  As you can see, first step, summarize the evidence; second step, identity local barriers implementation; third step, measure performance.  Then you come into this cyclical process of educating, engaging, executing and evaluating the implementation as it progresses.  So that’s an example of prescriptive or process model.  

An example of a mixed model is the PARIHS model and this is a framework that is sometimes referred to as a framework or model and I’m using these terms loosely.  I apologize.  This framework is used by many researchers in the many QUERI’s within the VA.  There, the basic premise of this particular framework is that successful implementation is a function of evidence context and an active component of facilitation.  So it’s this facilitation piece that is prescriptive because it calls out the importance of facilitation and then describes important attributes of facilitation.  Then I just also mentioned on this slide that there is a quantitative measurement tool called the ORCA and it was published by Helfrich and colleagues and then by the Alberta Context Tool.  

So how do you go about choosing theory for your particular study or your particular application?  You can start by considering the nature of the theory.  What would be helpful?  Are you focusing on process?  Are you focusing on explanatory really trying to discover or understand the levers of change or the constructs?  What is the context?  Are you working on a policy level?  Are you study a policy level or organizational level or even individual level?  What is your discipline?  Sometimes that drives the perspective or the theory base that you draw from, in terms of social science, psychology are just a couple of examples.  Consider the level that it will be applied, whether it’s the individual level, the team level, organizational system.  Consider previous finding and experience that best matches the context of your particular study.  What has the greatest potential for adding to the knowledge base?  

I want to show you an example of a multilevel model that actually brings out leaders and managers and staff and that’s another complication in doing implementation research is that we are often dealing at multiple levels simultaneously.  So just going back to the Step 3 case study example, I highlighted already earlier that we use two explanatory theories.  Then looking forward because we use these theories we were able to identify the constructs that we think maybe important levers of change around which we could design an implementation strategy.  Then what we need to consider is what level, whether organizational, team, individual, system wide, are all potential levels to target implementation strategies.  

So now I want to present a general framework.  This is iterative.  I don’t want to over simplify this process because it is a very complex and iterative process.  But in general, you first want to access your targeted evidence based practice change and the context.  So as I said earlier, what are the characteristics of your targeted change and what are the characteristics of the context in which you plan to implement that change?  As I showed in that Step 3 the MOVE evaluation, it is possible and desirable to use theory even to guide this assessment in the very first step.  

Once you have an understanding of context and how the intervention may relate.  Next is the task to select targeted theories and provide rational for those theories.  So I’m going to use an example study that has been published on the [inaudible] Implementation Science.  They chose to focus on preventable blindness and diabetes patients and they found that in evidence based practices, early detection and timely laser therapy.  And in Step 3 they found in their work, in their pilot work, they found that there was a lack of close follow-up of patients to ensure that they got the timely laser therapy.  So in Step 4 then they conducted a pilot implementation study to implement a strategy to improve follow-up and they had a system level implementation intervention which was to change performance measures.  Then they had a local or organizational level implementation strategy which was a tool which they called the progressive reminder and scheduling system.  A tool to help get patients scheduled appropriately.  They had an organizational level theory that drove their work and design of their implementation strategy.  

The next step, once you have your underlying theory is to develop your tailored implementation strategy based on both the theory and what you’ve learned about the local context.  So I also suggested this reading the citations at the bottom of the slide by French and colleagues and that was just recently published in Implementation Science.  I thought that they had a nice kind of flushing out of this process of how to move from once you’ve selected your targeted theory and then developing a tailored implementation strategy.  

First the question is who needs to do what differently?  Really understanding what that is the answers to that question.  In this particular study they found that they knew that providers needed to document follow-up visit timeframes.  Then the scheduler needs to schedule the appointment.  So these were the answers to the question to that particular study.  

The next question to ask yourself is which areas and facilitators need to be addressed?  One of the barrier that they chose to address was tension for change and to heighten tension for change as a strategy for kind of creating urgency and commitment to implementation or to change.  Then as I said earlier, they also developed a tool to help with scheduling. 

The next question is what behavior change techniques and I added organizational level techniques really could help overcome barriers and facilitators.  They chose a policy level advocacy is really the way that I could characterize their activities and then also their scheduling tool and follow up tool  

The next question is how can behavior and organizational change be measured and understood.  They did a mailed survey to staff and also conducted semi-structured interviews to try get a better understanding of what was going on locally.  

Your tailored implementation strategy then would depend on the answer to those questions as well as your theory that informs.  In this particular case study this is their conceptual model.  I’m not going to go through the details of this, but this process helps describes their approach to creating or developing this conceptual model. 

So now I want to go through available resources in terms of finding out or places you can go to select theories.  For those who are participating in the EIS training on the Sharepoint site there will be a, there’s not yet, there will be a document called Classification of Theories that just give a list of an example of several example theories and also there are some theory diagrams that are from example theories.  Then in terms of selecting specific techniques or strategies, there were the two papers that we recommended for reading either—for recommended reading, it wasn’t required for the training.  But at an organizational level [inaudible] colleagues recently did a synthesis of organizational level strategies for implementation and then Susan Micky’s work and her colleagues work on individual level techniques for individual change.  

So using theories, I just want to make a little sidebar here and just acknowledge that all models are wrong, but we must be alert to what’s importantly wrong.  There are not right models, but there’re no completely wrong theories either.  Really I should be saying theories here and so theories are useful, but there are better fitting theories that can explain why a specific strategy or mechanism causes the intended change.  So don’t try to get the perfect be all end all theory in your work.  Implementation strategies need to be operationalized from theoretical constructs.  So don’t just stop a kind of conceptual level really work to define and document how exactly you are implying those concepts in your study.  

The next step is of course to execute your tailored implementation strategy.  Ideally an evaluation is one of the topics of this training series and you ideally would want to do an evaluation through the course of implementation to access progress and to access impact of kind of ongoing implementation; so again, concurrent and monitoring and refining.  

The next step is to evaluate the effectiveness of your implementation strategy.  In the case of this case study of preventing blindness in diabetes patients, it took 5 years when they went back; policy level advocacy to change the performance measure was one of their strategies and in the VA it took 5 years to get that performance measure changed which was two years longer than planned and didn’t happen until near the end of the study.  This was one of the challenges of conducting implementation research in an uncooperative world sometimes; certainly challenging settings.  An there was a mixed commitment and lose of commitment over time in their study facilities.  There was the pressure exerted by the existing performance measure which was why they wanted to change it.  There were technical issues with the tool and also local priories and feuds kind of going on between services, but even though this was a—best characterized as a negative study perhaps.  But there were lessons that were learned in this and being applied in subsequent studies.  So in a negative study we can learn a lot from the experience of implementation.  

So the last step is then to access the fit of your finding with the initial theory that you chose coming into this.  I just really want to reiterate “don’t skip this step.”  There are not very many papers that are published that actually go back and review how useful was the theory.  You really need to do this in order to add we really want to add to the knowledge base so that we can develop theories and refine them and improve them across settings and across studies,  So in terms of accessing theories, questions to ask yourself is was it useful?  Does the theory still apply?  Are there modifications and refinements that are needed to that theory?  Make those recommendations in your discussions.  You also want to build validity.  This is another of course very important aspect of using theory implementation research is to establish or to continue validation of those theories through quantitative theory testing, through hypotheses testing or path analyses or in Klein, Conn and Sorra papers that is a reading for today.  Then qualitative theory testing; the questions you would ask for qualitative theories are whether the terminology or the language is coherent and useful?  Does it promote comparison of results across setting and studies over time?  Does it stimulate new theoretical developments?  

So, key points that I want to leave you with are using pre-implementation work to inform implementation.  So as the example with the MOVE Step 3 study that we identify kind of potential levers of change and that information can be used then to move into implementation.  Do use theory.  Of course that’s the primary thesis of this topic for today.  Do provide a clear rational for why you selected the theory or the rational for how you drew from multiple theories.  Clearly define your implementation strategy so that other can replicate what you’ve done.  So really focusing on internal validity which doing kind of traditional RCT’s  are good for that, but implementation research we’re often having to use other study designs and it’s very important.  Study designs are actually do well in establishing external validity or at least helping to contribute to external validity as well as internal validity and kind of managing those two challenges is a attention that is definitely present in implementation research.  Then of course you want to evaluate the usefulness of the theories that you’ve used in your work.  

So I hope that I’ve achieved some of these two objectives and you have a little better understanding of the role and value of theory in implementation research and that you can apply this framework and that this helps your thinking or help organize your thinking in how to apply theory in your work.  I just want to say that the field is really ripe for people to do or for implementation research.  I we have so many open questions that really need further research.  Even at basic questions like what is the dependent variable?  What do we mean by implementation success?  Then bringing clarity in terminology, as I’ve already demonstrated in terms of the loose use of terms in developing measures for constructs, then moving from the kind of the veneer term implementation is a challenging enough, but also keeping an eye toward longer term sustainability.  What can we do in the process of implementation that would help promote longer term sustainability which is really what we want to see?  We want to be able to develop predictive models and also develop tools for both researchers and people in operations to be able to predict success and also to be able to design implementation strategies that will help them be successful. 

The other thing is—just one moment here.   That really imbedding theory into the bones of your implementation work and then in this way it helps to set your work up for future synthesis and really building the knowledge phase.  Another kind corollary to this is that in your publishing to really be conscious and be comprehensive about and describing your implementation strategy in terms the components used and how you operationalize your theoretical constructs using mixed methods studies, quantitative and qualitative approaches, using additional files to add this additional information.  Our guidelines are out there that guide the publication of quality improvement studies that can be easily applied for implementation as well and that help kind of highlight the contextual variables that are very important to begin to get out into the literature so that we can really start to answer the question; what works where and why?  

So in terms of help there are lots of references in this presentation.  There’s a last slide in this presentation set that has a list of references.  For those of us in the VA there’s a CIPRS that is available for consultation.  If you know any QUERI researchers QUERI implementation research coordinators, you can go them and they can help link you in to consultation or help.  Then for those of you in the EIS training program, you have your mentors who should be available to you.  So the message here is that you are not alone and that together we can really make advances in the science of implementation.  Thank you.

Moderator:  Thank you very much Laura.  We are already receiving many questions.  People are very interested in this topic.  For those of you who still emailing in for the recommended reading, I am going to get to your request from the cyber inbox in just a moment, but first I am going to start moderating the questions that I’ve already been asked for today.  So do please stand by.  

The first question that came in:  Is there a list of “high priority” VA initiatives?  

Laura Damschroder:Is there a list of high priority initiatives in the VA?  I am not the best person to ask this question to.  If I can defer that question to maybe an email question, assuming we’re not going to be able to get to all the questions and we will provide a written response to any of the questions we’re not able to get through to.  There are several resources online that describe the VA’s high priority areas.  Some of them are associated with the T21 or the transforming, the system care priorities.  I can’t list them off the top of my head.

Moderator:  Not a problem.  Not a problem at all, that is completely understandable.  The next question—I just want to address everybody that’s emailed regarding the Sharepoint site;  unless you are a VA employee, no unfortunately you can not access it.  Yes we are batching all the material today and we will be updating the CIPRS and EIS public webpages.  Please do stand by, I will send an announcement when you get your archived email tomorrow.  Thank you for your patience.  In the meantime, as I mentioned, you can email cyberseminar@VA.gov and I will happily send you any attachments and resource materials I have.  

The next question;  Just curious what are Steps 3B and 3C in the MOVE case study slide?

Laura Damschroder:3B roughly is identity the gaps in practice.  So in this case we moved from administrative data that even though the MOVE program had been disseminated that there was a gap in practice in terms of many facilities that had not actually implemented MOVE.  Then Step 3C is really identifying barriers and facilitators related and trying to explain why there was that gap in practice.  So really trying to understand at an organizational level what other barriers and facilitators for implementing the MOVE program.  So Step 3 is really all about first of all identifying.  We have to understand what the gap in practice is and then to be able to understand what are the mechanisms that are related to that gap that help to indicate what implementation strategies might be most effective in closing that gap.  So that we can in this case get more facilities to fully implement MOVE.  

Moderator:  Thank you for that response.  The next question we received and this may be in your additional materials.  Where do we find this CIPRS and model reading materials?  Can I assume those were in the documents you sent me?  Well, here we are.

Laura Damschroder:  Yes, there is a—on a couple of the slides there is a URL on one of the CIPRS slides there’s an URL that you can put into internet explorer or whatever that will take you to implementation science and exactly the publication.  There’s also another URL that points to the Wiki that we have developed that has all of the contents of the published paper and there’s also on that Wiki site, there is the hyperlink also to the Implementation Science published article.  

Moderator:  Thank you.  Thank you for that response.  The next question and you may have addressed this just now; Somebody was asking for the stated literature slides 15 – 17 are those on here now.  

Laura Damschroder: Yes. If you look at that stated literature; 3 slides which is about in the middle of the slide that I have up now.  The references 1 – 4 are there.  

Moderator: Thank you very.  I appreciate you adding all these references it helps our attendees very much. 

Laura Damschroder:I apologize to for all of the references to references.  The topic of theories and the literature is just so rich and in this regard and it’s really hard to know how to hone in and I don’t want to over simplify the richness of this literature.  

Moderator:  I think that is the exact right decision to make given the time constraints of cyber seminars.  The next question:  The CIPRS framework has many domains and constructs and in this study, with an N of 4, how are these multilevel models analyzed?  

Laura Damschroder: Yes, actually as you imply with the sample size of 4 we took a qualitative approach with the CIPRS and this was the first study that we used to help kind of validate or test the usefulness of the CIPRS as kind of a guide for understanding barriers and facilitators.  So we did not take an exquisite or [inaudible] multilevel approach.  It was more exploratory.  So in the case for example our finding around teams, we didn’t go in explicitly looking for teams as a level of analysis, but rather they came out inductively in the data that we collected; and really related to the quality in nature of networks and communications within these facilities.  

Moderator: Thank you for that response.  I also just want to mention that we are having many comments coming in saying that you are just an amazing speaker on this topic and there are many thanks for presenting today.  

Laura Damschroder:You ware welcome [inaudible].

Moderator:  The next question.  Let’s see.  Okay.  This is a little bit of a long one.  It seems that the series/framework are quite inclusive across levels including context as well as change factors.  For example, PRISM is very comprehensive, how would one go about addressing some aspects of the model to test—I’m sorry can you refresh your screen or go back to the last slide.  

Laura Damschroder:Yeah, Hang on.  Okay.

Moderator:  Thank you kindly.  Okay.  I apologize.

Laura Damschroder:  I’m going to pull up the PRISM model.

Moderator:  Okay. Excellent.  I’ll continue the question.  How would one go about addressing some aspects of the model to test even as we acknowledge the other variables not under investigation or would a grant need to propose to address all levels at once?  That appears to require enormous resources and not the kind of “early investigator” application that some part maybe considering.

Laura Damschroder:  I would not try to tackle this all in one bite as an early investigator.  I think that s a very wise conclusion.  I would start relatively simple.  Have a single model like the implementation effectiveness model that showed earlier as an example or using the CIPR framework or using the PARIHS model.  There are examples in the literature using these theories.  I do want to say that the Implementation Science has—if you go to their homepage, which it’s just the implementationscience.com.  I believe. If you go to their homepage and just browse through some of the titles or search for the word protocol in the titles, there are many protocols implementation.  Implementation study protocols that have been published in the last year or two and they’re really a rich source of information about how researchers have operationalized various theories of models.  So I would recommend using that as a kind of a starting pointing and definitely start simple before you delve into a complex model like the PRISM.  

Moderator:  Thank you for that response.  Once again, I just want to remind all of our attendees that yes we will be updating the webpages later today and later this week, which will include all of the recommended readings for all the previous sessions.  So please do stand by for an announcement regarding those.  I will be sure to let you know.  

The next question that came in:  Can you please speak a bit more about how to identity barriers/facilitators need to be addressed, for example, the important ones.  Thank you and great talk.  

Laura Damschroder:  For example the important what?  There are several ways to approach this question.  I guess I’ll describe—I guess assume that perhaps this is an exploratory.  So let’s say you have an evidenced based practice like the MOVE program for example.  You want to explore what the potential barriers and facilitators are.  The first step is— I can use the CIPRS framework as an example.  I don’t want to say this is the be all end all framework, but of course because it’s one that we work with, it’s the one that I’m most familiar with and that I have the most experience with.  So it’s the easiest for me to talk about, but there are many other theories that could guide kind of a similar process.  The first think that we do when we’re looking to apply the CIPRS in a study is to interview stakeholders who are knowledgeable about the context in which the program the will be implemented and are also knowledgeable about the program itself.  That could include local stakeholders.  It could include policy level.  

For example, we work with the central office kind of the policy level office that was responsible for disseminating the MOVE program, their source of information as well as the local facilities and as well as their own experience.  So for example we’re doing an evaluation now of the telephone based coaching lifestyle program and we in part use our experience form the MOVE evaluation to help inform this other evaluation that is being implemented in very similar context within the VA.  So partly, it’s our own experience and we identify many a dozen, ten to twelve high priority constructs within the CIPRS and really focus on those.  But at the same time, we try to keep our eyes open for other barriers and facilitators, other important influences that may not be one of those 10 – 12 high priority constructs.  Then we develop—again we’re taking a qualitative approach to this.  A quantitative would be different.  I can say a word about that in minute. A qualitative approach—then we develop our interview guide.  If you go to that WIKI site that I had the reference to, there are a couple of example interview guides available on the WIKI site that are literally mapped with the questions or mapped to the CIPRS construct and so you can begin to get an idea of the kinds of topics or the kinds of details or how the operationalize those constructs.   

We also have information on the WIKI about inclusion rules and exclusion rules for qualitative coding.  Those are—we’re adding to those continually.  It’s kind of a living resource or online resource for people.  So that’s available to you as well in terms of a qualitative application of the framework.  In terms of the quantitative measures—really the quantitative measures are going to—ideally for example the ORCA measure that I mentioned earlier is based off of for example the PARIHS framework and so that measure kind of has an underlying theory drawing primarily from the PARIHS framework, but also a couple of other theories.  Those quantitative measures, we’ve also administered and used kind of a mix.  I mean the ideal is to use a mix of quantitative and qualitative measures so that you can continue to develop new quantitative measures and kind of refine your qualitative approaches as well.  They kind of inform each other.  I hope I’ve answered the question.  

Moderator:   Thank you very much.

Laura Damschroder:I guess the bottom line was then how do you identify the barriers and facilitators.  We actually have a process that we go through and rate the strength of the presence of these constructs which is a whole other talk in of itself.  Then we look for patterns of these strengths and valence of these constructs and look for patters that differentiate and look for constructs that differentiate between high and low.  It’s important as a qualitative analyst to be blind to whatever the outcomes of interest is, whether it’s the number of patient referrals or if it’s  weight loss  for a weight loss program.  Whatever your outcomes are that you be blinded to that and then in the end, then bring in your outcome information and look for a pattern based on highly effective or lower effective sites.  

Moderator:  Thank you for that response.   We do have several pending questions.  I’m wondering Laura if you are able to stay on and continue answering or if you need to end now that we’re at the top of the hour almost?

Laura Damschroder:  I can do 5 more minutes.

Moderator:  Okay thank you very much.  We truly appreciate it.  I will move quickly through these as many as we can.  For those of you that do have to leave at the top of the hour, we do try and get written responses for the remaining questions and we will post them with the archive along with the recording and along with all supplemental material.  Thank you.

Can you please explain the distinction between implementation research and action research?

Laura Damschroder:  Good question.  I think that—and this is my opinion and maybe partially not completely informed.  But I would say that action research may a type or can be —action research approaches can be used in implementation.  So action research really emphasizes engagement with stakeholders with local users with the community, if you are doing a community intervention.  And really emphasizing building that partnership and that partnership is so important in implementation.  I think it’s actually very important too and I’m really glad that you asked this question because I think it’s very important to embrace principles of action research in implementation approaches.  So some of the ground work in implementation and our implementation work actually happens in Step 0, before we actually launch the actual implementation study.

Moderator:  Thank you so much.  I just want to mention that many more positive responses are coming through.  Everyone says that this has been fantastic and that makes me want to mention that as you exit the session, you will have a survey that will upload onto your screen automatically.  Please take a moment and let Laura know directly how much you appreciate her content.  Thank you very much. 

Next question:  Can you please repeat really quickly the opinion leader study that was recently published or is that found in your references.  

Laura Damschroder:  I think I have the references for that and if I don’t, I can get them to Marle to include in the list of references.  

Moderator:  Excellent.  Thank you.

Laura Damschroder:I think I went too far.  Sorry.  Look on the last slide and it may be there and I don’t know if—hang on—yeah—I’ll make sure that you get it.

Moderator:  Thank you so much.  Do you have time for one more?

Laura Damschroder:  Sure.

Moderator: There and overlap between the field of industry psychology and implementation science.  Could you differentiate the two?

Laura Damschroder:  I think you mean industrial psychology and can I differentiate between the two.  I don’t really know a definition for industrial psychology.  I kind of vaguely know some examples of maybe industrial psychology type questions, but I think that industrial psychology would really inform implementation.  So industrial psychology would look at dynamics within the organization and kind of behaviors of individuals and groups of individual and the research I think that industrial psychologist do would help inform implementation. That’s about as good as I can get.  I can say the same about organizational researchers or researchers who are affiliated with business schools who really focus on organizational dynamics, organizational change.  All of those disciplines inform implementation research.  

Moderator: Thank you for that response.  Let me know if you would like to start concluding comments or if you’d like to take another question.

Laura Damschroder:  Let’s try one more.  

Moderator:  All right, because they are streaming in:  I’m planning an implementation of an EBP model for follow-up of [inaudible] with muscular skeletal disorders and have been wondering about using PARIHS.  Do you have any experience with that model?

Laura Damschroder:  Yeah.  This is with the PARIHS model so applying the PARIHS model there are a couple of published articles recently that I would point you to; one by Helfrich and colleagues.  Helfrich is the lead author.  I believe that is—I should know this, but I think that was published in the Implementation Science.  So if you look for PARIHS in the title and Helfrich as the lead author, and also I’m the second author on that as well, it should pop up on implementationscience.com.  There’s also another document that was published where Cheryl Stetler is the lead author and that document really goes through and operationalizes the constructs within PARIHS.  That should be a tremendous help to you in applying it to your work.  I can say that the mental health QUERI researchers used PARIHS and their work and the CHF QUERI and I believe to some extend the HIV health [inaudible] QUERI’s all used PARIHS to some degree in their research.  

Moderator:  Thank you.  Laura I do have a question that might be applicable to a couple of people.  Are you in charge of the post webinar session for today or should I have people contact [inaudible] directly?

Laura Damschroder:  What do you mean by post webinar?  The work group?  

Moderator:  Yeah, That answered my question.

Laura Damschroder:  Okay

Moderator:  That answered my question.  I can send people to [inaudible] for further information.  Thank you.

Laura Damschroder:  Okay. All right.

Moderator:  Okay.  Well I know that you have been very generous with your time.  People are asking if they can get written response to the questions.  I would like everybody to know that in additional to the recording and the archives, we do have a large transcription of today’s session.  We’ll try to get those posted as timely as possible, but there is a few days delay on those, but you can always access the archives within 24 to 48 hours.  At this time, I would like to allow Laura to give any concluding comments that you would like to.

Laura Damschroder:  I guess that’s all I have to say.  I just want to encourage you in this work.  I think it’s a very exciting place to be and wish you all the best and thank you for taking the time out to listen in.  

Moderator:  Thank you so much for lending your expertise.  It is clear by the amount of interest how many people really took a lot away from this and I know the archives and additional materials will be accessed promptly.  So thank you again and think everyone for joining us.  This formally concludes today’s HSR & D Cyber Seminar.  Thank you.  Please do fill out the survey upon exiting the session and let Doctor Laura Damschroder know how great she did.

Laura Damschroder:Thank you.

Moderator:  Thank you,.

(End of recording)

Cambridge Transcriptionswww.ctran.com

Cambridge Transcriptionswww.ctran.com