**Transcript of Cyberseminar**

**QUERI Implementation Seminar**

**Updated QUERI Implementation Guide**

**Presenter: Anne Sales et al**

**March 6, 2014**

*This is an unedited transcript of this session. As such, it may contain omissions or errors due to sound quality or misinterpretation. For clarification or verification of any points in the transcript, please refer to the audio version posted at www.hsrd.research.va.gov/cyberseminars/catalog-archive.cfm or contact**Anne.Sales@va.gov*

*Anne Sales:*My name is Anne Sales.  I’m one of the people who was involved in the original Guide to Implementation that QUERI put together in about 2003, and have been involved in the updated guide, which is what we’re going to be presenting today.

The original guide, as I said, was developed in about 2003.  It was a team effort at that point among the Implementation Research Coordinators for the QUERI Program, and this update has also been a team effort.  We’re not going to list everybody who was involved in it, but each of the sections will be presented by one member of the team who worked on that section.

Moving to our first slide—one of the questions that comes up, and that people ask, is why is implementation important?  I think that all of us are aware of how quickly things are changing, how rapidly new evidence and innovations are coming into the health care setting and sector.  Managing this flow of information and knowledge is really difficult.  People are finding it very tough to stay focused to understand what they need to do and how to do it.  You work in a very complex and routinized environment, so making changes to routines are very difficult.

All of this is in the domain of implementation research.  The guide talks mostly about the research and the importance of systematic approaches to assessing problems and areas, and finding approaches to dealing with them.  What the implementation research focuses on is understanding the reasons for gaps in performance, why care might be suboptimal, why care is not based on that current evidence, and then trying to develop reliable, effective, and efficient approaches to implementing evidence-based care.

VA has a particular ethic opportunity in this area, as well as some specific challenges, because as one of the largest integrated health care systems in the world, it has a huge number of sites—practices.  It covers a wide variety of health care products and services, and all of those create enormous complexity and opportunities for study and for implementation.

Very briefly, I’m going to give you a quick roadmap to the presentation.  This is actually a graphic that’s on the website where the guide is presented.  On the website, you can click on each of these areas and follow to the specific pdf that describes them.  What I’m going to start off by doing is just saying the name of the person who is going to be presenting in each of the sections, and then we’ll get started.

The first section is on Applying Theory, and Julie Lowery will be presenting that section.

The next section on Diagnosing Gaps and Designing Interventions will be presented by Teresa Damush.

The third section on Methods will be presented by Hildi Hagedorn.

The fourth section on Formative Evaluation will be presented by Tim Hogan.

The fifth section on Tools and Toolkits will actually be presented in two parts by Rani Elwy and Anju Sahay.

The sixth part, Resources, will be presented by Christian Helfrich and Amanda Midboe.

Without further ado, I’m going to turn this over to my colleague, Julie Lowery, and we’ll get started.

*Julie Lowery:*Great.  Thank you so much, Anne.

This particular section of the guide looks at terminology.  It also talks about *[audio cuts out briefly]* why QUERI’s theories, and models, and implementation research is important, and also discusses how to use them.  I’m going to touch very briefly on each of these areas, but obviously, the details can be found in the guide itself.

There are two domains or areas of implementation research where frameworks, theories, and models become important as evidenced in this diagram.  One is the Prescriptive Domain and one is the Explanatory Domain.  

Essentially, the Prescriptive Domain are using a framework, and a Prescriptive Domain guides how implementation should be planned, organized, and conducted.  In other words, it directly affects the implementation strategy.  

In the Explanatory Domain, we use frameworks, theories and models for guiding the design, data collection, and analysis of the actual implementation research, actually understanding why the implementation process did or did not work.  *[Audio cuts out briefly]* as a formative evaluation, why it might or might not work.

In the explanatory areas, then we’re trying to essentially focus on identifying the barriers and facilitators to implementation.  They tend to look at—this area tends to look at characteristics of the individuals involved in the implementation, and/or characteristics of the organization in which the implementation is being conducted.

Staying in the area of the Explanatory Domain for a few minutes, let’s look at the terminology.  This diagram sort of depicts the relationship among these terms.  At the highest level, we have the framework, which tends to be very general.  It’s pretty much a taxonomy or organization of the various constructs or variables that have the potential to affect implementation success.  At this level, we’re not really talking about, or proposing, how those variables or factors interact with each other, just that they are there, and these are things that should be considered.

At the Small “t” Theory level, we start now to look at, or propose, ways in which these constructs interact with each other, and how they interact, and why they affect implementation.  I say Small “t” Theory to contrast with Big “T” Theory, like the theory of evolution, which this is not.  The theory of evolution is proven through a ton of research.  The Small “t’ theory here in implementation, I think that we’re proposing and looking to test.  We haven’t really proven it yet, but it is a way of organizing our thinking about how things happen.

The model is the most specific level.  It generally takes a subset of constructs from a theory or a framework, and proposes or hypothesizes very specific relationships among those constructs, and how they affect implementation success.  I would say that most of the work that’s being conducted in QUERI these days is at the framework level where we’re looking to identify potential barriers and facilitators to look at a variety of constructs that affect implementation success.  Hopefully, this will guide our work towards more theory and more models down the road.  Once we identify those that seem to be popping up across studies, we can start then to propose relationships among these constructs, and look at testing specific models.

This slide just goes through some of the definitions that I just discussed.  This is a specific example at the framework level, again, staying within the Explanatory Domain.  It’s essentially a taxonomy of constructs potentially affecting implementation organized across five different domains, focuses at the organizational level, although there is a domain looking at individuals.

Another example is the Theoretical Domains Framework focused more at the individual level.  A number of constructs, it specifies and defines constructs within each of these 12 domains.

At the other end of the spectrum then is the model.  This one by Klein, Conn, and Sorra, which as I described, the model takes a subset, just a few constructs, hypothesizes how they interact to affect implementation success, and then looks at or tests this hypothesis in specific settings.  Models do tend to be context-dependent, where theories and frameworks do not.

Then just quickly back to an example at the Prescriptive Domain—I think framework is the most important term to use here rather than theory or model because we aren’t really hypothesizing relationships among specific constructs, but we’re laying out a framework or series of steps for conducting implementation work.  Here’s an example through intervention mapping, where you see examples of the steps that they recommend.

With that, I turn it over to my colleague, Teresa, who will talk about diagnosing gaps and intervention—with diagnosing gaps and designing an intervention.

*Teresa Damush:*Thanks, Julie.  This slide includes the tables of content for this section in the guide.  I’d like to thank my colleague, Dr. Arlene Schmid.

This chapter starts off with an introduction to systems thinking.  Systems thinking is important when designing interventions.  You can think of a system as formal or as informal as well.  The formal system is more of an objective mandated organization.  A lot of times the organization charts of an entity is an example of a formal system.  

The subjective system is more along the lines of existence from the observer.  For example, informal support that a clinician may receive is an example of such a system.  In our Stroke Center, a lot of our neurologists often get informal support from their academic affiliates.

Why is systems thinking important when you’re thinking about diagnosing and designing an intervention?  Well, it allows us to see where there are inefficiencies in the system.  One example of a tool to help diagnose where a system may not be functioning as it should is process mapping.  This is a tool that maps out the process or the performance, and it determines where in the system are your system is ineffective versus effective.  You can look at the gaps such as in work flow among the users of the system, like who owns that particular process, as well as looking at the transfer between the processes as well.  These can be the areas you focus your interventions.

This next slide is an example of process flow mapping.  Each of the boxes represents a process in the system.  The star-like shapes that you see on the screen here are examples of barriers that have been identified in the system for that particular process.  These represent an opportunity to design an intervention to smooth out the process so that you eliminate these barriers.

Intervention mapping, as Julie had mentioned, is a tool that can be used to guide your diagnoses and intervention planning.  This was created by Dr. Bartholomew and originally developed in the health care arena, Health Promotion Programs arena.

Just very quickly, as Julie had pointed out, there are six steps into intervention mapping.  Just to illustrate how you can go about using this to diagnose and plan an intervention, I’m going to provide an example of when we used this to develop an evidence-based secondary stroke prevention program in two VA centers.

We first did a needs assessment at the two facilities.  After conducting that needs assessment, we created a matrix of change.  This first one presents the program objectives across different parts of the system.  The first column represents the provider performance objectives, so what we would like the provider to do in this system ideally.  The second column represents community resources, so how do the users of the system interact with the community resources available for stroke risk management.

The third column represents the delivery system.  This includes the work flow, for example, discharge planning, where risk factor management is often a priority at that point.  There is also system alerts that can be placed inside the system for alerting the prescriber the results of the lab so that they can then prescribe the appropriate therapies for stroke risk factor management.

The third step, the next step is using both theory-based strategies and also practical strategies that were identified from your users of your system back in the needs assessment, so combining those strategies to come up with a planned intervention.  Again, this is an example for secondary stroke prevention where we use the theory of planned behavior and try to develop a perceived social norm that this is what the providers should be doing as models for the other clinicians.  Often we target—at the two sites, we targeted the Chiefs of Neurology so that they can model the appropriate behavior and then hopefully that would influence the other prescribers.

We also came up with practical strategies.  For instance, the neurology residents who are often responsible for the discharge planning wanted the guidelines posted at their work stations so that that would be readily accessible at the time that they need it.

This next slide depicts an example of how the system for secondary stroke prevention would look like in a performance model.  Each of these boxes represents an opportunity to look at the efficiencies and inefficiencies in the system, and then opportunities to design your intervention to make the system flow efficiently and properly.

At the end of the chapter are links to additional tools, and web resources.  This is my contact information, if you have any questions.

Now I’d like to turn it over to my colleague, Dr. Hildi Hagedorn, who will present on methods used in implementing research into practice.

*Hildi Hagedorn:*All right.  Thank you, Teresa.

This section provides an overview of the processes and frameworks that have guided much of QUERI’s work since its inception.  The Six Step Process shown here guides you through identifying a gap in quality, developing an implementation strategy to address that gap, and then evaluating the success of your strategy.

Each step has several sub-steps that are covered in detail in the chapter.  For example, Step 4 includes identifying potential strategies, which is Step 4a—adapting potential strategies, Step 4b—and implementing the strategy, Step 4c.  Steps 5 and 6 were originally separate with 5 focusing on assessment of the intervention’s feasibility, the implementation process, and the impact on health care processes, then with Step 6 focusing on the ultimate goal of improving the patient quality of life.

This is the Four Phase Framework, which is meant to describe the scale of the implementation projects that fall under Steps 4 through 6, implementing, and evaluating implementation strategies.  Following these phases is meant to insure the adequate development, refinement, and evaluation of implementation strategies prior to the investment of large amounts of resources that are necessary for regional and national roll-outs.  National roll-out efforts represent the point where well-tested strategies are handed off to VA operations, with the technical support and evaluation then still provided by QUERI investigators.

I wanted to provide a real world example of how a line of research might progress through the QUERI steps and processes in order to demonstrate that it is not all as neat and linear as we would like to hope.  This is from a Liver Health Initiative, which aimed to implement hepatitis screening, education, prevention services, and treatment referrals into substance use disorder clinics.

We identified a condition to target based on our clinical experience at our own site and also a literature review.  We then moved on to Step 3 to confirm that the gap existed throughout VA substance use disorder clinics by doing a survey of SUD clinic directors.  We then moved back to Step 2 to identify a potential strategy and to refine it based on input from stakeholders at our pilot site.

We then moved on to Step 4 and to Step 5, implementing and evaluating at our pilot site.  Once we had a successful strategy, we modified it for broader rollout, and repeated Steps 4 and 5 by hosting and evaluating a 10-site training program.  As you can see, the one pilot clinic was the Phase 1 project, which then informed the 10-site program, which was the Phase 2 project.  Based on the results from the Phase 2 project, we continued offering the training programs, and continued to cycle through training, evaluation, and refining.  

The last thing I wanted to discuss were the hybrid designs, which were developed by QUERI researchers in an effort to combine traditional effectiveness research with implementation research with the goal of moving promising clinical interventions through the research pipeline to clinical implementation more quickly.  

Hybrid 1 designs are basically standard effectiveness trials with a process evaluation added on to examine potential barriers and facilitators to implementation.

Hybrid 2 designs are evaluating the promise of an implementation strategy while also rigorously evaluating whether the clinical treatment retains effectiveness for these particular types of patients or in this particular setting.  An example of a Hybrid 2 would be the HI-TIDES trial, which took the TIDES intervention for depression that had been shown to be effective in primary care settings, and introduced it into HIV clinics for use with HIV patients.

Hybrid 3 designs primarily focus on testing the effectiveness of the implementation strategy or comparing the effectiveness of two or more strategies while also continuing to collect some patient level outcome data in order to confirm that the effectiveness of the intervention is maintained when implemented into standard practice.

That’s the conclusion of my section.  I will turn it over—oh, here’s our contact information for myself and also Jeff Smith, who assisted with this section.

I will turn it over now to Tim Hogan, who will discuss formative evaluation.

*Tim Hogan:*Thanks Hildi.

I’m just going to provide a brief overview today of a very integral part of implementation science—formative evaluation.  I would encourage everyone on the call to spend time with the full Implementation Guide as there’s a lot more details there about formative evaluation.

What is the purpose and need for formative evaluation in implementation?  Formative evaluation offers a rigorous assessment process designed to identify both potential influences and actual influences on the progress and effectiveness of implementation efforts.  

Formative evaluation is really a set of activities that’s focused on the process of implementation, rather than the outcome of implementation.  Formative evaluation helps to determine how various factors amplify, facilitate, mitigate, or impede implementation efforts in a given context.  It’s important to keep in mind that those factors may exist at different organizational levels or different levels of analysis.  

Lastly, insights garnered from formative evaluation can support more precise replication and further dissemination of evidence-based practices perhaps across settings, perhaps across other studies, and so on.

In the way of data collection strategies that are used in formative evaluation, there is a variety of strategies that can have a role in formative evaluation.  First, quantitative assessments can be used to collect data about stakeholder groups and implementation processes.  I’ve included just two examples here—surveys of individuals, perhaps those in the implementation setting, or surveys of those who might be exposed to the intervention that’s actually being implemented, and second, analysis of data from existing databases, so secondary data analysis.

Qualitative assessments offer a “deeper-dive,” typically with less individuals in an effort to understand more fully the factors that influence implementation, implementation strategies, and best practices.  Some examples of qualitative assessments might be semi-structured interviews, focus groups with small numbers of informants, and perhaps other strategies depending on the perspective sought.

In the way of formative evaluation research processes, there are actually a number of essential steps to conducting formative evaluation research.  The first and most important step is to really anchor your formative evaluation in a set of aims and a vision of how the formative evaluation informs your broader study goals.

It is also important to include the theoretical or conceptual framework that is guiding the broader study in the development of your formative evaluation.  We’ve heard earlier in this seminar about the role of theories and frameworks.  A thoughtfully selected framework can really suggest ways to focus your formative evaluation or highlight factors you might want to explore.

Subsequently, the evaluation team must also develop instruments and procedures to collect the data for the formative evaluation, collect the data, and then analyze and report that data, ideally in a rapid and continuous way.

I just wanted to mention a couple challenges that we often encounter in formative evaluation research.  There are a variety of challenges.  I just have two categories on this slide.  The first is challenges in data collection issues, so first being cognizant of selecting and effectively applying the appropriate approaches to the data that is actually needed, and then second, identifying the “right” informants for the formative evaluation.

The second category of challenges is just regulatory issues, given that the methods and activities of formative evaluation oftentimes look very different than what we might consider traditional research activities.  Describing formative evaluation to organizational entities like research boards or institutional review boards that may not be familiar with it can oftentimes be difficult.

That actually brings me to the last of the points that I wanted to make today about writing about formative evaluation.  Just a couple basic points here, but really important to keep in mind, the first is that knowing the needs and perspectives of your audience members when you’re writing about formative evaluation is absolutely critical.

A thorough description of formative evaluation will typically address the setting and the samples involved, data collection techniques used, data collection procedures, and then processes for managing and analyzing the data that actually results from the formative evaluation.

I want to just give a quick thanks to Jennifer Hill and Henry Anaya who helped work on this section and these slides as well.

At this point, I’m going to turn it over to Rani and Anju, who are going to talk about implementation resources.

*Rani Elwy:*Thanks, Tim.  This is Rani Elwy from the HIV/Hepatitis QUERI Center, and my colleague, Dr. Anju Sahay, from the Chronic Heart Failure QUERI Center.  I will be chairing this section of the guide.

This section is devoted to tools and toolkits that focus on the implementation of evidence-based practices to improve the quality of care for veterans and veterans’ health.  Many QUERI investigators have conducted projects focusing on translating evidence-based practices into routine care, and as a result, *[audio cuts out briefly]*, tools to assist in the implementation of these projects.

This section is divided into two parts.  Section A is on the VA HSR&D Cyberseminars, which you are all familiar with because you are attending one.  They cover a range of topics presented by many of the resource centers and researchers throughout the VA such as QUERI, VIReC, which have both the database methods and Clinical Informatics Cyberseminars, the Health Economic Resource Center, HERC, Women’s Health, PACT Demonstration Labs, Evidence Synthesis Program, and more, all present on the Cyberseminar.  

This is an example of one that’s upcoming on May 5th, the Traumatic Brain Injury Toolbox.  If you were a researcher wanting to do some implementation work around this area, then this would be something that you would want to attend.  Cyberseminars also have archived sessions, so this is a link that you can go and search archive sessions for.  This particular one that is upcoming in Traumatic Brain Injury is presented by VIReC.

This is the site, which you’re familiar with, with the link there, and with the QUERI Implementation Guide session that you are currently attending.  *[Audio cuts out briefly]* toolkits is from the HIV/Hepatitis QUERI.  It is the Hepatitis C Self-Management Program.  It was presented in one of the Timely Topics of Interest Cyberseminars led by Dr. Erik Groessl from VA San Diego.

This is a link here to the specific archive session that you can examine, but it basically provides you with the tools if you wanted to set up a Hepatitis C Self-Management Program.  It’s a six session, self-management program building on work by Kate Lorig and Albert Bandura in self-efficacy and care management at Stanford using principles of motivational interviewing and empowerment. In process your patients are really encouraged to take ownership for their health and to build these skills and become confident in being a patient who can manage their own health.

If you wanted to learn more about this evidence-based practice, you could go to this link and download the past slides.  The audio and transcript could be very helpful in your learning about how to move this forward and into implementation into practice.

Dr. Sahay will continue with Section B of this guide.  Just as a point, there are a lot of parts of this guide—of this section of the guide that would require regular updating, so make sure that you go back to Cyberseminars and search for upcoming sessions as you go along.

*Anju Sahay:*Thanks, Rani.  This is Anju with the CHF QUERI, or the Chronic Heart Failure QUERI.  I’ll talk about tools, toolkits, conferences and trainings, all targeting implementation research.

Let’s start with the implementation toolkits.  If you had a chance to go through the Implementation Guide, you would have seen that this guide lists several toolkits, which are available both within and outside the VA.  As an example, we’ll focus on one of these toolkits, which is the Agency for Healthcare Research and Quality, or AHRQ’s quality innovations toolkit.  This web link will take you to an exhaustive list of quality tools listed there.

Now one of these quality tools is the CHF QUERI’s Heart Failure Toolkit for Providers.  I’ve been integrally involved with the development of this toolkit.  We developed it in the... using existing tools developed at various VA facilities and non-VA organizations like the American Heart Association and the Heart Failure Society of America.

Here is a screen shot of the website for this Heart Failure Providers Toolkit.  Just below that, you can see the link if you want to go and access it later.  This toolkit has downloadable tools organized into 18 categories.  All tools are on one web page, which provides the ease by just scrolling down the page to browse or look for a specific tool.

Let me walk you through this slide, which is a screen shot only of the top part of the webpage copied here in two boxes.  The left screen shot shows the description of the toolkit, and the right screenshot shows the initial list of the 18 categories of tools that we have developed.  Within each category, there are folders specifying the source of those tools from VA and non-VA sources.  

Once you pick a folder, you can see the tools, which are either downloadable pdf documents or web links.  Since the Heart Failure Provider toolkit is an implementation toolkit, it is important to assess its use value and get feedback for continued improvement.

At the bottom of the left box or on top of the right box, which is just a continuation of this webpage, you can see three buttons for audience to suggest a tool, provide feedback about the downloaded tool, or provide comments.  We also use other methods to periodically make improvements in this toolkit.  

Here is a list of a few good conferences on implementation research.  I’ve attended each of these three and found them very helpful.  I also provided the web links for each conference over here.  The first is the National Institutes of Health conference called Dissemination and Implementation Conference.  The second one is the Global Implementation Conference led by Dean Fixsen.  The third is the Seattle Implementation Research Collaborative.

This slide provides a list of two good training opportunities for implementation research, both within the VA and outside the VA.  Given the short period of time, I’ll leave it here and later on you can—when you download the slides, you can go over and look at the web links, and look at more information about these training opportunities.

This is the end of our section.  Here is the contact information for Rani and myself.  Feel free to ask any questions later on.  Now I’ll hand over the presentation to Christian Helfrich to talk about the next section.  Thank you.

*Christian Helfrich:*Thanks, Anju.

My name is Christian Helfrich.  I’m with the Ischemic Heart Disease QUERI Center.

I want to talk about organizational partners.  Much of our work is implementation research, which either requires or is most effective when it’s conducted in partnership with operational stakeholders—by that, I mean that people who we ultimately hope to help change clinical or administrative practice to improve care quality and outcomes.

The Veterans Health Administration, as with any large complex organization, has many organizational units that have partnered with or are potential partners for implementation research.  In general, it’s helpful to develop an understanding and awareness of the different functions and missions that the different units play within the organization.

Often organizations have units responsible for institutional policy and planning, operations and management, monitoring and oversight, and training and development.  All of these represent potential partners.

What makes an effective partner, and how do you meet the needs of your organizational partner?

Effective partnerships typically built over time, developing long term relationships, helps you understand the operational partner’s needs and capabilities and better identify partnering opportunities that meet both their needs and capabilities and yours.  It also develops trust and allows you to respond quickly to emerging opportunities.  If your operational partner has a new initiative that they’re launching, and they need implementation of the initiative to be evaluated, if you have a track record with helping them, there’s a better opportunity that they’ll come to you, and give you a chance to do this work.

In terms of partner projects, the most effective partnered projects often share common characteristics.  First, they have a real question.  Not a question that has only one real palatable answer, such as how good is my new clinical program?  A real question has multiple potential answers, all of which are potentially useful to the partner.  For example, it might be which of these two or three programs is most effective, and in which clinical settings are they most effective, or what are effective implementation sites doing differently from sites that are struggling with implementation?

Effective partnered projects also have frequent communication.  The operational landscape changes quickly.  That can create challenges for meeting the needs and expectations of the operational partner and the researcher.  Frequent communication, while it doesn’t solve the problems, can go a long way to preventing problems.

I think in particular with an operational landscape, oftentimes partners have changing priorities.  At the beginning of an initiative or a program, they’ll have one set of priorities, and their institutional landscape changes.  With sufficient communication and lead time, you can respond to those needs, to those changing priorities without causing problems for your own research.

Finally, effective partnered projects often have an early win.  Nothing creates goodwill and builds trust like delivering a success.  Even relatively modest products that can be planned for—things like a thought piece in a journal, a cyberseminar presentation, or a summary of early lessons from pilot projects—those can all be good wins.  The key is to plan for them.

I would just note that contrary to conventional wisdom or popular opinion, operational stakeholders are often delighted with academic papers.  The challenge with academic papers is just having them published early enough for them to really represent an early win.

With that, I’m going to hand it over to my colleague, Amanda Midboe, with the HIV/Hepatitis QUERI Center.

*Amanda Midboe:*Thank you, Christian.

As Christian mentioned, I’m with the Hepatitis QUERI Center.  Today, I, in this section, will talk about some of the practical aspects and resources for getting your implementation research done, including resources that will help you as you are preparing your proposal and your research questions.  Some of this will echo and elaborate on some of the points in Christian’s presentation as well as Rani’s and Anju’s.  

To start, it can be helpful to orient yourself with the overall organization of the VA by looking at the org chart, which was recently updated in October of 2013.  I’ve included a link here.  I’ve also included a couple of links where you can find more detail about VA program offices, including a great recently developed manual that really in detail describes VA program offices that I find highly beneficial, and it’s the pdf link that I put in here.  It’s mostly beneficial because it’s often difficult to figure out who you should engage with in your work.

Then finally, there are ten QUERI Centers, nine of which are focused on high-prevalence and high-risk conditions, as well as a relatively newer QUERI Center focused on eHealth technology, known as the eHealth QUERI.  These centers can really help you with your research questions and your proposal, so contacting them and being in touch if your research questions fall within one of those domains, and sometimes if they fall within multiple QUERI domains, it can be really beneficial to involve them.

As you move forward with planning your proposal, you may also find it helpful to engage with or take advantage of resources provided by the Center for Implementation Practice and Research Support, or CIPRS for short.  It was mentioned by Chris, but just to reiterate, CIPRS provides education and technical assistance to VA implementation researchers, as well as support for VA implementation practice.  They also work to develop and further implementation science, theory and methods.

If you’re incorporating quantitative data that includes large VA administrative data sets, then you may find some useful information at the VIReC website, which is short for the VA Information Resource Center.

Then finally, cost and health economics have become a critical component of most implementation research.  Though it’s important to be aware of the Health Economics Resource Center, they have health economists who can consult or possibly serve as co-investigators on your project.  They have a number of resources on their website that can help both VA and non-VA researchers.

With my remaining time, I’d like to talk a little about the VA’s org chart, and how you as researchers can start to think about finding key partners in operations and policy.  The reason I want to spend some time on this is because it can be really confusing for early, and sometimes even more established, implementation researchers to figure out who to engage with, especially if you stare at the org chart and you see all these offices, but you don’t realize that there are several offices oftentimes within each division.

For example, within Patient Care Services, there are several offices that may be of interest to you.  This is just a few, not at all representative of all the offices within Patient Care Services, but if you’re doing any PACT-related research, or a Patient Aligned Care Team-related research, then the Primary Care Program Office may be of interest to you.

There is also the Office of Specialty Care Services, which is responsible for facilitating and implementing—sorry—facilitating implementation and evaluation of the SCAN-ECHO project, which Christian mentioned, as well as many other things in addition to that.  That’s just one example.

There’s also the Office of Mental Health Services, which includes the National Center for PTSD, and involves promoting just generally optimal mental health.  If you’re doing any implementation research focused on improving mental health, you may want to contact the Office of Mental Health Services.

Now I’d like to give you a concrete example from our QUERI, where, as it can often be so—I want to give you a concrete example from our QUERI, as it can often be very valuable to help establish relationships with partners as the QUERIs have often cultivated long-standing partnerships with them, so really leveraging—if you do have QUERI-related research, leveraging their existing partnerships.  

An example from our QUERI, the HIV/Hepatitis QUERI can be found with the Multi-VISN, HIV Testing Project.  Within our QUERI, our primary partners, the Office of Public Health, Clinical Public Health Group—again, just one of the offices within the Public Health Division—they were really instrumental to the success of our Multi-VISN and Quality Improvement HIV Testing Project, which Christian mentioned previously.  They provided us with help developing and implementing the clinical reminder for this HIV Testing Project, so for any investigators working with those, we really help them foster a strong relationship with the Clinical Public Health Group.

One final thing that I want to mention is that although the Clinical Public Health Group is our primary partner, we often work to help investigators establish and foster additional partnerships.  That’s something as implementation researchers you want to think about, not just finding one office or group with whom to partner, but sometimes you have research questions that branch multiple offices or divisions.

For example, one great recent example is actually an investigator in our QUERI who has been interested in looking at homelessness in HIV and hepatitis C.  That’s a key partnership for this investigator with not just the Clinical Public Health Group, but also the Office of Homelessness.  This has really been—these two key partnerships have really been instrumental to shaping his research questions and his grant proposal.

In closing, I really appreciated having an opportunity to talk to you about resources for implementation research.  If you have any additional questions, I would be happy to answer them.  Thank you.

*Moderator:*Great.  Thank you.  In fact, I’d like to thank all of our presenters today.  You did an excellent job.  For attendees, this is where we can now address your questions and comments.  I know a lot of you joined us after the top of the hour, so if you look on the right hand side of your screen, you will see a Q and A box.  Go ahead and submit your questions and comments through that console.  Just simply type your question or comment into the lower box, and press send, and we’ll get to them in the order that they are received.  We don’t have any in the queue at this time, so we’ll go ahead and wait for some of those to come in.

Would anybody like to give additional comments while we’re waiting for questions?  Here’s one right now.  What are some examples of other methods you use to update the Chronic Heart Failure Toolkit?

*Anju Sahay:*Yeah.  Hi, this is Anju Sahay, and I was covering that question.  We have formative evaluation methods where we are currently gathering data through surveys and also semi-structured  interviews, getting feedback about the toolkit, how they like the toolkit, how useful does it work, or which box I use for what else should be included, which type of providers are finding it useful, and similar feedback both using survey and open-ended interviews from members of the Heart Failure Network to whom we keep sharing this toolkit pretty regularly as well as non-members, so those who really haven’t heard it, want to find out from another VA provider that heard about it.  These are different ways we use or gather information to keep updating the toolkit.

*Moderator:*Thank you, Anju, for that reply.  When is an implementation study quality improvement, and when is it considered research?

*Anne Sales:*Yeah.  This is Anne Sales.  That’s not an easy thing to decide, frankly, and it depends on each study.  It depends quite often, particularly within the VA, on the funding source.  If a project is funded through research and development, it is research.  I think that piece is very clear.  That means going through all of the necessary regulatory components for research.

If it’s funded through an operational partner using medical care dollars, then it depends.  Many people, I think, within VA are familiar with the fact that new guidance was issued in 2012 for non-research quality improvement activities.  I think by contacting folks in the QUERI Program Office, if you are a VA person, they can help you find that guide and get information about it.  I’m going to leave it at that because it is a fairly complex question.

*Christian Helfrich:*This is Christian.  I was going to just add—I mean that’s exactly right that it’s really something that is complex and influx.  Oftentimes, I think, what we talk about though is generating generalizable knowledge in that research studies have that as the primary purpose.  Quality improvement has the primary purpose of improving the quality of care in the given setting.  It may still generate knowledge that can be used elsewhere, but that’s not the primary purpose; whereas with a research study, that is the purpose.

*Moderator:*Thank you both.  We appreciate those responses.  

We do have a comment that came in.  Thank you all very much.  You covered very important topics in a short period of time with great coordination and no overlap.  This was fantastic.  

Thank you for that.  Next question—what is your advice to operational partners who would like to work more closely with researchers?

*Anne Sales:*Christian, would you like to—

*Christian Helfrich:*Yes.  Sorry, I was fumbling with my mute button there.  Yes, and I think with operational partners, I think it is similar to the advice for researchers engaging early and understanding the sorts of capabilities and needs of the research community.  Certainly with implementation science, with new programs coming out, there’s a tremendous amount of opportunity to engage with implementation researchers because we’re very interested in understanding how new programs roll out, why there might be variation in how programs are implemented, and oftentimes, those are things, obviously, that are of great interest to the operational units.  I think engaging, early engagement, finding out what the research capabilities are with different units, that’s key.

*Anne Sales:*Thanks Christian.  This is Anne Sales.  I also want to just say that CIPRS, the Center for Implementation Practice and Research Support, to which there are links both on the QUERI home page and within the guide, is a place to go if someone is in operations and is looking for researchers who are interested in their area.  One of the things we try to do is provide a kind of knowledge-brokering function so that we can help link people together.  Amanda, I don’t know if you wanted to also say something about this topic.

*Amanda Midboe:*I was actually going to suggest contacting CIPRS, and definitely, if you have a question that is related to a specific QUERI Center, then please feel free to contact the QUERI Center directly.  We have had that happen with operations before, where we haven’t had people and operations connected with us before, but we’ve connected with them.  It has always been, during my tenure at least in the HIV/Hepatitis QUERI, a really fruitful and productive partnership.

*Moderator:*Thank you all.  We do have eight great questions in the queue.  The next one—are there any QUERI studies being done regarding distance treatment of hepatitis C?

*Amanda Midboe:*This is Amanda.  I can answer that.  Yes, there are.  There is the HCV SCAN-ECHO Project, which I briefly—I didn’t mention HCV SCAN-ECHO, but I briefly touched on.  Then Rani, if I’m missing anything, please feel free to chime in.

*Rani Elwy:*My first thought was the SCAN-ECHO Project, which is the main method that the VA is using.

*Christian Helfrich:*Yeah.  Then also the Office of Patient Care Services has a number of pilot programs, in addition to also doing pilots in SCAN-ECHO, which is a group—a televideo consult program.  There are also electronic consults that they’ve piloted, and there’s a new program called Specialty—generally under the label of Specialty Care Neighborhoods that is being piloted.  Basically, the idea is to have a team-based specialty care, and to try to provide more proactive outreach by specialty teams.  I’m not certain, but I believe that hep C was one of the programs that they were looking at for Specialty Care Neighborhoods.

*Amanda Midboe:*That’s really interesting.  Thanks Christian.  I should mention that Catherine Rongey out of the San Francisco VA is leading the service directed project that is looking at the HCV SCAN-ECHO in case you have further questions about that.

*Moderator:*Thank you all for those replies.  The next question—would it be possible to get assistance from the QUERI group about non-research projects, specifically for suggestions on implementing a grant program to provide service to veterans?

*Anne Sales:*Yeah.  This is Anne Sales.  I think the answer is that—I assume this is a VA group—that yes, I think they would certainly be interested in discussing it.  I think Amy Kilbourne is probably the best contact person initially for that.  Amy is the Acting Director of the QUERI Program within the Health Services Research and Development Service in the Office of Research and Development in the VA.  That’s where I would start with something like that.

*Moderator:*Great.  We have Dr. Kilbourne on the call, so she has fair warning.  Do you have suggestions for how to balance the value of tailoring site specific research in a multi-site study?  I mean when it’s a multi-site study, so you need standard measures across the sites.

*Christian Helfrich:*This is Christian.  I do think—I mean it sort of gets to the heart of what we do as implementation researchers, hopefully identifying those common sort of determinants, or factors, or constellations of conditions that are not site specific, but it’s actually very tricky.  I think that right now the state of the science is that we have—although we have ideas about the things that are important to tailor, and important to keep consistent, the truth is that we’re still at a very early stage, and that that’s actually a lot of the work of the QUERI teams.  I think there are opportunities there to try to understand that.  I don’t think that there are very good answers at this point about what to tailor and what to keep consistent.

*Anne Sales:*Yeah.  This is Anne Sales.  Thanks, Christian.  I think that this is a really important emerging area.  What I would say personally is that there are common measures that are being developed.  We have some validated tools and instruments now that are relatively new, and I would certainly encourage anybody who is interested in that, in those kinds of tools, please contact me.  I’d be more than happy to give you information and help connect you with others working in the area.

*Moderator:*Wonderful.  Thank you.  We do still have a couple remaining questions, which I will be reading aloud, but for our attendees, I’m going to put up your feedback form now, so please take a moment and answer these questions.  It is your opinions that help guide what topics we will have presented in the future, and we want to learn more about what you’d like to see more of.

With that, let me get back to my question section.  All right, give me just one minute.  All right, it looks like lots of feedback is streaming in, so that’s great to see.

Okay.  The next question we have is—my research interest is in the sustainability of interventions and QUERI implementation initiatives.  I see this as being closely tied with implementation research.  Is any work being done in this area within the implementation arena?

*Anne Sales:**[Pause]* Molly, we were so awed by the really amazing display of the questions that we were wondering—could you just repeat that please?

*Moderator:*No problem.  Remember, you don’t get to evaluate yourself, so don’t touch that.  My research interest is in the sustainability of interventions/QI initiatives.  I see this as being closely tied with implementation research.  Is any work being done in this area within the implementation arena?

*Anne Sales:*The answer to that is yes, definitely, but I would say sustainability and related, but slightly different, concepts that’s being called sustainment are both in their infancy.  What we have at this point is sort of early interesting suggested stuff rather than definitive answers.  It’s an area that I would personally say it’s sort of on the cutting edge of implementation research, and anybody working in this area, again, I think we would be interested in knowing about it, and having conversations.

*Moderator:*Thank you for that reply.  

*Presenters:**[Cross Talk]*

*Moderator:*Oh, go ahead.

*Rani Elwy:*Could I just add a little bit.  This is Rani Elwy from HIV/Hepatitis QUERI.  If somebody is interested in sustainability, they could look at the PRISM—P R I S M—model from Russ Glasgow and also the EBPAS model from Greg Aarons, so it at least incorporates some sustainability into those models.

*Moderator:*Thank you, Rani.  The next question—what is the difference between an IIR award and an SDR award, and what are the limits of funding available for each?

*Anne Sales:*Yes.  This is Anne.  This is probably a question that’s of most interest to VA researchers.  First of all, Service Directed—Investigator-Initiated Research is the sort of classic analog to, for example, an R01, R03, and in the NIH context.  It is what it says.  Investigators initiate the concept, and write a proposal that’s reviewed by a National Merit Review Board.

Service Directed Research, I think you might actually need Service Directed Projects.  SDPs are a standing funding mechanism that the QUERI Program operates.  They are intended to be partnered research, and involved with either one of the ten existing QUERI Centers, or partnered research that has strong organizational partnership links.

Both of those are opportunities that have regular deadlines, regular review cycles.  There is, I believe still, a category of research called Service Directed Research, or SDR.  That is actually very much an initiated... sort of  one-off initiated usually by a partner and not in the regular review cycle, but if you’re interested in either IIR or SDP funding mechanisms, information about both of those is available through the HSR&D website.

*Moderator:*Thank you.  We have one final question.  You may have already touched on it a bit, but I’ll read it aloud.  Does QUERI Program Office advise on smaller QI projects that may hold potential for spread?

*Anne Sales:*Again, I think this is probably a question that is best addressed to the QUERI Program Office, and to Amy Kilbourne, but I think the sort of short answer is that the QUERI Program Office is certainly interested in hearing from the field.  If people have specific questions and ideas, it’s a good resource to contact them or to contact CIPRS, and we can also help to ensure that you get answers to your questions.

*Moderator:*Great.  Thank you so much.  I really want to thank our QUERI Implementation Guide Writing Committee Team for presenting for us today.  You did an excellent job.  I also want to very much thank our attendees for joining us.

As I said, this session has been recorded, and you will receive a follow-up email with a link to the archive and the handout, so feel free to pass that along to your colleagues that couldn’t come.  I will leave this evaluation page up for a little bit longer, so feel free to take your time filling it out.

With that, this does conclude today’s presentation.  Thank you all again for joining us, and have a wonderful day.

*Anne Sales:*Thank you.

*[End of Audio]*

Page 1 of 18