**Transcript of Cyberseminar**

**QUERI Implementation Seminar**

**Developing Effective Partnerships for Implementation Research and Evaluations of Quality Improvement Programs**

**Presenters:  Amanda M. Midboe, PhD; Christian D. Helfrich, PhD**

**June 10, 2014**

*This is an unedited transcript of this session. As such, it may contain omissions or errors due to sound quality or misinterpretation. For clarification or verification of any points in the transcript, please refer to the audio version posted at www.hsrd.research.va.gov/cyberseminars/catalog-archive.cfm or contact**Christian.Helfrich@va.gov**or Amanda.Midboe@va.gov.*

*Moderator:*  And at this time I would like to introduce our two speakers for this QUERI implementation seminar. Speaking first we have Dr. Amanda Midboe and she is part of the HIV/Hepatitis QUERI at the Center for Innovation to Implementation in VA Palo Alto Health Care System. Joining her today speaking second we have Dr. Christian Helfrich and he is part of the VA Puget Sound Health Care System and he works on Ischemic Heart Disease QUERI and I know I just butchered that, I apologize.

*Dr. Christian Helfrich:*  That’s quite all right.

*Moderator:*  At this time I would like to turn it over to Amanda.

*Dr. Amanda Midboe:*  Thank you very much; I appreciate it. Again anyone let me know if you can’t hear me okay, I’m happy to pick up the handset if you have any problems hearing me. So first off I’m really excited to be here talking to you today about developing effective partnerships for implementation and quality improvement. I’ve been a co-implementation research coordinator with the HIV/Hepatitis QUERI 2010 and I’ve learned a great deal about the importance of partnerships in both research and quality improvement. 

I’ve also learned a great deal from my colleague Christian who has been implementation research coordinator with the Ischemic Crisis Disease QUERI for several years and I’m really pleased to get the opportunity to do this cyber seminar with him. As mentioned previously I’ll do the first half of the presentation which will cover some of the practical tips and strategies for initiating and maintaining partnerships. And then Christian is going to take over and discuss specific case study that illustrates the principles and strategies that I’ll outline initially.

So quick -- okay before we get started I would like to acknowledge the working contributions to this presentation by individuals listed on this slide, particularly Catherine Rongey who is located at the San Francisco VA and whose partnership work we will be highlighting. On the rest of the slide are key co-investigators who has shaped indirectly much of what we will be talking about today and key folks also in operations and they include folks from the Office of Specialty Care Services, the Denver-Seattle Specialty Care Evaluation Center and the Cleveland Specialty Care Evaluation Center. 

And before we dive into the presentation I would like to take this opportunity to learn more about you and your role in the VA. Many of us actually wear several hats so please select all of the roles that apply to you. 

*Moderator:*  Thank you; it looks like a lot of responses are coming in; we have a very engaged audience so we’re always happy to see that. And it looks like we have a varied audience. So we’ll give people a few more seconds to get their responses in and then Amanda you can talk through it real quick. We’ll go ahead and close the poll now. 

*Dr. Amanda Midboe:*  Okay so it looks like we do have a little majority of researchers for sure followed by a lot of folks doing quality improvements; so that’s great. And then gosh I wish I could figure out what the other is because that also sounds interesting. And then some operations and some clinical folks. Wow, this is a really great audience today; thanks so much for attending.

*Moderator:*  Sorry for interrupting but regarding those that selected other, when we put up the feedback survey at the very end of the session we will have a more extensive list of roles so you can probably specify it there. 

*Dr. Amanda Midboe:*  Thank you, awesome. Okay. All right, so we have a lot to cover; let’s go ahead and dive in. First of presentation goals. Today we hope that you will be able to leave this presentation with an understanding or at least an improved understanding for those who have more experience of the importance of partnerships and implementation and research and quality improvement. We also hope that we can help you with finding and using resources for developing partnerships as well as provide you with some key strategies that will help you to have a successful partnership. And finally we hope that many of you on the call today will feel more confident as you development partner implementation research or quality improvement evaluations and be excited to start your own if you have yet to do that.

So as we move forward through today’s presentation it is important to make sure we are on the same page in terms of what we are calling implementation research and how we define quality improvement. And I wouldn’t say these are perfect definitions so some of you may have some different understandings out there which may be completely valid but this is just to get us started for discussion purposes and for the purposes of this presentation.

Briefly implementation research often prioritizes testing falsifiable hypotheses, they usually prioritize implementation researchers usually prioritize internal validity over external validity which can mean well controlled design; even something as simple as comparing control intervention sites. But for QI or quality improvement we’re usually more concerned with will it work in a place like ours or is it generalizable? 

Quality improvement often tends to be continuous or iterative and rapid focusing a little bit more on practical issues, not that implementation research doesn’t but more essentially the majority of QI projects are a little bit more practical. The P < .50 is a reference to the significance level. In research we’re often worried about mistaking random variation for cause and effect; we’re constantly making various comparisons so that concern is understandable and operations often know their way to or perhaps more options and they’re looking for any evidence at all to make a decision. In some cases they’re actually really faced with a coin toss and anything that gets into less of a coin toss or less than -- or P < .50 is helpful. 

As any of you who have applied for research grants or even research centers such as the recent centers of innovation or coins can attest to there is an increasing emphasis on partnerships in your work and they are required in fact for receiving certain types of research funding in the VA and this is for good reason involving key partners in your research or evaluation projects has tremendous benefit to the design of your study and development of your work. 

It’s also critical to for the dissemination of your work as your operational partners are usually the ultimate kind of owners of your evidence based clinical practice and will be instrumental to further dissemination of whatever clinical practice you’re testing. And to begin to get a sense of where your work might fit the Org chart can be a good place to start and so I’m going to spend a little bit of time talking with you about its organization. And I know it’s not an ideal side but hopefully do have the handouts because a little more closely.

So this is the most recent organizational chart for the VHA. It provides an overview of different programs and offices and VHA operations and it’s updated on a somewhat regular basis particularly when there are new offices added or the organization occurs across program offices as I said a few years ago. It basically provides a high level overview of the programs and offices and operations and many of you on the call may be more likely to fall on the right side based on the survey which is here in research and development. And so it’s right there by Dr. Betzel’s signature. And there’s also a new office which has been added this fiscal year known as connective health so that’s a recent addition to this one. And many of you may not yet be familiar with the work that they are doing but my main caveat here in talking about this with you is just to point you towards the organizational chart because I by no means an expert on how all these pieces fit together. It looks a little straightforward, it looks mostly straightforward but it’s actually fairly complicated and there are several program offices that each of these boxes or overall offices. And so what I can do usually though is sort of figure out how to find relevant program office information and that’s what I’m going to hep you do shortly. 

But before I go any further I do want to highlight the two different sides of the org chart. The left side is what researchers in the VHA option refer to as the Field. It is sometimes referred to as 10N which is basically the mail stop code at VA central office for this field size. It includes all 21 VISN’s which means Veterans Integrated Services Network and a VISN generally includes five to 10 VA medical centers or healthcare systems and their associated outpatient clinics. And fields for practical purposes for this presentation they represent the folks who will hopefully be implementing their findings directly.

The right hand side of the org chart includes multiple program offices. Dr. Robert Jeffey (ph) is known as the principal deputy under secretary for health, which is located here or PISH (ph) and the mail code for the PISH is 10A and I only mention mail codes because you’ll often hear people talking and that type of speak as 10N, 10A, 10P and then Dr. Madu Aberwahl (ph) is ADUSH for health for policy and services located down here. And he’s actually part of 10P in reports to Dr. Jeffey. 

Just briefly I would like to highlight a couple of program offices that we will talk a little bit more about in our later case study which Christian will present. First you may note that there is a box on the left hand side that reads ADUSH for patient care services, let me just -- down there. Rogi Jane (ph) is actually the ADUSH or the assistant deputy under secretary for health for patient care services so many people who are doing partnered research at QI this is also an important office which is why I’m highlighting it. It includes mental health services, primary care services, office of specialty care services and tele-health services as well as many more groups and services. 

Just below that, which we will talk about briefly later is public health and that is office led by Vicki Daty (ph) and it contains several groups including the clinical public health group which contains programs on HIV/Hepatitis C, influenza, tobacco and health, healthcare quality and surveillance and research. Given my role as co-implementation researcher in the HIV/Hepatitis that is an office that we have a strong partnership with and work with often. 

Okay now that we talked a little bit about the big picture let’s start to dive into more details including individual steps that you can follow to develop and strengthen partnerships. First I cannot emphasize enough the importance of starting early particularly for those applying for grants. If you have an idea and think that it’s worthwhile, start your partnership work way in advance of the grant deadline. Many people have a lot of tasks and other projects that they’re juggling so it’s something that is often easier to say than to do but if you can really start this partnership work early it’s going to be incredibly valuable to you. One analogy that Christian brought up as we were preparing this presentation is that of tree rings. So think of the partnership that you’re building as growing rings around a tree, you want to build healthy core that will enable long term growth and future collaboration for years to come or future tree rings for years of productive collaborative work that builds on previous work.

Second and this relates to the Org chart that we were just talking about. You want to gather as much information about your partner before you initiate research or QI and there are two existing researchers within the VA if you’re working within the VA that will help you do that and I will show you some examples of those in a moment. In the meantime I also want to mention that asking colleagues for advice or input can be extremely valuable. They often have inside knowledge about perspective partners that you can’t get from reading just through pages or other entries. Also if you’re planning to apply for implementation research grants such as the rapid response proposed -- project proposal or service directed project proposal which are known as RRP or XDP for short then get in touch with your relevant QUERI’s that are -- as mentioned previously I’m with the HIV/Hepatitis QUERI, Christian is with the ischemic heart disease QUERI. But there are other QUERI’s such as substance abuse and mental health. And they can really be a wealth of knowledge as you would navigate this process and to really help you gather key information about your partners.

So let’s go ahead and look at a couple of the researchers I mentioned. And again this is going to be available to you and is available to you in handouts and will be available following this presentation so you’re going to not spend a ton of time and certainly not going to click through all these links. But the first link on slide 10 which is the previous slide will lead you to this site. And as you can see many of the VHA program offices have clickable links and they’re going to have varying levels of information. And then for example one of the offices listed is the ADUSH for patient care services, which I highlight on the screen. If you click on that link for 10P4 you are taken to the page that I’ve -- that I’m showing just beneath it. From there you can see there are clickable links to specialty care services, mental health services and many more offices within patient care services. So really you can see that you can start to dive deeply into different offices and it’s -- this particular site actually give  you a ton of information; so use the web resources. 

And I really recommend you spend some time on the main page and determine if your research or your QI benefit from partnerships across multiple program offices and sometimes we find one partner and then we think “Oh this is perfect, they said exactly what I need” but you often can have multiple partnerships and that if you work together can be incredibly valuable to your research project, to your quality improvement project.

The second link provided on slide 10 is a pdf. I was pretty excited when I saw this came out last year. It’s an extremely dense document and includes much more information than you will likely ever need; however if you’re someone who is really curious about overall VA structure and would like a document with an organized table of contents then this is a really good resource for you. I went ahead and highlighted section 4.3 which will have a lot of the information that we talked about on the right side of the org chart and actually may be a good starting place for some of you who are interested in a really comprehensive overview of a given program or office. Okay, all right.

Now we talked about resources available to gather information about your partner let’s get back to some other steps that are critical to developing a partnership. So once you’ve done your research contact the partner. Try not to send an overwhelmingly dense email or something too brief; moderation is really key here as is clarity. So it’s worth it to take some time to craft an email or think about a phone call that you might make. Tell them your aim and these don’t have to be perfect but a starting place for discussion. Give a little background of who you are; again don’t go overboard with details but provide enough information so they get a sense of who you are and your experience. It’s often helpful to send a CD along as a way of professional introduction. Again though avoid sending a 10-15 page plus CD with everything every time, your partners are really busy and they don’t have time to read all that information.

And finally be explicit in your own mind and with your partner about what they can help you with and how you can help them. And be really honest in your intentions and be careful to not convey attitudes they owe you or that this research that you’re doing is going to change their whole office because it’s generally not true. You guys will be mutually informative to one another. And when you do get to talk make sure to discuss relative priorities explicitly. So what do each of you want to get out of this partnership? How do you want to benefit veterans overall well being? How will you and your partner work together to contribute reciprocally to one another’s work and goals? 

Another key piece of this is that you’re going to likely have markedly different time horizons. The program office you’re dealing with may want to implement yesterday while you feel it’s your researcher or a QI evaluator that more time is needed for planning. So both of you are right. Now how you compromise to get both of your needs met while promoting veterans well being in a timely manner with evidence based work? Well sometimes that means building some like stability into your study design, perhaps ending a trial earlier than you planned because either deprecation or perhaps it’s really clear that it’s not working. 

Finally and this one is mainly relevant to research proposal draft a letter of support for your partners but do it early. They need time to digest and think through and edit it. It may also stimulate further discussions about relative priorities and leads further negotiation with study design and overall time horizons which I promise you don’t want to be doing a week before the grant is due because trying to short change a step will only be to those on either side, you and your partners being really frustrated.

So at this point you have survived and flourished while initiating and building a partnership, now how do you maintain it? Well here are some recommendations. Definitely do not forget to communicate regularly. As someone who has been in partnerships where I don’t hear from the other side for several months I find that I forget about them and I really appreciate it when they keep me apprised of progress in their projects. We are all really busy so this can be a difficult step to implement from a practical point of view, however I really want to emphasize that you want to try to do this. This is particularly important if you’re already on a funded research grant or you’re actively working on a QI project. I find it can be really helpful to actually schedule regular calls even if you cancel them every so often just to have that time booked out on your calendar where people think about it at the very least think about it coming up and you’re in their mind.

As I mentioned before you want to keep your study design flexible. If you didn’t take that into account when planning initially then definitely keep open to the idea of adjusting your study during -- the study design during the study. So as we’ve been made aware of recently we do not live and work in a static VA environment, policy and care practice is a change. As I mentioned we are observing that principle right now with an increase in care to address the wait list issues and we are having to address how veterans are receiving care. So again keep flexible; that’s key.

Also don’t hesitate to provide results to partners even if you haven’t hit your target sample size. Certainly in providing the results be clear that they are preliminary analysis but really here it’s really important and as a researcher I’ve fallen into this trap but do not let the perfect CD enemy of the good. You’re your partners and program offices will want to know if things look like they’re working, they will likely have specific questions they want answered and if you answer them even if they are not in the hypothesized or hoped for direction then that will make them very happy meaning plan early when with your partners.

Also within reason it may be wise to conduct unplanned analyses that your partners are interested in, particularly when unexpected results are emerging; however be careful to negotiate the partner. You want to make sure you’re still focused on the joint aims that you and your partners set out and don’t stray too much from those. Specifically it can be really easy to fall own through the rabbit hole in interesting analytic questions. Once you and your partner start talking and observing something perhaps in your study that was unexpected but the wise researcher and research operations partnership is one that keeps mindful of the aims and the original aims and stays on track. And another key point is to communicate with partners and program offices so that they can loop you in when their policy or operation changes coming down the road or that just occurs. You don’t want to find on several months later that the VA has completely changed direction and the questions asked by your study are no longer relevant. It is so much better to change steady course somewhere mid way than to find out you’ve been researching irrelevant set of questions or hypotheses for several months. And finally trust credibility and mutual respect are key. Your partners want you to be genuine with them as I am sure you want them to be with you as well. If your study results are contrary to what they were hoping, be explicit about it. Also don’t over comment; don’t say you can deliver unless you’re sure you can deliver. Be honest in your assessment of your resources for completing a project and in the generalized ability of those findings to other settings; so external validity.

Okay almost done before handing this over to Christian. I just want to share an example that Christian brought up as we were preparing this presentation; so Christian please feel free to chime in if I misinterpret this at all but I really like this example and thought it was a nice way to end this first section. This example really highlights some of the key communication and philosophical differences that can creep into their research operations partnerships. There are two brothers, one who is an education and one who is in business administration who wrote a book called *Made to Stick*which is about how to get ideas and messages heard. 

One of their ideas is that researchers or experts tend to move towards abstraction. So for example if you have some primary care physicians and leadership talking about how best to implement tele-health for example they may hone in and keep saying “Okay we need really to improve our leadership support and engagement; it’s really critical that we do that”. But what does that really even mean? This can often times means very little when communicating with someone who needs to implement it as it’s really just abstract words. In research these constructs or abstract terms are critical to communicating across researchers but in practice they have very little utility without concrete examples. So when communicating your ideas to your partners think about how to phrase it so it’s most meaningful. Also in this book they have examples of how Nordstrom excels at customer service, not just because they’re nice and thoughtful but because they do unexpected things. So one example is they’ll gift wrap someone’s purchase from Macy’s even though of course it wasn’t bought at Nordstrom’s. Another example is they accepted a return of tire chains from someone and gave them a full refund. Well Nordstrom doesn’t even sell tire chains so they were going above and beyond and doing the unexpected in terms of customer service. 

So how can we do this in VA care, how can we think outside the box to provide improved care? If we say that we need an increase in leadership support or engagement, let’s go beyond that as researchers and give our operations partners unexpected ways to do that. So can you challenge what leadership or support -- sorry, leadership support or engagement means? One example could be providing protected [Indiscernible 00:25:43] for physicians who work on a new QI project or research project and paying for .50 OSG for another provider to pick up the clinical work that this physician can no longer do because she or he is focused on implementation of a new intervention. 

And finally true, but useless, echoes what I was mentioning earlier and refers to trying to provide knowledge that an operations person can act on. Again labels are useful scientifically and generating generalizable knowledge but we need a way to make steps concrete so front line people can act on them. So don’t get stuck with your sort of esoteric ideas or the researchers think and communicate in practical and useful ways or think on both levels. Obviously you need to have your researcher have audit at some points in time, but at other times you need to be much more practical. 

Okay and now with that I will hand this over to Christian.

*Dr. Christian Helfrich:*  Great, thank you Amanda. Nice summary of the made to stick ideas. I believe we’ve got another poll question here. I’m going to talk about an example of a program of specialty care access network and want to find out how many of you, if any have been involved with scan-echo at tele-health or another specialty care electronic consults or e-consults. 

*Moderator:*  Thank you it looks like we’ve got a lot of responses coming in and they are still coming in so we will give people a little bit more time to get those. All right, it looks like it’s kind of leveled off. 

*Dr. Christian Helfrich:*  Yeah it looks like we’ve got a number of folks who are familiar with Scan-echo and e-consults and tele-health. Thank you.

So I wanted to talk about this program as an example of partnered research and typical in some respects and like a lot of the work we do it’s atypical in others, you know with a lot of these projects I think have this unique aspect. But this specialty care access network extension for community health outcome, you scan-echo is one of for initiatives that the office of specialty care services initiated in late 2011. It’s based on -- the reason why it’s called scan-echo it’s based on a very successful program outside of the VA called project echo developed by Dr. Aurora at University of New Mexico. And it was to address a problem that they saw in New Mexico where they had relatively high Hepatitis C rates and outlying clinics where the patients really did not have access to specialty care because of the geographic distances, small populations and you know again just travel distances between these outlying populations and the large, relatively large academic settings where they could get care. And the model -- I’ll talk about the model in a moment, but the basic idea was to provide for some knowledge transfer from the specialist to primary care providers who are located more proximally to the patients and could provide care. The idea was to use a cased base format to help build to that transfer; more of a problem solving. Something actually that I think the medical school in New Mexico is known for.

The initiative developed from a partnership between World Health and the office of specialty care services. The office of world health in the VA was very interested in this. The idea was to provide for some covered time for providers with the specialist and the primary care providers to participate and again, I’ll talk in just a second about the sessions. As part of the initiative there were two specialty care evaluation centers that were developed and they’ve been combined into a single center and then this I’m affiliated with. And one of the key duties area that was targeted early on for pilot program and this imitative started out as pilots in a number of sites. One of the key conditions was hepatitis C and this study that I’m sharing was actually led by Catherine Rongey and Steve Asch and affiliated with HIV/Hep C QUERI that Amanda is involved with. And again was sort of developed over time.

Just briefly the scan-echo model really comprises two pieces. It’s focus on a specific condition or issue so for example Hep C, dementia, diabetes is centered on a team of specialty care providers who represent all of the relevant specialty care knowledge bases that you want to bring to bear in developing a care plan for patient with that condition. So for example with Hep C maybe that would include an infectious disease specialist, a hepatologist such infuse specialist, mental health specialist and somebody from primary care, an experienced primary care provider. Often times a primary care nurse. And they meet with primary care providers, usually extended periods so many weeks or months having multiple live tele-health sessions. So not with a patient but with a specialist in the primary care providers communicating. And the sessions include basically two major components.

The most -- the essential one makes the program unique from typical tele-health is sessions where there are case presentations so primary care providers identify patients that they have in their panel with that condition. They send the specialty care team information about that patient, the specialty care team reviews the patient charts and information identifies any additional information for example labs that they may need. And then during the scan session they -- the primary care provider does a brief presentation of the case and then specialty care team provides recommendations and there’s some you know, there’s some dialogue, some back and forth and the primary care provider is presenting the case they learn about how to treat their own patient. But in addition their colleagues who are participating in this session also see how this case is being managed and the care plan that’s developed to make needs of that individual patient. And in that way we’ve been primary care providers who may have only a few patients with the condition. They learn how to treat perhaps dozens of patients and learn from the experiences of their colleagues. In addition some of what we’ve learned in the evaluation and some of the most successful centers too the most successful scan programs there’s also some tier learning among the primary care providers as they weigh in and share what they’ve learned in caring for patients with these conditions such as Hep C or dementia, diabetes, cardiology. And the sessions can also include presentations; so members of the specialty care team will present on specific topics such as maybe a new therapy that’s recently become available or changes to guideline therapy. 

The goal with Hepatitis C scan-echo program and really with the scan-echo program overall is create a network for specialist’s generalists colleagues; so to create this network between the specialty care providers and primary care providers. Hopefully with the result empowers the primary care provider to provide increasingly sophisticated care at their local site; so the colloquial term which many of us don’t particularly care for is to make the primary care providers specialists? But the idea is that your -- the program is basically providing them the knowledge and tools and confidence to provide a more sophisticated level of care and as a result also to better use the specialist time to focus on the most complex patients with that condition. It also is intended to empower the community based outpatient clinic or the VA and medical center clinic and it’s to help them provide that -- provide care to patients with that condition by creating some density of knowledge in that local clinic and hopefully where there’s some spill over from the participating provider to others. For example the community based outpatient clinic pharmacist or nurse, other members of the team. Ultimately the goal is to help coordinate care and I think it’s not a stretch to say hope to improve access, something that’s very timely as we’re in the midst of this access, wait list scandal.

Now Dr. Rongey’s team contacted us in -- us, being me, specialty care transformational initiative evaluation center in summer of 2012. They were working -- they were in the scan program and had some contact with the network 21 folks and were interested in doing an evaluation. They did their homework and learned that there was already an evaluation that had been ramped up by the office of specialty care evaluation; they reached out to us to find out what was already happening in terms of the national evaluation. And where there initial ideas about studying this program in VISN 21 where it might be able to fill gaps in what the national evaluation was doing. It reached out to a couple of program offices; perhaps the key one being the office of specialty care services which is overseeing the specialty care transformational initiatives. That’s headed by -- currently headed by Michael Duchas (ph) and one of the key heads of the specialty care transformational initiatives is Susan Kirsch (ph). Susan was instrumental in formulating and funding the specialty care evaluation centers. Dr. Rongey’s team also contacted the office of public health, that includes the clinical public health group which has programs on HIV and Hepatitis C and again, that is a key stakeholder in this.

So Dr. Rongey’s team gathered information about the different offices that are involved in the Hep C scan-echo program. They reached out to the program offices and again identified the evaluation center that was being organized, contacted us, quickly had an exchange of evaluation ideas, research ideas. They discussed some priorities around the QI project and a couple of the things that we identified early on, one was with the specialty evaluation centers that were by the office of specialty care services. It wasn’t necessarily going to be an evaluation -- we need to cover all the pilot programs which included many different clinical conditions, many different geographic sites across the country that are 15 in the pilot programs for scan-echo program. And it was going to be a limit t how much the evaluation to this and we’re going to be able to explore the specifics of the disease condition and the specifics of Hep C as a scan-echo condition and it was also in critical time since the specialty care evaluation centers has been extended. But at the time it was not clear to what extent the evaluation center was going to be able to do long term assessment of the sustainability of the initiatives and particularly in the period of time that you’re two years, maybe three years after the program went from a pilot program or funding from central office was subsidized by central office to limit transition to be supported wholly by the other networks, by the VISN’s or local facilities.  So those are things that we identified that Dr. Rongey and her team included in their proposal.

The eventually will receive funding through the VA quality research initiative funding program, its called service directed projects, SDP’s. These are large scale implementation oriented grants so grants that have a goal of contributing to implementation science and also to actually supporting implementation of evidence based clinical changes in the system. And their team has produced a number of interim results and produced results iteratively and I’d like to spend a few minutes here in mentioning these and also move back to what Amanda had talked about before with personal knowledge. The idea of producing findings that are true but useless. 

Two things to note here, one Dr. Rongey’s team produced interim results which then as Amanda said did not wait for the end of the study, did not wait until they had their final findings to feed back results to their operational partners, so the results back iteratively. Second the findings that they focus on and I’ll just mention that the top two here were very concrete and very actionable. These are not theoretical constructs. IT developed -- just getting the IT system up and running the tele-video system, making sure that the primary care providers of the outlying clinics have video connections on their computers and are in locations where they’ve got you know, privacy so they can discuss patient cases, that was a challenge. Dr. Rongey’s team found 18% of the scan-echo calls were actually focused on IT issues; so a lot of time being spent essentially trouble shooting the medium. 

Second as the program was transitioning out of the pilot phase it was concern that that was not a good time to decrease funds, that reducing the resources at that time could be critical and again, you know just hitting on the importance of communication with operational partners and others working on this. One of the things that was happening at the same time that Dr. Rongey was producing these findings the evaluation center was finding that across specialties there was very positive responses from primary care providers, especially at rural clinics which was one of the key groups that were being targeted by this initiative. Primary care providers were reporting that they were finding participating in the sessions, very engaging and morale boosting. In some cases really talking about that experience being a Seminole change in their clinical practice in terms of sort of sustaining them intellectually and you know, engaged at outlying clinics. So in that context we have a program that appears to be addressing some needs, especially for the clinics that were you know most -- the highest priority for being supported and in that context we were looking at having some reduction in funding and so they were able to provide some input on that in a very timely way.

This partnership has been successful in a number of ways and I think just touching on some of the indicators that have been successful. They’ve provided also some actionable feedback on communication between the primary care providers and the specialists around the initial consult in helping the office of specialty care and VISN 21 better measure the quality of the communication. Maybe one of the key indicators as it’s been a successful partnership is that Dr. Rongey now splits her time between the clinical public health service and research at the San Francisco VA. And I think that you know our hope is across Dr. Rongey’s work, some other work being done on scan-echo by other services, R & D researchers and at least by the evaluation center is that this work will help this program, the scan-echo program will actually be funded and supported by the field in a sustainable way. We’ve certainly seen some really positive indications with its effect in the early days.

So some key take always -- maybe the key one is there are many opportunities for implementation research and quality improvement partnerships certainly in the VA and I think increasingly in the community systems as well. Key aspects to the successful partnership and this really goes to much of what Amanda had talked about. Again find the right partners and stakeholders, learning about them early on and really understanding ownership of what are the clinical changes. Partnerships are long term; they should be nurtured like we like to say they should be nurtured in tree rings. You’ve got to somewhere, you’re going to start with something and that can hopefully -- you’re developing long term partnerships that are going to build on each other. And then finally with the researcher’s goals and knowledge as researchers we tend to move to abstraction, which isn’t bad. That’s what our goal is; we want to understand those underlying generalizable patterns but in doing so we move away from concrete findings which is what operations needs. They don’t need theoretical constructs; they need actionable findings that they can move on at the end of the day. I think scan-echo is a prime example of the partnered implementation research and quality improvement that represents the best of partnered research and I think is likely to be very common in the future.

And with that I think we would be happy to take some questions, comments, feedback.

*Moderator:*  Excellent, thank you so much to both of you. I know a lot of our attendees trying to stop at the top of the hour; I just want to mention to submit a question or comment for our presenter’s just use that Q & A block that’s in the lower left -- lower right hand corner of your screen. Simply type into the lowest box and press the speech bubble and we will get it answers. The first question we have -- this one came in when Amanda was speaking. Our IRB told us that QUERI implementation -- I’m sorry. That QI needs to be a projects that focuses on how the project improved your own institution and not to generate data for possible generalized ability; can you address that?

*Dr. Amanda Midboe:*  Yes, I did see that question and that is exactly right. When you do quality improvement and you ask your institution can I count this as QI versus research that is what they’re likely to say and I -- this is good feedback for me in terms of phrasing QI versus research but -- so I’m not going to give a perfect answer but yes in general QI is really just meant to improve functioning your own system. Does that mean it doesn’t have practical implications for other facilities? No it may. What they’re really driving at is with implementation research and research in general actually is you want to generate the generalizable knowledge. For example, the construct and the abstract QUERI’s that we were talking about. When you’re doing QI it’s much more practical. So you -- you often times you would do some QI work and it’s going to be small and you want to generalize within your facility and you’re less worried about controlled variables or randomized controlled trials, that’s what I meant when I was referring to generalized ability is often valued over internal validity. But that wording is misleading. But yes absolutely in general most facility R&D offices will tell you that is exactly what quality improvement our QI is is that it’s not intended to generate generalizable knowledge. But it may have findings that are generalizable to your facility and inadvertently maybe other facilities.

*Dr. Christian Helfrich:*  Yeah actually and I’d like to just add onto that that I think this is something that is research community we need to be more active in challenges if that’s the right word, but maybe the -- more expecting about this. I think the essence of what Amanda said is that you know for research has the primary purpose of producing generalizable knowledge and in fact may not produce knowledge in the traditional research, may not produce knowledge that is immediately actionable at the -- you know in the settings where the research is being done and I think that is an important distinction that QI should have for it’s primary purpose to improve the quality of care outcomes of the system in which the work is being done. But I think it’s -- it really just doesn’t pass the test to suggest that work then cannot be generalized to you know, or should be in fact -- QI we should be looking in the system like the VA we should absolutely be looking to generalize across clinic settings. It’s insane to be doing work in one clinic or in one network and not try to understand the implications of that work for another part of it at work and another part of the system. One of the things that I think has been lost in that R&D and IRB discussion is we end up focusing on the definition of research where his quality improvement and we move our focus away from human subject’s protections, which I think is -- that’s where our attention should be focused. And it’s certainly possible for quality improvement program or implementation research to expose our providers other employees, patients to [Indiscernible 00:51:11] that they otherwise would not be exposed to. That’s where our attention should be focused. Is what’s the potential risk or harm that’s introduced by the activity and that’s something that you know should apply to quality improvement and research so to speak? And I think it’s something we’re not going to solve it on this call and I’m sure this is something that’s going to take you know, years to evolve. But if -- definitely a discussion we as a research community need to be actively promoting, that’s my two cents.

*Moderator:*  Thank you to both of your responses. Actually we got some two cents from quite a few of our audience members as well; so I’d like to read their comments aloud. The first one is that Greg Orbrink (ph) has an article out in IRB Ethics and Human Research, the title is *An Instrument to Differentiate Between Clinical Research and Quality Improvement* and that is -- one second; and it does provide a more specific direction. Also another submitter wrote there is a helpful handbook on what is research versus QI, a VHA handbook 1058.05. I believe you two are familiar with that one. And yes, somebody just wrote in I had to miss some of these talks -- I’m hoping the auditory transcript will be available soon. Yes, we do transcripts, we do audio of the transcripts usually come up within two weeks of our archived catalog. Please be patient for those.

Okay we’ve got a whole long list of questions so we will get right to them. What are opportunities for non VA scientists to partner with VA network? What “other networks” would VA engage with?

*Dr. Christian Helfrich:*  That’s a great question. And the -- probably the easiest way for non VA researchers to engage in with the VA is through one of the health services R & D or health services research and one of the health services R & D centers, centers of innovation. If you go to VA.gov I think it’s -- if you go to VA.gov and search health services R & D you’ll come up with that page. And I’m embarrassed to say there used to be 14 centers of excellence and I can’t remember are there 18 centers of innovation now? 

*Moderator:*  I believe 19, but don’t quote me.

*Dr. Christian Helfrich:*  They’re around the country; those would be the places to contact instead of the investigators at their centers. The VA has an internal -- a couple different internal grant funding mechanisms and non VA researchers can be funded through those but generally with a few exceptions generally not as principle investigators. We need to collaborate with a VA investigator to put in a grant that there are opportunities there.

*Dr. Amanda Midboe:*  I mean I would also encourage folks to read -- to focus on implementation research. I would not -- I would recommend they go ahead and contact relevant QUERI centers if they’re interested in implementation research and it’s a big domain such as HIV or Hepatitis C. Just because it can be difficult sometimes making sense of the complex network that is the VA that they can find local points, sometimes QUERI’s can kind of facilitate that and I -- I don’t feel overrun by people outside the VA; so maybe I’ll take that back at some point. 

*Dr. Christian Helfrich:*  That’s an excellent suggestion, the quality enhancement research initiative or QUERI centers would also be an excellent resource for implementation oriented work. 

*Moderator:*  Thank you for throwing that out because I continue to say we’ve never once filled that out that QUERI stands for quality enhancement research initiative; thank you for sharing that. 

*Dr. Amanda Midboe:*  Helpful for the non VA folks on the cal.

*Moderator:*  One of the last side notes for quality communication how does one measure affective communication within the health care landscape and feel free to reverse back in the slides if you like.

*Dr. Christian Helfrich:*  Yeah I will go to that and I am also going to punt and say that’s something that I would have to defer to -- or refer the viewers to Dr. Rongey. I was going to try to find where that slide go -- there we go. Actually I don’t know how the -- what quality metrics end up coming up with provider communication, just that that was one of the products that developed.

*Dr. Amanda Midboe:*  They’re still refining it; so this is -- they’ve been in their STD project for about a year and this is still a work in progress. So there are key entices, hard and fast for provider communication quite yet but I’m sure he’d be happy to share them with you if you contact her directly. 

*Moderator:*  Thank you for that reply. The next question we have -- give me just one second please. All right. Getting through a whole lot of them here. This one when conducting a double blind RCT or efficacy do you break blinding and peek at the data to show if clinical partners early on; we have been told that we shouldn’t peek with a study design for scan-echo and a blinded RCT.

*Dr. Christian Helfrich:*  Now this is an important distinction between we’re really talking about quality improvement research or implementation research which is picking up where there is -- at least the reasonable expectation of efficacy essentially. So with the scan echo program you know there was some -- had demonstrated it had been efficacious in New Mexico for Hep C and it’s also a problematic change rather than you know, clinical procedure or medication and that sort of thing. And so this was not a randomized control trial and you know I think that would be a very different situation with double blinded RCT.

*Moderator:*  Thank you for that reply. The next question -- let’s see. Okay this one is very closely related to implementation research. On the last slide is that half dome, I said no it’s probably rainier in Washington but I’m curious too. 

*Dr. Amanda Midboe:*  I actually replied on it yes it is half dome which is where I was early last week.

*Moderator:*  Excellent. Great well that proves someone’s surprise because I went off the fact that Christian is based in Washington. 

*Dr. Amanda Midboe:*  I am near Yosemite though, that is why.

*Moderator:*  All right well that is the final question that comes -- came in so I’m going to put the feedback for our attendees. Please do take a moment to fill this out. I will leave it up well past the top of the hour so take your time. It is your opinions that help guide which sessions we have presented for us and which topics you’d like to hear more about. And if you two have time feel free to make any concluding comments you’d like to now.

*Dr. Christian Helfrich:*  Yeah well Molly thanks so much for organizing the session and the opportunity to present and everyone for attending and you know I think I can speak for Amanda in saying that folks have additional comments, questions please feel free to follow up.

*Dr. Amanda Midboe:*  Yeah absolutely; we try to target this to be practical and be for either new or more experienced folks who kind of could learn a few more things or be reminded of a few things but if we’ve missed anything or if you have follow up questions I would be happy to follow up with you.

*Moderator:*  Wonderful and I’d like to thank you two for lending your expertise to the field and also to our participants for joining us. Please do check your -- check our registration catalog for future sessions you may be interested in and we’d love to see you at the next one; so as I mentioned I will be leaving this feedback survey up for quite some time so feel free to take your time filling it out, but we are the top of the hour so this does officially conclude HSR&D cyber seminar. Thank you.

*Dr. Amanda Midboe:*  Thank you.

**Page 1 of 15**