**Transcript of Cyberseminar**

**HERC Cost Effectiveness Analysis Seminar**

**Budget Impact Analysis**

**Presenter: Neil Jordan, PhD**

**July 2, 2014**

*This is an unedited transcript of this session. As such, it may contain omissions or errors due to sound quality or misinterpretation. For clarification or verification of any points in the transcript, please refer to the audio version posted at www.hsrd.research.va.gov/cyberseminars/catalog-archive.cfm or contact herc@va.gov.*

*Todd Wagner:*I just wanted to introduce Neil Jordan who is going to be talking today on “Budget Impact Analysis”. I told Neil that I pulled a little information of him off the internet to introduce him. He is actually an Irish filmmaker, fiction writer, what you may not know he won an Academy Award for the *Crying Game*so he is highly prolific. And in between films, he is a Professor at Northwestern and a collaborative with the VA so I am just thrilled that he gets in scientific writing as well as all this fiction. His work spans several areas including quality and costs for persons with chronic illness; financing and outcomes mental health and child welfare services and health economics. It is a great pleasure to introduce Neil Jordan to you. So Neil I hope all that information on the internet was right. 

*Neil Jordan:*Todd, thank you very much, that is the best introduction I have ever gotten and clearly the funniest. I want to thank everyone for joining us this afternoon or this morning depending upon where you are sitting. Okay we are going to talk about budget impact analysis with a particular focus on, let me go to the next slide and tell you where we are going to go. 

Three objectives here. Number one, I want to talk about the purpose of a Budget Impact Analysis, why do we use it and what cases do we use it. Talk a little bit about some of the methods that we use within this broader method called Budget Impact Analysis as well as the types of data that we need to collect and where we might find them with a particular emphasis on how we might find data to conduct Budget Impact Analysis within VA research. Then finally at the end, we will spend just a little bit of time and I will offer up some additional resources if you like to learn more about the things that we cover today. 

Before we get into the content, I want to get a better sense for who is in the audience and so the first question I want to ask is – what is your primary role where you work. Are you a clinician; are you in operations; are you a researcher or are you in some other setting. Let us go ahead and vote. Folks who have done this polling before know that there is no science to when we stop, we kind of wait for things to trickle down, it looks like they have trickled to the end. Let me just kind of read back. The majority of folks, about sixty percent are researchers; it looks like about one-sixth are in operations; about one-sixth are doing some other kind of work and then about ten percent are clinicians. 

Great, the second question I have, again I want to get a sense for exposure to the material. Have you ever worked on a study that included a Budget Impact Analysis and the answer possibilities are: yes; no, do not know or have not worked on a research project. So take a moment and respond to that poll if you would please. Okay and I will stop the poll right there and again I think this is very good. About a third of you have worked on a study that included a Budget Impact Analysis and about sixty percent have not so that is good. It sounds like we have some folks in the audience who have some knowledge of what we are going to cover and then for many other folks this will be much newer. Let us get into it then. 

Let me start with the punch line here which is – what is the purpose of doing Budget Impact Analysis? What does it tell us? I think the main takeaway point here is that this is an economic method that we use when we want to analyze the expenditures associated with a program. We do it over a short period of time usually one to three years and we are really interested in not only what it costs to roll out that program or implement that program but we are also interested in considering the effect of any offsetting savings associated with the implementation of the program that we are assessing. Some other things to keep in mind and we will touch a lot more on these points as we proceed we are evaluating a scenario rather than a single action. There is a comparison here that we are going to do and this will be familiar to folks who have participated in some of the other cyberseminars in this series around economic evaluation. Then finally, Budget Impact Analysis like most other methods also includes sensitivity analysis. 

A principle purpose of doing Budget Impact Analysis is that we want to, it enables us to estimate the feasibility or affordability of implementing some sort of intervention. That is going to be useful for a few reasons – number one, for organizations that may want to implement something new, knowing what it costs and whether it is feasible is going to influence budgeting and forecasting. Similarly and I think increasingly importantly for those who do research in the VA partners more and more are asking what will it cost to implement this new intervention. What will it cost now? What will it cost next year? What might it cost much further down the road? Budget Impact Analysis is a method that enables us to start to answer those kinds of questions. 

Now the perspective of a Budget Impact Analysis, over the next few slides we will start to draw some distinctions here between some of the key tenants of Budget Impact Analysis versus the key tenants of Cost Effectiveness Analysis. For starters, Budget Impact Analysis typically takes the perspective of the buyer or the provider or the payer or the organization who is going to be implementing the intervention for which we are interested in evaluating the Budget Impact Analysis. This of course is different than in a Cost Effectiveness Analysis where often the perspective of the analysis is something broader. Often we adopt a societal perspective so there is on key distinction. 

Another important distinction is the time horizon for which we conduct this analysis. Budget Impact Analysis with this method we typically focus on a much shorter time horizon than we would in Cost Effectiveness Analysis. We are typically limiting that time horizon to a few years at most, one, two or three years is what we most typically see. What this means is that there is no need to do long term modeling like we might do in Cost Effectiveness Analysis. We do not discount the costs in a Budget Impact Analysis equation and we are not focused on the longer-term savings that may be associated with implementing an intervention. so again, in Cost Effectiveness Analysis in particularly when we are analyzing the cost effectiveness of say a prevention intervention, where the benefits may not accrue until many, many years later, in a Budget Impact Analysis we are not really considering those benefits of savings in the far off future. We are much more focused on the costs and consequences in a much shorter term. 

Then I think the final point to make in terms of distinguishing Budget Impact Analysis from Cost Effectiveness Analysis is that we are really not measuring effectiveness and in particular, we are not measuring utility. Which again for folks who have participated in some of the previous seminars in the series, you know that utility is that really great measure that we get where individuals are providing a personal reflection on their relative value or preference of different health states again a very common outcome that we use in economic evaluation. In Budget Impact Analysis, we are not measuring utility so consequently we do not to survey patients, we do not need to calculate quality-adjusted life-years. When we do Budget Impact Analysis, we already assume that we know the outcomes and more often than not, we know the outcomes are favorable because again the question we are trying to answer is – given that this intervention is effective what will it cost to implement and roll out and administer over a short period of time. 

Let us start to shift gears a little bit and start to talk about how we do Budget Impact Analysis. I think we want to start by noting that there are several costs that we want to estimate. For starters, we want to estimate the actual cost associated with the intervention. We want to understand the changes in staffing, changes in schedules, what sort of technology is going to be employed because there will be costs associated with all of those elements. We will also want to be mindful of changes that effect patients and in particular would the intervention lead to changes in access for patients or demand for services or how patients actually work through the system and get services. We are interested in understanding the costs of operating whatever environment is where the intervention has been implemented. And as I mentioned at the beginning we are also trying to be mindful of where there may be potential savings associated with this new intervention when conducting Budget Impact Analysis. 

I want to now show you a framework, kind of a schematic diagram that does a nice job of characterizing all of the different elements that could impact the costs associated with an intervention. Here is the way to think about it, I realize that the graph has one of these cool effects where it rolls out things gradually so let me try to match my talking with the graphics here. What you can see on the left side the current environment, these are the aspects of the population, resources and utilization, sort of what exists in the environment, what is the existing intervention. Again you might want to think about this in a more classic way which is whenever we are trying to evaluate the effectiveness of a new intervention, we always do it with regard to an existing intervention, right care as usual; placebo. So when we do Budget Impact Analysis we are again starting from the same frame except what we are really interested here is on understanding the different components that can drive the costs of the current environment. Again, these have to do with the incidence or prevalence of a particular condition that the intervention is tied to; the proportion of patients that are diagnosed and treated; the way we treat and of course the costs associated with all of those things. We start from that basic framework and basic understanding and from there we start to think about what sort of impacts will the new intervention have. Again, the new intervention could have impact on the incidence of a particular condition and particularly when we are looking at the budget impact of a preventive intervention. Similarly, the implementation of a new intervention could impact diagnosis and treatment, the complexity of services that patients receive. Again, there could be some new therapy or procedure that is a consequence of _____ [00:12:18] [audio skipped] that is being tested. The point from all of this is that all of the potential impacts of the new intervention could change lots of the inputs that are ultimately going to drive us to understanding the cost of illness and the cost of the associated intervention. 

Let me just give you an example of how to pick apart some of the pieces of this visual. You could imagine that if we were doing a Budget Impact Analysis of an intervention that expanded HIV testing, that could impact the proportion of patients that were diagnosed with HIV and treated. Let us contrast that with a Budget Impact Analysis for a new telephone based psychotherapy intervention for veterans with anxiety. There you could imagine that the focus would more be on the kinds of services, the utilization and again the costs associated with those. 

Now, the reason that this framework can be very helpful is that when we conduct Budget Impact Analysis we are doing it very locally. That is to say that the result that we get from doing a specific Budget Impact Analysis will not apply to all decision makers or settings. In addition to coming up with a Budget Impact Analysis estimate, a total cost figure if you will, the other common product of doing a Budget Impact Analysis is to develop a model. And we often do this by creating an excel spreadsheet that other payers or organizations can use to estimate the budget impact of implementing the intervention in their own setting. You could imagine having a spreadsheet where it is your particular VA, you could input your own values, and you could input the values for your population and for the resources and the costs and come up with an estimate specific to your own organization. 

I touched on many of these points just by going through the diagram but just to elaborate on a few more items from that slide. Again having a reference point the current environment is a very important starting point for conducting Budget Impact Analysis and we want to pay special attention to the number of patients that are getting care because often implementing a new intervention is going to change that number. Sometimes it means that more patients will be treated and that is going to have a direct impact on our estimate of budget impact. Who gets care is really really important and in particular understanding the clinical characteristics, secondarily sometimes understanding the VA priority category or the VERA category that may have implications for our result. Then understanding the size of the target population. 

We also want to know what treatments are the current population getting and how will that change by the implementation of the new intervention that we are conducting the Budget Impact Analysis for. Then what other healthcare resources might that current population be getting. 

Let us talk a little bit more now about the intervention itself because again I think you heard me say enough times now in the first fifteen minutes that Budget Impact Analysis is really about understanding the budget impact to a payer on the intervention. What is the intervention? Of course, it can be many different things, but the kinds of things that we want to know about is number one in what settings is the intervention being carried out. How often is it provided? Who is providing the care? What sorts of staff are actually involved in delivering the intervention? Then what do the providers do when they are actually providing the care? 

Once we know all of those things, we want to start to think about how will all of those elements change relative to the reference scenario how will these different inputs change? In particular we are really interested in understanding whether there will be some impact on the demand for care that is to say the number of patients that are seeking care because oftentimes a new intervention draws new patients into the system. We know again, the in the VA that many patients are dual users and seek care outside the VA system but when new interventions are brought in sometimes it attracts new patients. Sort of conversely, will new patients become eligible for contract care or home care or other kinds of services outside the VA? Again, from a budget impact standpoint it is really important to know about those kinds of patients. Then also from the perspective of being able to understand the total number of patients seeking care, will the intervention be associated with a change in the incidence or prevalence of the condition or conditions that can be treated or affected by the intervention. We also want to know relative to the reference scenario will there be changes in the future need for care? Will it have an impact on the co-payments collected or the VERA payment that is received?

Digging down even a little bit deeper, we really want to start to pay attention to some of the particulars of these inputs. How will the staff providing services change? Sometimes there are interventions that lead to a change in the next resources used by physicians, nurse practitioners and registered nurses and of course, with those kinds of changes there will be changes in the costs of providing care. Relatedly, if the new intervention associated with a change in space or other kinds of costs that we think about as overhead. So if there is a need for an expansion of the clinical space or a change of the clinical space, will the new intervention compare to sort of the reference intervention require less space, which happens sometimes. Especially in the context of the kinds of telehealth, interventions and some of the other more mobile web based handheld device based interventions that have been implemented in the VA in recent years. If new space is required will it have to be  rented, purchased or built understanding that each of those is associated with different costs. Is there technology associated with the intervention and again will that technology be purchased. What is the cost of developing it? What is the cost of repairing or maintaining it? These are all elements that will be important to understand. Again, particularly in comparison to the reference scenario that is sort of the current state of care, when conducting Budget Impact Analysis. 

Now, let us come back to this issue of perspective, which we talked about briefly at the beginning. The perspective here as we said is the perspective of the payer, the implementer, the organization. In the VA, setting what we mean is we are interested in the VA’s costs, we would not include patient costs in a Budget Impact Analysis. Again, unlike a Cost Effectiveness Analysis where we often are interested in the impact of an intervention on earnings, whether there are transportations costs associated with engaging in the intervention or time costs, we would not count those costs in a Budget Impact Analysis. Then we would not be calculating or accounting for societal costs, so the costs incurred by other payers, by employers, by caregivers so we are really focused on the cost of the payer of the implementer. Again, in the VA, which is a closed system the payer and implementer are really one and the same thing. What we want to be able to do here is to estimate the change in units per type of care and then to estimate the change in the cost per unit of care. 

Let us gear a little bit now and talk about some of the data sources that we might use to conduct Budget Impact Analysis within the VA. Again I want to sort of start by thinking about some of the different components and costs that we want to measure. We talked before about the potential impact of an intervention on healthcare encounters experienced by patients. In order to start to understand the costs of those encounters, one place to go would be to look at the national data extracts within a decision support system. So again, you could imagine going into the inpatient files particularly if the intervention that you were conducting the Budget Impact Analysis for would have some impact on hospitalization use.  So we could use those files to look at discharge records and to look at bed section records. Similarly, if the intervention was expected to have an impact on outpatient care, we could go into the outpatient NDE files and look at again the encounter files and look at the pharmacy records. 

From the standpoint of coming up with the costs associated with those encounters, there are other places that we can go. If we were interested in looking at the costs associated with a hospitalization, we could go to the HERC Average Cost Data and we could actually link those data with the patient treatment file discharge files. Then similarly, if we were interested in drawing distinctions between medical/surgical discharges and non-medical/surgical discharges, we would similarly be able to do that by linking the HERC Average Cost Data to the patient treatment file/bed section file. In order to come up with the actual dollar costs of the outpatient encounters we could link the HERC Average Cost Data with the outpatient clinic data. Now for pharmacy that is going to be a little bit trickier. We actually there need to use the DSS or the PDM pharmacy data in order to come up with the cost associated with pharmaceutical use unless the care is delivered in the clinic. 

A little bit more about the HERC Average Cost Data. Again, some folks may have some experience with this, the HERC Average Cost Data and the DSS data are different in the sense that the HERC Average Cost Data uses the Medicare Relative Value units not the Relative Value units that you find in the DSS data. What we see here is less granularity so I guess the consequences that you will see more similarity in costs across encounters. Again, there is much more you could say about this, but I would really guide you, HERC has some terrific publications that provide a lot more details about this. Again, if you go to the HERC internet website, click on publications, click on technical reports you can learn much more about this. 

Now, let us talk about actually moving from the costs of the inpatient, outpatient and pharmaceutical encounters, let us move to costing the intervention itself. There are sort of three broad methods for doing this. One is to do what we call direct measurement where we observe and track what happens and assign a cost to each of the activities that occur. The second method for costing out an intervention is to use what we call pseudo-billing, which is to use billing codes that are associated with services and then assign costs based on the services. Then the third method is to use what we would call cost regression and this is a technique that we can use to estimate the marginal costs of a new intervention relative to an existing intervention what the costs of care are known. Let us say a little bit more about each of these. 

For Direct Measurement what you can imagine is we want to identify all the elements of the intervention. You could imagine this being a very labor-intensive activity. It would require one or more project staff, actually observing all of the activities that happen, tracking them in a log with a particular emphasis on trying to capture the amount of time spent, carrying out different parts of the intervention, capturing the materials that are used, the supplies that are used. Then what we would do with that information is use the labor costs to actually and multiply those by the amount of time each provider type spent carrying out the intervention. If we multiply all the time spent for each particular role, by the labor costs associated with that role, and add it all up, that will give us a sense for the cost of delivering the intervention. Again, pun intended labor-intensive task but really a very good way of understating the true costs of carrying out, implementing an intervention. 

The second technique Pseudo-billing so here what we would do is we would actually again based on the kinds of services that a patient would receive because of an intervention, we would use CPT codes, those are the outpatient procedure codes or other utilization codes to estimate the cost associated with the intervention. We can do this, we cannot always do this, it really works best when the CPT coding is clear and fits well with the kinds of procedures associated with the intervention. Of course we want it to have good faith validity, we want to be sure that the CPT four codes that we want to use are consistently used in this fashion to identify service use. Then we would combine that service utilization data with Medicare payment data or other payment schedule data. Again, this is really about multiplying years of service by some cost of the service and that would give us again a sense for the cost of the intervention.

The final technique, Micro-costing the Intervention again this is really one that we can only use in a subset of cases and we have to have a lot of data in place actually in order to do this. We need to have a good amount of data about the costs of the current practice because we are going to use statistical techniques basically a regression model to estimate the marginal costs and the marginal effect in intervention. And we really cannot use this if we are trying to estimate the budget impact of a new technology or if we do not really have good data on the accounting costs. Again, this sort of nuts and bolts costs of carrying out an existing intervention. 

Now a little more, we talked a few slides ago about measuring the staff activities associated with implementing an intervention so just to say a little bit more about where could we get the cost data that we would need to multiply by the staff hours. There are a few places where you can find average hourly staff cost data for more than seventy occupation categories. One place you can find this information is in the DSS Account Level Budget or Cost Center and then the VA’s financial management system also has this information. The other place to go is there is HERC technical report #12 supplement, and again our colleagues at HERC have actually figured out these hourly staff costs for many, many occupation categories. The data are a little bit dated but again, I think this could be kind of a faster way to get into the hourly staff cost data that you would need for conducting Budget Impact Analysis. 

With regard to supplies and machine, how might we come up with the costs of the supplies and machines and technology and other equipment that are part of intervention for which we are conducting a Budget Impact Analysis. One example is and I cite this example because it is something that is managed where I am at the Hines VA and that is the National Prosthetics Patient Database. So you can imagine if you were doing a Budget Impact Analysis that involved prosthetic equipment, you could go to this database, which captures the purchase price of all of the items that are ordered, basically through this group. Again, it has all kinds of stuff, not just prosthetics or sensory aids. Again, these data are stored and handled by the MPPD Data Manger who is located at Hines. The other place you can go and in particular, for the costs of other supplies and equipment that are not prosthetic you could get that information from your local A&MMS  purchasing officer. 

We talked about identifying the impacts of the intervention, we talked about how estimate the costs associated with each element so at this point we would be able to actually calculate a total cost or total budget impact value. We are not quite done yet though because like with many other analyses we need to conduct sensitivity analyses. Like for many other analysis types the reason here is we want to test the robustness of what we have done. We want to determine how sensitive is the total cost we have estimated to changing some of the assumptions in our model. Again, this is really important because as you can imagine if you think back to that very complicated slide that laid out all of the different inputs we are not always going to be able to measure each input precisely. So sensitivity analysis gives us the opportunity to see how our estimate will change based on changing some of the input values in our model. Now there are a few ways to do it so we could do what is thought of as univariate sensitivity analysis where we take an input and change each input one at a time. It is easy to do, it takes a while, it may not really get us there, in this day and age it is really not very state of the art but it is certainly something you can do. What folks are more commonly doing now is approaching sensitivity analysis in a more multivariate way by using software to actually change multiple assumptions at once basically the input values for multiple inputs. It has much higher credibility, the software that is out there actually allows you to generate very useful graphs, tornado diagrams are one of the more popular kinds of graphs where you can actually show how sensitive your results are to changing some of the inputs. 

Let me summarize because we are actually getting fairly close to the end now. Budget Impact Analysis provides really good information about how we might expect changes in expenditures or the level of changes in expenditures for a healthcare provider or a system after we implement a new intervention. The reason that this technique is very useful is that increasingly the VA and other payers want to understand what it will cost to implement a new intervention. Budget Impact Analysis is still a relatively young methodology but the guidelines for conducting BIA are well developed and continue to get better. In a moment I will point you to several resources and there is one in particular a really good one that is has just been updated earlier this year that is a great source for understanding and thinking about how to carry out a Budget Impact Analysis. Then as we have seen the DA is really a great place to do Budget Impact Analysis because as a closed system and as a system with really good data there are a lot of useful and relevant data sources within the VA for conducting Budget Impact Analysis. 

In terms of resources, I want to point you again to the HERC website, really a first and best place to go. The internet site has guidebooks and technical reports that also address some of the issues that we talked about today and I think, have not looked at the FAQ’s lately but I think there are some FAQ responses to questions about Budget Impact Analysis as well. The VIREC  website of course is always a great resource particularly for looking at information about health services utilization within the VA and how to understand what is available to us when conducting these kinds of analyses. I would in particular point you to the Research User  Guides and some of the technical reports that are available. Some may wonder, where could we read some papers that are write-ups of Budget Impact Analysis? There are three journals that I can think of where you might find many Budget Impact Analysis published in particular medical decision-making, health economics. Value in health I think even more than the other two is a destination where there have been lots of Budget Impact Analysis published and it is also the journal whereas methods developments unfold you are most likely to find those developments there in the journal called *Value in Health*

Let me just go back a second, one more comment about journals, I would be remiss if I did not say that because many Budget Impact Analysis are conducted on clinical interventions, you will also see Budget Impact Analysis papers published in relevant political journals. It is not to say that these papers are only published in economic journals and clearly, if we want to reach the audiences that most care about these results, I think increasingly we are going to want to see Budget Impact Analysis published in clinical journals. 

The really newest and best and greatest paper out there if you only were going to look at one paper that was written by Sean Sullivan, Joe, Mauskop and others again came out of the journal *Value in Health* earlier this year. Sean and Joe lead a task force that has really taken on the development and dissemination of Budget Impact Analysis methods. They call it the *Principles of Good Practice* and this is a really, really good tool that talks about Budget Impact Analysis methods. There is also although it is getting a little bit dated now, there is a very nice paper from the *Journal of Internal Medicine* about seven years back that was funded by the VA and it is kind of an earlier literature review on Budget Impact Analysis. I want to call that to your attention. It is also worth noting that some colleagues and I are working on a white paper now that talks more specifically about Budget Impact Analysis methods in the VA, something that is going to be a little bit more VA specific. And that is a report that will be coming out later this year and we look forward to sharing that with interested folks so that will be coming down the pike as well. I want to also point out that in terms of more information about cost data sources we talked just a little bit about overhead costs and again I just want to point you to a few very good resources. There is a HERC technical report that talks about how to calculate the indirect costs of specialized VA mental health treatments. Again maybe help believing if the intervention that you want to do a Budget Impact Analysis for is not mental health related and then similarly there is a real nice journal article that goes back a ways by Bob Rosenheck and colleagues about how to estimate the cost of innovative mental health programs. Again, it focuses on estimating some of the indirect costs that you might also want to include in a Budget Impact Analysis. 

We have reached the end I really thank you for your interest, for joining us today and I think we have lots and lots of time for questions so I would really welcome the chance to answer questions. 

*Todd Wagner:*The first question that has come in is more of an admin question for Heidi is how do we obtain a copy of this presentation?

*Moderator:*I will put the link up for everyone right now. 

*Todd Wagner:*So just to guide people as they ask questions, there is going to be a Q&A screen, there is the Q&A screen and you can type in your questions and I can read them to Neil or he can see them and answer them. 

Maybe I will start Neil because I was actually working on and presenting a Budget Impact Analysis yesterday. One of the things I struggled with and I do not know if you have any advice on this is that I often find the results are somewhat less than satisfying. So the example that we had yesterday was a situation in which we are using health coaches to encourage people to get disease management for their care. These are people with hypertension and diabetes. So we can measure and we use direct measurement so we knew the exact precise amount of cost, there is an actually cost of measuring it, the trial is about four hundred and forty people and we did not see any significant change in downstream costs at one year. There is a fair amount of uncertainty around those estimates and I think that is where everybody was that is not what we hoped to see but it is the case that you are left with. We know it has a known cost and an unknown effect or at least a smaller than anticipate effect. Have you run across that and how do you deal with that or is that just life?

*Neil Jordan:*Yeah I think Todd that is a common problem and two thoughts come to mind. Number one is this health coach is intervention, what is the reference case? What were health coaches an improvement on that was in existence beforehand?

*Todd Wagner:*That is a great question so it is a randomized trial of just usual care and this is in a county health system. Depending on how you see county, relatively unmanaged and sort of high-risk population compared to these trained outreach workers that are trying to get people connected and stay connected to their services. In many cases, it is medication management trying to make sure that they are getting their refills as needed and so forth. 

*Neil Jordan:*So, again, I think one interesting thing to look at and perhaps you did is is to compare. What was the additional level of resources needed to improve medication management. Again, maybe that is sort of more broadly, what are the costs of having these health coaches in place compared to usual care where it is as you say it is wild, wild west unmanaged patients are on their own. The question is  I imagine you did or started to do some sensitivity analysis and can you point to particular elements where you really feel like the uncertainty is greatest and can you talk about, back to your question, sometimes when I have done this we talked about ranges. Formally, confidence intervals less formally sort of a range of costs based on different input levels for the things you are most worried about uncertainty. 

*Todd Wagner:*Great questions, sort of what we ended up with and I think this generates some questions that people typed in on is one was in the near term did it change the patterns of appropriate and inappropriate care. In your talk, you talked about linking to different types of services whether it is pharmacy, medical/surgical so you could imagine a situation where with diabetes and hypertension where you are hoping if you manage these things, they are going to have fewer inpatient stays. That would a good shift even if you do not see an immediate savings from it, we checked for that. We checked for changes in emergency department utilization and in neither, we saw a commensurate change between the two randomized groups. We know that we have also tracked hemoglobin A1C in hypertension we see small and significant benefits there so we ended up on this fall back plan which is to say well if we had the money we could do a full blown cost effectiveness analysis to say at what point would this recoup savings. Again, everybody sort of recognized we did not have the extra two years of funding to do that. I think that is sort of where we end up a lot of these interventions are a little bit more expensive and if it is a small gain in hypertension control or small gain in diabetes control what does that real mean? 

*Neil Jordan:*Right. I think Todd I am glad you said that because I think it points to one of the limitations of this particular kind of analysis which is it tells us what something will cost in the short term. but it is not always going to give us an answer that is going to convince folks that the intervention is ultimately worth adopting.

*Todd Wagner:*I think that is exactly where we ended up. One of the questions – did we do risk adjustment in the analysis? We are sort of benefitted because of the randomized trial and so as we look for balance across the two arms as one would we found incredible balance. We did not find strong differences when we did risk adjustments, we also looked at, you mentioned sensitivity analysis, we had their baseline spending in their prior year so we said maybe it was most effective with the highest cost patients. We see just a glimmer, it was a small sample there, a glimmer of hope but nothing significant. 

There was another question that says – it seems we should also include indirect avoided costs in the new intervention assuming outcomes measures bear this out. For example if new self-management trading helps to limit need for Clinic X then a voided cost  may be fewer FTE’s needed but also the space where they did not have to occupy. That really gets to the question of if you were to do services a distinguish between short-term costs and long tem costs, how does Budget Impact Analysis handle that? 

*Neil Jordan:*Again I think this is one of those things where the time horizon really matters. A lot of times when we conduct Budget Impact Analysis and I think I will be preaching to the proverbial choir for folks who collaborate with operations folks you know that one of the questions we always want to ask when we are evaluating something is what time horizon matters to you. I think this is sort of an example where if you only look one year out versus looking two or three years out, you may get a very different answers. I think that any time horizon is reasonable as long as it is relatively short and as long as it matches the budget impact that is of interest to the payer or the healthcare leader. 

*Todd Wagner:*I guess it relates a bigger question I have for Neil and I do not know the answer to this, but at some point let us say you reduce utilization a little for a treatment group, are healthcare systems like the perfect gas and they are just going to fill those beds no matter what. In some senses is the savings elusory? 

*Neil Jordan:*That is a very thought provoking question Todd. For me what it sort of gets at is thinking about how the VA system is structured and we could really think about almost any other closed system the same way. You are right if an intervention winds up reducing the need for particular kinds of service utilization, perhaps requires less space, then the question becomes what is the next best use of those resources. Historically the VA has not been at overcapacity, if anything it has been a under capacity so I think it is reasonable to believe that as cost effective or what we might think of as budget worthy interventions are implemented, that is going to create the opportunity to provide new, more additional services and to fill the space. Again, whether that is truly the most efficient thing that should occur is I think a question to be debated. I think it is a really good point Todd. 

*Todd Wagner:*There are two people that have follow up questions. One is about if you are a young professional starting off a career in this area, where do you head and where would you find training and mentoring in this area. It actually relates to a follow up question, I think I recognize the fellows name, who has gone through some of the training programs for example at the University of Washington. I think there are very strong training programs out there and also MDM, what would you add to that as Neil?

*Neil Jordan:*A few things, there are Graduate programs that have [skipped audio] in me who do this work. Frankly I think the best way to learn is to identify an opportunity to work alongside one of us on an existing Budget Impact Analysis . There is an awful lot of art mixed in with the science of doing this and I think folks probably have a sense for it in the way we have been talking about it. Budget Impact Analysis  projects, or Budget Impact Analyses are sometimes done as companions to implementation projects within query. Probably not as much as folks would like but there certainly have been some of those. I think again for folks who are interested I would be happy to talk to anybody Todd, I imagine you would be too. We might be able to figure out based on interest who else around the country may be doing this work and who might be someone who you could potentially learn from and be mentored by on an existing project. 

*Todd Wagner:*Those are great and I think you also mentioned value and health and they have ISPOR the related International Society for Pharmaecoeconomics and Outcomes Research. That is also a great outlet for meeting people both and they are probably one of the best meetings that span industry and academics. So I tis a very nice blend in that regard. There is a question do you have examples of state of the art publications that you can refer to and maybe you and I, Neil, can come up with some that we can put up on the HERC website. 

*Neil Jordan:*I think that is a great question and I have a few in mind. Rather than talking about them Todd you and I should probably offline put together. We actually have a list and that is part of that white paper I was describing. that a few of us are working on, that is one of the things that we are actually doing is covering, picking out some of the best ones to put them forward as examples of really well done analyses that feature particularly strong elements of BIA. 

*Todd Wagner:*I will just reiterate what you said Neil that there is art and science it is both and often the journal articles are too short by design to include all of the details that go into these things. We can put these together but it is hard to have any that are perfect. 

*Neil Jordan:*Agreed, a hundred percent. 

*Todd Wagner:*A lot of people would love to have your contact information Neil and probably my introduction of it describing you as acclaimed Academy Award winning director and film writer was not accurate because that is a different Neil Jordan. I do not know if there is an easy place or maybe Heidi can put your contact information. 

*Neil Jordan:*Yeah that would be fine. 

*Todd Wagner:*I apologize if I confused people in the intro. Another question has come up does the VA have a standard cash flow modeling analysis tool. 

*Neil Jordan:*I am going to kick that one back to you. I am not aware of one but are you? 

*Todd Wagner:*I am not either. There are many groups, the VA is huge so there are many groups in VA so there may be one that exists I just do not know of it in the research services. Almost everybody I bump into doing this operationally is using Excel but I could be dated and off, I apologize. 

*Neil Jordan:*I was going to say I am going at pitch in and help with the questions. One of the questions is – is it possible to list potential mentors, proctors, researchers on the HERC website. If I understand the question, if the question is – is there a list of economists in the VA that you can find on the HERC website that might point you to folks like Todd and me who do this work, the answer is yes and Todd can you remind folks where on the HERC website that is. 

*Todd Wagner:*I think it is under facts, but I would have to find it myself while we are talking here. I will say that we are increasingly seeing people who are not, there is a stereotype that only an economist can do this and I would like to break that stereotype down. I think in many cases even someone with a Master’s Degree or a well-trained Bachelor’s person who understand the implications can do it. Maybe structuring the right design for a study would require a Ph.D., but in many cases you do not have to be a Ph.D. to do this. 

*Neil Jordan:*I completely agree. One of the things that may not have been apparent from the presentation today is that while carrying one of these analyses out it is very intricate. It does not require as advanced a level of analytic prowess as lots of other things do. I mentioned that one of the by-products of a Budget Impact Analysis is often a spreadsheet that would enable somebody to input values and come up with a cost calculation. That in many ways mirrors some of the key features of carrying out a Budget Impact Analysis. It is really about estimating the different cost components and elements as well as precisely as you can with as little uncertainty as possible then  being able to combine those in a way that makes sense. The sensitivity analysis I think is a little bit more sophisticated, but again, as Todd says, somebody with good analytic training even at the advanced Bachelor’s or Master’s level could carry out one of these analyses 

*Todd Wagner:*I do not know if people can see it, someone is moving it so that is even better. On the slide is a HERC link that says resources and experts so you can see people who put their names there. Of course you can contact Neil or myself and we will do our best to sort you to people. It is a good mentoring and it takes a lot of time, so there are no promises that we can do everything to help you but we will do what we can. Any other questions for Neil? This has been great Neil, thank you so much and thanks everyone for participating in today’s HERC course. As you can see on the slide, the next one is July 9 by Paul Barnett and “How Can Cost Effective Analysis be Made More Relevant to US Healthcare”. 

*Moderator:*And on that note, I am going to put up our feedback form. If you could take just a few moments to fill this out. There is not a submit button so once you have entered your information we do have that captured. If you could take just a few moments to fill this out, I know we at CIDER and everyone at HERC we really do read through your responses here. If you could take a few moments we would definitely appreciate it. Neil I want to thank you so much for pairing and presenting for today’s session, we really appreciate you being able to fill in for today’s session. Todd, thank you so much for helping out for today’s session I always appreciate that. 

*Todd Wagner:*You bet, thank you Heidi.

*Moderator:*To our audience, thank you for joining us today. If you did not catch that link for next week’s session a registration information was sent out earlier today. We hope that you can join us next week for Paul Barnett’s session and if not we hope to see you at a future HSR&D Cyberseminar. Thank you everyone. 

*Neil Jordan:*Thanks folks. 

*Todd Wagner:*Thank you, thanks again Neil.

*Neil Jordan:*You are welcome Todd, thank you pleasure doing this with you. 

*Todd Wagner:*You bet. 

Page **11** of **12**