**Transcript of Cyberseminar**

**VIReC Clinical Informatics Seminar**

**9 Factors that Influence Prescribers’ Response to Alerts at the Point-of-Care: Implications for VA Order Check Design**

**Presenter: Alissa L. Russ, PhD**

**January 15, 2013**

**9 Factors that Influence Prescribers’ Response to Alerts at the Point-of-Care: Implications for VA Order Check Design**

**January 15, 2013**

Margaret:Welcome, this session is part of the VA Information Resource Center’s ongoing clinical informatics cyber seminar series. The series’ aims are to provide information about research and quality improvement applications in clinical informatics and also information about approaches for evaluating clinical informatics applications. Thank you to CIDER for providing technical and promotional support for this series. Questions will be monitored during the talk in the Q&A portion of Goto Webinar and VIREC will present them to the speaker at the end of the session. A brief evaluation questionnaire will appear when you close Goto Webinar. We would appreciate if you would take a few moments to complete it. Please let us know if there is a specific topic area or suggested speaker that you would like us to consider. 

At this time I would like to introduce our speaker for today, Alissa L. Russ, PhD. Dr. Russ is a research scientist at the Roudebush VA HSR&D Center of Excellence on Implementing Evidence-Based Practice. She also holds appointments with Regenstrief Institute and the Purdue University College of Pharmacy. Without further ado may I present Dr. Russ.

Dr. Alissa Russ:Thank you, Margaret and thank you all for taking the time to attend this cyber seminar today. The topic discussed today are nine factors that Influence Prescribers’ Response to Alerts at the Point-of-Care and the Implications for VA Order Check Design. 

I thought I would get started by showing a little bit of my background. My formal training is actually in engineering and I apply human factors engineering methods to health services research. So what is human factors engineering? It really focuses on the design of technology, processes, and work systems to be compatible with human cognitive and physical capabilities and limitations. That is a bit of a mouthful, but essentially it is focusing on designing tools to support human characteristics. When you apply that to health services research in this case, I am interested in supporting prescribers, their decision-making, and ultimately medication safety for our veterans.

Now that I have had a chance to share a little bit about my background I am interested in learning more about your background. At this point, would ask that you complete the poll question that Heidi is going to pull up here. 

This questions reads: What is your primary role in the VA? Is it informatics? Patient care? Research? Other? Or perhaps you work outside the VA. Just take a couple of minutes to complete that and we will see the results here shortly. 

Okay so it looks like we have the results in front of us. Pretty good distribution across informatics, patient care, and research, so pretty evenly spread that way. There are a few that are outside the VA so thank you for your interest. 

Here we go. This is an overview of what I will be talking about today. First I will present just a few basics about VA alerts so we are on the same page. Most of my time will be spent presenting results from field observations and interviews that we completed with VA prescribers. These results will be presented along with a framework that describes the human-computer interaction between prescribers and alerts. Throughout the talk and also at the end, I will describe potential implications for VA Alert Design and Medication Safety. 

According to the Institute of Medicine there are over one and a half million preventable adverse drug events that occur in the United States each year.  Computerized Medication Alerts have been implemented in many healthcare systems including the VA to try to reduce the number of adverse drug events. However, we know from the literature that these alerts have some problems associated with them. One is that they promote alert fatigue. There are so many alerts and so many false alarms that over time prescribers can actually become desensitized to them and tend to bypass them. There has also been research groups that have conducted database analyses of override rates to understand why alerts are being overridden and there are surveys conducted by various groups to get input from prescribers. At the time of this study, we were unable to identify any research groups that had really taken a close look at how prescribers interacted with these alerts in the context of care. The aim of this work was to identify factors that influence how prescribers perceive and respond to alerts at the point of care.

This is one example of a medication alert. In the VA these are formally known as order checks. I will be referring to them as alerts throughout the presentation just for ease of presenting. The one shown here is a duplicate drug order for Lisinopril.

This is another example of one of the adverse reaction alerts to Penicillin. 

Some alerts in the VA system require an override justification for the prescriber. This is a free text option so they can enter an explanation of why they are bypassing the alert. There are close to 20 different types of alerts for medications in the VA system. 

This is an overview of where the alerts appear during the ordering process in the VA’s electronic system. At the beginning, the prescriber can initiate a medication order and at that point after entering the dose and other information alert can appear. Then later on during the ordering process when a prescriber goes to initiate a signature to finalize the order an alert can appear again and is sent to the pharmacy. This is really a simplification of the actual alert system, but the basic flow of the alert in the ordering process. 

This leads me to question two, a key component of this work is derived from human factor science. The question is: according to human factor science, what is the primary way that providers learn about how health IT is designed and intended to be used for patient care? So go ahead and bring up the poll here. Heidi should be assisting us with this. The poll should be open. The options for this question are:  do you think that the main way providers learn about health information technology is through operations manual, or their peers, through training sessions, through software interface design or do they talk to the developers? This is kind of a fundamental concept from this work. Looks like our audience response is 70% saying it is from their peers; 21% is training sessions, and seven percent are guessing that it is software interface design. 

Actually from human factors science we know that software interface design is actually the primary way that providers learn about health information technology, the end-users learn how to use new technology. It is true that providers learn from their peers and training sessions are also important part of helping people learn new technologies. The interface is really the key link between the programmer and the end-user, and individuals naturally try to learn by interacting with their surrounding world, and in this case it is the interface. 

So the methods for the study that we used: we conducted field observations and interviews at a single but major VA Medical Center, and we observed prescribers for half a day each. We looked at three different prescriber types: nurse practitioners, physicians, as well as clinical pharmacists who have prescribing privileges in outpatient care. We conducted a qualitative analysis of our data. We took an inductive and really looked through the data to examine and identify emergent themes. We did not have a pre-defined coding list. These themes were refined and identified through a team consensus process and we also used MAXQDA to facilitate coding. 

Altogether we analyzed over 300 pages of typed notes. This included information from 146 patient encounters and over 300 alerts. This data was analyzed across 30 different meetings. I would like to take a moment to recognize the other individuals who are part of the qualitative analysis team. It was myself, Jason Saleem, Alan Zillich, and Sue McManus. 

This table represents the recruiting results. We recruited 30 prescribers: 20 from primary care and these prescribers were spread across five primary care clinics. Then we also recruited ten prescribers from specialty clinics and these included eight different specialty clinics. Average age was 42 and there was a range of experience with the VA computerized provider order entry system, this ranged from one to 24 years with a mean of ten. We had a similar number of men and women. 

This is a quick overview of the results. Through analysis we identified 44 themes that affect how prescribers interact with alerts. We further organized these themes into nine overarching factors. Then we integrated these factors into a framework. I am actually going to start with the framework. 

I want to point out that the underlying framework is derived from mental models concepts in human factors science. The underlying framework is not new and we adapted that model specifically for prescribers. We spent quite a bit of time going through this underlying framework. On the left we have the programmer’s mental model. The mental model refers to that individual’s understanding of the system functions and features and how the system is intended to work and be used. The programmer’s mental model is reflected in the system design so in how they set up the programming rules and other features of the software. The system design has been conveyed by a system image or by the interface to the end-user, in this case to the prescriber. The system image is key as we talked about a few minutes ago that is the primary way that the end-user learns how to use the system. It also influences the end-user’s perception and in this case the prescriber’s mental model. That is the prescriber’s understanding of how the system functions and is intended to be used. That in turn influences the prescriber’s input back into the system and how he or she responds to the interface. This model here is really applicable across a variety of software applications. 

In our particular study we were focusing on alerts. Adding to this model I am going to add in the nine overarching factors that we identified in the study. The first two were alert system logic and alert system redundancy. 

So I’m not going to present all of the associated themes for each factor. In this talk I have just chosen to highlight a few. If you see an underlying term that refers to one of our emergent themes. 

For  Alert System Logic, one of the emergent themes was External Crosschecks. In this case it refers to instances where the system is comparing medications across different VA medical centers and triggering alerts. Then also comparing VA medication lists against medications that are entered under non-VA medication lists. Prescribers really perceived this as a strength of the alert system that they could check across these different lists and trigger alerts. This was an unexpected perceived strength of this system. 

Another theme under alert system logic was detection. We included both over detection and under detection in this theme. We also identified some gaps where prescribers wanted more alerts. Oftentimes in the literature we hear there are too many alerts, they want less. We definitely got a lot of feedback in that regard as well. But there are also some cases where prescribers wanted additional alerts. Many of these were related to laboratories, values, or different types of medical conditions. There are a few of them listed here such as lithium and alerts for liver function. 

The second factor was Alert System Redundancy. Within this factor one of the interesting themes was repetition within an encounter. We certainly know that alert fatigue is a problem that we need to address. This was kind of a unique instance that could promote alert fatigue. I have an example from our observation. In this first one the nurse practitioner ordered niacin and received an alert for niacin and pravastatin. She then signs the order, an alert appears again. Now she is ordering pravastatin and receives an alert for pravastatin and niacin. She then signs the pravastatin order and receives an alert for the fourth time. The observer noted: we received the same alert four times in the last ten minutes or less. These are some cases of repeat alerts. We could potentially reduce to try to minimize the likelihood of alert fatigue. Some of these cases may be easier to address. 

The third and fourth factors I will talk about are alert display and alert content. These closely relate to the system image or interface of the software design. These are particularly key for the prescriber’s understanding of the alert system. For alert displays identifies by the emergent themes, which are listed here such as format and the timing of the alert displays. I am not going to go into detail today on each of these, just to say that these are strong themes and we had a pilot project funded to look at this more closely. This project was entitled “Redesigning Medication Alerts to Support Prescriber Workflow”. In this project we actually redesigned some of the display features of alerts and compared two different alert designs in a simulation study with prescribers. This was a VA funded effort and we are currently in the dissemination phase of this work and hoping to have some publications soon. 

The fourth factor was alert content. This was referring to the information presented by alert. One theme under alert content was specification, which we define as an explanation of why an alert was triggered. This theme included several cases where prescribers were uncertain about why an alert was appearing. This is one example from our observation. An order check appears it is for 

duplicate drug class alert. And this physician responds “I don’t even know what that means. It says expectorants, but it doesn’t say what the other medication is. It says non-VA medication guiafenesin, but you’re getting that here, right?” That is one example of confusion about what the alert was intending to convey. 

This was an unexpected theme for us and the next audience question is, you want to go ahead and open polling, Heidi. Out of the 30 prescribers in the study, data from blank indicated that there was confusion about why alerts were triggered. So just take a guess at how prevalent the theme was across the 30 prescribers. Do you think it was: less than 5; 5-9; 10-19; or more than 20? Do your best guess and we have 48% saying more than 20 and that 48% is correct. We had 45% saying ten to 19. 

So we had evidence of this from more than 20 prescribers, actually from 21, just slightly over. This is a very strong theme that we did not expect. One of the implications of this is that it can pose a substantial barrier to resolving alerts. These are cases where prescribers had difficulty figuring out why the alert was appearing; what the hazards were; what they should be aware of in terms of their prescribing decision. There is certainly more work needed on the clinical content presented by alert. 

The fifth factor I’ll talk about today are cognitive factors. These relate to prescriber perception. We are not going to talk about cognitive factors real in-depth today, this is just going to be one click audience poll. Heidi you can go ahead and open up the poll if you would like. 

Heidi:Sorry, just a second.

Dr. Alison Russ:Oftentimes we hear about how alerts are not helpful but we found several cases where alerts can be helpful for providers. The question reads: alerts sometimes supported awareness by providing new information for prescribers. Alerts were found to be particularly helpful for: new patients; new medications; medications they rarely prescribed; allergies documented by someone else, or all of the above. We will give a few seconds for people to chime in. Okay. 

Seventy-six percent got that all of the above examples were cases where prescribers found alerts useful. These are all cases where alerts supported awareness for prescribers. The theme that we identified was awareness and all of these situations are places where it is particularly key for prescribers to receive alerts. One of the implications from this finding is that these are cases where we should make sure that prescribers are receiving alerts and maybe a broader range of alerts. You may need to reduce alert fatigue certainly for prescribing in general but these are cases where we really want to present perhaps more alerts for prescribers. 

Factor number six was pharma--[inaudible] and this relates to the prescriber’s mental model. This theme was also really interesting for us it was a very strong theme and there was a subtheme within  this related to the three different prescriber types that were supported by all three types of prescribers. 

We had physician here on the lower left. Physicians are saying things like: I talked to the clinical pharmacist to resolve order checks. I like having the pharmacist in the room here with me. Similarly nurse practitioners were saying “I’ll ask the pharmacist in the clinic if it’s an important interaction.” Then we were hearing from pharmacists things like “If I’m not in the room the doctors don’t know what to do with the alert.” So all three prescribers had seemed to recognize kind of a gap between the knowledge of physicians and nurse practitioners compared to pharmacists especially as it relates to resolving alerts. 

The specific theme within this that I am referring to is pharmacist consultation and proximity. We found that individuals consulted pharmacists to help resolve the alerts and they preferred to have the pharmacist close by so they could consult them in real time by rotating their chair and looking over their shoulder and asking the pharmacist on their team. One VA physician stated that “physicians are not trained like pharmacists. We have to learn what a significant clinically relevant interaction is”. 

There are some implications of this; a key implication is that this indicates a gap between the alert content and language and then intended end-user group. Indicates that we need to revise the content and language of alerts to better support physicians and nurse practitioners. You might think well why don’t we design separate alerts for physicians and nurse practitioners and then have a different set of alerts for pharmacists and kind of tailor it to each of those user groups? That is a potential approach I think that becomes very difficult to sustain over time if you tried to incorporate evidence and become more complex to manage. An alternative approach based on what we know from human factors is to develop a more universal design for alerts to support a wider range of prescriber types in terms of training and levels of experience. The universal design is really designing for the more novice users all the way up to the end-users and having features on the alerts that more advanced users can utilize, but then also having basic features for the more novice users. 

Factor seven and eight are medication management and workflow. Now we are starting to get more into some of the broader factors that encompass a larger look at the interaction between the system and the prescriber. One theme under medication management was the design of the prescribed provider order entry system. We actually found some cases where this CPOE design sometimes hindered alert resolution. This is an example from one of our observations where a physician orders mycophenolate, and receives a duplicate drug class alert for mycophenolate and azathioprine. The physician overrides the alert and goes on to explain that “I could not discontinue the azathioprine because it is coming from another VA. I can only tell the patient to stop it.” This is a case where the physician was intending to discontinue the azathioprine but due to limitations of the CPOE system was not able to do so. 

I think this finding has implications for the CPOE design especially as we start to exchange more data from VA to non-VA that we will need more advanced CPOE systems that can detect alerts across different medication lists which we are doing a good job of at the VA currently. Also so that we can manage ideally a single medication list for the patient. 

The eighth factor was workflow and this is our last audience question. I am not going to spend a lot of time on the workflow piece but wanted to highlight this point. The question is: in this study after computer delays to reach [blank] prescribers began expressing frustration. Do you think it was after ten to 15 seconds; 16 to 30 seconds; 31 to 60 seconds, or 61 to 90 seconds? We will give a little time for people to weigh in on this question. 

Here are our results: 68% selected ten to 15 seconds and you are correct. No one selected 61 to 90 seconds; we would all be expressing frustration at that point. Here we go, after ten to 15 seconds was when we first noted that prescribers began expressing frustration and I think this highlights their workload and also their expectations for the computer system and this threshold is somewhat low. I think it is important to recognize the importance of and impact on workflow. 

The final factor number nine is alert system reliability. This is looking even more at the big picture of things that influence prescribers’ trust in alerts and how they use the alert system more as a whole. One theme under alert system reliability was common care practices. Oftentimes you read in the literature that an alert may apply broadly but does not apply to a specific patient. For example, the alert applies to Mr. Jones but not Mr. Smith because Mr. Smith is tolerating two different medications so the drug interaction does not apply to him. In this case this theme we actually found cases where alerts inappropriately warned against practices that apply to broad patient populations. Now it is not that alert was not relevant to that individual patient, it was actually contradicting practices that are applicable to a large population of patients. This should hopefully become more clear as I provide some specific examples. 

I am going to show three basic examples here. On the left we have the alert trigger from our observation and on the right we have the prescriber’s response. This first example relates to medications related to diabetes. That is insulin and metformin. The prescriber response is “there are tons of diabetic patients on this combination and this is safe”. We also observed this theme for patients prescribed multiple inhalers. Then finally the last example is from a clinic where there was a duplicate drug class for antiretrovials and the prescriber responded that “each patient is on at least three antiretrovials, the cocktail is how we treat the patient”. 

To summarize this theme the implication of this is that we need to reduce alerts that conflict with evidence. This is important not only to reduce alert fatigue, but also because these types of alerts may undermine prescribers’ trust in the alert system. 

Those are the nine factors and this is the final framework that we had from our study. In the earlier factors, so like think about one through four, tend to influence the later factors. 

In summary there are several design implications I am just going to highlight three here. One is that we found that the alert interface was often problematic for prescribers. The interface design should be addressed along with efforts to reduce alert fatigue. I think that there are efforts needed simultaneously to reduce the number of alerts and also improve the interface. Second point here is that we need additional work and knowledge on how to present clinical information on the alerts so that we can better aid prescriber decision-making. This is much easier said than done but I think that there is a clear need to enhance the clinical content of alerts. The final point, which I mentioned earlier, is we need to start preparing these alert systems for increased data sharing across healthcare organizations. 

In conclusion, this is one of the first studies to examine alerts real-time at the point of care and results yielded a novel [inaudible] that describes the prescriber alert interaction. Our hope is that these findings will inform alert redesign and ultimately enhance patient safety. The ultimate goal really is patient safety. 

As I begin to wrap up the talk I want to highlight a key quote that we collected in our data and this was from a VA physician who said that “Some are critical interactions for example, nitrates and phosphodiesterase inhibitors. It has happened before where I didn’t catch this interaction, but the computer did.” This is really the essence of what these alert systems were intended to do. 

If you are interested in learning more about the study and our results I’ll refer you to our publication in the I*nternational Journal of Medical Informatics* that was published in 2012. This work was supported by the VA and would not have been possible without VA support. This includes support from Health Services Research and Development as well as several career awards. 

I would like to acknowledge the individuals that participated in this study as well as the project team members for making this work possible. I would also like to thank other individuals who helped us along the way. 

This is the list of references that I cited throughout the presentation and with that I would like to thank you for your time and open up the floor for questions.

Margaret:Thank you Alissa, it was an excellent talk. There are a few questions that have already been typed in and I imagine a few more will be coming. First is something of a comment. When I was involved in designing the initial Vista CHCS program, 1987, the programmers told me that the criteria for screen refresh was two seconds. We were averaging over 50 seconds at the time. I tried to explain that any noticeable delay after pressing the key will result in a great deal of frustration over time even the then unattainable two-second delay prescribed. Comment I guess on triggering frustration. 

Dr. Alissa Russ:Yeah that is a really interesting comment that is fascinating. Thank you for sharing that. 

Margaret:Okay and here is the next long probably lot of comment. I actually believe the order checks have made the entire system less safe. I honestly cannot remember the last time I ordered a medication where I didn’t get an order check. The noise is so great and worse than two years ago that any signal has been lost. I believe, I believe we even get order checks on renewing current medications. That is just silly. In my ten years of VA work the only consistent property of CPRS is that it is growing harder to use and consumes more of what our patients truly want – our time and listening ear. Quite a comment.

Dr. Alissa Russ:Yeah. Thank you for that candid comment. I certainly agree that alert fatigue is a big burden on prescribers. We found evidence of that in our observations. I think there are certainly individuals who found value in the alert system. We also found evidence of that as well in our study. We agree with you that we have a long way to go to really make the system highly effective. 

Margaret:Okay. Why do expired meds show up as duplicate now? You have to get out again to look at the long list to figure out just expired. This makes one ignore more alerts. I guess maybe the real question – why do expired meds show up as duplicate now?

Dr. Alissa Russ:Yeah, I don’t have an answer for that in terms of the exact details of why that is happening. That would be a question more for the local or national developers. I think there has been a challenge to decide at what point the medications fall off the list. So should they not be reviewed by the alert system as soon as they become inactive or expired or what is the time period where a patient may still be taking that medication and it should still be compared against other medications and reviewed for alerts. I think that has been a challenge of figuring out the timeframe that alerts should be reviewed after they are expired.

Margaret:Okay. How do you address balancing patient safety with alert fatigue?

Dr. Alissa Russ:I think the main thing to address is we in the VA and other healthcare organizations really need to make strides to reduce alert fatigue. I guess if the question is how do you balance the number of alerts versus safety, the clear evidence that you have too many alerts and basically cannot be useful then you negate any of the patient safety intent. I think there is likely going to be restrictions on the number of alerts and we need to really focus on alerts that are more likely to be of high severity and applicable across a large patient population. It is not an easy problem that is why there is a lot of different research groups looking at this. Unfortunately it is not a simple solution, but we definitely do need to decrease the number of alerts and increase their specificity as well as when they are triggered.

Margaret:Okay. Is there any word on how the adverse reaction and allergy field will be separated to help providers discern between serious allergic response versus drowsiness or nausea? 

Dr. Alissa Russ:Thank you for that comment. I do not know the details of what the plans are for the allergy package moving forward with the alerts. I think that is something that needs to be addressed. We actually did a little bit of work with that in terms of interface display for our pilot grant, we redesigned interfaces like for allergies that we said people could see what the previous reaction has been from the alert itself. I agree that we also need to continue to push forward on the design of that allergy package. 

Margaret:Okay. Here is a follow up to the duplicate order just a comment from the provider. If an inhaler expires and I reorder it, it says duplicate. So adding more to that conversation. Okay, go ahead.

Dr. Alissa Russ:First, I do not have additional information on that and thank you for testing that out and providing that detail. I think we really do need to provide feedback at our local facility and at the national level. People are interested in feedback and they do take that into account. I think that is something to provide feedback on. 

Margaret:Okay. Another long comment. I am on the allergies new term committee that responds to requests for new terms or changes in the way the alerts work. Frequently there are legitimate clinical requests that we cannot honor because the pharmacologic classification system is not designed to handle the particular clinical cases. The request to redesign the pharmacy classification system requested by pharmacy experts was made a few years ago, 2009 or 2010 as I recall and yet no approval for funding has been given for this. Designing better alert systems is complex and requires understanding from those at the top that make funding decisions. So far it has not happening in the VA. 

Dr. Alissa Russ:Thank you for offering your insight from your experience within the VA. I know there certainly are a lot of challenges with implementing some of these changes. We do see changes continually rolled out for alerts which is encouraging but I agree there are a lot of barriers as well. 

Margaret:Can the severity or importance of the alert be ranked in order? There is a material difference between a med marked never for drug interaction and lesser EBIs. 

Dr. Alissa Russ:There are things that we can certainly do to prioritize alerts. For example, like black box warnings should be higher priority. Like most anything in clinical care it is not cut and dried. One alert may be high priority for Mr. Jones but low priority for Mr. Smith so it depends on patient factors some of which we might be able to add into the alert logic. So looking at patient lab values and information about the patient’s history but some of that is going to really require a lot of clinical judgment by the prescriber. I think we do need some focus on reducing the number of alerts to really hone in on sort of the high severity alerts. But again there has not been a clear consensus about what active alerts should be. 

Margaret:Okay. This was an excellent presentation. Has this been presented to other groups? I would suggest the IEHR, ICPT groups for orders and alerts notifications. Will copies of slides be available and would you be interested in presenting to other groups?

Dr. Alissa Russ:I guess I am certainly open to presenting to other groups and copies of the slides will be available from this talk. I believe we will also have an audio recording with the slides available. Is that correct Margaret? 

Margaret:Yes it is. And Alissa your email is on your slide, can people contact you?

Dr. Alissa Russ:Yes, feel free to contact me. The audio recording and presentation will be available after this talk on the VIREC site. 

Margaret:It is actually on the, well HSR&D cyber webinar website in the archives in 24 hours I think. We will also be sending an email off to everyone with the direct link to that archive recording. 

Dr. Alissa Russ:Okay great. 

Margaret:Next comment: it sounds like a system redesign requires incorporating your research as well as all levels of providers. 

Dr. Alissa Russ:Yes, it does involve incorporating not only this research but I think also the research of other groups in the VA as well as considering evidence from outside the VA. Yes, I agree that the alerts need to be designed for a broad range of prescribers. 

Margaret:Okay another comment: In response to the question about distinguishing severe allergic reactions from trivial ones, there is a nature of reaction field in CPRS that enables one to designate anaphylaxis, did I say that right, etcetera, interoperability standards also require severity codes, which VA has not traditionally required, but hopefully we will in the future. 

Dr. Alissa Russ:Thank you for adding additional information to that discussion about allergies. 

Margaret:Okay. I would appreciate any comments about alternate approaches to presenting alerts. Do you think it would be effective to provide additional patient-specific information to the provider? Are there non-narrative approaches to presenting effective reminders? What VA and non-VA research groups are leaders in researching pharmacy alerts?

Dr. Alissa Russ:Okay so I try and take those one by one. I heard a question there about how the text is presented to alert or by the alert and then there is a question about non-VA and VA research groups. What was the first part of the question?

Margaret:The first question yeah I was going to read it again, Alissa. Do you think it would be effective to provide additional patient-specific information to the provider?

Dr. Alissa Russ:Okay I will start with that one. Yes, definitely, that would be beneficial. That was one of the themes that we found. I did not report that in the presentation today but it is in the paper. There certainly are patient information such as lab results and even the adverse reaction alerts are previous symptoms that could be presented on the alert. I think there are some plans underway to enhance the adverse reaction alerts. Certainly the information that a prescriber needs to resolve the alert ideally needs to be available on the alert itself.  We have to be careful we do not add too much information but the key components needed for decision making as much as possible should be on the alert itself. That is an area where we need further improvement. I think we jump to the second part of that question Margaret.

Margaret:Yes. Are there non-narrative approaches to presenting effective reminders?

Dr. Alissa Russ:There are non-narrative approaches and actually non-narrative so I am thinking when I think of narrative I think of more of a sentence structure. There is actually evidence from human factors literature that when you are presenting warnings to individuals it is better to have more of a bullet structure short statements rather than a sentence approach structure. I do not know how many alert systems out there have a sentence structure versus a bulleted list at this point but that was a key thing we looked at in our pilot grant. We redesigned some of that text and shortened it to present it more concisely to the prescriber. There is evidence that presenting short statements can be more effective than long prose text. 

Margaret:Okay. The last part of the question was what VA and non-VA research groups are leaders in researching pharmacy alerts?

Dr. Alissa Russ:Okay. There are a wide variety, there are several groups that come to mind, Vanderspeed [sp] has done a lot of work on alerts. David Bates and his group has also worked on alerts and then Dan Malone has some ongoing work on alerts. Pete Glassman has some work on alerts in the VA. We looked at different aspects of alerts to my knowledge there are not as many human factors specialists working on alerts and I think we actually need more of them. But to really pick out the key VA and non-VA I think is difficult, there is a large body of literature evidence and I think we need to start incorporating more of that evidence into alert design. 

Margaret:Okay are there any studies on patient safety with the alert system? I believe that the plethora of alerts is decreasing patient safety because the critical alerts are getting buried under the vast majority of alerts that are not significant. I honestly doubt many providers read any of the alerts anymore. 

Dr. Alissa Russ:I certainly agree that prescribers have across the board can be subject to alert fatigue and have that propensity. We actually did observe individuals in the study though that were really waiting for that alert to appear. It is kind of like it gave them peace of mind as to whether or not an alert occurred and they seem to be dependent on alerts that really can use them as a key part of their ordering process. I think it is not accurate to say that no one looks at alerts because we certainly found individuals who did and described cases where alerts have been helpful to them. In terms of studies that have looked at alerts and outcomes not aware about work the VA, there are a lot of challenges to doing that. We are trying to push in that direction but that has not been easy. Outside the VA I think there has been some work done looking more closely at that but still not a lot of looking at alerts and outcomes. If you would like more information you can email me and I will try to find some literature related to alerts and outcomes to send to you. 

Margaret:Okay. Another question. Do you think it would be effective to generate patient alerts? If yes, how might this be accomplished? Could alerts be integrated with VA’s blue button patient access system?

Dr. Alissa Russ:I think there are challenges with integrating into something like the blue button. If I understand that correctly the patient would have to log in to MyHealtheVet to see the blue button. Really you want it tied to the medication and you want to prevent the patient from getting the medication in the first place. I think given the challenges we are seeing with presenting these alerts even to physicians and nurse practitioners who do not have the same specialized training as pharmacists I think it is going to be very difficult to present especially computerized alerts to patients. Now there may be some other things we could do in terms of the medication label, maybe to highlight on the medication labels like what adverse reactions are potential side effects to watch for in terms of a drug/drug interaction. That would really take additional work to see if that is feasible and effective. 

Margaret:Okay, thank you very much. Right now there are no more questions. I think maybe people have finished typing their questions and there certainly were plenty of them. Alissa, thank you very much for presenting this talk. I want to just let our audience know about our next VIREC clinical informatics seminar, it will be Tuesday, February 19, presented by Drs. Mitchell Mallin from the VA and Joseph Finkelstein from Johns Hopkins. They are presenting on A Chronic Disease, How Automated Telemanagement System for Patients with multiple sclerosis. Still no more questions so I think we are done. Thank you very much everybody, have a good day. 

Dr. Alissa Russ:Thank you. 

Page 1 of 13

Page 13 of 13