**Transcript of Cyberseminar**

**VIReC Clinical Informatics Seminar**

**Effect of EHR Use on Patient-Provider Communication**

**Presenter: Zia Agha, MD**

**December 17, 2013**

*This is an unedited transcript of this session. As such, it may contain omissions or errors due to sound quality or misinterpretation. For clarification or verification of any points in the transcript, please refer to the audio version posted at www.hsrd.research.va.gov/cyberseminars/catalog-archive.cfm or contact the VIReC Help Desk at virec@va.gov.*

Joanne StevensWelcome everyone. Either good afternoon or good morning depending on your time zone. This section is part of the VA Information Resource Centers ongoing clinical informatics cyber seminar series. The series aims are to provide information about research and quality improvement applications in clinical informatics and also information about approaches for evaluating clinical informatics applications. We’d like to thank CIDER for providing technical and promotional support for this series. As Molly mentioned questions will monitored during the talk of Adobe Connect and will be presented to the speaker at the end of this session. When we close the session a brief evaluation questionnaire will pop it. If possible please stay until the very end and take a few moments to complete it. Let us know if there is a specific topic area or suggested speaker that you would like us to consider for a future session. At this time I’d like to introduce or speaker for today. Dr. Zia Agha. Dr. Agha is Director of Health Services in Research and Development at the VA San Diego Health Care System. He is also a staff physician at the VA San Diego Health Care System and Professor of Medicine at the University of California San Diego. Without further adieu I present Dr. Agha.

Dr. Zia Agha: Hi can everybody hear me?

Molly:Yep.

Dr. Zia Agha: Great. Well first of all thanks for inviting me to give this presentation. It’s always a pleasure. I think I’ve done a few of these in the past on different topics and it’s always been a great interaction. So I’m going to try to keep this to the point and I’m hoping we can generate some discussion around the poll questions. Without wasting anymore time let me dive into the topic of today’s talk. Really it’s talk about EHR usage and how that affects the commissions workflow and in that workflow we talk about patient-provider communication. I’m not going to get in too much detail about the communication aspect today but focus more on the workflow and the EHR usage. This is work that was funded by HSR&D so there are no conflicts of interest to disclose here. Honestly this is work done with a large number of collaborators. Just to give a little bit of background let’s talk with the term usability and this is a term that is being discussed a lot in popular media and with EHR’s being adopted and the high tech act. You think of interoperability which is the lifecycle property of data. Usability is also a lifecycle property of software. The definition of usability really is effectiveness, efficiency and satisfaction with this user simple form they intend to pass using the Health IT tools. While usability, science or measuring usability is conducted in different ways whether it comes from the laboratory setting or clinical settings. Our approach is to do that in a clinical setting. But then improving usability is obviously achieved through either re-engineering the EHR or re-engineering our workflow. Let’s talk a little bit about that also. So what are the learning objectives for today’s discussion. Really I want to describe how the EHR usage occurs in one setting, which is in our VA and the primary care clinic. How that effects staff efficiency and we will do this by using a very detailed time-domain process of time-stamping different events. We will try to recognize challenges that area posed by the current technology to providers who are often who are faced with multitasking between EHR work and addressing the patient’s needs. We will try to identify and rank-order some of the most commonly used EHR components and the reasoning for doing for that is to help focus our energy for redesign in those areas. We will try to correlate EHR usage with communication patterns and then we will try to summarize by doing targeted studies of data to help improve EHR design future projects. 

So let me tell you a little bit about the PACE study that was funded by HSR&D and the motivation of that study. The primary motivation was while there was some data we are looking on EHR usage in a clinical setting it was not sufficient. It was either time motion studies that—purely observational without any quantitative data or they were quantitative studies in lab experiments. So we wanted to combine the combine the ability to connect quantitative data from real clinical settings. We also wanted to get as realistic workflow and ratability that exists in clinical interactions. So a basic observation of study, the study was conducted in the VA Medical Center in San Diego in its outpatient clinic. We targeted all our patients visits to primary care providers. These are all established patients. They were not new patients and these are routine follow up visits. We collected time-stamped visit process data or EHR activity or non-verbal or clinical workflow and then for verbal communication. In addition we collected patient-provider satisfaction data, agreement data and conducted interviews and focus groups.

In terms of the EHR activity data, this was collected by usability software called Morae which logs the time-stamp mouse activity of the EHR and the screen capture of what’s on the EHR screen as the primary care provider is working within it. The screens, obviously provide you with information of the notes section, the labs, medications, reports, reminders etcetera as they were maneuvered. We can also look at various tasks such as viewing the screen or doing order entry, whether we’re looking at a current document or a prior document. We can also study the modes of interaction. So these are the interacting with a text box versus a radio button on the scroll bar or entering free text and then we can also focus on some specific areas of the EHR such as order entry, or alerts and reminders. Let me highlight one fact while we’re here while this this multi-level data is helped by direction from Morae software, most the data is not contextualized, i.e. we have mouse clicks linked to events but they’re not contextualized so we had to develop a coding schema which would help annotate each mouse click with specific activities. I am going represent that data in two formats. These are pretty typical for these types of usability’s studies. Practical task analysis basically rank ordering of data and then the sequential task analysis to show you the sequence of events, such as most frequent transitions between screens. We would also some temporal analysis looking at time-at-task profiling and so this has the ability because our data is linked to exclude certain sections or to create certain more important areas of the visit. For instance when you’re looking at interaction with the EHR we can exclude the physical exam or interruptions when the doctor left the office. That way we are able to look at EHR workflow when the provider is sitting on the computer. We’re also able to do verbal analysis of data through different time segments of the visit. So for instance in one analysis we looked three minute of the highest and lowers EHR activity during a visit and to compare what happens during those visits in terms of communication.

Just a quick outlay of the data collection process. So there’s a room video camera, which is collecting the interaction of the patient and the provider. From there you can get non-verbal and clinical workflow information and obviously you can get the vocalization and verbal discourse also. Then the usability software which is connecting the mouse click activity and the EHR screen. We wanted to collect keystrokes also in the study but could not do it because of the privacy concerns. At that time the software would have captured keystrokes, log-ins and passwords. There was no way to de-identify them. 

All right so just giving you a sample of some of our data and I wanted to show this for two reasons. One to get you familiarized with how we are we going to present out data. So rather than presenting our data in summary format, which you will do also, we chose the course to do more data discovery, a knowledge discovery process where we try to keep the data intact. We did not summarize it into means and medians right off the bat. We did a lot of good exploration using some very nice synchronization tools. Here’s an example of two samples of this showing you visual format. Visit one is on the top and visit two is on the bottom and these are chromograms. Chromograms can be used to show timed events, which are happening at specific times. On the top bar you see the reds, yellows and blues. Those are the mouse clicks. Then events that are happening in a sequence continuously and these are the non-verbal or the clinical workflow behaviors and the demographics of the blue is when the physician is interacting with the computer creating something and trapping information. Light blue is physical exam. Pink is them interacting with a paper artifact or paper document these two channels of data are connected separately. One is from the room video the other is from the Morae usability software and obviously this data has to be synchronized to make sense. That synchronization was done by our team by using a time-stamp on both the software and recording those times and synchronizing them. 

If you look at the categories of EHR user interactions already you can broadly categorize them into four areas. Physicians then use the EHR they are interested in information retrieval which includes browsing, searching for information and some decision-support tools. Documentation is a huge chunk of work of progress notes, reports. We do a lot of order entry which includes ordering medications, labs and so forth. We also use the EHR for coordination of activities such as reminders, care team communication. Even though some of these communication and coordination activities are not inherently enabled in the current EHR physicians have found work arounds and physicians have found work arounds to use the EHR to coordinate their care. The one area that is missing over here and we should talk about that is all the administrative tasks that are done. These are mostly for billing and coding purposes. I’m going to stop here before we delve into sharing some of the data. So at this point just giving you an overview of the study and I’m curious, when we started doing the study we had it on 22 primary care providers who were involved in our samples and I asked them how much time do they think they spend on the EHR or more specifically how many mouse clicks does it take for them to conduct one patients visit. We got a broad range of numbers from people and I’m just curious how you guys think how much effort goes into interacting with the EHR so this is sort of....

Moderator:Dr. Agha looks like we’ve got a lot of our respondents writing in. About 47% estimate 100 clicks. About a third of our audience estimate 250. About 8% each for 20 and 500.  Thank you to our respondents. 

Dr. Zia Agha:Okay so that’s great. You know when we did this with our physicians a lot of physicians thought it was going to be in the 20’s, 30’s, or 40’s clicks and when we looked at the data and shared it with them they were very surprised that the average number of mouse clicks was closer to 250. So I think in the group, in the audience we had around 40% of people saying that C is the correct answer. So I think in our study that was the median number of clicks per visit and of course, I’m sure that number varies a lot across providers. 

So here’s one way of looking at the variability in EHR activity. This is a pretty busy graphic and I’m going to over it with you slowly. On one axis you can the sections of the EHR. So this is the section EHS so you got a note section, orders, labs, meds, reports, so forth. On the other axis you’ve got provider numbers, provider 1, 2, 3, 4, 5, 6. The gray circles give you a sense of the number of mouse clicks in that section. The largest circle is 300 mouse clicks. The next smaller circle is 100 mouse clicks, then we go into ten mouse clicks and the individual dots are one mouse click each. You can see that there is definitely a big distribution in terms of how many mouse clicks are occurring in different sections and the total number of mouse clicks per visit. If you look at provider number one approximately 100 mouse clicks each in notes and orders and then the tens and 20’s and that’s labs but if you jump over to provider number five this provider had close to 100 mouse clicks in notes, 300 mouse clicks in orders. We don’t have the information here but let me tell you that on average each provider had four to five interactions with their patients. So this is not just one visit. This is a summary data across four to five of the provider that we are providing to. So clearly this helps us identify in a hierarchical way where most of the activity is happening and it also gives us an idea of variation across providers. You can also look at that data as what happens to provider interaction with EHR of clicks and scroll counts on one axis and then to the visit. Surprising that even though these are routine followup primary care visits you will notice that most visits were in the 30 to 40 minute range. So these are fairly long visits that are occurring. This is a busy slide again but the colors are denoting each provider so that you can seen the pink, there are four visits from one provider. This provider on average, more than 400 mouse clicks and on one visit goes almost 900 mouse clicks on one extreme and with them so much longer, 40 to 50 minutes. Similarly some providers have shorter visits in the 20 minute range. The light blue color here and had fewer mouse clicks. We can also go ahead and look a the distribution of the whole sample of data from all providers combined to help identify which are the EHR sections that providers work mostly in. It seems like the notes section is the central hub and that’s where most of the activity happens. Notes account for almost 40% of the mouse clicks and later I will show you that is more times than [inaud.]. Orders are the second largest category and then medications and labs. When you look within notes what’s happening you can look at the view tab of the notes, 83% of the time the provider is interacting with the current note for that visit. Only 7% of the time does the provider look at his own previous note or the notes of other providers. We were surprised by that information. We thought that providers were viewing other people’s notes or reading their own prior notes much more. It seems like most of the time is spent documenting the new note versus reviewing and reading of the previous notes. We can also say okay well that is more a picture of mouse click data, what’s happening in the room. Where is the attention focus of the provider. The reason for looking at that is because the [inaud] of providers are multitasking. They are paying attention to the patient and the discourse of what’s happening with the patient, doing decision making, getting a history, at the same time also working with the computer. If you look at all the visits in our study the total number of time spent with different activities, the EHR is the predominant activity. Almost 40% of the time is spent working with the EHR and we coded that as EHR because we recorded that the providers visual attention was.... Provider were making eye contact with the patient around 35% of the time and then conducting, 16% of the time of physical exam and other activites such as extracting the paper records.

So while that was a summary view of the data here is a sampling of the raw data. We have this obviously in the paper in print with all our samples but I failed to show that in the slide. So I’m going to show you a sampling of low EHR activity and high EHR visits across a couple of providers. The bars show a time line based chromogram so we have time zero beginning of the visit and then time moving on to the end of the visit. Each click denotes an interaction within the EHR. So for provider for patient number one there was hardly any mouse clicks and these are only acting in the notes section. Where it says provider 14, patient number 79 has a very dense bar graph showing mouse clicks happening very frequently. The other thing that you will notice if you go down and look at number 76 patient. There’s a section at time 20 where there’s a lot of mouse clicks of different colors i.e. the provider is multitasking between different screens going from meds to orders, to notes, to labs, and back and forth. We see a lot of that happening with it especially those who were high users  of HER.

How do we contextualized this EHR workflow with sort of a clinical workflow where the physician physical orientation and attention is the patient or the EHR. The same visits that I showed you before focusing on the mouse click data, we can show you those samples in terms of non-verbal [inaud.] I’m sorry about the color. Some of these colors are very similar so  it’s difficult to show them but pay attention to the green, which is provider’s attention to either the patient or patients companion. The dark blue, which is providing attention to the EMR, the computer. Light blue is the physical exam and pink is interacting with the paper record or some physical artifact like a medication bottle. You will see that in the first four sampling of visits these are shorter visits but you’re again seeing the differences across their style. Clearly visit number 24, the provider spends much more time looking at the patient, very little time looking at the computer. With visit five, which is 501, provider one, he looked liked he had fewer mouse clicks and the visit was short. A lot of the time was spent looking at the computer and just sort of multitasking going back and forth. Very short glances to the computer and then back to the patient. If you look at the sequence below that the next five visits obviously you see much more interaction with the computer so a lot of blue with some sustained green activity or hardly any green activity in some segments. There are minutes go by before the provider turns away from the computer to address the patient. 

I wanted to stop here and ask our audience to chime in what do you think would be the variability factors?

Moderator:Thank you.

Dr. Zia Agha:[ overtalking]

Moderator:Sorry go ahead.

Dr. Zia Agha:Go ahead yeah. 

Moderator:For audience members you’ll see some annotation tools in the upper left hand corner. Click on that capital T and then you can click anywhere on the screen and start typing your responses. 

[00:22:07 - 00:22:17 waiting for responses]

Moderator:Give me one more second I might need to change a setting here looks like. Okay. One second. Thank you for your patience everybody. Okay now you should be able to click right on the screen. Click the capital T and then click right on the screen and you can type your examples there. Thank you looks like some people have got the hang of it. Might have to move around your responses a little bit so we can see all of them. Once again you’re just going to click on that capital T in the upper left hand corner. And then you can click anywhere on the screen and begin typing your response. 

[00:23:00 - 00:23:14 waiting for responses]

Moderator:Dr. Agha just let me know when you’d like to move on.

Dr. Zia Agha:So I’m going to spend some time discussing these comments from our audience. Obviously that’s my goal is to try to discuss them. These are all the things that we have thought of in our own study and I’m seeing some new things also. So I would like to have a discussion of this at least from my perspective and then we can probably address some of these things in the Q&A session later. 

So I’m going to go through some of these—as they are showing up on the screen. Type of visit and context of visit of course. So in terms of the type of visit and context I think what they’re talking about here is whether this is a new patient visit or an old, followup visit. Most of these are followup visits. We’re able to control that. Clearly even within followup visits there are more complex visits that have three or four chronic problems that need to be discussed versus a simple visit where there’s only one acute or chronic condition that needs to be taken care of. We have looked at that information and in terms of trying to quantify based on the CPT codes that are assigned to these visits but there isn’t much discrimination in that value. Most of these visits are coded as an intermediate complexity level by the providers. We have also looked at the number of medical problems the patient has in terms of like number of medications and there is definitely some correlation between the complexity of the visit and the length of visit. [inaud.]

Time of day. That’s an interesting point. I haven’t looked at that and I think the assumption here is that maybe providers who had visits at the end of the day were now getting tired and writing shorter notes so maybe they had less interaction but that’s a good point. I will look at that. 

Provider comfort with technology. That’s important. When we did our inclusion criteria we made sure that every provider in our study had been, at least for one full year in the VA system i.e. they had one year of experience of interacting with CPRS. Clearly there are some providers who are much more comfortable with technology and have very sophisticated technology users versus some that are not. 

Specialty type. These are all primary care. Although within primary care we had nurse practitioners and physicians but there was no difference between those two systems of care.

Communications. Now that’s a really important one. There was some providers who basically really did our focus group and we could talk with them and said so I cannot focus on the patient. I cannot communicate with the patient if I have to interact with EHR. Hence I defer my EHR workload and do it after the visit. I just write very brief notes and do it there. The other thing, the variability in providers skill in using the EHR, one of the things I’m focusing there is the ability to take notes in real time while still paying attention to the patient. So clearly those people who are able to type really fast and do touch typing and not have to look at the screen are able to engage better with the patient. 

Provider’s interpersonal style. I think that’s the same as the communication style that you are talking about. Some providers are very patient focused and refuse to let the EHR get in the way and others are trying to juggle both the EHR and the communication with the patient.

Age of the clinician. We did not find much difference as most of our clinicians are in a very narrow age range. 

Use of templates versus pre-text. So we did see that most of the providers in our sampling were using some form of template or they were doing a cut and paste of a previous note and then editing it. There are very few providers who wrote the note de novo completely from free text. Now the complexity of the templates vary. Some templates are very basic with just a SOAP header of SOAP and then the data was imported and they fill in the blanks and some templates are much more involved. So clearly documentation style of the provider is an important factor. Some providers have much detailed documentation style and they address every preventive measure, past medical history, the [inaud.] systems in their notes where other providers have more brief notes. 

We also wanted to look how much time to providers spend outside of the visit completing their notes or preparing their notes. While we found certain styles, different providers come in early in the morning, maybe at 7 a.m. and prewrite some of their notes or review their notes to get themselves prepped so the have less time in the visit, interacting with the EHR. There are some providers who will take brief notes, take some paper notes and then fill in the blanks later. When we discussed the difference why they adopted both methods the most common concern from ot preparing the visit note beforehand but providers were essentially, just time. I don’t have time to do it. Why don’t they defer most of this work to after the visit and some providers felt if they did that they would forget information. They would forget relevant pieces of data and also make mistakes so there were concerns for safety. Hence they wanted to do all the EHR work in real time. 

All right so I think you got really points. I probably haven’t addressed all of them but we can leave some of them for the Q&A session.

Moderator:Thank you. I’ll switch back to your slides now. I apologize we got...got the wrong ones up.

Dr. Zia Agha:All right.

Moderator:Just one second. Thanks for your patience everybody. All right they should be up in just a moment. 

Dr. Zia Agha:Okay. Just like we saw variability in the amount of time physicians spent on EHR we were also interested in whether the physician shared the EHR screen with their patients. Either sharing the screen physically, showing them what’s going on the computer or sharing it verbally. I bring the patient’s attention to something that they’re doing on the computer. For instance the provider may say okay now I’m looking up your labs or I’m entering your medication orders. Using some kind of way of engaging the patient while they’re doing the computer activity. If you look the number of times providers shared the EHR, so first of all, all providers did not share the EHR. Provider number 18, 20, 19, 23, 15 and all three that really had any significant instances of sharing the EHR and the width of the column shows the number of visits in which they did it. So some providers pretty much did it in every visit. So provider 18 shared the EHR screen in every visit with his patient. Some providers do it in one of their five or six visits. When we asked the providers why they like to share and what they like to share with their patients most of the time they liked to share lab data and they liked to turn the screen to the patient and show the patient some of the test results. They also like to review medications with patients together to help reconfirm medications. Then the next one is notes. They will point to something in the notes and ask the patient to confirm it. The other screens are not shared obviously because it’s mostly administrative stuff and orders.  Now within labs and meds we asked the providers why don’t they share the screen more often and one of the issues that was brought up was that the CPRS, the way the user interface is set up, is not very graphical, it’s not very colorful, it’s not intuitive. They look around it’s texty,  they tend to find it less useful to share information with the patients during the visit. 

Okay let’s talk about another area that we explored which was the transitions between EHR screens. So because we have the time-stamp data for all these visits and all these actions we are able to look at not only which screens the provider works in but where do they go from one screen to the next. So this is a diagram showing the transitions between EHR screens and this is a sub-sample of all visits. We’re not showing all the raw data here because there was literally thousands of different transitions. This is the top 80% of transitions. Can you guys still hear me?

Moderator:Yeah.

Dr. Zia Agha:Okay. I just got an incoming call that I have to decline now so I can stay with you.

Molly:We appreciate you staying with us.

Dr. Zia Agha:So as you see here in the center of the graphic is the note section. It seems like that—most of the work is happening in the notes and then from notes providers will wander off to reports, labs, consults, cover sheet, orders and then come back into notes. This requires [inaud.] CPRS because you can’t have multiple windows live, it requires opening and closing of windows. So that’s one issue The other reason is why do providers have to go to these places to get information is because it’s required for any task that you’re documenting or doing within our EHR, we need to search information from different areas. For example let’s say a patient is on a certain medication like an Ace inhibitor and I’m going to be changing that medication or renewing it, I need to go look up the patients graph and order his labs. I want to make sure that he doesn’t have a need of something else. I want to look at his kidney function and so it’s requiring us to go into different areas and pulling that information together. You can look at the similar transitions but now by viewing or ordering activities and again, you see that notes is the central hub. Most of the activity happens in the notes and then providers go off to view information or to interact with the EHR and order information in different sections.  So this multitasking we were talking about was within the EHR. Now you can of course look at multitasking across all the channels of data that we have collected. So in this interesting graphic that I tried to communicate four different things and this is for one visit. We chose a particularly busy visit to make a point here. On the bottom level is the communication channel. The bottom graph is showing who is talking to whom and this is in real-time. This was a visit length of ten minutes, 20 minutes, 20 minutes, 40 minutes, and 50 minutes showed. Green is when the patient is talking. Blue is when the provider is talking. Black is under the pause, nobody is talking and red is when they’re both talking together at the same time. So you can clearly see that there’s a lot of back and forth and a lot of communication happening. There’s hardly any silence in this visit. On the bar above that they’re showing the physicians non-verbal behavior data and again blue is geared toward the computer and green geared toward the patient and light blue is physical exam. You will see that the alternating gears between [inaud.] the patient is talking at beginning of the visit. The physician is engaged with the computer possibly entering all of this data that the patient is telling him and on top of that, the third level, I’m showing the EHR mouse clicks by their context. Rather they are interacting in notes, labs, and so forth. So red is notes, yellow is orders, green is lab and blue is medication. The labs graphic above that, doesn’t show really well, is essentially just for the order entry mouse click and it’s showing you an order of how many events or how many tasks were completed. So the height of the bar tells you how many medications or how many orders were placed by the provider. Let’s not focus on that. Let’s focus on the three bars that are below the bottom three bars and you can clearly see now that there is tremendous amount of multitasking happening during this visit. Both in terms of paying attention to the patients need and communicating with them, interacting with the EHR and then specifically ordering medications or reviewing information such as labs, which is information retrieval. What does this mean for us? Explanations. Well in our new study they are really interested in studying how does this back and forth and multitasking introduce cognitive challenges. So is there sort of cognitive load that is being developed which potentially leads to missed opportunities or mistakes or safety issues. When we talk about health safety we always have traditionally focused on safety from the point of view of complete data. Mistakes, drug interactions. Here we are sort of almost looking up a new type of safety so when physicians are very heavily engaged with the EHR there may be a safety issue of they may not be listening to all of the information that the patient is providing to them. They may not be getting all the non-verbal queues from the patient. They have maybe missed opportunities for picking up on things or picking up on communication or affect related issues. Also we have seen in our visits in the video just missed information. Patient may have told you something, you’re too busy doing an order and you forgot to record that. The other thing of interest is when you look at these multitasking episodes is that there is a disconnection sometimes that happens. I’ll give you an example of how that can happen. So you may be talking to a patient about his heart and the fact that he needs to take cholesterol medication and lower his cholesterol. At that time you start entering his cholesterol medication order into the computer. While you’re ordering the cholesterol medication order, the patient may decide to move onto the next topic of interest for him and may start discussing his depression with you or his knee pain. You haven’t finished the cholesterol order entry and so you’re now focused in two different areas almost. You’re trying to listen with one ear on his new complaint while still addressing the prior complaint that you started working on. This also poses an opportunity for mistakes and safety issues that we want to explore in our new study. 

One way of looking EHR activity is also not just looking at the number clicks but the time-at-task so we have done that in this study. What I want to highlight here is the time-at-task. If you look at this graphic the left bar is the mouse clicks, the right bar is based on time-at-task. So if you look at most of these visits even though we said that most of the mouse clicks happen in notes, in term of time-at-task the writers spend even more time within the notes section because mouse clicks is not the only way of tracking, there’s a lot of typing and reading. So literally while a lot of mouse clicks happen in orders, orders are more efficient and require less time than notes and respectively the bar is smaller. This is just an example of how you can measure both a count of mouse clicks but also time-at-task to help do some analysis.

I’m about to wrap up. I’m going to show a few more slides in order entry activity and how we have some coding and analysis with that. Here’s an example of looking at order entry sequences that are aligned along the timeline of the visit. Essentially in this sequence we taking all the mouse clicks except we’ve left the ones that are related to physicians ordering activity. Here’s another way of taking that same data but putting it in counts. Each of these rows is one visit. The numbers, and I’m going to focus on 023-04. The mouse clicks that are denoted are just for order entries for medications. So number one happened five times, so it’s five clicks to order one medication. Then the number two is when he starts ordering the second medication and there are another 17 clicks to complete that. So 22 clicks to order two medications. We can go to visit number 028-05. Total number of 165 mouse clicks to order six medications. Each medication is taking close to 20 mouse clicks in this instance. So you can see the sequence of clicks. All the ones are for the first medications, all the two’s are for the second medication and so forth. This gives us the opportunity to both identify certain medication order entry screens that are more problematic than others but also it gives you the sense of the overall work load for something that physicians do very frequently, which is ordering medication.

Now we can look at things like labs, imaging and reminders and of course similar patterns, multiple mouse clicks to conduct one task and sometimes multiple tasks are conducted. You can present this data as summaries for the full visit giving you the number of mouse clicks per unit ordered and there’ a linear relationship obviously, the number of units of medications you order the more mouse clicks you’re going require. The slope of this graphic tells you the efficiency. So medications require many more mouse clicks per unit ordering than labs. Labs require many fewer mouse clicks per unit order entry. 

So how are we planning on using some of this data? One of the ways we are using this data is in my role as helping with efforts that has informatics initiative. In the HI2 initiative we have taken some of this data and help to drive the re-design of the EHR that we are working in. I’m focusing on working in the area of developing better order entry systems and better documentation systems because to us it seems like, one of the two areas where there’s more interaction happening during the visit. Obviously when you talk about notes the biggest problem is that writer is now dealing with an instructed ASCII file. You want to be able to have a structure in the clinical notes. When you have structure things can become searchable. They can be linked. Things that have auto completion and you don’t need to copy and paste information, redundancy. Those reductant notes that we see in our office is that it’s a lot of work but also it creates bad information. Most doctors these days when they read a clinical note have to really hunt hard to find the two lines of important information versus 80 lines of whatever else has been put in there. We want to automate the direct entry of orders. We don’t believe in the pull down list or menu driven system. We are working with [inaud.] driven system where we imagine a physician could just write an order like he used write on a prescription pad and using some smart technology that information will be cross dated into an order that can be fulfilled by a computer. That would require parsing the field that the physician is writing or dictating into the computer, matching them to certain data bases like RxNorm, it puts the medication order entry, running some business logic and business rules, making sure all the checks—or validity and completeness are happening. All of that can be done on the backend with today’s IT infrastructure. We’re also helping, at least understand better ways of visualization of health care data. At the simplest level just better formatting will help you enable to share information better with patients and with colleagues. Also thinking about putting data in some sort of timeline browser where information is gathered in different sections such as that in imaging.  So the idea is that healthcare information is contextualized best on a timeline because patients medical history happen on a timeline and all the information should be able to browsed in a timeline browser that allows you to zoom in or zoom out into any specific event. Most of this work right now is at the prototype stage. I’d be happy to share some our findings at a later date in that effort. 

The other thing I wanted to talk about is while we were doing the usability studies we realized the challenges of doing this on a real time basis because you have to do a lot of coding and capturing data using usability software and then contextualizing it to the clinical work. One of the projects that I’m working with VA today is trying to develop a usability analytic platform, which essentially if you think of it, it would be monitoring in real-time all the user interactions that happen with an EHR and contextualizing graph to some degree and automating some of these task analyses and interactions in real time for us. It will be the equivalent of a click stream analysis that Google Analytics does on its pages. So that’s a very interesting technology that we are working on trying to set up the metrics that would be available to the side of a analytic platform but also how to connect that data and contextualized it. I’m going to stop here. We have around 12 minutes left for discussion. 

Moderator:I just want to make a quick announcement before we get started. For those of you that joined us late if you have any questions or comments please use the Q&A box in the upper right hand corner to submit those and we will get to them in the order that they are received and I will turn it over to you now Joanna. I’m sorry Joanne.

Joanne Stevens:Thank you Molly. Thank you Dr. Agha. The presentation and the work that you’re doing is extremely interesting and I would imagine that a lot of physicians and other staff are very interested in trying to streamline their work, as well as, this is just my own comment, taking care of the patient at the same time. As people are thinking about what they’d like to write in, actually we do have a question that I will read. Let me just open up the screen. Could you talk about the data visualization software you used. What did you use?

Dr. Zia Agha:Yeah. So the data visualization that we showed were made in Mathematica Dr. Alan Calvitti is our sort of main statistician, data visualization expert. It required a lot of work on Alan's part. He has to write code in Mathematica to do this so it’s not sort of a plug in play software that you just import data and give me a graphic like this. If anybody is interested in learning more about it I can put them in touch with Alan who really is the expert in doing this.

Joanne Stevens:Thank you. Can you talk about how you were able to obtain the time stamp in the context where of where on the record they were and how you were able to coordinate that with the communication?

Dr. Zia Agha:Yeah. Good question. So the time stamps for what’s happening in the medical record are available through the Morae software that records this video of the screen and the context and sort of puts them on a similar timeline. Contextualizing them is not done by Morae that’s done by human coder. So when we are in Morae we see an action happen on a medication. We have to note okay this is medication one. So we have developed an extensive coding scheme and code these events. The reason we can code them is because we’re actually watching the screen briefly. So we’ve got a screen capture of the whole event plus we have the audio and video of the whole event and by replaying those two things together you can recreate all the events in the visit and code for them. Similarly the second part of the question was how do you contextualized what’s happening in the room. Again, we’re able to take the video and audio stream of what’s happening in the room, put it on the same screen as what’s happening in the computer, time align it and review it as one file. In our new study we’re actually adding two more tools, we are looking at eye tracking and really using connect for motion and movement. We’re using a new software that allows us to now look at four of these trails of data on the same timeline fashion.

Joanne Stevens:Okay. Thank you. Another question, first a comment. Partly the EHR will only show 100 notes at one time. How will the timeline browser improve that function of CPRS?

Dr. Zia Agha:So the timeline browser is a concept we are borrowing from other areas. If you think of a timeline browser and to think of what we call zoomable user interfaces, you can add a current dimension. So if you take, for example, Google Maps. You can zoom out on a Google map of the United States and only see the outlines of the states and maybe the states capitols listed. You can zoom one level in and maybe you’re focusing on one state and you can see all the counties. You can zoom further in and it will re-image the data and give you different details. It may start giving you details down to a street. You can zoom further in to even a house. It’s the same idea. The zoomability of interfaces, imagine a timeline of a patients history but the zoom out view shows you the biggest event, hospital admission and office visits. You can zoom in on to one hospital admission and it’s giving you data for each day now. You can zoom into any one day and it will give you all the data for that one day. Having a zoomable interface gives you the ability to restructure the data that is going to be made visible based on that time frame.

Joanne Stevens:Okay. Thank you. The next question is, this person would be interested to know about CPRS education received by the provider’s amount of time and depth. Were they instructed on how to create personnel quick orders or templates for instance?

Dr. Zia Agha:So you know as our institution, when people join they get a very sort of basic introduction to CPRS. However that’s not to say that doctors don’t pick up on a certain templates or thing. I think a lot of the education is happening at a peer-to-peer level where Dr. Smith has some better skills and is able to share them with his own clinic colleagues. So a lot of the education is really normal. Learning through your mistakes and learning through your small network of colleagues versus specific models or specific course work that would be given on an ongoing basis. The other thing is that CPRS does not have a very intuitive Viki or help function. So if you don’t know something it’s very difficult to go find it. It’s not easy to go to a Viki page or support page to answer a question that you have.

Joanne Stevens:Thank you. The next person indicates your presentation provided a very nice discussion of what happens when physicians interact with the EHR. Can you briefly discuss the relationship between the discourse of the physician-patient interaction as it relates to the physicians EHR use?

Dr. Zia Agha:Yes, I didn’t have time to show those slides. We have, working with Dr. Rick Street and Yunan Chen who are both communication experts. We have done two or three different types of coding scheme for the communication. With Rick Street we are focusing on the discourse analysis. We are seeing what type of communications are the providers engaging in? Is it information sharing? Is it shared decision making? Is it patient centered communication? We focused on rather than coding the whole visit, we coded the high three and low three minutes of each or that the high three are where the most EHR interactions and the low three where there are the lowest interactions. We are definitely finding correlations and that data that then physicians are not engaged with EHR. They are more patient centered, are able to do more patient education versus when they are highly engaged in the EHR. All they are doing is, if they are communicating they are basically gathering information, asking vital medical questions, requesting vital medical information from the patient and not focusing on patient education or communication or empowering the communication angle. We are also looking at trying to ask providers focus group questions and interview questions to look at how they prepare for this so we ask them how that effects communication and we are finding that most physicians, and we’re looking, this not analyzing, what we’re trying to see is those physicians who set an agenda beginning of the visit if they’re able to perform better in both their communication ability but also their EHR work. The hypothesis being that agenda setting helps prepare the visit, makes it more organized and they’re able to better manage both their communication tasks, their physician making task, and their EHR tasks. 

Joanne Stevens:Thank you. The next question is to what extent are providers interested in speaking their notes. Would they find this more efficient than typing and clicking?

Dr. Zia Agha:So in our sample we had three providers who used Dragon Speech. So those providers, we did do some small analysis in looking at their mouse click behaviors and surprisingly their mouse click behaviors are no different. Part of the reason is while they’re speaking into the computer they are documenting information but for all the information retrieval tasks and all the order entry tasks, they still have to use a mouse to move between the menu driven, drop down driven environment.  Now if you were to go to a feedback data entry environment that would be proposing prototype. You could speak an order into the text and it could be converted by a software into an order that could be executed. That is definitely a huge opportunity to use dictation, the ability. There are some providers who disagree with that in the sense that they feel talking to a computer is going to be actually even more destructive to their workflow. So imagine if you’re talking to a computer but not naturally, not just dictating the story but you think, select medication lisinopril choose dose 500, order 30 tablets or three refills. That in itself, that workflow can become more destructive to the communication.

Joanne Stevens:Okay. Thank you. The next person comments this is fascinating research. Can you discuss briefly the IRB process for this. We’re both providers and patients consented?

Dr. Zia Agha:Yes. Both providers and patients were consented. Patients also provided HIPPA authorizations. I think the biggest challenge with IRB concent was getting permissions to do the date logging of the EHR but it was actually the screen capture and mouse click capture but we were able to work with our software vendor to ensure that everything was captured securely and stayed within the VA environment.

Joanne Stevens:Thank you. The next person indicates you think the high flow of three minutes is a kind of “thin ice” method, which is useful but often leads to sampling decisions that are contextual. Do you have a sense of what context e. g. activities those thin slices occur.

Dr. Zia Agha:Yeah. So there’s definitely issues of sampling. We could have randomly sampled this or could have studied the whole with or we could have used the sampling scheme. The reason we went with the high-low sampling scheme was for two reasons. One we are leveraging our data. We have the data on the mouse clicks which tells you the highest or the lowest used history in the visit where’s there’s the highest or lowest interaction with the EHR. We wanted to leverage that information to help answer the question, what happens to the communication during these types of instances? You could decided to do it in a different way easily and our data would allow you to do that. I think that our issue is whenever you do sampling whether the sampling is going to give you a biased result. But in this situation we tried to make as transparent as possible and then the selection of the high and low windows themselves were done by a computer program that we wrote that randomly selected based on a visit, a window of time, that gave you the highest concentration of mouse clicks. So we did try to prevent any bias and sort of cherry picking segments that we were studying but by contextualizing it whether it be the highest interactions with the mouse and the lowers interactions with the mouse click and EHR is a fairly objective measure that we can go after.

Joanne Stevens:Thank you. That concludes the questions that have been written in. I’d like to thank Dr. Agha for taking the time to develop and present this talk. So we’d ask that you please forward any remaining questions to Dr. Agha or to VIReC help desk. virec@va.gov and we will be happy to forward those on. Our next session is scheduled for Tuesday, January 21st. It is online training for providers in prolonged exposure for PTSD. Our speaker is Dr. Ken Rigario. I believe that Molly will be putting up our questionnaire shortly but I would like to thank everyone for attendance and attention and hope to see you next time. Thank you very much.

ModeratorGreat thank you Joanne and thank you Dr. Agha and for our attendees your survey will pop up now so please take a moment and fill it out. Thank you.

1 of 13