VIReC Clinical Informatics Seminar- 12 -Department of Veterans Affairs

August 21, 2012

**Department of Veterans Affairs**

**VIReC Clinical Informatics Seminar**

**Qualitative Analysis of VA Personnel Experiences with the Electronic Medical Record**

**Laura Bonner, Ph.D**

**August 21, 2012**

Moderator:  At this time I would like to introduce our speaker, Laura Bonner, Ph.D.  Dr. Bonner is a clinical psychologist in the VISN 20 Geriatric Research, Education, and Clinical Center known as GRECC and the VA Puget Sound HSR&D Center of Excellence.  Her interests include dementia detection and prevention, evidence-based care for depression and PTSD, and healthcare informatics.  Without further ado, may I present Dr. Bonner.  Thank you.

Dr. Bonner:  Thank you.  Well, I'm excited to be here today speaking about qualitative analysis of VA personnel and their experiences with the electronic health record in the VA.  As was just said, I am a psychologist in the GRECC VISN 20 and also have the opportunity to work with several research teams in HSR&D so I am very fortunate to be part of both of those groups.

And before I get started with my presentation, I want to acknowledge the co-authors of the paper that much of this talk is drawn from: Carol Simons, Louise Parker, Becky Yano, and JoAnn Kirchner.  The paper that I will be referencing was published in *Implementation Science* last year, and I do provide the reference at the end of this talk if anyone wants to read the full paper.

And just to provide some overview of what I'm going to be talking about today, my goals are, first of all, to provide some context for why it's so important to understand more about the VA's electronic health record and why research like this is really interesting to everyone in the healthcare informatics arena; and then more specifically, to increase our understanding of how VA personnel evaluate the electronic health record within the VA, the advantages that they see, the disadvantages or concerns that they have, and the barriers that they see to uptake and implementation that are associated with the VA's electronic health record; and then finally and importantly, to discuss kind of what I would suggest are some potential solutions to address those disadvantages and barriers.

So these screen shots are drawn from various sources, and I wanted to talk about them just briefly to help begin putting the VA's electronic health record into a more broad context.  So the first picture that you see is the VA's own MyHealtheVet.  And MyHealtheVet, as most of you are probably aware, is actually the VA's patient portal that is still kind of under development but that patients have the potential to manage and access some of their own healthcare information.  And we won't be talking today specifically about any data related to the VA's MyHealtheVet, but I wanted to put that in some context, that the electronic health record is a very broad concept.

The second screen shot that we see is from NBC News, and it tells us there's Facebook misconduct, med students cross the line and try and friend their patients.  And I'm sure that it's not just medical students who struggle with the issue of how to appropriately use technology that is very rapidly evolving, that changes, and where there really aren't a lot of rules because the technology hasn't been even in existence before.  Again, we're not going to be talking about Facebook or other social media, but understanding how people interact with technology in the healthcare arena has a lot of implications.

And then finally, down at the bottom the *New York Times* had an article, this is a couple of years ago but still relevant, telling us that doctors and hospitals say goals on computerized records are unrealistic, that although there's been a lot of national attention given to use of electronic health records and kind of in the national debate about healthcare financing that a big push has been made for all of the advantages that go along with electronic health records, but there are some concerns that maybe we're not going about things in the right way.  So again, this is just a context and we'll be talking specifically about the VA's electronic health record.

So as I said, there's been a lot of emphasis lately on kind of both good and maybe potentially not so good things relating to the electronic health record in the VA and in a lot of other settings.  So the electronic health record, very broadly defined, is any way of managing health information using computerized systems.  It can be something very basic like just putting in basic notes about what happened during a clinical visit.  It can be something maybe more sophisticated such as presenting imaging that was done so that I can easily see – if a patient got a CT scan or an MRI, I can easily review those results and many other things as well.

So the electronic health record is a broad concept, and people think that it really has a lot of potential.  The literature is full of the positives, that it can improve the quality of care, that you can help promote guideline-concordant care by, for example, having clinical reminders that remind providers in their very busy day to remember to do certain things, for example, a diabetic foot exam, with a patient who would need that so that it can improve the quality of care, and that because things can be done potentially more quickly, things don't get missed, the patients don't get into trouble further down the line.

And it has the potential at least to improve cost effectiveness.  It is certainly an upfront investment, but it's seen as having the positive impact of improving cost effectiveness down the line.

Also facilitating communication.  We all know that it can be difficult, because people are busy, to get a hold of our colleagues if we have a question or to learn what happened on a different shift from our own.  And so the electronic health record, everyone can look at and multiple providers can access it at the same time.  It can facilitate and improve communication.  And again, that it can empower patients.  It's not just for providers, but patients as well can track their own health information using an electronic health record.

So these are all really wonderful things that have been cited in the research and kind of in marketing literature as well.  But there are – as in any human endeavor, there are some concerns, one of which is that maybe if we have a lot of clinical reminders and a lot of templates that we use, that people will get to be overly reliant on them and not use their good clinical judgment.  And certainly physicians and other providers say, "I went to school.  I have a lot of training that I have developed my own clinical judgment, and I also know my patients a lot better than the computer does."  And so people are concerned about being kind of encouraged to use templated care rather than what they think is most appropriate for the individual patient that they're sitting in the room with.  So that is one concern.

Another is time management, that yes, in some ways, there is a potential for the electronic health record to make things more efficient, but it does also take time.  You do have to write a note, you do have to click through a reminder, you do have to sign documents, and all of those things take time, whether you're using a paper chart or an electronic health record.  And perhaps it might even be seen that some things about the electronic health record take more time.  So that's one concern that has been cited in the literature and that people have.

Another concern that people have is if I'm interacting with a computer, then what am I not doing.  Well, I'm not interacting with my patient.  And most people who get into the healthcare field didn't get into this field because of a love of computers so much as really enjoying working with their patients.  And so there's a concern about the loss of interpersonal relationships and, again, that's been described in the literature.

Another thing that has been described in the literature is the possibility for unintended consequences, and these can be very wide ranging and could potentially have some really negative effects.  A really simple example might be if you can order medications that might have some type of interaction and the computer is supposed to catch it but it doesn't.  And so then the provider trusts that there's not going to be an interaction and both of those medications go to the person, it could potentially cause harm.  And there are certainly many other examples that we could think of.

So this kind of very brief review of the literature shows that while people are excited about the good things that the electronic health records can do, we also have some concerns about what might happen as they're being implemented.  So we have a brief poll just so that we know what people have experienced.  Do you or have you used CPRS?  And go ahead and click yes or no.

Moderator:  And the responses are coming in.  Unfortunately, it's not dynamic so you can't see it on your screen.  We're at about 63 percent right now.  I'll give it a few more seconds and then close it out and show the responses here.

Dr. Bonner:  Great.

(Pause)

Dr. Bonner:  Okay.  So about two-thirds of the audience has used CPRS which is the VA's electronic health record.  And I will say I think that, having used CPRS myself as a psychologist, it really has opened my eyes to some concerns that people may have that I was not aware of just from reading the literature.  So I'll try and give some information based on clinical experiences.

So again, just to provide some context specifically related to the VA, the VA's electronic health record was first implemented in a large scale about 15 to 20 years ago beginning in the mid-1990s.  There had been various electronic health components before.  Various sites had different electronic health record offerings.  But throughout the VA kind of universally beginning in the mid-90s there was a decision made to really create a comprehensive electronic health record, CPRS, the computerized patient record system.  And today it is, for those of you who are clinicians, you know this, that it is essentially universal, that really it's hard to find a site or a provider who doesn't interact with it on pretty much a daily basis.

And it provides basic services like documentation.  I can put in a note and I can see all the notes from the patient's other providers at my facility and at other facilities as well.  You have the option to search notes if a patient has been seen at another VA as well.  It also provides things like decision support.  So for example, for myself as a psychologist, if I'm working with a patient who may have a risk for suicide, there is a templated note that I can use to document what are the patient's risk factors for suicide, what are the protective factors that are going on in their life, and then use that to kind of help me determine, well, what is their risk and what is the kind of plan that we need to be thinking of to keep this patient safe.

So some decision support, as well as support for more kind of basic things like billing, scheduling, placing orders.  So it's quite comprehensive.  And I'm not pretending that this is an overview of every feature of CPRS, but it's a comprehensive electronic health record.  And it has actually been cited as something that makes the VA really good.  The VA is able to provide kind of coordinated care because all providers have access to this health record.

And for those of you who do use CPRS, I would like to know what is your favorite feature of CPRS, just for fun.  And your options are tracking patient progress over time, exchanging information with other providers, placing orders and consults, or other.  And unfortunately, if it's other, we don't have any [use] to specify what that is, but just for fun, I'd like to see what people think.

Moderator:  And I actually went a little different from how you set it up and I'm letting people – they can actually pick more than one option.  So it may add up to more than a hundred percent.  Just so you're not confused, that's why that happened.  So we'll give it a few more seconds, and we're at about 48 percent right now.  So give it a few more seconds and then close it out and show the results.

(Pause)

Dr. Bonner:  Okay, so it looks like tracking patient progress over time is the winner as a favorite feature, and then about 24 percent for exchanging information with other providers and placing orders and consults.  And then about a third of folks said other so I wonder what that might be, but that is interesting.  So people like to be able to track patients over time.

So I want to provide some context for the data that were collected in this particular project and kind of how we came to evaluate the respondents to this project and their relationship with the electronic health record.  So to do that we have to go back about ten years or so to the beginning of TIDES.  I bet a lot of people are familiar with TIDES because it has really become pretty prominent throughout the VA.  But TIDES was, for those who are not aware, was a quality improvement project for depression care that promoted collaborative care for depression.  And it was first rolled out in 10 VAMCs or larger CBOCs throughout three VISNs.

So it was a pretty big quality improvement project, although of course, it's grown a lot since then.  But again, specifically collaborative care for depression.  And it featured management of patients with depression within primary care with some provider education so that providers had kind of ongoing support, so that providers had some training in how to manage patients with depression, when it was appropriate to do so, when it might be more appropriate to refer a patient to specialty mental healthcare or to receive psychotherapy.  Another really important component of TIDES was the nurse care manager who monitored patients over time and spent about a six-month period calling patients – it was almost always done by phone – calling patients and tracking their progress, providing patient education, patient activation, and addressing things like barriers to medication adherence or to making other changes that the patients needed to make in their lives.

So TIDES is kind of the broader context.  The parent study from which the paper that I'm referencing and the data I'm using in this talk was drawn – was the COVES project.  COVES stands for Cost and Value of Evidence-based Solutions for depression care, and it was an evaluation of the TIDES implementation effort.  We did have both a cost analysis that was led by Dr. Fen Liu in Seattle and a qualitative analysis led by Dr. Kirchner in Little Rock.

And the participants in this qualitative analysis were 72 front-line clinicians and clinical managers, nurses, other stakeholders in the VA.  Sixty-seven interviews were done in person and five, just because of travel and schedules and other issues, had to be done by phone.  And we included physicians, nurses, psychologists, and administrators.  We did not, importantly, break results down by participant type so in the results that I'm going to be presenting, we didn't specify themes that were mentioned mostly by physicians or mostly by nurses for several reasons.  One is that there was not really a strong pattern that emerged.  It didn't seem like all physicians kind of had one opinion or all psychologists had another opinion.  And also although 72 participants is a pretty robust sample size for a qualitative study, when you start separating people into categories, of course, you do get a smaller sample size for each group.  So people sometimes ask about that, and I wanted to explain our reasoning there.

So as I said, these were interviews that were done in a semi-structured format so there were certain questions that we wanted to get answered for every single participant, but the interviewers had the flexibility to follow up on things that the participant mentioned that maybe we had not even thought of and to kind of probe a little bit more and ask them some more in-depth questions about that.  The interviewers included experienced health services researchers so psychiatrist, psychologist, social worker and two other health services researchers.  And we did interview VA patients as part of COVES, but they very rarely mentioned informatics in any context.  And often it was simply I don't even have a computer or I don't even use the internet.  So we didn't include VA patient data in this paper or in this presentation because they just really did not have a lot to say about informatics, although they had information about other questions that we were interested in.

So some sample questions – and again, different interviews would have followed somewhat different paths depending on what the interviewee was saying.  But some sample questions would be how similar or dissimilar do you believe depression is to other chronic illnesses such as diabetes and heart disease which is a really important question.  It tells us a lot about the participants' views of the appropriateness of collaborative care for depression and the ease of management in primary care.  Another question would be can you tell me what your facility did as part of the TIDES program.  And certainly we expected that most people who were participating in this study would be aware of TIDES, but there are certainly different levels of awareness and detail for different participants.  And did you ever have a suggestion for how to improve the program or something that you thought might not have been working.  A very broad question and people could have answered in a way that was related to depression care or to aspects of the structure at their site or to the informatics or all of the above.  So these are some of the questions that might have been asked.

The qualitative analysis – and for those who are familiar with or have done some qualitative analysis know that it is a time-consuming process but that you get really, really rich data from the interviews that you do.  So several of the research teams read all of the interview transcripts and derived some themes, some things that were mentioned repeatedly by a lot of the respondents, and we developed 22 top-level codes.  And one of those codes was informatics.  The other codes could relate to things like views of depression or views of quality improvement or views of the organizational dynamics at their particular site.  So a pretty broad coding scheme, but one of those was informatics.  And again, that could be informatics related specifically to TIDES and depression or any other aspect of informatics that could be mentioned.

We analyzed the interviews as a whole, not question by question but really everything that was said in the interview.  And we used Atlas.ti as a software program to manage the coding process.  And the coding process is really – I feel like I always need to emphasize this – it's really done by the researchers.  It's not done by Atlas.ti which is a wonderful tool, but it's no more magic than any other software program.  It is a good tool to help you manage the work that you are doing with your qualitative analysis.

The coding team included three of the authors on the paper that I cited and another member of the project team.  And then the subcoding, just for the informatics code, we developed subcodes relating to different aspects of informatics because we knew that informatics is something kind of unique that people were mentioning.  And that was myself and Carol Simons, the second author of this paper as well.

We reviewed transcripts – after each person had coded about half the transcripts, we reviewed them for coding consistency, which was pretty high, and then talked about codes where – or quotes where we perhaps had disagreed.  And then we really got very detailed and conducted word searches to ensure that we were not missing any content.  For example, email is something that came up when people were talking about informatics.  And so we searched just all of the text for email and made sure that every single utterance that involved the word email had associated with it the code for informatics.  Sometimes people ask can a single quote have more than one code associated with it, and yes, that certainly can be done and often is because a single statement can have really rich context and provide a lot of information maybe about informatics and about depression care and about quality improvement all mentioned in one sentence.  So yes, one block of text can have more than one code or more than one subcode associated with it.  But we were very careful to make sure that we were coding accurately.

And so what did we find?  Well, we found that people talked about implementation in general, implementation of quality improvement projects, they talked about depression, a lot of things, and they talked about the role of informatics.  Informatics support was used in TIDES.  For example, the care managers that I mentioned had to use different informatics tools, spreadsheets, etc. to track when patients were due for follow-up calls.  But the discussion of informatics wasn't necessarily about TIDES.  It could have been about any aspect of VA informatics.  And when these data were collected, it was about 2004 so the VA's electronic health record was, at that time, really pretty well established and so most people were pretty familiar with it.

And let's start off with the benefits that people saw.  So our participants really liked that the VA's electronic health record could speed documentation, could improve communication among members of the treatment team, and improve quality of care.  So they saw some real advantages to having the electronic health record and those were some of them, which is pretty consistent with what had been already published in the literature.  But they had some concerns which were also, again, somewhat consistent with what had been published in the literature before.  And they had some pretty strong statements that they thought the informatics system could or did interfere with the provider/patient relationship and they were concerned about that, and also that it had the potential at least to take time away from clinical tasks because of the need to interact with the electronic health record.

And I'm going to use some of the actual data that people told us to illustrate these points.  So the first point about communication, someone said "Our computerized record system makes it awfully easy for the mental health and primary care to work with the other," which, of course, is what we wanted them to be doing in a collaborative care setting.  So that was encouraging, that the computerized record system was there, people could use it to communicate and work together well.  So that was a positive thing to hear.

They also thought that there was some really good advantages for quality of care.  The clinical reminder reminds us to talk with people and ask are you feeling depressed.  If we didn't have the reminders, we may not take the time to do that, particularly – and this is somewhat of my own opinion, but I would say particularly with something like depression where people may be a little bit reluctant to talk about it.  But if the clinical reminder is there – and it's just like any other clinical reminder, that patients are used to answering these questions, providers are used to kind of routinely asking them – it can really open up a conversation and ask the question and get an answer and really learn something important for the patient's care.

Another statement was "You can have a consult form that asks questions or builds in information and has force fields.  So you say here are the diagnostic criteria, here are the screening criteria, has your patient met these, have you done this kind of assessment, do they have contra-indications, have you tried this initial intervention."  So again, from my own perspective working in a consult clinic, you want to not have every patient in the hospital referred to your clinic; you want patients who are really in need of your services come into your clinic.  And so if it's something that can be managed, for example, have you tried this initial intervention, have you done this kind of assessment, if you ask those questions of the referring provider when they're putting in a consult, they may have the confidence to then kind of manage that patient themselves because they realize I can do that kind of assessment, I can try this initial intervention, and then refer patients where, yeah, I've done that assessment, I've done this intervention and it really hasn't worked out, refer those kind of more complicated patients to the specialty care clinic.  So as I think this participant was stating, right in the consult form you can be providing almost a mini-provider education just by the way that you write a consult form.  And that's very helpful for the primary care provider and the specialist and does help them at least potentially work together well.

But what were some of the barriers?  What were some of the concerns that people had?  Well, time.  Always, universally throughout healthcare we are concerned about time.  And so someone said, "CPRS is great, but it takes time to use.  Providers have to see very complicated patients in 20 minutes, and with every point and click on a computer, it's less time they spend with a patient.  They generally just want to take care of the patients."  And of course, as I was saying before, people get drawn into the healthcare field because they do want to take care of patients.  And so it does have the potential, as stated by this participant, to draw attention away from the patient because you're focusing on the computer screen.  Along some of the same lines is "... how many clicks it took to take care of a diabetic patient.  I clicked through all of the reminders.  It's hundreds."  So again, every time you're clicking on the computer, you're not devoting your full attention to the patient, and that was a concern that a participant had.

Another concern was not so much time but the impersonal relationships that could be engendered.  "All of these checklists, clerks should be doing that.  Doctors need to sit there and look someone in the eye, what's really bothering you," which is in contrast to the quote that I showed a few slides ago where someone said the checklists really open up the conversation and get us to ask questions we may not have even asked.  But this participant had a little bit of a different take and said the checklists are fine but someone else could do them.  The doctors need to forget the checklists and look at the patient and open up a conversation that way.  So two different views.

Another person, this person was a nurse who said, "I'm asking physicians to know me and trust me simply by what they have read in my progress notes and most of them have not met me personally."  So if you are working with a patient maybe in a tele-health setting where there's someone who's monitoring the patient but not at the same site as their primary care provider, there has to be a certain level of trust between the primary care provider and the nurse or whoever else is providing healthcare kind of remotely.  And it's a little bit harder to have a trusting relationship with someone where you don't have a personal relationship so that's a concern that was expressed.

And an additional concern was kind of misuse of the communication tool.  Someone said, "There's been a couple of times that providers will respond back to me as if they're forgetting that they're in a patient's medical record and will say what would you like me to do where that's not appropriate."  We all know that there's conversations that we would have in the hallway or – well, hopefully not in the hallway but metaphorically, but conversations that we would have, kind of a curbside consult with patients, where we wouldn't use the same kind of informal shorthand in the patient's formal medical record because we know that that's a really important permanent document.  And so I think this person was saying that people were getting lulled by the fact that it was almost like a secure messaging system and they could ask questions by kind of adding people on as a co-signer to notes but some people were forgetting that these innocent questions are really going to be permanently part of the patient's medical record.  And so perhaps an unintended consequence that people were not remembering the formal language that they should have been using.

Someone else was not really complaining about the medical record itself but more about the context in which it was being implemented.  And they said, "We don't flunk in depression screening ..." – specifically about depression – "... but we flunk in follow up," which is something that has kind of been supported by some research literature that depression screening is great and if the electronic health record prompts you to do that, that's wonderful.  But that's not enough.  To really have high-quality care, you need to move forward and care for the patient from screening through follow up through appropriate treatment.  So that was a concern that one person had.

Another person said, "If the providers are overwhelmed with clinical reminders, they become somewhat numb to them.  It's also a system issue."  So again, you have to be selective with the reminders or it kind of becomes something that people may not notice, that oh, this person screened positive for whatever, I need to really follow up on that and get on it.  If there's too many positives, it's hard for the provider to figure out where to best direct their attention.

So I have a summary table that summarizes both the positives that people have seen and also some of the concerns that they have.  And this is adapted from the paper that we published.  It's not exactly as in the paper, but it's quite similar.  So some valued attributes of the electronic health record: people like that they could have asynchronous communication, that I can write a note right now and maybe the patient's psychiatrist or neurologist needs to see it, they're not available until tomorrow, but they can view at a time that's convenient for them.  So we can communicate more rapidly than kind of waiting outside their door until they're free to talk to them.  But speaking of time, it takes time to go through reminders and that can be a bit of a burden.

Another valued attribute was that there's support for appropriate documentation, not just writing a note but a template that tells me exactly what I need to put in so that it's useful data for other people who are going to read that note.  On the other hand, maybe there's less appropriate conversation that can get into the patient's medical record and so everyone needs to be careful and mindful about what they're writing.

Another valued attribute, people can easily alert other providers about a patient's status.  However, I can alert someone, will they trust me if they don't know me, if they're only reading this note.  That's a concern.

Reminders can prompt providers to initiate important conversations, and we talked about that, just having that reminder can get the conversation going.  But on the other hand, if you're focused more on clicking kind of mechanically through the questions and a reminder, maybe you lose some of the interpersonal connection that the patient values and that the provider might value.

And then finally that structured consults and reminders give us guidance about what is the state of the evidence, what we should really be doing, but we can't just rely on that.  We need to use evidence-based care as a process.  We screen and we treat and we follow up with the patient.

And some suggestions that I personally recommend based on my reading of the literature, people do have concerns about time.  And so if you have concerns about time or if you're working with someone who has concerns about time and specifically how the electronic health record might impact your usage of time, it's important to remember that there are some ways in which the electronic health record makes things more efficient and potentially may save time for us.  For example, as I said, I don't have to wait outside someone's door until they're ready for me to ask them a question about a patient.  I can add them on as a co-signer to a note and then they can get the information and tell me their thoughts.

Templated notes so that I may be able to just click yes or no to certain symptoms rather than relying on my memory to put those into the notes.  So it can be more efficient and potentially higher quality.

Another barrier is concerns about the effect on our relationships with patients.  And one thing that has been cited in the literature, it seems kind of simple but it's important, is if you're using the computer, how can you physically position that computer so that you don't have your back turned to the patient the whole time and looking at the computer, can you put the computer kind of to the side so that you are looking at the patient and at the computer and it's not as disruptive.  And also think about the positive aspects of having clinical reminders, that they might open up important conversations so remembering that, yes, this is a concern about our relationships with patients but that there are some positive aspects of the electronic health record.  And when people have concerns about the effects of the electronic health record on our relationship with colleagues, like are people putting things that maybe shouldn't go in the electronic health record in there or are people not as able to trust each other because they don't have an in-person relationship, again, emphasize the benefits of improved communication, that everyone can see everyone else's notes and so everyone can know what is going on with the patient.

This slide is very similar to one that I showed at the very beginning of this talk, and I said that there were some things that had been cited in the literature as benefits or potential benefits of electronic health records and then some things that had been cited in the literature as concerns.  So the lines that are highlighted in red are information that was given by our participants that kind of reinforces what we had already found in the literature.  So there's kind of widespread concern that the electronic health record – or widespread feeling that the electronic health record can improve the quality of care, and that's something that our respondents mentioned.  That it can improve cost effectiveness, which was not mentioned as much by our respondents for whatever reason.  They weren't specifically asked about that.  That it can facilitate communication and again, yes, our respondents thought, yes, it could.  And that it could empower patients.  Again, we didn't use patient data and patients didn't really provide much data about that.

And some concerns in the literature that providers might over rely on templates and not as much on kind of interpersonal relationships, and that was mentioned by our respondents.  That time management could become an issue, certainly that was mentioned.  And that interpersonal relationships with other providers or patients could be affected, and that was certainly mentioned.  Our participants didn't talk as much about unintended consequences, but that is something that's out there in the literature certainly.

And where are we going in the future?  Well, we know, we can easily predict, that electronic health records will continue to be important and increasingly important.  And we know that our relationship with technology is evolving, that the electronic health record will continue to change, that patients' expectations will change, things like when the data for this study were collected, things like Facebook weren't even around or widely used, but now there's a whole concern about the proper use of social media in the healthcare system.  So we always have to be aware of new issues that emerge.

And so my final statement would be that it's just so important for researchers and funders to prioritize assessment of the relationship between technology and the clinician and patient relationship.  It just is very important because of all the real benefits that people have mentioned and all the real concerns that I'm sure we could all think of additional concerns to that.  But there's both sides to it, and we need to continually assess that.

Before I close, I'd like to acknowledge the other members of our study team: Lisa Rubenstein and Ed Chaney, Fen Liu, Mona Ritchie, Jackie Fickel, and Penny White, and the wonderful participants who gave their time to participate in the COVES interviews, and thank you to QUERI for providing the funding.

The reference for the paper that I've mentioned is provided, as is my email.  If you do want to contact me after the session with any questions, people are certainly welcome to do that.

So this is the end of my presentation, and we do have some time for question and answers.

Moderator:  Thank you very much.  That was excellent.  I have one question.  Often people take the time now to type in their questions.  Question: Can you please describe the subcoding process that you used?  Also did the subcoding team consist of different team members than the coding team?

Dr. Bonner:  Okay.  So that is a really good question.  So to answer the second question first, the subcoding team specifically for that informatics subcode was really myself and Carol Simons.  At the time we were actually working on another informatics-related project which kind of piqued our interest in learning all we could from the COVES data related specifically to informatics.

And so the subcoding process was very similar to the coding process that I described, but it was done by the two of us specifically on quotes that had already been coded as informatics by anyone who was involved in the top-level coding, which we were both involved in as well.  The subcodes involved things like use of email or use of the electronic health record, concerns, barriers, positives, kind of the things that we've been talking about today.  I don't actually have a list of the subcodes in front of me, but those are the kinds of things that we did use as subcodes.

Moderator:  Thank you.  Here is another question and comment: Please re-state how to get these great slides to study further.  Slides, I think, may still be available now.  Maybe Heidi can let us know about that.  In any case, they will be posted on the HSR&D cyber seminar archive, both the internet and intranet.  I think it takes about 24 hours, and people can go to the archive to get them.  Heidi, can people still get a hand out from ...

Heidi:  Yes, there was a link included in the reminder that was sent out this morning, and they can use that link to download the handouts immediately.

Moderator:  Okay.  Great.  I'm waiting for some more questions.  Let me just take a moment to tell our audience what our next session will be.  Our next September session will be Tuesday, September 18.  Speakers will be Drs. Neil Evans and Deyne Bentt.  They're from the Washington, D.C. VA, and their title for their talk is Mobile Display of Patient Data: The iPhone/iPad iHealth Adaptor.

So with that said, I don't have more questions here, Laura.  We could wait another minute, or if people are still formulating their questions, please feel free to follow up with VIReC, the VIReC help desk, at VIReC@va.gov, indicate the questions you have, and we will forward them to Dr. Bonner.

Here is a question: What do you see in the future for nurses and informatics?

Dr. Bonner:  I think that that is a wonderful question.  I think certainly a lot of the informatics development that was done in the VA has involved nurses as clinical applications coordinators or as developers of the software, and I think that we know that nurses are such a key player in so many aspects of the healthcare team.  So I think nowadays in CPRS, nurses enter notes but it may be a little bit limited.  But I certainly see an ongoing role for nurses in development as well as using the health care record.

Moderator:  Okay.  Thank you.  That is – oh, here we go.  What is Dr. Bonner working on now?

Dr. Bonner:  That may take another hour to answer that question, but several different projects.  One of them is a dementia prevention project which provides care management for patients at high risk of dementia because of vascular factors like diabetes and hypertension.  And interestingly, many of them are depressed as well, and that's kind of an ongoing project that is just winding down, as well as I do have still some ongoing involvement with the TIDES team and with a primary care mental health [degration] and collaborative care for PTSD.  So several different projects.

Moderator:  Keeps you busy.  Here's another question: It seems like the face of VA informatics has changed a lot recently.  There is much more functionality in MyHealtheVet.  There now is secure messaging and a move to develop web applications.  Are you planning a follow-up study for further research in this area?

Dr. Bonner:  At present, I don't have any plans for that, but I think it is very important for someone to do it.  So it would be something that I would do or someone else could do.  I think it's, like I said, such an evolving field that I think that it would be really good to do something like that.

Moderator:  I think I have to tell you, Laura, that your second author submitted that question.

Dr. Bonner:  ... know.

Moderator:  Anyway, could be some more questions.  I don't have any yet.  Oh, here we go.  Getting down to – why did you choose to publish your study in an open-access journal?

Dr. Bonner:  I think the fact that it was open access is certainly nice, but I don't think that was really the determining factor for choosing that particular journal.  It seemed like topic wise it was a good fit.

Moderator:  Okay.  Every time I just say um, another question comes in.  No more questions yet.  Let me just take a moment to say when you all exit from GoToWebinar, we would really appreciate it if you would answer – fill in the questionnaire evaluation.  And it looks like no more questions are coming in.  Again, if you do have some questions, submit them to the VIReC help desk, VIReC@va.gov, and we will forward them to Dr. Bonner.  Thank you so much for a lovely talk today, and hopefully we'll see you all in a month.  Bye bye.

**[End of Recording]**