**Transcript of Cyberseminar**

**VIReC Clinical Informatics Seminar**

**Using Patient-Facing Kiosks to Support Quality Improvement at Mental Health Clinics**

**Presenters: Amy N. Cohen, PhD; Matthew J. Chinman, PhD; Alison B. Hamilton, PhD, MPH; Fiona Whelan, MS; Alexander S. Young, MD, MSHS**

**September 17, 2013**

*This is an unedited transcript of this session. As such, it may contain omissions or errors due to sound quality or misinterpretation. For clarification or verification of any points in the transcript, please refer to the audio version posted at www.hsrd.research.va.gov/cyberseminars/catalog-archive.cfm or contact the VIReC Help Desk at virec@va.gov.*

Arika:Welcome, everyone. This session is part of a VA Information Resource Centers ongoing Clinical Informatics Cyber Seminar Series. The series’ aims are to provide information about research and quality improvement applications and Clinical Informatics and also inform about approaches for evaluating Clinical Informatics applications. Thank you to CIDER for providing technical and promotional support for this series. Questions will be monitored during the talks in the Q&A portion of Adobe Connect and VIReC will present them to the speaker at the end of this session. A brief evaluation questionnaire will pop up when we close the session. If possible, please stay until the very end and take a few moments to complete it. Let us know if there is a specific topic area or suggested speaker that you would like for us to consider for future sessions.

At this time, I would like to introduce our speaker for today, Dr. Amy Cohen. Dr. Cohen is a Licensed Clinical Psychologist and a core investigator and health services researcher at the VISN Mental Illness Research Education Clinical Center, MIRECC, and at the VA HSR&D Center of Excellence for the Study of Healthcare Provider Behavior. Without further adieu, may I present Dr. Cohen.

Dr. Amy Cohen:Thank you so much, Arika. So I am happy to be here today to be part of the VIReC Clinical Informatics Seminar Series and I am going to be presenting on Using Patient-Facing Kiosks to Support Quality Improvement at Mental Health Clinics. Alex Young, Alison Hamilton, and myself are affiliated with the VISN 22, which is the VA Desert Pacific Mental Illness Research Education and Clinical Center, as well as the VA HSR&D Center of Excellence for the Study of Healthcare Provider Behavior. Our Center of Excellence will be becoming a COIN on October 1 and be the Center for the Study of Healthcare Innovation, Implementation, and Policy. Matthew Chinman, who also worked on this project with us, is now at the VISN 4 MIRECC. And Fiona Whelan is the Department of Biostatistics at UCLA.

So I have four key questions that I hope to cover in our hour together today. How can we work in a partnered way to improve care quality? Can care in specialty mental health even be improved? What is the role of data in quality improvement? And how do we get those data and from whom? We will return to these questions at the end of the hour and, hopefully, we will feel pretty comfortable answering them. 

But I want to start today with a pull question because that is what we do in cyber seminars. So let me read it aloud to you. Oh, she is going to put it up first. Alright, there we go. In which of the following areas do you have experience? And you can answer as many as are applicable: Mental Health, Quality Improvement, Health Information Technology (development or implementation), or none of the above. I see the answers are coming in. 

Arika:And it looks like things are slowing down there, Amy, if you want to read through the results.

Dr. Amy Cohen:Sure. So close to 88% of the audience has experience in mental health. And somewhere between 50 and 60% of you also have experience in quality improvement and health information technology. So this is an audience that really knows a lot about what I am going to talk about today. So I thank you for answering and I will move on to the next slide.

So I am going to take you back to when we began together. That is a picture of us, oh, about 10-12 years ago. That is Matthew Chinman on the left, Alex Young, myself, and Alison Hamilton. And we are a multi-disciplinary team with Matthew Chinman and myself as Clinical Psychologists, Alex as a Psychiatrist, and Alison as a Medical Anthropologist. And we have been thinking about and trying to improve care and functional outcomes for our veterans with serious mental illness, particularly schizophrenia. So schizophrenia is the most common serious mental illness. It is a chronic disorder of thought. And individuals with schizophrenia have considerable cognitive deficits and these deficits are in the areas of attention, memory, information processing, and executive functioning. And this population also has limited literacy and difficulty advocating for their care. This population accounts for 10% of all permanently disabled people, 12% of all healthcare costs, and about 100,000 veterans with schizophrenia are treated annually in VA.

Great news is that evidence-based practices exist. These include the use of Clozapine, supported employment services, psychosocial weight intervention, intensive case management, assertive community treatment, social skills training, cognitive behavioral therapy. And outcomes really are good when patients access these services. But the reality is they are often not available or when they are available, they are not used by a majority of the population. And, therefore, we see outcomes poorer than expected, given the efficacy literature. And this is true both inside and outside the VA.

As I mentioned, our group has been focusing on thinking about and attempting to improve care in Specialty Mental Health. And efforts in this area generally to improve care have often had limited or no success. And the research in this area has lacked data on the process of implementing evidence-based practices. Now research has provided us with information on the challenges. Implementation needs to be tailored for the patient population with cognitive deficits, limited literacy, and for those with difficulty identifying and advocating for needed services.

The providers often lack competencies to implement the services to a gold standard. And the medical records for this population often have zero of very limited information on patient preferences, specific psychosocial service needs, or measurable outcomes. Without data, our policymakers cannot identify unmet patient need nor evaluate the effectiveness of the care that is being provided. And improvement is hampered by a system that allows for a limited time in the clinical encounter and limited dollars to implement new services.

So, as a group, we have been identifying and trying to address these challenges using some innovative methods. In an early study from our group, we examined the medical records of this patient population and showed that there was really no data that could be used for quality metrics. We began then to explore the idea of gathering data directly from patients, those with schizophrenia, and we started to design tools that were tailored to their cognitive deficits. Matthew Chinman led two papers reporting on the psychometrics and then the feasibility of such a system. We then went to the next step to try to explore where such data, gathered directly from patients, could be used to support care improvement. And then, lastly, we wondered how we could get those data to providers in a timely manner and began to develop software systems that could interact with the clinician. 

So at the same time we were working away in Los Angeles on these issues, there were important reports and movements that influenced our thoughts and our direction. First, in 2001, the Institute of Medicine published a report “Crossing the Quality Chasm,” which I am sure many of you are familiar with. This called for greater use of health information technology to support care coordination. 

Second, in 2003, the President’s New Freedom Commission and the VHA Strategic Plan set forth a plan for mental healthcare that was recovery-oriented and patient-centered.

Third, in 2006, the*Journal of Implementation Science* was started. And one of the two founding editors was our own Brian Mittman from the HSR&D Center of Excellence here in Los Angeles and the QUERI resource center, CIPRS, the Center for Information Practice and Research Support. With CIPRS, which started in 2008 and the *Journal of Implementation Science* starting in 2006, there was a lot more attention on conceptual models and research methods to support innovative work in implementation of evidence-based practices across disorders.

So in the midst of this, starting in 2001, we started the HSR&D Funded Equip Study, where we piloted a method for collecting data directly from patients and then using that data to identify gaps in care. And then pilot some methods for closing those gaps. The project was important, but really laid the groundwork and the evidence for a larger trial, a regional implementation of those methods, which was the second Equip study.

So I am going to focus today on some of the details about our Equip Study, Enhancing the Quality of Care in Psychosis, which was funded by VAHSR&D QUERI. The specific aims of this project was to assist four medical centers across four VISN’s to implement and sustain evidence-based care for schizophrenia. We wanted to evaluate the effect relative to usual care of care model implementation on both service utilization and patient outcomes. And we used mixed methods, both quantitative, and qualitative assessments, to evaluate the processes of and the variations in care model implementation and effectiveness. 

So this was a clinic level control trial. Hold on. Sorry about that. This was a clinic level control trial that involved 801 patients with schizophrenia and 201 providers. And this study would not have even been possible or had any chance of success without the research network partnerships that we built across four VISN’s and had their input. So the four VISN’s involved were VISN 3, VISN 16, VISN 17, and VISN 22 for a total of eight medical centers. One intervention site and one control site in each VISN. And we used a strategic planning process to design on the targets for care improvement. 

So I want to spend a minute talking about these partnerships because they were really critical to the success of this study. So in VISN 3 on the upper left-hand corner of this slide, we had Mara Davis at the at the VISN level in VISN 3 as well as Eran Chemerinski and Bruce Levine and Claire Henderson at the Medical Center level at the implementation site, which was the Bronx. We also had Helen Rasmussen, who was the local recovery coordinator at the site. And these people were critical in starting and developing local evidence-based quality improvement teams at each site. And I am going to talk about that a little bit later.

In VISN 16, we were lucky to have Kathy Henderson help us at the VISN level with Anna Teague, Vance Hamilton, and Deborah Mullins at the Medical Center level. And Christy Gamez-Galka as the local recovery coordinator. 

In VISN 17, we had Kathryn Kotrla, and Wendell Jones, both at the VISN level, with Max Schubert and Paula Hicks at the Medical Center level. With Sherry Fairchild as our local recovery coordinator.

And in VISN 22, our home VISN, we had Peter Hauser at the VISN level, with Chris Reist, Larry, Albers, Kirk McNagny at the Medical Center, and Stacey Maruska as our local recovery coordinator.

Now we came to know these individuals and their vision for change for specialty mental health population over the course of this study. And we had an ongoing dialogue with them, which was critical to accomplishing our QI goals that we set together.

So the design was to implement a chronic illness care model to increase the use of evidence-based practices for individuals with schizophrenia and to use evidence-based quality improvements to support moving that research into care practices. We compared this to usual care and, as I mentioned before, we did both quantitative and qualitative assessments with patients and providers. Now we started this strategic planning by talking with people at each VISN and Medical Center that were involved in the study about the care targets. And we actually gave them a menu of five evidence-based practices for this population that we felt we could support. And, interestingly, all four VISN’s picked the same two targets, which were Weight Management and Supported Employment. And I believe that this represented high priorities for this population at the time, and actually, which continue to today.

And for the purposes of our talk today, so that I can have an example of the impact of the work that we did, I’m going to focus on outcomes in weight services. Now weight services are very important in this population because obesity is a serious problem. And this has become a real problem recently because weight gain is a very common side effect of these second generation antipsychotics with up to weight gain of 10 pounds per month. So people with schizophrenia die 11 to 17 years prematurely. And this is mostly due to cardiovascular disease and cancers. And this population has not benefited from improvements in primary care seen in the general population over the past few decades. But there are interventions that can help. These include changing to a different antipsychotic medication, augment with a weight loss medication, or providing access to a psychosocial intervention for weight.

So there is good evidence. The reviews and meta-analyses indicate that there effective psychosocial interventions specifically designed for individuals with schizophrenia who are overweight or obese. These seven randomized controlled trials show that the intervention group does better than the control group. You can use different formats. It needs to be from three to six months long. But over a year, you do see modest weight loss of about six pounds. But even that modest weight loss has been associated with health benefits.

So the evidence of efficacy of these programs existed. And, as a result, the report guidelines were updated in 2009 and included a recommendation that individuals with schizophrenia who are overweight or obese should be offered a psychosocial intervention for weight. And what we wanted to know was how to move these practices from labs to usual care and support that implementation. So we relied on evidence-based quality improvement or EBQI. EBQI is a structured form of continuous quality improvement that, one, incorporates a research clinical partnership, uses both top-down and bottom-up features to engage senior leaders and quality improvement teams in adapting and implementing care improvements. It focuses on prior research evidence regarding clinical guidelines for treatment, previous validated care models, and behavior change methods that we use with providers in order to promote adherence to appropriate treatment. And the overall goal of EDQI, which is exactly what we were doing in EQUIP, was to translate research on care delivery models into routine practice.

So in EQUIP, we did several strategies, evidence-based quality improvement strategies, including building evidence-based quality improvement teams locally at the intervention sites, and, as I mentioned before, these were led by local recovery coordinators. We also worked to gather leadership support to identity and support a clinical champion at each site, build education for providers, establish a quality manger who was a registered nurse at each site, to gather routine data, educate patients, and provide feedback on the performance to providers.

Now when you look at these, I want you to note that several of them either rely completely on health information technology or use it quite substantially. And those include the Quality Manager, obviously gathering the data. Patient education can also happen through health information technology and data from this also is used for performance feedback.

We relied on the conception model the Simpson Transfer Model from Duane Simpson at Texas Christian University to guide the transfer of research into practice. And I want you to focus on that box within the yellow box. So there are really four main steps to the Simpson Transfer Model. The first one is exposure, which is the introduction and training in the care model. Then we move to adoption, which is the intention to try the care model through a program of leadership decision and subsequent support. Then there is the implementation phase, where there is exploratory use of the care model. And finally, the practice phase, where there is routine use of the care model likely with the help of some customization or tailoring for the care model to the local site. 

And for the purposes of our conversation today because we are focused on the use of health information technology and quality improvement, I am really going to focus on the phases of implementation and practice. So in the implementation phase of EQUIP, we conducted routine assessment of patients each time they came to the clinic using patient-facing kiosks. This provided data that powered the QI efforts. The data included not only information about the care targets, work, and weight, but also use of services, psychiatric symptoms, and other side effects. Also quality of life and some measures of functioning. We educated patients and providers on a regular basis about the care targets and related services. We had a Quality Manager who was a nurse who received and monitored these data from patients using care management software we designed specifically for this purpose. We gave provider feedback routinely about patient level data, specifically those patients on their panel. And this feedback was delivered by a local clinical champion who also provided support and expertise on the care targets.

We gave manager and administrator feedback routinely about clinic level data, identifying the needs of the clinic population as a whole and providers who might be early adopters and other providers who might need extra support. We supported the formation of local evidence-based quality improvement teams, as I mentioned, led by the local recovery coordinator, which learned to use the data from the patient-facing kiosks to identify other gaps in care outside of work and weight and to attempt to solve these. These EBQI teams that we supported really promoted a general atmosphere of quality improvement and empowered line staff to take a role in improving their own clinic. And they were pretty excited about it.

As I said, the quality improvement was driven by routine assessment of patient needs and preferences using patient-facing kiosks that I am showing to you here. The kiosk, which we call the Patient Assessment System or PAS, was in the waiting room of the specialty mental health clinics enrolled. Patients were directed typically by a clerk to use the kiosk each time they came to the clinic. And the kiosk included a touchscreen computer, there were no keyboards, headphones, a color printer, and there was a scale next to the kiosk.

The PAS was designed for the cognitive needs of this population and here’s a screenshot of a typical question in the assessment. The questions were read aloud to the patient as they were presented with the questions on the screen. We used simple sentence structure. The answers were presented in words and as visual representations. You can see the pie charts here on the picture. So, for example, extreme difficulty is the whole pie, no difficulty is an empty pie. There were big, clear buttons for questions to be repeated or to return to a previous question. Also the system prompted them if there was a short period of inactivity. The assessment in total actually only took ten minutes or less and did not seem to interfere with the clinic flow. Upon finishing the assessment, the kiosk summary report printed, which the patient then used in a clinical encounter that day.

So here is a screenshot of a typical kiosk summary report. So this printed out right at the kiosk each time the patient finished using it. And I really want to draw your attention to three things on the kiosk report. The report helps the patient through both education, helps them support their advocacy, and self-monitoring. So, as you will see there, it says, “Your weight. Your body mass index is 27.62,” for this particular patient. That means for your height you are overweight. Medications you are taking for your illness could be making this worse. So it helps them then advocate for their needs and services because they will have this sheet with them when they go to see their clinician right that day. So what can you do? Talk to your doctor about switching to medications that do not have weight gain side effects. Or talk to your doctor about a referral to local wellness program. That program can help you lower your body mass index by helping you eat a balanced diet and get enough exercise. So this helps the patient have words to advocate for services that might help them combat the risk of their weight problem.

Then another thing I want to draw your attention to is that box at the bottom where it says, “weight in pounds.” This allows the patient to self-monitor and notice if there has been some weight gain before too much weight is gained. So it shows their weight today and their weight at the last two appointments and has a line of their ideal weight.

Every quarter this also printed out of the kiosk and this was routine education for the patients. It told them, “How can I tell if I am overweight? Why should I be worried about it?” And listed some of the common medical problems. Many of our patients already had these issues. It had a BMI chart – a body mass index chart – on the back. We did the color-coding of red, yellow, green. Green is ideal weight. So we could say like, “You are in the red area. You want to start moving towards yellow and your goal is to get to the green.” You will also notice there were some portion size tips at the bottom and they are very concrete. For example, three ounces of meat is a deck of cards – to try to help them, for these people who have more cognitive deficits.

We also provided education to providers. Now this is routine usually at clinic meetings. We made it one page, no back, so that it was really succinct. But we wanted to draw their attention to the care in this area, identify who should be referred, so overweight and obese patients, and provide them the treatment options for overweight individuals. So we were working the system at both ends – educating the patients, helping them advocate, and educating the providers. 

We also provided routine data and monitoring for the Quality Manager. Remember, this is the nurse and individual providers. This is a screenshot from the care management tracking software that we had and you could sort these tracking reports by a psychiatrist and just print out one psychiatrist clinic panel. Or you could sort it by patients who were coming that day. You could also sort it by which patients had weight issues there on the right or work issues. And this was constantly updated from the past. So when the kiosk was used each day, this software then updated the case management software. So this was constantly changing.

We also used the data from the past to provide information for managers and administrators. If you look on the left-hand side, this would be a report we would give a manager or an administrator about clinic-wide issues. So, for example, it tells them how many patients are overweight or gaining weight currently in the enrolled sample. And then if you look to the last two lines on the bottom, how many had been referred to a weight group – 62% in this example – and how many were actually going to a service for weight and that was 28%. So they can quickly see where are the gaps. There is a gap in how many were referred and then there is a large gap in how many are going. And so we need to work on both of those, the referral and then supporting patients once they are going to the wellness group. We also provided managers a comparison to other sites that were enrolled in the study so that they can see the progress at other sites compared to their progress. 

So the evidence-based quality improvement strategies as well as – which was central to that, the patient-facing kiosk – led to a new care flow diagram that included weighing each patient at each visit using the scale at the kiosk, immediate information on their weight for this session and the last two sessions, routinizing referral to weight programs and reminding providers about that via the educational sheets, and routinizing feedback on progress towards goals both to the patient, to the provider, and to the administrator. We trained staff to lead an evidence-based weight management program. We freed up staff time to deliver programs. And, again, this relied heavily on the partnership with the VISN and Medical Center leadership. We helped identify rooms large enough for groups. And we also identified other weight and exercise programs that existed at the Medical Center. Again, all of this was in partnership with the VISN and Medical Center.

So in the practice phase, again from the Simpson Transfer Model, we wanted to teach people at the sites how to maintain the kiosks. So if they would break down, be offline, something was missing, we wanted them to be self-sufficient and be able to maintain that and, therefore, keep it in routine use. We also had the routine education to providers be part of the usual clinic meetings and we supported the clinical champion in printing out and handing out the quality reports to providers and helping support them with the people we knew were having trouble across their panels. We continued tailoring the implementation using formative evaluation data and provider and leadership input. And we continued the local evidence-based quality improvement teams who were using Practice-Do-Study-Act cycles with data from the kiosks.

And, lastly, the sustainability phase, which was not on the Simpson Transfer Model photo I showed you, but is an important part of quality improvement projects. We really tried to integrate the kiosk into regular care, educate when new hires and new patients entered the clinic, continue to use the quality reports and support the clinical champions, and really integrate the evidence-based quality improvement teams into the system. So if people wanted to rotate off of the evidence-based quality improvement teams, we worked with others to join the team. These teams, again, I just want to emphasize – they really supported an atmosphere of quality improvement at the local site.

So now I am going to give you some outcomes as I sort of promised at the beginning. And these are going to be the weight outcomes from this study. So there were 801 patients with schizophrenia who were enrolled in this study and of that, 571, a large percentage, were eligible for weight services due to being overweight or obese. Of that eligible subsample, they were mid-fifties, largely male, typically VA sample, had some high school or college education, split between white and African-American. A good proportion of them were obese. And they really were not using weight services in the year prior to baseline. So those who actually went to weight services in the year prior to baseline averaged three appointments in a year. Remember, we saw that a good psychosocial weight management program needs to have visits weekly for three to six months. So we wanted to get that up. And also, the rate of having at least one appointment for weight service in the previous year was low and comparable at intervention and control sites at baseline. So we are talking less than 20% of the people who were overweight or obese had even just one visit in a year.

So the intervention status being in an intervention site versus a control site as part of this project was a significant predictor of having a weight management visit during the study year after controlling for demographics and weight category. In fact, overweight individuals at the intervention sites were 2.3 times more likely than controls to have a weight service appointment during the study year. So individuals receiving the interventions were more likely to use weight services. Intervention status was also a significant predictor of the number of days to the first weight management visit after controlling for demographics and weight category. Individuals at control sites averaged 136 days to the first weight visit. While individuals at the intervention sites averaged 98 days, showing that individuals receiving the interventions started to use weight services five weeks sooner.

Intervention status was also a significant predictor of the number of weight management visits during the study year. Control sites, pre-study, had four visits post, four visits, no change. Intervention sites increased from three visits pre-baseline to 12 visits during the intervention year, showing that individuals receiving the intervention continued to use the service three times more than controls.

So those were utilization outcomes. What about impact? Well, the control group was on average 13 pounds heavier than the intervention group at the end of the study year, showing that individuals receiving the intervention maintained weight and stopped gaining. 

So what about the acceptability of the kiosks? Well, from patients we heard sitting at the computer was one of the highlights of this project. Another person said, “It helped me see my progress in black and white.” Others noted how it promoted self-reflection saying, “It kept me in check with myself.” Or “It helped me connect the dots.” Providers also like the kiosks. They said, “The availability of the computer has made it easy for patients to monitor how they are doing with their weight.” Another person said, “Well, we were not doing a bad job before, but now we are doing an enhanced job.” Other providers commented that getting to the clinicians was critical. 

So, in conclusion, this is the largest QI in VA Specialty Mental Health to date and we worked in a partnered way that was critical to the success of this study. Evidence-based quality improvements, including integration of routine data from patient-facing kiosks, resulted in timelier and greater utilization of services and improved patient outcomes. The kiosks were central to the care reorganizations. They were feasible in the usual care clinics and acceptable to both patients with schizophrenia and their providers. 

So I now want to go back to our four key questions that we started with. So can we work in a partnered way? Absolutely. Without a strong partnership, our experience is the QI efforts will fail and health information technology will not be integrated or used. We spent considerable time finding interested individuals at the VISN level who then identified those at the Medical Center and clinic level and continued to work with us throughout this three-year study. There was an ongoing relationship with them and they actually saw that their ideas and recommendations drove the study and implementation – what we actually did. And it was tailored to their sight and vision. 

So can care in Specialty Mental Health be improved? Well, I am thrilled to say yes, but you must have data to know that. And in Specialty Mental Health that means you need to develop a system to routinize data collection. The health information technology that we used here was perfect because patients liked it. It was cheaper than an employee in terms of money and time. 

And what was the role of data and quality improvement? Well, it was central. It provided specific direction, told us what needed to be done, and the metrics for benchmarks and comparison. Data helped providers see patient level needs, helped managers see clinic level needs, and policymakers make best use of their limited dollars.

And where do we get those data? Well, even if you could get it from medical records or from clinicians, we would still suggest that data directly from patients is very valuable and their voice should be heard and integrated into care planning.

So what is next for patient-facing kiosks? Well, it is being used as a frontend interface for the VA Mental Health Package. We are also using it to deliver an evidence-based care weight management service. So we have actually moved the weight management intervention itself onto patient-facing kiosks. And we are also trying to use it to gather and rank patient treatment preferences. Clearly I am at the end of the talk and my mouth has stopped working. So I hope we can open it up and have a good conversation now, see if anybody has anything to say. Here is our funding support. Here is some references. And I put a yellow arrow towards the article that this talk is based on. 

Arika:There are no questions at this point. If anyone does have questions, please type them in the Q&A box on the right-hand side of your screen. Dr. Cohen, we will give them a little time just in case.

Dr. Amy Cohen:Okay. That is okay. There are a lot of mental health experts out there. They may have some thoughts about using health information technology with this population. Or maybe I lulled them to sleep. 

Arika:Or are there any questions you would like to ask the audience if no one is –. They may all be typing really long, complex, interesting questions for us.

Dr. Amy Cohen:Oh, now, now. Now, now. Looking down the list, a lot of these people I know have been involved in this area and doing very good work. And, see, now the questions are flooding in.

Arika:They are. Here is one question.

Dr. Amy Cohen:Okay.

Arika:If we want to implement something like this at our site, how do we get started?

Dr. Amy Cohen:That is a very good question and I wish I could say it was super easy. Moving these kiosks from research into clinical practice has been challenging, but they could definitely get in touch with either Alex Young or myself. We are also using this as a frontend to the Mental Health Package. So if they are talking about in a mental health population, we can also see that we can integrate that at their site.

Arika:Perfect. Thank you. Here is another. “I can see this works with something measurable as weight. But what about less measurable recovery-oriented outcomes like hope?”

Dr. Amy Cohen:That is a very good question. I mean, I think one of the things that I really value in the kiosk is that it is a way to ask patients where they are at on different things. So, for example, you could have the mental health recovery measure delivered via the kiosk and you could get their score on that and you could see where is their hope? How are they feeling about it? Do they feel like their providers are recovery oriented and treating them as equals? And sometimes that quick measurement allows you to start a conversation that is really critical. And with limited time in the clinical encounter, we do not get those data regularly. So I believe things like hope and their idea about where they are in their recovery transformation could be gathered from the kiosk. I mean, we know that there is not going to be a provider sitting down and doing mental health recovery measure with a patient or maybe they are asking them about hope. I would like to think so, but I am not sure. But this would allow you to start the conversation. And we found that patients with schizophrenia actually liked using the kiosk more than talking with a nurse about these questions because they felt it was more private and personal. And so maybe you would actually get some answers that you would not get if you asked them directly. And it starts the conversation, which I think is pretty critical.

Arika:Great. Thank you for that. Here is another question: Has patient-facing kiosks been implemented with veterans who have substance abuse or other mental health diagnoses?

Dr. Amy Cohen:We have not excluded people with substance abuse in our projects, so there are people who are duly diagnosed in our projects with schizophrenia, schizoaffective disorder and a substance use disorder. We also have recruited at clinics that are heavily based on substance abuse treatment, but that the individuals also have schizophrenia. So I think we have used it with that population.

Arika:Okay, a couple more questions. Is there any evidence that the patient assessment and feedback to the provider method is superior to the direct education and dashboard feedback approach, such as the pharmacy academic detailing initiative?

Dr. Amy Cohen:Tricky, mostly because I do not know about the pharmacy blah-blah-blah-blah initiative. But my guess is that they both provide feedback to the providers about their clinical panel. Can that person tell us that? I mean, basically one of the main problems in this population is that we just do not have metrics that are in the Vista system. So we do not know, for example, like diabetes that says, you know, “This person’s numbers have increased or decreased.” And that you could get from the pharmacy system. But especially in mental health, we do not have those measurements and metrics about their symptoms and side effects. And that is really why we built this system. Feedback to providers that could be gathered from the system that this individual is talking about and the system we are providing, I think is critical. Because providers want to know how their patients are doing, but it is very hard for them to get those data. Can you read that question one more time and see if I have missed anything in it?

Arika:Sure. And the person just added, “Yes, the dashboard provides metabolic measure.”

Dr. Amy Cohen:Oh, okay, so that is great. So in weight that would be terrific because it would probably give them their weight and any inches on their waist, any waist circumference data. That would be great because the providers could see over time the metrics and whether it was changing or not. But for other things in mental health, we just do not have that measurement. You could also go to CPRS and look at their weight over time. That is listed there as well as their BMI. But it is not always entered regularly. So the benefit of this kiosk is that it is really feasible, patients like it, and then you get routinely data every single time they come. They do it at the kiosk, they print it out, they walk into the clinical encounter, they hand it to the provider, and it is right there in front of them to start that conversation. But I think this person is right that the bottom line is that providers need and want data about their patients and whatever way they can get that and have time to access that, all the better for our care improvement.

Arika:Great. Here is another. Did your clinics have the kiosks already? If not, how expensive were they and what funding paid for them?

Dr. Amy Cohen:Good question. The clinics did not have them. So we bought the computers and the desks and, in some cases, the chairs. The kiosk itself – the whole package can be around $2,000 each kiosk. Some clinics we had – I think one clinic in Houston we had three kiosks. But usually we had two kiosks and it was never a problem with people waiting. Two kiosks were fine. And also in the clinics, that is about all they could handle in terms of space. And they were bought with HSR&D research dollars. But there is nothing special about the computers, other than the fact that we use a touchscreen rather than a regular screen. And so I believe that VAIT could buy something like that. The CPU’s were sort of the lowest level CPU you can get because we did not need a lot of power. 

Arika:Perfect. Here is another. Does the kiosk software have VAY IT accreditation, or is approval for its use limited to the test sites?

Dr. Amy Cohen:Well, during this project it was limited to the test sites. But since then, Alex Young, who is the Director of our Health Services Unit here at the MIRECC and Associate Director at our Center of Excellence, he got an innovation grant to develop the software into Class 1 software. I believe that is complete now and that is why it is being used with the Mental Health Package. But, again, we can talk with him or you can contact him directly to see if you wanted to use it, which is what I think the person is leading to. Can they use it in a regular VA clinic?

Arika:Great. Starting with a comment: Interesting that patients consider kiosks as more personal.

Dr. Amy Cohen:Mm-hmm, I know, I thought that too. Go ahead. I am sorry.

Arika:Do you have response data concerning patients’ acceptance of the kiosks?

Dr. Amy Cohen:We do, we do. And we published some of that. On the reference page, which is up right now, if you look at the second article, the Chinman, Hassell, Magnabosco, Nowlin-Finch article, there is some feasibility and qualitative outcomes there. But, again, like I said, the patients really preferred it. So we did a study where half the patients got a nurse interview of the exact same questions first and then did the kiosks. And then we had half of the group do the kiosk first and then the nurse. And the data was the same, so they gave the same information, but patients reported that they preferred using the kiosk more. And some of them did report that they felt like it was more private. They felt like they could reveal more. And that was sort of surprising to me too. I think we felt like they are going to sort of wonder where is this data going? What about the security of the data? But, in fact, another reason we need to ask patients, they felt very differently.

Arika:Great. Here is another. What are the financial and time burden considerations for training the participants and the providers?

Dr. Amy Cohen:In the kiosks, I am assuming they are talking about. So we had a research level individual, BA level individual there the first time each patient used the kiosk. This could have been a clerk could have done it. But because it was a research project we had a research assistant there. Once patients used it one time, they did not need help again. I mean, it was very rare that we would have someone who might need help again and usually that was because they had not come in a really long time, maybe six months. But the interface is really intuitive. The providers did not actually directly interact with the case management software other than the nurse. So the nurse and the clinical champion, which was typically a psychiatrist, would print out the reports at the panel level reports and the clinic-wide reports. So they did not actually interact with the care management software directly. But, again, in training the care management nurse and the clinical champion, very simple. Typical – similar to many of the other interfaces that the VA has, like the pharmacy interface that I am sure one of our audience members talked about. Relatively intuitive.

Arika:Great. Could patient-facing kiosks also be used for self-administration of clinical reminders?

Dr. Amy Cohen:Yes. So I think this person is saying could we use the patient-facing kiosks to help patients self-monitor other parts of their care? So, for example, could they check in each time and it would say, “Remember that it is the month that you typically get your flu shot.” Or “Remember, this is the month where you need to get your mammogram.” And certainly, I think that that could be built in. Because when the patients check in, they type in something that identifies that it is them. So the system could then keep information about that individual and feed it to them, maybe as part of the summary report that prints out. So maybe at the top of the summary report you could have a box with like reminders. You know, in the next three months do these three things. And I think that is a great idea sort of to help them self-monitor and navigate their care. I like that idea lot actually.

Arika:A couple of more questions, if that is okay.

Dr. Amy Cohen:See, a little encouragement. Alright, I am good.

Arika:Okay. Can this program be used with the VA kiosk check-in system?

Dr. Amy Cohen:Ah, the one that is being used in primary care, I think the person is talking about. So I think that system is very similar to what we are using. I think that it uses a touchscreen in primary care. Maybe that person can tell us if that is true. But I think it is a very similar system. I mean, this system is different – the system I presented today is different in the sense that it is specifically designed for people with cognitive deficits. But those design elements would not hamper a person without those deficits from using it. So using a touchscreen, a lot of our older veterans would appreciate that because it is easier. Using simple sentences, everybody would appreciate that. Less complicated interface is appreciated by many people. But I do not know enough about the one in primary care to know if it takes care of the kind of deficits that we need in this population.

Arika:Okay. How does the EQUIP kiosk compare to Pat Deegan’s Common Ground software, which is designed to facilitate consumer-provider interactions?

Dr. Amy Cohen:So I know Pat Deegan’s work, but I do not know that actual interface, so I cannot really comment on that.

Arika:Okay, great. That looks like the end of our list of questions.

Dr. Amy Cohen:That is great, except for the fact that I could not answer that question, but yes.

Arika:Perfect. Thank you to the attendees for the questions. And thank you to Dr. Cohen for taking the time to develop and present this talk. Please forward remaining questions to VIReC’s Help Desk at VIReC@va.gov. 

Dr. Amy Cohen:Thank you every –

Arika:Our next session is scheduled for Tuesday, October 15, “The Patient Search Tool CPRS Extension,” and will be presented by Rachel Cornett. Did you have something else, Dr. Cohen?

Dr. Amy Cohen:I just wanted to thank everybody for attending.

Arika:Yes. And we hope that you can join us in October.

Dr. Amy Cohen:Good-bye.

Arika:Bye, thank you.

Page **1** of **15**