Using Patient-Facing Kiosks to 

Support Quality Improvement at 

Mental Health Clinics 

VIReC Clinical Informatics Seminar

VA HSR&D Center for the 

Study of Healthcare 

Provider Behavior

Amy N. Cohen, PhD 

Matthew J. Chinman, PhD

Alison B. Hamilton, PhD, MPH

Fiona Whelan, MS

Alexander S. Young, MD, MSHS

September 17, 2013
Key Questions for today

 How can we work in a partnered way?

 Can care in specialty mental health be improved?

 What is the role of data in quality improvement?

 How do we get those data and from whom?
Poll Question 

 In which of the following areas do you have 

experience (answer as many as are applicable):

 Mental Health

 Quality Improvement

 Health Information Technology (development or 

implementation)

 None of the above
When we began....

Matt, Alex, Amy, Alison
Schizophrenia

 Most common serious mental illness

 Chronic disorder of thought

 Cognitive deficits: attention, memory, information 

processing, executive functioning 

 10% of all permanently disabled people

 100,000 Veterans with schizophrenia treated annually

 Evidence-based practices exist

 often not available or used; outcomes poorer than 

expected 
Improving Specialty Mental Health Care 

 Efforts to improve care have often had limited or no 

success

 research has lacked data on implementation process

 Challenges

 patients: cognitive deficits, limited literacy, poor advocates 

 providers: often lack key competencies

 medical records: no data on patient preferences, specific 

psychosocial services, outcomes

 policy makers: cannot identify unmet patient needs or 

evaluate the effectiveness of care

 system: limited time in clinical encounter, limited dollars

Institute of Medicine: Crossing the Quality Chasm

Health Information Technology (HIT)

President’s New Freedom 

Commission 

VHA Strategic Plan

Recovery Movement in Mental 

Health + Patient Centered Care

Implementation Science Journal

Methods in Quality Improvement Research

2006

2001 2003

2001 EQUIP1

2005 EQUIP2
EQUIP2: 

ENHANCING QUALITY OF 

CARE IN PSYCHOSIS 

Funded by VA HSR&D QUERI (MNT 03-213)
EQUIP Specific Aims

 Assist 4 medical centers to implement and sustain 

evidence-based care for schizophrenia

 Evaluate the effect (relative to usual care) of care 

model implementation on service utilization and 

patient outcomes.

 Using mixed methods, evaluate processes of and 

variations in care model implementation and 

effectiveness.
EQUIP Design

I: Long Beach

C: Los Angeles

I: Waco C: Temple I: Houston C: Shreveport

I: Bronx

C: Northport 

 Clinic-level controlled trial

 801 patients with schizophrenia; 201 providers

 Research-Network partnership in 4 VISNs

 1 intervention ,1 control site in each VISN (8 medical centers)

 Strategic planning for evidence-based care targets
EQUIP Team

VISN 16

Anna Teague, MD (PI: Houston)

Dean Robinson, MD (PI: Shreveport) 

Kathy Henderson, MD

Vance Hamilton, MD

Deborah Mullins, PhD

Avila Steele, PhD

Christy Gamez-Galka, PhD

Ethel Williams, RN

VISN 3

Eran Chemerinski, MD (PI: Bronx)

Charlene Thomesen, MD (PI: Northport) 

Mara Kushner Davis, CSW

Ann Feder, LCSW

Bruce Levine, MD

Claire Henderson, MD, MPH

Deborah Kayman, PhD

Helen Rasmussen, PhD

Amy Look

VISN 22

Christopher Reist, MD (PI: Long Beach) 

Peter Hauser, MD

Larry Albers, MD

Kirk McNagny, MD

David Franklin, PsyD, MPH

Stacey Maruska, LCSW

Kathy Allan, RN

VISN 17

Max Shubert, MD (PI: Central Texas) 

Kathryn Kotrla, MD

Wendell Jones, MD, MBA

Paul Hicks, MD

Staley Justice, MSW

Sherry Fairchild, PhD

Kathryn McNair, RN

Alexander S. Young, MD, MSHS (PI)

Fiona Whelan, MS 

Youlim Choi

Amy N. Cohen, PhD (co-PI)

Alison Hamilton, PhD, MPH

Paul Jung

Los Angeles MIRECC (Coordinating Site)
EQUIP Design (continued)

 Intervention: implement chronic illness care model to 

increase use of evidence-based practices for individuals 

with schizophrenia; use evidence-based quality 

improvement to support moving research into practice

 Weight services

 Supported Employment

 Control: usual care

 Quantitative Assessments: 

 Patients and Providers: 0, 7, 15 months

 Qualitative Assessments: 

 Patients: 15 months only

 Providers: 0, 7, 15 months
Obesity is a Serious Problem in 

People with Schizophrenia

 Weight gain is the most common medication sideeffect; up to 10 lbs/month

 People with schizophrenia

 die 11-17 years prematurely

 mostly due to cardiovascular disease and cancer

 have not benefitted from improvements seen in general 

population over past decades

 Potential interventions

 change to different antipsychotic medication

 augment with a weight loss medication

 provide a psychosocial intervention for weight
Evidence for Psychosocial 

Weight Management 

 Reviews and meta-analyses indicate there are effective 

psychosocial interventions specifically designed for 

individuals with schizophrenia 

 7 RCTs indicated:

 intervention > control

 individual or group format

 3-6 months

 modest weight loss; mean = 6 pounds

 modest weight loss has been associated with health benefits

Alvarez-Jimenez M, Hetrick SE, Gonzalez-Blanch C, et al., (2008). Non-pharmacological management of 

antipsychotic-induced weight gain: Systematic review and meta-analysis of randomised controlled trials. 

British Journal of Psychiatry, 193: 101-107. 
Schizophrenia PORT Guidelines 2009

Individuals with schizophrenia who are overweight or 

obese should be offered a psychosocial intervention 

for weight

Dixon LB, Dickerson F, Bellack AS, et al., (2010). The 2009 Schizophrenia PORT Psychosocial Treatment 

Recommendations and Summary Statements. Schizophrenia Bulletin, 36(1): 48-70.
Evidence-Based Quality Improvement (EBQI) 

 Structured form of Continuous Quality Improvement that,

1) incorporates a research/clinical partnership

 2) uses top-down and bottom-up features to engage 

organizational senior leaders and quality improvement 

teams in adapting and implementing improvements

3) focuses on prior research evidence regarding clinical 

guidelines for treatment, previously validated care models, 

and provider behavior change methods that promote 

adherence to appropriate treatment

GOAL: translation of research on care delivery models into 

routine practice

Rubenstein LV, Chaney EF, Ober S, et al., (2010). Using evidence-based quality improvement methods for 

translating depression collaborative care research into practice. Fam Syst Health, 28(2): 91-113.
 

 

 

Evidence-Based Quality Improvement (EBQI)
 

EBQI 

Clinical champion 

Quality manager 

Routine data 

Patient education 

Performance feedback 

Leadership support 

Provider education 
Simpson Transfer Model

Simpson DD. (2002). A conceptual framework for transferring research to practice. J Subst Abuse Treat.;22(4):171-82.
Implementation Phase 

Strategies and Tools

 Routine assessment of patients

 Education (Patient and Provider)

 Quality manager (Nurse)

 Care Management software 

 Routine provider feedback (patient-level data); Clinical 

Champions for support/education

 Routine manager/administrator feedback (clinic-level data)

 Local EBQI teams (led by Local Recovery Coordinator)
Routine Assessment of Patient 

Needs and Preferences

 Kiosk in waiting room for 

patients’ use at every visit

 Touchscreen, headphones, 

color printer, scale

Patient Assessment System (PAS)
Patient Assessment System (PAS)

 Audio, computer assisted

self-interviewing
Weight Status (education)

Overweight, and describes risks 

due to this

What you can do (advocacy)

You should discuss with your 

doctor about changing 

medication and/or referral to 

wellness program

Kiosk Summary 

Report

Weight tracking (self-monitoring)

Weight now and at last 2 

appointments + ideal weight 

indicated 
Routine Education of Patients 
Education of 

Providers

By local experts and 

opinion leaders
Routine Data and Monitoring 

for Quality Managers and Providers
Routine Data and Monitoring 

for Managers and Administrators
Care Reorganization

Developed new care flow diagram that included 

 weighing of each patient at each visit (scale at kiosk) 

 immediate information on weight for this session and 

last 2 sessions 

(Summary Report, care management tracking)

 routinized referral to weight program

 routinized feedback on progress towards goals 

(Summary Report, care management tracking)
Care Reorganization (continued)









Trained staff to lead evidence-based weight 

management program (16 sessions)

Freed up staff time to deliver program

Identified room large enough for groups

Identified other weight and exercise programs that 

exist at the medical center
Practice Phase 

Strategies and Tools

 Kiosk maintenance; routine use

 Routine education to providers

 Monthly Quality Meeting/Quality Reports; support

 Continue tailoring from formative evaluation data and 

provider and leader input

 Continue local EBQI teams; Practice-Do-Study-Act 

(PDSA) cycles
Sustainability Phase 

Strategies and Tools

 Kiosk integrated into regular care

 Education to new hires; new patients

 Quality Meetings/Quality Reports; support

 Local EBQI teams integrated into system
WEIGHT OUTCOMES
Patient Characteristics 

 N=571 eligible for weight services

 N=801 patients with schizophrenia

 Average age = 54

 91% male

 45% white; 46% African-American 

 68% HS or some college

 44% were overweight; 56% were obese

 Weight service utilization in year prior to baseline:

 Average 3 appointments (SD=4.5)

 Rate of having at least one appointment in previous year is comparable 

at intervention (13%) and control sites (18%) (p>0.05)
Having a Weight Management Visit

 Intervention status (intervention vs control) was a 

significant predictor of having a weight management 

visit (χ2=10.5, p<0.01) after controlling for 

demographics and weight category (overweight or 

obese)

 Overweight individuals at intervention sites 2.3 times

more likely than controls to have a weight service 

appointment

Individuals receiving intervention more likely to use 

weight services
Time to First Weight Management Visit

 Intervention status was a significant predictor of the 

number of days to the first weight management visit 

(t=2.0, p=0.05) after controlling for demographics and 

weight category

 Individuals at control sites averaged 136 days (SE=17)

 Individuals at intervention sites averaged 98 days (SE=15)

Individuals receiving intervention start to use 

services 5 weeks sooner
Number of Weight Management Visits

 Intervention status was a significant predictor of the 

number of weight management visits (t=-4.6, p<0.01) 

after controlling for demographics and weight 

category

 Control sites: pre=4 visits; post=4 visits

 Intervention sites: pre=3 visits; post=12 visits

Individuals receiving intervention continue to use the 

services 3 times more than controls
Weight Management

 Control group was, on average, 13.4 +/- 7.6 lbs 

heavier than the intervention group at the end of 

the study year (F=4.83, p=0.03)

Individuals receiving intervention maintained 

weight; stopped gaining
Acceptability of the Kiosks

Patients: 

 76% reported they enjoyed using the kiosk

 “Sitting at the computer was one of the highlights of the project.”

 71% reported they like getting the Summary Report

 “It helped me see my progress in black and white.”

 Noted that kiosk questions promoted self-reflection

 “It kept me in check with myself.”

 “It helped me connect the dots.”

Providers: 

 “The availability of the computer has made it easy for [patients] to monitor 

how they’re doing with [their weight].”

 “We weren’t doing a bad job before, but now we are doing an enhanced 

job.”

 Another commented that giving data to clinicians was essential.
Conclusions

 This is the largest QI effort in VA specialty mental health 

to date. Working in a partnered way was critical to the 

success of this study.

 Evidence-based quality improvement, including integration 

of routine data from patient-facing kiosks, resulted in 

timelier and greater utilization of services and improved 

patient outcomes.

 The kiosks were central to the care reorganization; were 

feasible in usual care clinics and acceptable to both 

patients with schizophrenia and their providers.
Key Questions and the Answers

 How can we work in a partnered way?

 Can care in specialty mental health be improved?

 What is the role of data in quality improvement?

 How do we get those data and from whom?
What’s next for patient-facing kiosks?

 Front end interface for the VA Mental Health 

package 

 Delivery of evidence-based care (weight 

management services)

 Gather and rank patient treatment preferences 
 Funding Support

 VA HSR&D QUERI (MNT 03-213)

 VA Desert Pacific Mental Illness Research, Education and 

Clinical Program (MIRECC)

 VA HSR&D Center of Excellence for the Study of 

Healthcare Provider Behavior

 For further information

 Amy Cohen

 Amy.cohen@va.gov 

 MIRECC, West Los Angeles VA Healthcare Center,

11301 Wilshire Blvd. (210A), Los Angeles CA 90073
References 















Chinman M, Young AS, Schell T, Hassell J, Mintz J: Computer-assisted self- assessment in persons with severe mental illness. Journal of Clinical Psychiatry. 2004; 65: 1343-1351. 

Chinman M, Hassell J, Magnabosco J, Nowlin-Finch N, Marusak S, Young AS: The 

feasibility of computerized patient self-assessment at mental health clinics. 

Administration and Policy in Mental Health. 2007; 34: 401-9. 

Brown AH, Cohen AN, Chinman MJ, Kessler C, Young AS: EQUIP: Implementing 

Chronic Care Principles and Applying Formative Evaluation Methods to Improve 

Care for Schizophrenia. Implementation Science. 2008; 3: 9. 

Hamilton AB, Cohen AN, Young AS. Organizational Readiness in 

Specialty Mental Health Care. Journal of General Internal Medicine. 2010; 25: 

27-31. 

Cohen, AN, Glynn, SM, Hamilton, AB, Young, AS. Implementation of a Family 

Intervention for Individuals with Schizophrenia. Journal of General Internal 

Medicine. 2010; 25: 32-37. 

Cohen AN, Chinman MJ, Hamilton AB, Whelan F, & Young AS. (2013). Using 

Patient-Facing Kiosks to Support Quality Improvement at Mental Health Clinics. 

Medical Care, 51: S13-S20.

Hamilton AB, Cohen AN, Glover DL, Whelan F, Chemerinski E, McNagny KP, 

Mullins D, Reist C, Schubert M, Young AS. (in press). Implementation of Evidence- Based Employment Services in Specialty Mental Health. Health Services Research.