Introduction to the Mental Health 

Information System (MHIS) Dashboard

Presented by 

Jodie Trafton, Ph.D. and Jeanne Schaefer, Ph.D.

VA Program Evaluation and Resource Center (PERC), 

VACO Office of Mental Health Operations

Menlo Park, California

1 
 Presentation Outline

• Office of Mental Health Operations (OMHO) Mission 

• Development of the MHIS Dashboard

• Facilitating and evaluating OMHO’s nationwide quality improvement program

– Site visits and technical assistance

– Action plans

– Best Practice Dissemination

• Additional Resources

• Questions and Discussion

2 
Key Goals of Mental Health Operations 

• Facilitate and ensure implementation of the 

Uniform Mental Health Service Handbook

– A comprehensive policy document which lays out 

required mental health treatments that must be 

available as well as details about how, where, and 

how promptly care must be delivered. 

• Reduce variability in mental health treatment 

access and quality across VHA’s 141 health 

care systems

3 
Task

• To develop and information system to guide 

– 1) VISN and facility level implementation and 

quality improvement efforts

– 2) a national mental health site visit program 

– 3) a national technical assistance program

– 4) Sharing and dissemination of best practices 

across VA facilities

4 
Goals of the MHIS

• To assess level of implementation of key elements 

of the Uniform Mental Health Services Handbook

– Handbook is divided into domains which describe key 

principles or requirements for patient populations, 

specialty programs, or processes.

– Prior evaluation consisted of survey based 

assessment of self-reported fact of implementation of 

key requirements 

• To detect and decrease variability between 

facilities as well as VISNs

– Identify negative and positive outliers

• Track implementation over time 
Polling question

6 

Have you used the MHIS? Yes/No
Development of the MHIS Dashboard

• Extract all unique requirements from the Handbook for 

each domain

• Enumerate the concepts that need to be operationalized 

in order to construct a metric that maps onto each 

requirement. 

• To the extent possible, matching metrics to handbook 

language and structure, we operationalized concepts 

with diagnostic, clinical, pharmacy, and other 

administrative data

7 
8 

“All facilities must make medically-supervised withdrawal 

management available as needed, based on a systematic 

assessment of the symptoms and risks of serious adverse 

consequences related to the withdrawal process from alcohol, 

sedatives or hypnotics, or opioids. Although withdrawal 

management can often be accomplished on an ambulatory 

basis, facilities must make inpatient withdrawal management 

available for those who require it. Services can be provided at 

the facility, by referral to another VA facility, or by sharing 

arrangement, contract, or non-VA fee basis arrangements to 

the extent that the veteran is eligible with a community-based 

facility.”
Concepts from this example

• Both inpatient and outpatient services required

– Determined which files and data sources we use to look for 

withdrawal treatment services

• For patients with alcohol/sedative or opioid withdrawal. 

– Determined which diagnoses to look for, and which medications 

to look for as signs of withdrawal treatment

• Facility is responsible for ensuring that patient receives 

withdrawal management. They don’t necessarily have to 

do it themselves.

– Because this was a general principle across most of the 

Handbook, we developed a home facility methodology for 

assigning credit for service received by a patient to a facility. 

Credit for all services delivered, regardless of location of delivery 

was given to the facility where the patient receives the majority 

of their care. 

9 
Development of the MHIS Dashboard

• After initial metric specifications were drafted, we 

involved a larger group of clinical experts and policy 

leaders to provide feedback on the dozens of choices 

that were made in operationalizing each construct. 

• Other Issues:

– Standardizing concepts across PECs

– Assigning patients to facilities

– Time frames

– Constructing item thresholds and domain scores

10
Thresholds for Measures

• Set by policy-based program goals if available. 

• In the absence of policy-based program goals, 

thresholds are based on distributions.

– Identify facilities that were low outliers in terms of 

implementation or quality goals. 

• Distribution-based thresholds required expert 

consensus

– Individually discussed with policy leads in MHS, 

program evaluators in OMHO, and content experts.
Dashboard Domains

 Handbook Implementation 

Survey

 Specific Services

– Population coverage

– Screening and follow-up

– Specific required positions

– Care transitions

– Inpatient 

– Residential

– High intensity ambulatory 

services

– Emergency services

– General ambulatory services

 Specific Services (cont’d)

– Ambulatory services for SMI

– PTSD

– SUD

– Evidence-based psychotherapy

– MH care for homeless 

Veterans

– Integrated MH with medical 

services

– Services for older adults

– Services for women

– MST services

– Suicide prevention

– Medical care for mentally ill

12
14 Measures in the SUD Domain

• Meets SUD Handbook requirements by survey 

• Percent SUD Diagnosis 

• SUD patients receiving specialty SUD treatment 

• SUD patients receiving any intensive SUD treatment

• SUD patients that receive intensive residential SUD 

treatment 

• Length of intensive residential SUD treatment

• SUD patients that receive intensive outpatient SUD 

treatment 

• Length of intensive outpatient SUD treatment 

13
Measures in the SUD Domain

• Inpatient medically managed withdrawal for alcohol or 

opiates 

• Follow-up after inpatient medically managed withdrawal 

for alcohol or opiates 

• Outpatient medically managed withdrawal for alcohol or 

opiates 

• Follow-up after outpatient medically managed withdrawal 

for alcohol or opiates 

• Pharmacotherapy for alcohol use disorder 

• Pharmacotherapy for opioid dependence 
To access the MHIS: 

Direct link

http://reports2.vssc.med.va.gov/Reports/Pages/Report.aspx?Ite

mPath=%2fMentalHealth%2fMHInformationSystem%2fMHInf

ormationSystem

 

From the main VSSC page

http://vssc.med.va.gov/

15
Go to main VSSC site 

(http://vssc.med.va.gov/) 

Select “Mental 

Health” 

Select “Office of MH 

Operations MH Evaluation 

Center Information”

1 2 3 

MHIS Access from main VSSC site
The MHIS Interface
REPORT FILTERS

18

Fiscal Year drop-down

Location 

selection 

drop-down

1 

2 

3 

Fiscal Year drop-down
Outputting Results

19

Expand/collapse (+/-) buttons

Click “View Report”
Exploring Results

20

Expanded 

results for 

SUD Metric 

Grouping 

(Domain)
Frequently Asked Questions 

21

Q. Who has access to the MHIS?

 

All VA employees.

 

Q. Where does the data in the MHIS come from? 

• Administrative (workload) data 

•National Patient Care Database (NPCD),

•Decision Support Systems (DSS), 

•Pharmacy Benefits Files. 

• Handbook Survey

• EPRP reviews, and other sources and developed by OQP.
Comprehensive MH Site Visits

• Baseline assessment of implementation of the

 UMHSH 

 site-visits conducted at 140 VA health care systems in 

FY12

 areas for growth and best practices identified

• Over the course of 2 days, site visitors met with:

– health care system and MH leadership 

– frontline mental health staff 

– Veteran patients and their families

– community stakeholders

22
• The workbook is split into two sections 

 Pre-work 

 Site-visit meetings

• The pre-work includes data on mental health services and 

staffing

 Filled out by evaluation center staff and the TAs based on the MHIS 

data 

 Identify areas of strength and concern

• The site visit meeting section contains questions to be asked 

during the site visit

 Site Visit Workbook
• During meetings, the site-visit teams asked about:

– Mental health services domains (e.g., inpatient and 

residential treatment, general ambulatory care, PTSD, SUD, 

etc.)

– Pre-work strengths and concerns for each domain as 

identified in the MHIS

– Specific questions related to each domain

Site Visit Meetings
Site Visit Pre-Work Process

1. Focus

• Areas of concern

• Areas of strength

2. Report includes:

• Flagged areas

• Non-flagged weak areas

• Strong areas

• Trends 

25
Summary of Strengths and 

Concerns based on MHIS data* 

Site Visit Report Recommendation 

and Action plan Progress* 

Evaluation of MHIS data

Meeting Links

Evaluation of Briefs and Reports

Summary of Concerns based 

on Briefs/Reports* 

*Auto populates 

meeting section 

from pre-work 

section.
27

Link to meetings 

covering this domain

Mental Health 

Information 

System metrics

How does it 

compare to 

national 

average?

Is the score 

improving 

over time?
• There are 22 site visit meetings which cover:

– Mental health services domains 

– Pre-work strengths and concerns for each domain

– Specific questions related to each domain

Site Visit Meetings
Introduction: Purpose, 

Attendees, Meeting Length & 

Script

Link to Pre-Work section

Strengths and Concerns from Pre-Work 

– Response options* 

Site Visit Report and Action Plan Notes 

from Pre-Work– Response options* 

Site Visit Meeting Questions

*Auto populated from 

pre-work section to 

meeting section
30

Strengths, concerns 

and Site 

Visit/Action plan 

notes are autopopulated from 

pre-work

Button links back 

to pre-work

 Pre-Work Section
 Meeting Questions

Checked 

boxes 

indicate 

content 

should be 

covered

Multiple 

boxes can 

be 

checked
Exit Summary

• Upon completion of the 

Site Visit, site visitors 

 outline: 

 5 overall areas of concern

 5 overall areas of strength

• Reviewed with facility

• Submitted to OMHO

32
Site Visit Final Report

• In-depth review of all MH domains

– Areas of strength and concern based on:

• Data (i.e. MHIS, ARC, NEPEC Vacancy, etc)

• Site Visit Interviews

– Includes recommendations for improvement

– Suggested Resources

• Reviewed by OMHO and shared with facility

• Facility develops a strategic action plan to address report 

recommendations

33
Strategic Action Planning Process

• Designed to guide the facility’s quality improvement process 

in response to site-visit report recommendations. 

• VISN MH leadership, facility leadership, & facility MH 

leadership work create a strategic action plan to address 

report recommendations with input from OMHO.

• Facilities submit quarterly reports to provide progress on the 

plan.

• OMHO staff track facilities’ progress and offers additional 

technical support if progress is not satisfactory. 

34
Strategic Action Plan 

• Facilities develop action plans based on final report 

recommendations 

– Each recommendation has associated:

• Action steps

• Milestones 

• Deliverables/measures/targets

• MHIS data - used to measure progress in meeting milestones

• Quarterly reports to OMHO indicate facilities’ progress on action 

steps, milestones, deliverables/measures/ targets 

35
Strategic Action Plan

• Designed to guide the facility’s quality 

improvement process in response to identified 

recommendations from the site visit report. 

• VISN MH leadership, facility leadership, facility 

MH leadership discuss creation of strategic 

plan with OMHO.

• Quarterly updates are submitted to provide 

progress on the plan. 

36
Strong Practices Program

• Putative areas of strength are discussed in the site visit

• Innovative practices that improve care or care delivery 

are explored by site visitors or offered by staff

• The programs are evaluated for impact on care delivery 

using MHIS and other data sources

• Strong practices based on this review are described 

and shared on a Strong Practice Website:

• https://vaww.portal.va.gov/sites/OMHS/omhostrongpr

actices/default.aspx 

37
38

MHIS data –helps 

to identify facility 

strengths and 

concerns

Strengths and 

concerns inform 

site-visit interview 

questions

MHIS used to 

assess progress in 

meeting strategic 

action plan 

milestones

MHIS: Integral to the Site-Visit and Strategic Action Planning Process

MHIS provides 

facilities ongoing 

feedback on their 

quality 

improvement 

efforts 
For Questions, Comments, Suggestions:

OMHO Program Evaluation Center leads

Jodie Trafton or Alex Harris (PERC)

Rani Hoff or Greg Greenberg (NEPEC) 

Fred Blow or John F. McCarthy (SMITREC)

Site Visit and Action Plan Reporting Tool 

Development

Jeanne Schaefer and Sara Tavakoli

39