Quality Improvement and 

Implementation Support to Improve 

use of Guideline Recommended 

Practices for Chronic Opioid Therapy

Jodie A. Trafton, Ph.D.

Director, Program Evaluation and Resource 

Center, Mental Health Operations

Principal Investigator, SUD QUERI & Ci2i
Making Clinical Practice Guidelines (CPG) 

Codeable

• Clinical practice guidelines include written 

recommendations about how to deliver clinical care

• Not written in terms of standard data elements in medical 

administrative records or electronic medical record fields

• Here, I will discuss 

– (1) methods used to convert the Clinical Practice Guideline for 

Opioid Therapy for Chronic pain into codeable elements, and 

– (2) ways in which these definitions have been worked into 

implementation, decision support and quality improvement 

tools

– (3) examples of ways HSR&D funded projects have been 

incorporated into operations and policy initiatives
Uses of CPG definitions

• Clinical decision support systems

– Bring personalized guidelines to clinicians reviewing specific 

patient cases

• Quality Improvement metrics – Identify, prioritize, address, and evaluate system issues that limit 

use of guideline recommendations

• Electronic Medical Record (EMR) specifications

– Design EMRs to systematically collect and display information 

needed for high quality care 

• Panel Management tools

– Provide tools for cohort-based management and risk mitigation

– Target and evaluate clinical education efforts

• Research and Validation

– Identify process/outcome relationships, test 

intervention strategies, define high-risk populations
Strategies for Operationalizing CPGs

General strategy combined:

1) Analysis of guideline to identify:

1) Key recommendations

2) Concepts that require definitions

2) Review of available EMR and administrative data

1) Examine and filter universe of used data codes

2) Annual code updates

3) Guidance and definition review from Guideline authors and 

clinical experts

1) Discussion to refine guideline concepts

2) Iterative review of putative definitions by expert panel

3) Discussion of points of contention

4) Field review and feedback by target audience

1) Testing of definitions by comparative chart review by clinicians

2) User validation of patient cases

5) Assess associations with expected outcomes
Clinical Decision Support System

• Intended use 

– Case review and treatment planning for patients 

with chronic pain considered for opioid therapy 

– Provides reminder and summarized and 

personalized information to facilitate and guide 

good care decisions
Clinical Decision Support System: 

ATHENA-Opioid Therapy

• Development, usability, and pilot testing funded 

by SUD QUERI. Updates and expanded pilots 

funded by National Center for Patient Safety

• Created definitions for filtering EMR data for 

clinical use

• Created an algorithm for generating individualized 

recommendations based on a patient’s existing 

EMR data

• Created a system for providing clinicians with 

summarized information and recommendations 

to reduce risk and increase effectiveness of opioid 

therapy around the time of a clinic visit
ATHENA-OT Features

• Point of care patient-specific guideline –based 

recommendations and warnings for PC clinicians

• Gathers and provides information commonly needed for 

opioid prescribing like demographics, diagnosis, prescriptions, 

labs and adverse events. 

• Standardized pain assessment with write-back to medical 

record

• Checklist of good clinical practices

• Resources include

– Information about local, non-opioid treatment options (e.g., MH 

treatment, rehab therapies)

– Drug conversion calculator

– Suggested responses to aberrant medication use behaviors

– Patient education materials
Cautions

Patient data

Treatment 

Checklist

Patient specific guideline- based 

recommendations for opioid therapy, 

alerts if patient is high risk for misuse 

and more!

Tools as drop down menus

Summary Screen
Patient Recommendations

Click the double arrow to 

collapse all 

recommendations
Quality Improvement Metrics

• Goal: To identify system-level problems with guideline implementation 

– Focus on issues such as global access to treatments, coordination of care, use of 

standard protocols

• Get use of good opioid prescribing practices prioritized

– Provide supervisory or administrative assistance and pressure to motivate 

clinicians to take the time and effort to use good prescribing practices

• Guide development of quality improvement initiatives and tools to 

monitor impact of QI efforts over time

– Identify local weaknesses

– Identify best practice sites

– Monitor progress over time

• Encourage 

– Providers to attend to pain and opioid management 

– Administrators to ensure resources and training are available to enable providers 

to follow guidelines
Administrative data based metrics to assess 

adherence to clinical guideline recommended 

practice

• Metrics development and validation funded by SUD 

QUERI. Dashboard and maintenance funded by National 

Center for Patient Safety and Office of Mental Health 

Operations

• Created measures to look at adherence to key CPG 

recommendations at the facility level

• Identified practices in key domains that could be identified 

in “billing” data

• Developed measures of practices or outcomes in these 

domains

• Creating a data report (updated quarterly) to allow 

facilities 

– to track their performance over time

– allow comparison to performance at other facilities and 

national performance
Opioid prescribing practices 

metrics 

• Propensity to prescribe

• Side effects management

• Serious adverse effects

• Dangerous drug interactions

• Minimizing misuse risk

• Appropriate follow-up

• Avoidance of sole reliance on opioids

– Psychosocial treatments 

– Other pharmacotherapies

– Rehabilitation medicine

– Complementary and alternative medicine treatments 

• Absolutely contra-indicated opioid prescriptions 

• Medication management/pharmacy 

reconciliation 

• Appropriate lab tests
Opioid prescribing practices examined 

in 4 subgroups

• Hierarchical subgroups based generally on perceived 

risk of various opioid types

– 1) Long-acting opioids (e.g. methadone, fentanyl patch, 

oxycontin) 

– 2) Chronic short-acting opioids (Chronic is defined as 

greater than 90 days supply in a FY) 

– 3) Acute short-acting opioids 

– 4) Tramadol only

• Allows one to examine practices in areas where there is 

not clinical consensus regarding whether the practice 

should be standard for acute, or non-traditional 

opioids.
Dashboard

• Intended use 

– Identification of problem areas

– Design and evaluation of local quality improvement 

efforts

– Identify model programs

– Share best practices

Available at: 

https://securereports3.vssc.med.va.gov/Reports/Pages/

Report.aspx?ItemPath=%2fMentalHealth%2fMHOpioid

%2fOpioidMatrixReport 

Electronic Medical Record specifications

• Optimizing design of a electronic medical 

record requires understanding what 

information needs to be collected and 

presented to best document care needs 

and plan and monitor treatment.
Example 

• CMS develops “Meaningful Use Criteria” that 

outline the standard types of information that an 

EMR must collect and make available to clinicians 

to meaningfully support care delivery. 

• Data elements for the quality improvement 

metrics and decision support system were used 

to guide recommendations for CMS Meaningful 

Use Criteria to promote development of EMR 

software that supports good opioid prescribing 

practice. 
Meaningful Use Criteria Recommendations

• Experience and measures from encoding the Opioid 

Therapy guideline was used to:

• 1) identify important guideline concepts that 

currently are not captured in EMRs

– Recommendations propose addition of structured data 

collection elements for these items as a standard 

component of EMRs

• 2) Propose standard alerts and the data rules that 

would drive them for EMRs

• 3) Propose standard quality metrics to guide facilities 

in identifying care gaps 

• These were used as a basis for a solicited proposal to 

CMS for Meaningful Use Criteria from a federal 

interagency workgroup on adverse event prevention
Panel Management Tools

• Goal: To identify higher-risk patient cohorts for case 

review and treatment augmentation or modification 

when appropriate

• Here, clinician specific patient cohorts, or high-risk 

cohorts that might require similar follow-up are 

identified. 

• Used by clinicians to review and address target risk 

factors or guideline non-adherence

• Used by administration to identify providers, clinics, or 

patient clusters where training, reorganization or 

process improvement is warranted
Examples

• Opioid therapy definitions and metrics were shared with the 

Academic Detailing Program and the Opioid Safety Initiative to 

facilitate their development of panel management tools.

• Academic detailing:

– Developed facility tool with provider-level metrics with patient drill-down 

– Set facility goals for improvement

– Worked with individual clinicians to train the on the tool and educate 

them on evidence-based practice change

• Opioid Safety Initiative

– Developed provider-level lists of high risk cohorts

– Asked VISNs to review and develop strategies to address high- risk prescribing
Measure Validation

• Examine whether defined concepts have 

expected relationships to outcomes and other 

concepts

• Guideline outlines why key concepts should 

be examined and what they should predict 

– E.g. risk of adverse outcome, better patient 

outcome, need for clinical intervention to prevent 

adverse outcomes

– 
Multivariate model predicting suicide attempts in 

the chronic short-acting opioid prescribed 

population 

• Patient Predictors (Odds Ratio)

Age (0.74)

Gender (0.89)

Married (0.87)

Frail (1.22)

Drug use disorder (2.42)

Alcohol use disorder (1.99)

Mood disorder (3.55)

Traumatic Brain Injury (1.31)

• Treatment receipt (odds ratio)

Mental health assessment (2.37)

Sedative co-prescription (1.11)

Urine drug screen (2.67)

Medication Management (1.55)

Rehabilitative treatments (1.36)

Anti-inflammatory (1.07)

Tricyclics (1.09)

SNRIs (1.08)

Anticonvulsants (1.25)

• Facility Predictors (Odds Ratio)

Urine Drug Screen (0.17)
Multivariate model predicting suicide attempts in 

the Long-acting opioid prescribed population 

• Patient Predictors (Odds Ratio)

Age (0.77)

Married (0.85)

Frail (1.17)

Drug use disorder (2.83)

Alcohol use disorder (1.87)

Mood disorder (3.87)

TBI (1.50)

• Treatment receipt (odds ratio)

Mental health assessment (2.45)

Sedative co-prescription (1.20)

Urine drug screen (2.29)

Medication Management (1.61)

Rehabilitative treatments (1.30)

SNRIs (1.15)

Anticonvulsants (1.20)

• Facility Predictors (Odds Ratio)

Follow Up (0.18)

Urine Drug Screen (0.34)

Sedative co-prescription (20.32)
Multivariate model predicting overdose in the chronic 

short-acting opioid prescribed population 

• Patient Predictors (Odds Ratio)

Age (.92) 

Married (1.17)

Frail (1.77) 

Drug use disorder (4.41) 

Traumatic Brain Injury (1.37) 

• Treatment receipt 

(odds ratio)

Mental health assessment (1.32) 

Sedative co-prescriptions (1.10) 

Urine drug screen (1.99) 

Medication Management (1.52) 

Rehabilitative treatments (1.92) 

Anticonvulsants (1.27) 

• Facility Predictors (Odds 

Ratio)

Urine Drug Screen (0.15)
Multivariate model predicting overdose in the 

Long-acting opioid prescribed population 

• Patient Predictors (Odds 

Ratio)

Gender (1.58) 

Frail (1.82)

Drug use disorder (3.90) 

Mood disorder (1.19) 

• Treatment receipt (odds ratio)

Mental health assessment or 

psychotherapy (1.23)

Sedative co-rx (1.18) 

Urine drug screen (2.20) 

Medication Management (1.68) 

Rehabilitative treatments (2.10) 

Anticonvulsants (1.32) 

• Facility Predictors (Odds Ratio)

Urine Drug Screen (0.21) 
Key take homes

• Encoding guidelines provides numerous opportunities to 

expand guideline recommendations into quality 

improvement or implementation tools and strategies

– Can we develop standardized and efficient ways of incorporating this 

into the guideline development process?

• Variable and measure development, which is part of most 

research projects, has operational value to VA and other 

health care organizations

– Developing ways of sharing research developed data definitions and 

databases with operations may increase the clinical impact of HSR&D 

research

• Existing VA platforms provide opportunity for relatively 

flexible development of tools to facilitate quality 

improvement and good clinical practice

– Little research has been done on the impact of these tools and factors 

that moderate their effect on clinical practice