Clinical trial NCT06810076: Developing and Evaluating a Machine-Learning Opioid Overdose Prediction & Risk-Stratification Tool in Primary Care
Official title: Developing and Evaluating a Machine-Learning Opioid Prediction & Risk-Stratification E-Platform (DEMONSTRATE)
Study type: INTERVENTIONAL
Overall status: RECRUITING
Phase: NA
Conditions: Opiate Overdose
Opioid-Related Disorders
Narcotic-Related Disorders
Substance-related Disorders
Chemically-Induced Disorders
Mental Disorders
Keywords: Opiate overdose
Opioid-Related Disorders
Risk Evaluation and Mitigation
Machine Learning
Medical Order Entry Systems
Decision Support Systems, Clinical
Lead sponsor: University of Pittsburgh
Sponsor class: OTHER
Collaborators: name: National Institute on Drug Abuse (NIDA)
name: Applied Decision Science
First posted: 2025-02-05
Enrollment: 2000
Sex: ALL
Minimum age (years): 18
Standard ages: ADULT
OLDER_ADULT
Accepts healthy volunteers: no
Allocation: NA
Intervention model: SINGLE_GROUP
Intervention model description: This single-arm clinical trial employs a pre- and post-implementation pilot evaluation design to assess the usability, acceptability, and feasibility of implementing a ML-driven overdose CDS tool across thirteen UF Health primary care clinics (3 internal medicine and 6 family medicine clinics in Gainesville, Florida). The CDS tool will generate an Overdose Prevention Alert (OPA) when a PCP signs an opioid order in Epic® for patients at elevated risk of opioid overdose identified by ML algorithm.
Primary purpose: HEALTH_SERVICES_RESEARCH

Brief summary: This clinical trial aims to evaluate the pilot implementation of a machine-learning (ML)-driven clinical decision support (CDS) tool designed to predict opioid overdose risk within the electronic health record (EHR) system at UF Health Internal Medicine and Family Medicine clinics in Gainesville, Florida. The study will use a pre- versus post-implementation design to compare outcomes within clinics, focusing on measures such as naloxone prescribing rates and opioid overdose occurrences. Researchers will also assess the usability, acceptability, and feasibility of the CDS tool through qualitative interviews with primary care clinicians (PCPs) in the participating clinics.

Detailed description: This clinical trial evaluates the pilot implementation of a ML-driven CDS tool designed to predict opioid overdose risk within the electronic health record (EHR) system at thirteen UF Health internal medicine and family medicine clinics in Gainesville, Florida.

The implementation process involved backend and frontend development and integration of the CDS tool. For backend integration, the investigators reviewed clinical workflows, designed a data flow plan to incorporate risk scores into patient charts, and collaborated with UF Health IT and Integrated Data Repository (IDR) Research Services to address alert implementation, data flow, server specifications, and responsibilities. Risk assessments approved by UF Health IT and the institutional review board (IRB) ensured secure access to patient health information (PHI) and enabled EHR integration. For frontend development, the investigators used a user-centered design approach to create the CDS tool prototype, incorporating feedback from PCPs during formative interviews to refine the user interface and ensure timely, actionable alerts through the EPIC system without disrupting clinical workflows.

The study primarily aims to assess the usability, acceptance, and feasibility of the CDS tool six months post-implementation through mixed-method evaluations. Researchers will use semi-structured interviews and an online questionnaire to collect feedback from PCPs, focusing on alert usability, preferences, and outcomes. Quantitative analyses will evaluate alert penetration, usage patterns, and PCP actions, while qualitative analyses will explore themes and insights from override comments to guide tool optimization. Researchers will also explore secondary patient-level outcomes using EHR data such as naloxone prescriptions.

Eligibility criteria: Inclusion Criteria:

For PCP level outcomes assessment

* PCPs
* practicing in any of the 13 participating clinics (10 UF Health Family Medicine clinics and 3 UF Health Internal Medicine) in Gainesville, Florida.

For patient level outcomes assessment:

Inclusion criteria: Patients who seen in any of the 9 participating UF Health clinics who

* are aged ≥18 years
* received any opioid prescription in the past year prior to their clinic visit.
* are identified as being at elevated risk for overdose by the ML algorithm. Exclusion Criteria: Patients who
* had malignant cancer diagnosis or hospice care prior to study enrollment

Primary outcomes:
measure: Composite patient-level outcomes related to opioids
timeFrame: From enrollment and up to 12 months (3, 6, 12 months) post implementation of the OPA
measure: PCP's use feedback of the Overdose Prevention Alert (OPA)
timeFrame: From enrollment and up to 7 months post implementation of the OPA
Secondary outcomes:
measure: Receipt of a naloxone order or prescription fill
timeFrame: From enrollment and up to 12 months (3, 6, 12 months) post implementation of the Overdose Prevention Alert (OPA)
measure: Absence of opioid overdose diagnoses and naloxone administration
timeFrame: From enrollment and up to 12 months (3, 6, 12 months) post implementation
measure: Absence of ED visits or hospitalizations due to opioid overdose or OUD
timeFrame: From enrollment and up to 12 months (3, 6, 12 months) post implementation
measure: Absence of overlapping opioid and benzodiazepine use
timeFrame: From enrollment and up to 12 months (3, 6, 12 months) post implementation
measure: Absence of high-dose opioid use (average daily morphine milligram equivalent ≥50)
timeFrame: From enrollment and up to 12 months (3, 6, 12 months) post implementation
measure: Receipt of referrals to non-pharmacological pain management (e.g., physical therapy, chiropractic care
timeFrame: From enrollment and up to 12 months (3, 6, 12 months) post implementation
Other outcomes:
Locations:
city: Gainesville
country: United States
facility: University of Florida Health Internal Medicine and Family Medicine
state: Florida
Study officials:
affiliation: Division of General Internal Medicine, School of Medicine, University of Pittsburgh, Pittsburgh, PA
name: Wei-Hsuan Lo-Ciganic, PhD
role: PRINCIPAL_INVESTIGATOR
References:
citation: Lo-Ciganic WH, Donohue JM, Yang Q, Huang JL, Chang CY, Weiss JC, Guo J, Zhang HH, Cochran G, Gordon AJ, Malone DC, Kwoh CK, Wilson DL, Kuza CC, Gellad WF. Developing and validating a machine-learning algorithm to predict opioid overdose in Medicaid beneficiaries in two US states: a prognostic modelling study. Lancet Digit Health. 2022 Jun;4(6):e455-e465. doi: 10.1016/S2589-7500(22)00062-0.
citation: Lo-Ciganic WH, Huang JL, Zhang HH, Weiss JC, Kwoh CK, Donohue JM, Gordon AJ, Cochran G, Malone DC, Kuza CC, Gellad WF. Using machine learning to predict risk of incident opioid use disorder among fee-for-service Medicare beneficiaries: A prognostic study. PLoS One. 2020 Jul 17;15(7):e0235981. doi: 10.1371/journal.pone.0235981. eCollection 2020.
citation: Lo-Ciganic WH, Huang JL, Zhang HH, Weiss JC, Wu Y, Kwoh CK, Donohue JM, Cochran G, Gordon AJ, Malone DC, Kuza CC, Gellad WF. Evaluation of Machine-Learning Algorithms for Predicting Opioid Overdose Risk Among Medicare Beneficiaries With Opioid Prescriptions. JAMA Netw Open. 2019 Mar 1;2(3):e190968. doi: 10.1001/jamanetworkopen.2019.0968.
citation: Militello LG, Diiulio J, Wilson DL, Nguyen KA, Harle CA, Gellad W, Lo-Ciganic WH. Using human factors methods to mitigate bias in artificial intelligence-based clinical decision support. J Am Med Inform Assoc. 2025 Feb 1;32(2):398-403. doi: 10.1093/jamia/ocae291.
citation: Nguyen K, Wilson DL, Diiulio J, Hall B, Militello L, Gellad WF, Harle CA, Lewis M, Schmidt S, Rosenberg EI, Nelson D, He X, Wu Y, Bian J, Staras SAS, Gordon AJ, Cochran J, Kuza C, Yang S, Lo-Ciganic W. Design and development of a machine-learning-driven opioid overdose risk prediction tool integrated in electronic health records in primary care settings. Bioelectron Med. 2024 Oct 18;10(1):24. doi: 10.1186/s42234-024-00156-3.