Clinical trial NCT06823765: Can Feedback From a Large Language Model Improve Health Care Quality?
Official title: A Pilot Ptudy of an LLM Tool to Support Frontline Health Workers in Low-Resource Settings
Study type: INTERVENTIONAL
Overall status: RECRUITING
Phase: NA
Conditions: All Conditions
Keywords: Large Language Models
Primary Care
Nigeria
Lead sponsor: Yale University
Sponsor class: OTHER
Collaborators: name: EHA Clinics Nigeria
name: World Bank
name: University of Pennsylvania
name: George Washington University
First posted: 2025-02-12
Enrollment: 500
Sex: ALL
Standard ages: CHILD
ADULT
OLDER_ADULT
Accepts healthy volunteers: no
Allocation: NA
Intervention model: SINGLE_GROUP
Intervention model description: For the main analysis, the study employs a within-patient comparison of two patient notes created by the CHEW; one during the initial patient consultation, and one after the LLM feedback was received. The patient is also seen by a fully trained medical officer who is in charge of patient care. The MO conducts a blinded review of the CHEW's patient notes to measures changes in the CHEW's care as a result of the LLM feedback. The data comes from the information captured in the electronic medical record (EMR) of the patient and from survey data collected from CHEWs, reviewing MOs, and a panel of reviewing Medical Doctors.
Primary purpose: HEALTH_SERVICES_RESEARCH

Brief summary: The goal of this study is to learn if computer-assisted advice can help improve patient care in Nigerian health clinics. The main question it aims to answer is: does giving healthcare workers instant computer feedback help them make better decisions about patient care?

Researchers will compare patient care notes written by healthcare workers before and after they receive computer feedback to see if the feedback improves care quality. A doctor who doesn't know if feedback was given will review these notes.

Participants will:

* Be seen by a community healthcare worker who uses the computer feedback system
* Be treated by a fully trained medical doctor
* Get tested for malaria, anemia, or urinary tract infections if they have certain symptoms

Detailed description: This project tests whether Large Language Models (LLMs) can improve patient care in Nigerian primary care clinics by giving customized and instant feedback to the provider in natural language. An LLM-based tool integrated into an electronic patient record management system provides "second opinions" to community health extension workers (CHEWs) at two clinics in Nigeria. These second opinions are intended to mirror what a reviewing physician might advise the CHEWs after seeing or hearing their initial report on a patient.

For the main analysis, this study employs a within-patient comparison of two patient notes created by the CHEW; one during the initial patient consultation, and one after the LLM feedback was received. The patient is also seen by a fully trained medical officer who is in charge of patient care. The MO conducts a blinded review of the CHEW's patient notes to measures changes in the CHEW's care as a result of the LLM feedback. The data comes from the information captured in the electronic medical record (EMR) of the patient and from survey data collected from CHEWs, reviewing MOs, and a panel of reviewing Medical Doctors.

Eligibility criteria: Inclusion Criteria:

* Patient is at the clinic for outpatient consultation
* Parent/guardian consent is required for individuals under 18

Exclusion Criteria:

* Patient does not require emergency care
* Patient is not at the clinic for a checkup (e.g. weight, blood pressure, follow up after recovery)
* Patient is not a trauma patient (visit is not for an accident, wound or injury)
* Patient is not at the clinic for a scheduled procedure or a birth

Primary outcomes:
measure: Indicator for an Error in the Treatment plan (with the Potential for Harm)
timeFrame: Through study completion, an average of six months
measure: Indicator for an Error in the Treatment Plan that Causes a Loss of at least X Quality-Adjusted Life Days
timeFrame: Through study completion, an average of six months
measure: Indicator for the Better Treatment Plan (as Determined by the MOs)
timeFrame: Through study completion, an average of six months
measure: Indicator for whether Treatment is Consistent with a Predetermined "Standard of Care"
timeFrame: Through study completion, an average of six months
Secondary outcomes:
measure: Indicators Denoting Diagnosis and Treatment Alignment Between CHEWs and MOs
timeFrame: Through study completion, an average of six months
measure: Alternative Indicators for Treatment Misallocation
timeFrame: Through study completion, an average of six months
measure: Relationship of QALY Loss to Severity of Patient Condition
timeFrame: Through study completion, an average of six months
measure: Indicators for the Appropriateness of Medical Testing Decisions
timeFrame: Through study completion, an average of six months
measure: Average and Distribution of DALY Lost
timeFrame: Through study completion, an average of six months
measure: MO Evaluation of SOAP Notes: Deviations from the MO's SOAP
timeFrame: Through study completion, an average of six months
measure: MO Evaluation of SOAP Notes: Types of Harm Incurred
timeFrame: Through study completion, an average of six months
measure: MO Evaluation of SOAP Notes: Measuring Healthy Time Lost in DALY
timeFrame: Through study completion, an average of six months
measure: MD Evaluation of CHEW and MO Notes: Flagging MO Error
timeFrame: Through study completion, an average of six months
measure: MD Evaluation of CHEW and MO Notes: SOAP Note Rating
timeFrame: Through study completion, an average of six months
measure: MD Evaluation of CHEW and MO Notes: LLM Review
timeFrame: Through study completion, an average of six months
measure: Indicator for the Appropriateness of Triage Decisions
timeFrame: Through study completion, an average of six months
Other outcomes:
Locations:
city: Kano City
country: Nigeria
facility: EHA Clinics REACH Community Clinic, Gyadi Gyadi
state: Kano State
city: Kano City
country: Nigeria
facility: EHA Clinics, 33 Lamido Crescent
state: Kano State
Study officials:
affiliation: Yale University
name: Jason Abaluck
role: PRINCIPAL_INVESTIGATOR
References: