Clinical trial NCT06624605: Enhancing Interdisciplinary Understanding of Ophthalmology Notes Through a Local Large Language Model
Official title: Enhancing Interdisciplinary Understanding of Ophthalmology Notes Through Local Large Language Model
Study type: INTERVENTIONAL
Overall status: COMPLETED
Phase: NA
Conditions: Communication
Artificial Intelligence (AI)
Artificial Intelligence Technology
Interdisciplinary Communication
Lead sponsor: Mayo Clinic
Sponsor class: OTHER
First posted: 2024-10-03
Enrollment: 851
Sex: ALL
Minimum age (years): 18
Standard ages: ADULT
OLDER_ADULT
Accepts healthy volunteers: no
Allocation: RANDOMIZED
Intervention model: PARALLEL
Intervention model description: Randomized study assessing the efficacy of Large Language Model-generated Plain Language Summaries of Ophthalmology notes in increasing non-ophthalmologist note understanding. Notes were randomized to intervention (Summary + Original Note) or control (Original note only) arms. Non-ophthalmology providers were sent surveys assessing note comprehension for both control and treatment arm notes, and Ophthalmologists were sent surveys for intervention arm notes regarding Summary accuracy and other attributes. Conducted as a Quality Improvement initiative.
Primary purpose: HEALTH_SERVICES_RESEARCH

Brief summary: This prospective, randomized controlled trial evaluated the efficacy of adding Large Language model (LLM)-generated Plain Language Summaries (PLSs) to Standard Ophthalmology Notes (SONs) in enhancing comprehension among non-ophthalmology providers. The study utilized surveys to assess non-ophthalmology providers\&#39; comprehension and satisfaction with the notes and ophthalmologists\&#39; evaluation of PLS accuracy, safety, and time burden. An objective semantic and linguistic analysis of the PLSs was also conducted.


Eligibility criteria: Inclusion Criteria:

* Evaluation by an Ophthalmologist in the Mayo Clinic Department of Ophthalmology in Rochester, MN in either the inpatient or outpatient setting between Feb 1, 2024 - May 31, 2024.

Exclusion Criteria:

* No in-network non-Ophthalmologist care providers in the medical record
* Procedure-only visits containing only a procedure note

Primary outcomes:
measure: Non-Ophthalmologist Note Comprehension and Satisfaction
timeFrame: From enrollment to 8 weeks after enrollment
Secondary outcomes:
measure: Ophthalmologist Evaluation of Summary Accuracy and Time burden
timeFrame: Assessed at a single time point &lt;24 hr after enrollment
measure: Semantic and Linguistic Quality of Plain Language Summaries
timeFrame: Assessed at a single time point &amp;lt;24 hr following enrollment
measure: Semantic and Linguistic Quality of Plain Language Summaries
timeFrame: Assessed at a single time point &amp;lt;24 hr following enrollment
measure: Semantic and Linguistic Quality of Plain Language Summaries
timeFrame: Assessed at a single time point &amp;lt;24 hr following enrollment
measure: Semantic and Linguistic Quality of Plain Language Summaries
timeFrame: Assessed at a single time point &amp;amp;lt;24 hr following enrollment
measure: Semantic and Linguistic Quality of Plain Language Summaries
timeFrame: Assessed at a single time point &amp;amp;lt;24 hr following enrollment
Other outcomes:
measure: Presence of Errors, Type of Errors, and Potential Harm of Errors
timeFrame: Assessed at a single time point &amp;amp;amp;amp;amp;lt;24 hr following enrollment and again at a single time point 8 weeks after enrollment
Locations:
city: Rochester
country: United States
facility: Mayo Clinic
state: Minnesota
Study officials:
affiliation: Mayo Clinic
name: John J Chen, MD, PhD
role: STUDY_DIRECTOR
References: