Clinical trial NCT06748170: Al to Improve the Diagnosis of Rare Rheumatic Diseases
Official title: Al to Improve the Diagnosis of Rare Rheumatic Diseases
Study type: INTERVENTIONAL
Overall status: COMPLETED
Phase: NA
Conditions: Rheumatic Diseases
Keywords: Large language models
Diagnostic decision support
Clinical reasoning
rheumatology
Lead sponsor: Philipps University Marburg
Sponsor class: OTHER
First posted: 2024-12-27
Enrollment: 68
Sex: ALL
Minimum age (years): 18
Standard ages: ADULT
OLDER_ADULT
Accepts healthy volunteers: yes
Allocation: RANDOMIZED
Intervention model: PARALLEL
Intervention model description: The trial will be designed as a randomized, two-arm, single-blind parallel group study.
Primary purpose: DIAGNOSTIC

Brief summary: This trial aims to assess the impact of providing medical students with access to ChatGPT, a state-of-the-art large language model, in comparison to conventional diagnostic decision support tools, on their diagnostic accuracy for rare rheumatic diseases.

Detailed description: Advanced artificial intelligence (AI) technologies, particularly large language models such as OpenAI's ChatGPT, hold significant potential for enhancing medical decision-making. While ChatGPT was not specifically designed for medical applications, it has shown utility in various healthcare scenarios, including answering patient inquiries, drafting medical documentation, and aiding consultations. Despite these advancements, its role in supporting diagnostic reasoning-especially among less experienced medical students-and for complex rare diseases remains underexplored.

Diagnostic reasoning is a multifaceted process that combines pattern recognition, knowledge synthesis, and probabilistic thinking. Tools like ChatGPT could potentially alleviate cognitive burden, enhance diagnostic accuracy, and ultimately accelerate the diagnosis for rare diseases. However, ChatGPT is not tailored for diagnostic reasoning and lacks comprehensive validation in this domain. Additionally, it is susceptible to generating misinformation or plausible-sounding but inaccurate responses, which may hinder rather than support clinical decision-making. Therefore, understanding how medical students utilize such AI tools is essential before they are integrated into educational or clinical workflows. This study will also assess a standardized prompt to facilitate ChatGPT usage and will give students direct access to enable a realistic scenario.

This study will investigate the impact of ChatGPT on the diagnostic accuracy of medical students when tackling cases of rare rheumatic diseases. Participants will be randomized into two groups: one with access to ChatGPT and one using conventional diagnostic tools. Each participant will analyze diagnostic cases by providing up to 5 differential diagnoses and and rating the diagnostic confidence. Independent reviewers, blinded to group allocation, will evaluate the accuracy and quality of their responses. This study hence aims to provide insights into the potential benefits and limitations of integrating AI tools like ChatGPT.

Eligibility criteria: Inclusion Criteria:

* Medical students having started with clinical subjects (Internal medicine)

Exclusion Criteria:

* Not being a medical student

Primary outcomes:
measure: Diagnostic accuracy of top diagnosis
timeFrame: during evaluation
Secondary outcomes:
measure: Diagnostic accuracy of top 5 suggestions
timeFrame: during evaluation
measure: Diagnostic reasoning
timeFrame: during evaluation
measure: Diagnostic confidence
timeFrame: during evaluation
measure: Time spent for diagnosis
timeFrame: during evaluation
Other outcomes:
Locations:
city: Marburg
country: Germany
facility: Institute for Digital Medicine, University Hospital of Giessen and Marburg, Philipps University Marburg
Study officials:
References: