Clinical trial NCT06963957: Automation Bias in Physician-LLM Diagnostic Reasoning
Official title: Trust or Verify? Automation Bias in Physician-LLM Diagnostic Reasoning
Study type: INTERVENTIONAL
Overall status: RECRUITING
Phase: NA
Conditions: Diagnosis
Keywords: clinical reasoning
large language models
automation bias
computer-assisted diagnosis
Lead sponsor: Lahore University of Management Sciences
Sponsor class: OTHER
First posted: 2025-05-09
Enrollment: 50
Sex: ALL
Standard ages: CHILD
ADULT
OLDER_ADULT
Accepts healthy volunteers: yes
Allocation: RANDOMIZED
Intervention model: PARALLEL
Intervention model description: The trial will be designed as a randomized, two-arm, single-blind parallel group study.
Primary purpose: DIAGNOSTIC

Brief summary: This study aims to systematically measure the extent and patterns of automation bias among physicians when utilizing ChatGPT-4o in clinical decision-making.

Detailed description: Diagnostic errors represent a significant cause of preventable patient harm in healthcare systems worldwide. Recent advances in Large Language Models (LLMs) have shown promise in enhancing medical decision-making processes.

However, there remains a critical gap in our understanding of how automation bias -- the tendency to over-rely on technological suggestions -- influences medical doctors' diagnostic reasoning when incorporating these AI tools into clinical practice.

Automation bias presents substantial risks in clinical environments, particularly as AI tools become more integrated into healthcare workflows. Although LLMs such as ChatGPT-4o offer potential advantages in reducing errors and improving efficiency, their lack of rigorous medical validation raises concerns about potentially amplifying cognitive biases through the generation of incorrect or misleading information.

Multiple contextual factors can exacerbate automation bias in medical settings: time constraints in high-volume clinical settings, financial incentives that prioritize efficiency over thoroughness, cognitive fatigue during extended shifts, and diminished vigilance when confronting diagnostically challenging cases.

These factors may interact with psychological mechanisms that include the diffusion of responsibility, overconfidence in technological solutions, and cognitive offloading---collectively increasing the risk of uncritical acceptance of AI-generated recommendations.

This randomized controlled trial (RCT) aims to systematically measure the extent and patterns of automation bias among physicians when utilizing ChatGPT-4o in clinical decision-making. The investigators will assess how access to LLM-generated information influences diagnostic reasoning through a novel methodology that precisely quantifies automation bias. In this study, participants will be randomly assigned to one of two groups. The treatment group will receive LLM-generated recommendations containing deliberately introduced errors in a subset of cases, while the control group will receive LLM-generated recommendations without such deliberately introduced errors. Participants will evaluate six clinical vignettes randomly sequenced to prevent detection patterns. The flawed vignettes provided to the treatment group will incorporate subtle yet clinically significant errors that should be identifiable by trained doctors. This will enable investigators to quantify the degree of automation bias by measuring the differential in diagnostic accuracy scores between the treatment and control groups.

Prior to participation, all physicians will complete a comprehensive training program covering LLM capabilities, prompt engineering techniques, and output evaluation strategies. Responses will be evaluated by blinded reviewers using a validated assessment rubric specifically designed to detect uncritical acceptance of erroneous information, with greater score disparities indicating stronger automation bias. This naturalistic approach will yield insights directly applicable to real clinical workflows, where mounting cognitive demands may progressively impact diagnostic decision quality.

Eligibility criteria: Inclusion Criteria:

* Completed Bachelor of Medicine, Bachelor of Surgery (MBBS) Exam. The equivalent degree of MBBS in US and Canada is called Doctor of Medicine (MD).
* Full or Provisionally Registered Medical Practitioners with the Pakistan Medical and Dental Council (PMDC).
* Participants must have completed a structured training program on the use of ChatGPT (or a comparable large language model), totaling at least 10 hours of instruction. The program must include hands-on practice related to LLM's aspects, specifically prompt engineering and content evaluation.

Exclusion Criteria:

* Any other Registered Medical Practitioners (Full or Provisional) with PMDC (e.g., Professionals with Bachelor of Dental Surgery or BDS).

Primary outcomes:
measure: Diagnostic reasoning
timeFrame: Assessed at a single time point for each case, during the scheduled diagnostic reasoning evaluation session, which takes place between 0-4 days after participant enrollment.
Secondary outcomes:
measure: Top choice diagnosis accuracy score
timeFrame: Assessed at a single time point for each case, during the scheduled diagnostic reasoning evaluation session, which takes place between 0-4 days after participant enrollment.
Other outcomes:
Locations:
city: Lahore
country: Pakistan
facility: Lahore University of Management Sciences
state: Punjab
Study officials:
affiliation: Lahore University of Management Sciences (LUMS)
name: Ihsan Ayyub Qazi, PhD
role: PRINCIPAL_INVESTIGATOR
affiliation: Lahore University of Management Sciences (LUMS)
name: Ayesha Ali, PhD
role: PRINCIPAL_INVESTIGATOR
affiliation: King Edward Medical University
name: Muhammad Asadullah Khawaja, MBBS
role: PRINCIPAL_INVESTIGATOR
affiliation: Lahore General Hospital
name: Ali Zafar Sheikh, MBBS
role: PRINCIPAL_INVESTIGATOR
affiliation: Children's Hospital, Lahore
name: Muhammad Junaid Akhtar, MBBS
role: PRINCIPAL_INVESTIGATOR
References: