Role of ChatGPT in the Differential Diagnosis of Polyneuropathies and Comparison of Its Performance With That of Peripheral Neuropathy Specialists and Non-specialists
Trial ID: NCT07493681
Status: COMPLETED
Study type: OBSERVATIONAL
Start date: 2023-06-12
Completion date: 2025-03-01
Sponsor: Istituto Clinico Humanitas
Collaborators: Fondazione I.R.C.C.S. Istituto Neurologico Carlo Besta
Conditions: Polyneuropathies
Interventions: ChatGPT-4o; chatGPT
Locations: Humanitas Research Hospital
Countries: Italy
Enrollment: 100
Age range: None - None
Sex: ALL

Summary:
BACKGROUND AND PURPOSE Polyneuropathies are diseases affecting the peripheral nerves that occur in approximately 1% of the general population, rising to up to 13% among older adults. Despite their prevalence, accurate diagnosis is often challenging and requires specialist expertise that is not uniformly available. Patients evaluated in primary care or non-specialist settings frequently experience diagnostic delays or misdiagnoses, highlighting the need for innovative tools to support clinicians at critical points in the diagnostic process.

Artificial intelligence (AI) large language models (LLMs), such as ChatGPT, are increasingly being explored as potential aids in clinical diagnosis. These tools can process complex clinical information and generate diagnostic suggestions at low cost and with broad accessibility. However, their performance in specialised neurological conditions, particularly complex polyneuropathies, has not yet been rigorously evaluated in real-world settings.

STUDY OBJECTIVES This study aims to evaluate the diagnostic performance of ChatGPT-4o on real-world polyneuropathy cases and to compare it with that of peripheral nerve disease specialists and non-specialist neurologists. A secondary objective is to assess whether exposure to ChatGPT-4o outputs influences and potentially improves neurologist diagnostic accuracy.

STUDY DESIGN This will be a comparative diagnostic accuracy study conducted at two tertiary referral centres for peripheral neuropathies in Milan, Italy. One hundred patients with confirmed polyneuropathy diagnoses will be randomly selected from consecutive outpatients. Each case will be summarised in a standardised format including demographics, symptom history, neurological examination findings, nerve conduction study results, and screening laboratory data. Only cases with a diagnosis confirmed after at least 12 months of clinical follow-up will be included.

ChatGPT-4o will be presented with each case using a structured prompt, and will be asked to provide: (1) a leading diagnosis, (2) two alternative differential diagnoses, and (3) a single recommended confirmatory diagnostic test. The model will be run in two independent trials to assess response consistency.

The same 100 cases will also be reviewed by neurologists from multiple international centres. Participants will be classified as either peripheral nerve disease specialists, neurologists routinely practising in tertiary polyneuropathy centres, or non-specialists, including general neurologists or those sub-specialised in other fields. Neurologists will first provide their own diagnostic assessments independently, and will subsequently be shown ChatGPT-4o's output with the option to revise their responses.

EXPECTED SIGNIFICANCE This study will provide evidence on whether AI-based LLMs can serve as reliable diagnostic aids in complex polyneuropathy cases.

Eligibility criteria:
Inclusion Criteria

* Confirmed diagnosis of polyneuropathy of any etiology
* Final etiological diagnosis confirmed after at least 12 months of clinical follow-up
* Availability of sufficient clinical, electrophysiological, and laboratory information in the medical record to allow preparation of a standardized case summary

Exclusion Criteria

\- Absence of a confirmed etiological diagnosis (idiopathic neuropathy)

Outcomes:
Diagnostic Accuracy of Leading Diagnosis