Clinical trial NCT06726733: Assessing Intensive Care Unit (ICU) Indications: Human vs. ChatGPT-4o Predictions
Official title: Evaluation of the Accuracy of Intensive Care Unit (ICU) Admission Indications in Emergency Department Patients: A Comparison Between Clinical Decisions and ChatGPT-4o Prediction
Study type: OBSERVATIONAL
Overall status: NOT_YET_RECRUITING
Conditions: Intensive Care Unit (ICU) Admission
Emergency Department Patient
Artificial Intelligence (AI)
Clinical Decision-making
Keywords: AI
Intensive Care Unit
Clinical decision-making
Artificial Intelligenc
Lead sponsor: Bursa Yüksek İhtisas Education and Research Hospital
Sponsor class: OTHER_GOV
First posted: 2024-12-10
Enrollment: 500
Sex: ALL
Minimum age (years): 18
Standard ages: ADULT
OLDER_ADULT
Observational model: CASE_CONTROL
Time perspective: RETROSPECTIVE
Sampling method: NON_PROBABILITY_SAMPLE
Study population: 500

Brief summary: This retrospective study evaluates the accuracy of ICU admission indications by comparing clinical decisions with predictions from ChatGPT-4. Patient data, including demographics, vital signs, laboratory results, imaging findings, and clinical decisions, will be retrospectively collected and documented systematically using Case Report Forms. The model will be trained using ICU admission guidelines and tasked to predict ICU needs based on collected patient data. This study aims to systematically assess the alignment between AI-based predictions and clinical decisions for ICU admissions.

Detailed description: This study has a retrospective design. The medical data of patients admitted to the emergency department and consulted to the anesthesiology and reanimation clinic for ICU indications will be collected retrospectively. Demographic information, vital signs, laboratory results, imaging findings, and clinical decisions of the patients will be recorded. These data will be systematically collected for each patient using an individual Case Report Form.

Inclusion Criteria for the Study:

Patients aged 18 years and older who were consulted to the anesthesiology and reanimation clinic for ICU indications from the emergency department will be included in the study.

Exclusion Criteria for the Study:

Patients consulted to the anesthesiology and reanimation clinic for ICU indications from inpatient services.

Patients consulted to the anesthesiology and reanimation clinic from the emergency department for reasons other than ICU indications.

Patients consulted to the anesthesiology and reanimation clinic for ICU indications from the emergency department but with insufficient recorded data (patients with data loss).

Model Training and Prediction Analysis:

ChatGPT-4 will be trained according to the guidelines in "Yoğun Bakım Hasta Kabul Kriterleri (Rehberleri)" by Çiftçi B, Erdoğan C, and Demiraran Y (5). The collected patient data will be presented to the ChatGPT-4 model to obtain predictions regarding whether the patients require ICU admission. The predictions made by ChatGPT will be compared with clinical decisions, and accuracy rate, false positive rate, and false negative rate will be analyzed.

Statistical Analysis Methods to Be Used in the Study:

Accuracy Rate: The rate at which ChatGPT correctly predicts ICU indications will be calculated.

False Positive Rate: The rate at which ChatGPT predicts ICU need for patients who do not require ICU admission will be evaluated.

False Negative Rate: The rate at which ChatGPT predicts no ICU need for patients who require ICU admission will be analyzed.

Kappa Statistics: The agreement between ChatGPT predictions and clinical decisions will be measured.

ROC Curve and AUC: The performance of ChatGPT will be evaluated using the ROC curve and AUC.

The Case Report Form used for each patient ensures detailed and systematic data collection of clinical information, aiming to meaningfully compare the alignment of ChatGPT's predictions with clinical decisions.

Eligibility criteria: Inclusion Criteria:

* Patients aged 18 years and older who are consulted to the anesthesiology and reanimation clinic for ICU indications from the emergency department will be included in the study.

Exclusion Criteria:

* Patients consulted to the anesthesiology and reanimation clinic for ICU indications from inpatient services.
* Patients consulted to the anesthesiology and reanimation clinic from the emergency department for reasons other than ICU indications.
* Patients consulted to the anesthesiology and reanimation clinic for ICU indications from the emergency department but with insufficient recorded data (patients with data loss)

Primary outcomes:
measure: Accuracy rate of ChatGPT-4 in predicting ICU indications
timeFrame: 3 month
measure: False positive rate
timeFrame: 3 month
measure: False negative rate
timeFrame: 3 month
Secondary outcomes:
measure: Kappa statistic
timeFrame: 3 month
Other outcomes:
Locations:
city: Bursa
country: Turkey
facility: Bursa Yuksek Ihtisas Training and Research Hospital
Study officials:
affiliation: ceylanilkay@yahoo.com
name: İlkay Ceylan
role: PRINCIPAL_INVESTIGATOR
References: