Clinical trial NCT06814327: Assessment of Organ Failure Risk Predictions in ICU
Official title: Prospective Assessment of Risk Predictions of Organ Failure in the Intensive Care Unit - Comparing Accuracy of Human and AI Risk Predictions
Study type: OBSERVATIONAL
Overall status: RECRUITING
Conditions: Circulatory Failure
Respiratory Failure
Renal Failure
Keywords: Machine learning
Intensive care
artificial intelligence
Lead sponsor: ETH Zurich
Sponsor class: OTHER
Collaborators: name: Insel Gruppe AG, University Hospital Bern
First posted: 2025-02-07
Enrollment: 690
Sex: ALL
Minimum age (years): 18
Standard ages: ADULT
OLDER_ADULT
Accepts healthy volunteers: no
Observational model: COHORT
Time perspective: PROSPECTIVE
Sampling method: PROBABILITY_SAMPLE
Study population: Adult Intensive Care Unit at the University Hospital Bern

Brief summary: During this observational study, the investigators aim to assess the ability of ICU clinicians to predict the risk of impending organ failure and retrospectively compare it to the performance of previously published machine learning models. The central hypothesis of this study is that the treating physician can predict impending organ failure in adult ICU patients with similar accuracy as the best previously publishes machine learning models.

Detailed description: In this observational study, clinician's (physicians and nurses) assessment of the estimated imminent organ failure risk in an ICU setting are prospectively collected. Circulatory failure is investigated in the primary objective, and respiratory failure, renal failure, and mortality are investigated in secondary objectives. These assessments investigate the predictive performance and influencing factors for clinician prediction. The assessments will be collected in questionnaires and be performed by the clinicians directly involved in the patient treatment and by clinicians who are not actively responsible for the patient treatment. Furthermore, this study aims to benchmark these risk assessments made by healthcare professionals against retrospectively generated AI risk scores for the same patients and timepoints. The AI risk scores will be calculated retrospectively from a set of models from a systematic search of the current literature. The AI models that will be employed for this analysis will be identified as indicated by a systematic review protocol and must satisfy the following two criteria: they do not require any data beyond what is routinely collected during an ICU stay and may be accessed as open source. Such a comparison is vital for the understanding of the relative accuracy and reliability of AI-based predictions in the context of organ failure risk compared to human performance. The data and findings from this study are anticipated to provide evidence for the clinical utility of AI-based risk scores and pave the way for future research into the optimization of AI systems for healthcare applications.

Eligibility criteria: Inclusion Criteria:

* patient minimum age of 18 years
* emergency admission to the ICU
* arterial line in place

Exclusion Criteria:

* documented refusal (on the general consent form) to participate to clinical research
* patients with neurologic conditions that impair the patient's level of consciousness (including, but not limited to stroke, traumatic brain injury, intracranial hemorrhage, CNS infections; except polytrauma)
* patients on mechanical circulatory support systems (IABP, VA-ECMO, Impella, VAD) or extracorporeal membrane oxygenation (VV-ECMO) at any time during their ICU stay;
* patients receiving end-of-life care or are admitted for the sole purpose of evaluating organ donation

Primary outcomes:
measure: Clinician prediction of circulatory failure within 8 hours compared to published ML models
timeFrame: Assessments are collected within the first 72 hours following admission.
Secondary outcomes:
measure: Clinician prediction of respiratory failure within 24 hours compared to published ML models
timeFrame: Assessments are collected within the first 72 hours following admission.
Other outcomes:
measure: Exploratory analysis of clinician prediction of renal failure within 48 hours compared to published ML models
timeFrame: Assessments are collected within the first 72 hours following admission.
measure: Exploratory analysis of clinician mortality prediction compared to published ML models
timeFrame: Assessments are collected within the first 72 hours following admission.
measure: Exploratory analysis of predictive accuracy of treating physicians versus treating nurses
timeFrame: Assessments are collected within the first 72 hours following admission.
measure: Comparison of predictive accuracy of treating versus non-treating physicians
timeFrame: Assessments are collected within the first 72 hours following admission.
measure: Calibration analysis of clinician prediction scores
timeFrame: Assessments are collected within the first 72 hours following admission.
measure: Exploratory analysis of patterns (e.g., systematic errors/biases) of predictive performance in human and machine assessors
timeFrame: Assessments are collected within the first 72 hours following admission.
Locations:
city: Bern
country: Switzerland
facility: University Hospital Inselspital, Berne
Study officials:
affiliation: Insel Gruppe AG, University Hospital Bern
name: Martin Faltys, Dr. med.
role: PRINCIPAL_INVESTIGATOR
References: