Clinical trial NCT06018545: AI Assisted Reader Evaluation in Acute Computed Tomography (CT) Head Interpretation
Official title: AI Assisted Reader Evaluation in Acute CT Head Interpretation
Study type: OBSERVATIONAL
Overall status: ACTIVE_NOT_RECRUITING
Conditions: Intracranial Hemorrhages
Acute Ischemic Stroke
Hydrocephalus
Cerebral Infarction
Cerebral Edema
Cerebral Injury
Keywords: Radiology
Head tomography
Emergency medicine
Radiographer
Artificial intelligence
Lead sponsor: Oxford University Hospitals NHS Trust
Sponsor class: OTHER
First posted: 2023-08-30
Enrollment: 33
Sex: ALL
Standard ages: CHILD
ADULT
OLDER_ADULT
Accepts healthy volunteers: yes
Observational model: COHORT
Time perspective: RETROSPECTIVE
Sampling method: NON_PROBABILITY_SAMPLE
Study population: Setting:

Readers will be recruited from the following four hospital Trusts (secondary and tertiary level:

* Guy's \& St Thomas NHS Foundation Trust
* Northumbria Healthcare NHS Foundation Trust
* NHS Greater Glasgow and Clyde
* Oxford University Hospitals NHS Foundation Trust

Participants:

30 volunteer participant readers will be selected from the following groups:

* Emergency Medicine Consultants and Registrars (5 Consultant, 5 Registrar (ST3-6), 5 junior (F1-ST2)
* General Radiologist Consultants and Registrars (5 Consultant, 5 Registrar)
* 5 CT Radiographers

Brief summary: This study has been added as a sub study to the Simulation Training for Emergency Department Imaging 2 study (ClinicalTrials.gov ID NCT05427838).

The purpose of the study is to assess the impact of an Artificial Intelligence (AI) tool called qER 2.0 EU on the performance of readers, including general radiologists, emergency medicine clinicians, and radiographers, in interpreting non-contrast CT head scans. The study aims to evaluate the changes in accuracy, review time, and diagnostic confidence when using the AI tool. It also seeks to provide evidence on the diagnostic performance of the AI tool and its potential to improve efficiency and patient care in the context of the National Health Service (NHS). The study will use a dataset of 150 CT head scans, including both control cases and abnormal cases with specific abnormalities. The results of this study will inform larger follow-up studies in real-life Emergency Department (ED) settings.


Eligibility criteria: Inclusion Criteria:

* Radiologists/Radiographers/ED clinicians who review CT head scans as part of their clinical practice

Exclusion Criteria:

* Neuroradiologists.
* Non-radiologist groups: Clinicians with previous formal postgraduate CT reporting training
* Emergency Medicine group: Clinicians with previous career in radiology/neurosurgery to registrar level

Primary outcomes:
measure: Reader performance: Sensitivity, specificity, comparative between with and without AI assistance.
timeFrame: During 6 weeks, which is the period for reading or reviewing the cases/scans.
measure: Reader performance: Positive and negative predictive value, comparative between with and without AI assistance.
timeFrame: During 6 weeks, which is the period for reading or reviewing the cases/scans.
measure: Reader performance: Area Under Receiver Operating Characteristic Curve (AUROC), comparative between with and without AI assistance.
timeFrame: During 6 weeks, which is the period for reading or reviewing the cases/scans.
measure: Reader speed: Mean time taken to review a scan, with versus without AI assistance.
timeFrame: During 6 weeks, which is the period for reading or reviewing the cases/scans.
measure: Reader confidence: Self-reported diagnostic confidence on a 10 point visual analogue scale, with vs without AI assistance.
timeFrame: During 6 weeks, which is the period for reading or reviewing the cases/scans.
measure: qER (AI algorithm) performance: Sensitivity and specificity
timeFrame: During 6 weeks, which is the period for reading or reviewing the cases/scans.
measure: qER (AI algorithm) performance: Positive and negative predictive value.
timeFrame: During 6 weeks, which is the period for reading or reviewing the cases/scans.
measure: qER (AI algorithm) performance: Area Under Receiver Operating Characteristic Curve (AUROC).
timeFrame: During 6 weeks, which is the period for reading or reviewing the cases/scans.
Secondary outcomes:
Other outcomes:
Locations:
city: Oxford
country: United Kingdom
facility: Oxford University Hospitals NHS Foundation Trust
state: Oxfordshire
city: Glasgow
country: United Kingdom
facility: NHS Greater Glasgow and Clyde
city: London
country: United Kingdom
facility: Guy's & St Thomas NHS Foundation Trust
city: Newcastle Upon Tyne
country: United Kingdom
facility: Northumbria Healthcare NHS Foundation Trust
Study officials:
affiliation: National Health Services in the United Kingdom (NHS UK)
name: Alex Novak, MSc
role: PRINCIPAL_INVESTIGATOR
affiliation: National Health Services in the United Kingdom (NHS UK)
name: Sarim Ather, PhD
role: PRINCIPAL_INVESTIGATOR
References:
citation: Juszczyk K, Ireland K, Thomas B, Kroon HM, Hollington P. Reduction in hospital admissions with an early computed tomography scan: results of an outpatient management protocol for uncomplicated acute diverticulitis. ANZ J Surg. 2019 Sep;89(9):1085-1090. doi: 10.1111/ans.15285. Epub 2019 Jun 17.
citation: Chan J, Fan KS, Mak TLA, Loh SY, Ng SWY, Adapala R. Pre-Operative Imaging can Reduce Negative Appendectomy Rate in Acute Appendicitis. Ulster Med J. 2020 Jan;89(1):25-28. Epub 2020 Feb 18.
citation: Greenhalgh R, Howlett DC, Drinkwater KJ. Royal College of Radiologists national audit evaluating the provision of imaging in the severely injured patient and compliance with national guidelines. Clin Radiol. 2020 Mar;75(3):224-231. doi: 10.1016/j.crad.2019.10.025. Epub 2019 Dec 19.
citation: Lin E, Yuh EL. Computational Approaches for Acute Traumatic Brain Injury Image Recognition. Front Neurol. 2022 Mar 9;13:791816. doi: 10.3389/fneur.2022.791816. eCollection 2022.
citation: Sheth SA, Giancardo L, Colasurdo M, Srinivasan VM, Niktabe A, Kan P. Machine learning and acute stroke imaging. J Neurointerv Surg. 2023 Feb;15(2):195-199. doi: 10.1136/neurintsurg-2021-018142. Epub 2022 May 25.
citation: Yeo M, Tahayori B, Kok HK, Maingard J, Kutaiba N, Russell J, Thijs V, Jhamb A, Chandra RV, Brooks M, Barras CD, Asadi H. Review of deep learning algorithms for the automatic detection of intracranial hemorrhages on computed tomography head imaging. J Neurointerv Surg. 2021 Apr;13(4):369-378. doi: 10.1136/neurintsurg-2020-017099. Epub 2021 Jan 21.
citation: Chilamkurthy S, Ghosh R, Tanamala S, Biviji M, Campeau NG, Venugopal VK, Mahajan V, Rao P, Warier P. Deep learning algorithms for detection of critical findings in head CT scans: a retrospective study. Lancet. 2018 Dec 1;392(10162):2388-2396. doi: 10.1016/S0140-6736(18)31645-3. Epub 2018 Oct 11.
citation: Guo Y, He Y, Lyu J, Zhou Z, Yang D, Ma L, Tan HT, Chen C, Zhang W, Hu J, Han D, Ding G, Liu S, Qiao H, Xu F, Lou X, Dai Q. Deep learning with weak annotation from diagnosis reports for detection of multiple head disorders: a prospective, multicentre study. Lancet Digit Health. 2022 Aug;4(8):e584-e593. doi: 10.1016/S2589-7500(22)00090-5. Epub 2022 Jun 17.
citation: Lee JY, Kim JS, Kim TY, Kim YS. Detection and classification of intracranial haemorrhage on CT images using a novel deep-learning algorithm. Sci Rep. 2020 Nov 25;10(1):20546. doi: 10.1038/s41598-020-77441-z.
citation: Arbabshirani MR, Fornwalt BK, Mongelluzzo GJ, Suever JD, Geise BD, Patel AA, Moore GJ. Advanced machine learning in action: identification of intracranial hemorrhage on computed tomography scans of the head with clinical workflow integration. NPJ Digit Med. 2018 Apr 4;1:9. doi: 10.1038/s41746-017-0015-z. eCollection 2018.
citation: Davis MA, Rao B, Cedeno PA, Saha A, Zohrabian VM. Machine Learning and Improved Quality Metrics in Acute Intracranial Hemorrhage by Noncontrast Computed Tomography. Curr Probl Diagn Radiol. 2022 Jul-Aug;51(4):556-561. doi: 10.1067/j.cpradiol.2020.10.007. Epub 2020 Nov 15.
citation: Wardlaw JM, Mair G, von Kummer R, Williams MC, Li W, Storkey AJ, Trucco E, Liebeskind DS, Farrall A, Bath PM, White P. Accuracy of Automated Computer-Aided Diagnosis for Stroke Imaging: A Critical Evaluation of Current Evidence. Stroke. 2022 Jul;53(7):2393-2403. doi: 10.1161/STROKEAHA.121.036204. Epub 2022 Apr 20.
citation: Finck T, Moosbauer J, Probst M, Schlaeger S, Schuberth M, Schinz D, Yigitsoy M, Byas S, Zimmer C, Pfister F, Wiestler B. Faster and Better: How Anomaly Detection Can Accelerate and Improve Reporting of Head Computed Tomography. Diagnostics (Basel). 2022 Feb 10;12(2):452. doi: 10.3390/diagnostics12020452.
citation: Warman R, Warman A, Warman P, Degnan A, Blickman J, Chowdhary V, Dash D, Sangal R, Vadhan J, Bueso T, Windisch T, Neves G. Deep Learning System Boosts Radiologist Detection of Intracranial Hemorrhage. Cureus. 2022 Oct 13;14(10):e30264. doi: 10.7759/cureus.30264. eCollection 2022 Oct.
citation: Dyer T, Chawda S, Alkilani R, Morgan TN, Hughes M, Rasalingham S. Validation of an artificial intelligence solution for acute triage and rule-out normal of non-contrast CT head scans. Neuroradiology. 2022 Apr;64(4):735-743. doi: 10.1007/s00234-021-02826-4. Epub 2021 Oct 8.
citation: Mallon DH, Taylor EJR, Vittay OI, Sheeka A, Doig D, Lobotesis K. Comparison of automated ASPECTS, large vessel occlusion detection and CTP analysis provided by Brainomix and RapidAI in patients with suspected ischaemic stroke. J Stroke Cerebrovasc Dis. 2022 Oct;31(10):106702. doi: 10.1016/j.jstrokecerebrovasdis.2022.106702. Epub 2022 Aug 19.
citation: Andralojc LE, Kim DH, Edwards AJ. Diagnostic accuracy of a decision-support software for the detection of intracranial large-vessel occlusion in CT angiography. Clin Radiol. 2023 Apr;78(4):e313-e318. doi: 10.1016/j.crad.2022.10.017. Epub 2023 Jan 11.
citation: Zech JR, Badgeley MA, Liu M, Costa AB, Titano JJ, Oermann EK. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study. PLoS Med. 2018 Nov 6;15(11):e1002683. doi: 10.1371/journal.pmed.1002683. eCollection 2018 Nov.
citation: Huang SC, Pareek A, Jensen M, Lungren MP, Yeung S, Chaudhari AS. Self-supervised learning for medical image classification: a systematic review and implementation guidelines. NPJ Digit Med. 2023 Apr 26;6(1):74. doi: 10.1038/s41746-023-00811-0.
citation: Hillis SL, Obuchowski NA, Schartz KM, Berbaum KS. A comparison of the Dorfman-Berbaum-Metz and Obuchowski-Rockette methods for receiver operating characteristic (ROC) data. Stat Med. 2005 May 30;24(10):1579-607. doi: 10.1002/sim.2024.
citation: Obuchowski NA. Sample size tables for receiver operating characteristic studies. AJR Am J Roentgenol. 2000 Sep;175(3):603-8. doi: 10.2214/ajr.175.3.1750603.