Title: Interactively trained ‘human-in-the-loop’ deep learning approach to improve cardiac CT and MRI assessment for accurate therapy response and mortality prediction
Funder: Department of Health and Social Care

Abstract: Background: A comprehensive assessment of all four chambers of the heart is critically important to inform the diagnosis and treatment of individuals with cardiac and pulmonary disorders. We have recently developed a deep learning-based method trained on scans with high variability that has achieved fully automated and accurate segmentation of the left ventricle on cardiovascular magnetic resonance imaging (CMRI). There is an urgent need to improve existing deep learning models to automate contouring of all four cardiac chambers to derive a comprehensive cardiac assessment. Aims and objectives: Aim: Develop an interactive human-in-the-loop multi-stage deep learning tool to measure the health of the heart. Objectives: Extract automatic physiological parameters by interactive human-in-the-loop deep learning using a multistep approach. Assess the prognostic significance of the approach Develop a prototype for trials Project plan and methods used: Perform multiple interactive steps of manual editing and deep learning on four chamber contours and landmarks using 600 CMRI / 600 CT scans to achieve ideal deep learnt segmentation. Derive automatic physiological parameters and assess the scan-scan repeatability. Assess the prognostic significance of automatic physiological parameters and deep learnt shape and/or motion analysis in well phenotyped cohorts such as the ASPIRE registry, Sheffield 3D lab and National Pulmonary Hypertension Cohort (>14K CT and MRI). Success shall be judged at receiver operating characteristic analysis 0.80=ideal. We will develop the user interface/prototype, seek feedback from end users/individuals and engage with an industrial partner. Timelines for delivery: In work package 1 (WP1) we develop the automatic physiological parameter extraction approach (M12), in WP2 we will assess the prognostic significance (M18) and in WP3 we will develop the prototype (M24). Anticipated Impact and Dissemination: Precise, rapid and repeatable prognostic and therapy response assessment. Dissemination will be achieved through professional bodies, journal publications, guidelines, and patient associations.

Lay abstract: Cardiovascular and pulmonary diseases account for close to half of all deaths in the UK. Diagnosing individuals with such conditions correctly and predicting whether they will respond to treatment is important to guide therapeutic intervention and achieve optimal outcomes. Cardiac magnetic resonance imaging (CMRI) and computed tomography (CT) scans are key to cardiac and pulmonary diagnosis. Current assessments are made by manual measurements (contours or landmarks) to derive meaningful information of heart function, or physiological parameters that are time-consuming and suffer from variability between individuals due to human error. Artificial intelligence using deep learning is pushing the limit of image analysis to an unprecedented level. Recently, a deep learning-based method trained on scans with high variability has been shown to achieve fully automated assessment of the left ventricle (LV) on CMRI. However, this level of automation has not been achieved for the right ventricle (RV) or the atrial chambers. RV failure is the key determinant of death in many cardiac conditions, but accurate RV assessment by human observers is challenging due to the more subtle anatomy and the thinner structures. There is a real clinical need to develop a fully automated assessment of the RV and all four cardiac chambers to derive meaningful physiological parameters. The aim of this project is to develop an interactive human-in-the-loop multi-stage deep learning method to measure the health of the heart in large cohorts of patients with different cardiac and pulmonary conditions, scanned on different MRI/CT machines. This method will automatically generate physiological parameters and assess their ability to automatically predict treatment response and early death. In addition, we will assess the ability of machine learning to directly assess complex shape or/and motion of the heart to make a prognostic assessment from the automatic drawings on the heart. Existing contour and landmark detection algorithms (known from our pilot assessments to be suboptimal) will be applied to scans of a large cohort of patients from different hospitals with diverse cardiac and pulmonary conditions. The measurements will be edited by a team of experienced consultants, these revised contours and landmarks will be fed back into the training to achieve an improved segmentation, and the process will repeat three times to achieve ideal accuracy. Feedback will be sought from doctors, imaging experts and patients.The deep learnt assessment of the heart will provide important measurements for predicting therapy response and survival in individuals with cardiac disease. The automatic physiological parameters and assessment of complex shape or/and motion produced by the deep learning approach could revolutionise disease assessment, and improve treatment delivery and patient care. Key advantages are rapid assessment, minimised human error and more consistent reporting of cardiac physiology and function.

Research activity codes: 4.1 DISCOVERY AND PRECLINICAL TESTING OF MARKERS AND TECHNOLOGIES;7.3 MANAGEMENT AND DECISION MAKING
Health categories: CARDIOVASCULAR