Title: Patient-specific care at the cutting-edge of paediatric cardiology practice: a 3D window on anatomy at the intersection of medical imaging, computer science and 3D printing
Funder: Department of Health and Social Care

Abstract: BackgroundDespite advances in 3D cardiac magnetic resonance (CMR) imaging, structural appreciation of congenital heart disease remains hampered by presentation on a 2D computer screen. Advanced methods of medical image presentation, such as 3D printing, holography and virtual reality, have been proffered as solutions. In particular, many report that 3D printed models can provide a fully 3D, tactile appreciation of patient-specific anatomy and disease morphology. Understanding is quickly realised without image interpretation, and easily conveyed to others by inspection.Despite their potential for treatment planning, these techniques are yet to become part of routine care. We assert that their common dependence on image segmentation precludes their growth outside of the largest teaching hospitals. Current approaches rely on semi-automated procedures and normally include a significant manual component, demanding time and specialist expertise that are rarely available to clinical staff.Deep convolutional neural networks (DCNNs) are rapidly becoming the foundation to many tasks in medical image processing. In particular, DCNNs have outperformed conventional image processing approaches in the segmentation and measurement of ventricular volumes in CMR images. The ability of DCNNs to learn abstract representations from images makes this approach robust to the variation in anatomy and image quality that is observed in the clinic.Aims To explore the use of DCNNs to perform CMR image segmentation and achieve accurate, 3D presentation of congenital heart defects To assess the clinical feasibility of this approach Research Questions For what fraction of CMR scans can a DCNN derive patient-specific 3D geometries that are representative of congenital heart anatomy and disease morphology such that they are suitable for clinical presentation as a 3D model? How do segmentation by DCNN and manual operation differ in:Geometrical agreement Duration Manual complexity Cost Plan of Investigation Collection of training data CMR image data will be retrieved and segmented retrospectively for 300 patients with a range of conditions that reflect the mix of congenital heart disease seen at Evelina London Children's Hospital. Image segmentation will be performed using: (i) a largely manual pipeline performed using Mimics Medical software by Materialise; (ii) a semi-automated pipeline using conventional image processing methods developed in house. DCNN development by deep learning 240 of the 3D cardiovascular geometries collected in phase 1 will be used to train a DCNN to perform image segmentation using deep learning algorithms. Experiment the DCNN (method (iii)) will be used to perform image segmentation in the remaining 60 patient data sets that were segmented using methods (i) and (ii) in phase 1. Methods (i), (ii) and (iii) will be statistically compared in terms of their geometrical agreement, duration, complexity and financial cost of image segmentation. Benefits to PatientsAutomated image segmentation may increase the use of 3D models to care for those with congenital heart disease, improving surgical and interventional planning, and providing safer access to bespoke treatment options. Furthermore, 3D models have been used as a consenting aid to help patients and their families make informed decisions about the direction of their care.Benefits to the NHSAutomated segmentation will provide a low cost, clinically feasible and technologically based path to a branch of patient-specific care for a challenging disease cohort.More broadly, a deep learning approach to image segmentation could be applicable to other areas in which patient-specific 3D printed models have been employed. These include maxillofacial, orthopaedic, oncology, neuro and transplant surgeries to name but a few from a growing list. We hope that our findings will support the growth of bespoke treatment options that place the patient at the centre of care across a range of NHS services.

Lay abstract: Personalised care for children with heart disease: a 3D window on anatomyWhy is this research being carried out?What is congenital heart disease?Congenital heart disease is a group of conditions in which the heart's valves, chambers or major blood vessels do not grow correctly while the baby is in the womb. To help these patients, doctors must fully appreciate the size, shape and arrangement of defective and healthy anatomical structures. When these are complicated, 3D models of the heart can provide greater understanding.How can we make a 3D model of the heart?A 3D model can take different forms. It might be a physical object fabricated using a 3D printer; a hologram shown using specialised computer screens; or an interactive, virtual reality simulation. In each case the 3D model is extracted from a scan of the body such as magnetic resonance imaging (MRI). Importantly, this provides doctors with a model of the heart that is specific for a particular patient. This approach allows treatments to be tailored for individual children and promises to improve the way doctors care for their patients.What's the problem?For all the potential of these technologies, they are yet to be adopted as part of mainstream care. We suspect that this is partly due to the difficult process of converting the scan into a 3D heart model. Unfortunately MRI scans cannot be introduced directly to a 3D printer, for example. Instead a process called image segmentation is undertaken to pick only the relevant anatomy. Problematically, this process needs specialist computing expertise and resources, and can take hours to perform. This is not feasible for the busy clinical staff who work in the hospital.What are our aims?Machine learning is a branch of computer science in which a computer learns how to perform new tasks after being shown examples of how that task is done. We will explore whether this approach can be used to perform image segmentation automatically and prepare 3D models of congenital heart disease.How will we do this?We will firstly collect MRI scans from different patients with congenital heart disease. Each will be inspected to find the relevant cardiovascular anatomy for the 3D model. Working from these examples, we will use machine learning software to establish a mathematical relationship that captures the image segmentation procedure.In the second half of our project we will apply the procedure learned by the computer for a set of new patients. We will perform an experiment to compare its performance with the existing method, as performed by hand. We will check: Do the two approaches produce models that have the same size and shape? Which approach produces a model more quickly? Which approach demands the least expertise to prepare a model? Which approach is cheapest for the NHS? What can we achieve?Ideally, the computer will learn to prepare models automatically, in a way that is consistent with the manual process and faster, easier and cheaper. This would remove the technical barriers that currently prevent hospitals from using this technology. It would make the use of 3D models to care for patients with congenital heart disease more likely. At the conclusion of our project we hope to advise other hospitals on the best way to incorporate 3D models into their practice.What would this mean for healthcare?How would patients benefit?The wider availability of 3D modelling techniques could mean improved treatment planning. This in turn could enable the design of personalised operations that are tailored to a patient's particular needs. However, there are also advantages for the patient's family. Other researchers have shown that 3D printed models can help relatives to understand a complicated congenital heart defect.How would the NHS benefit?This technology would allow the NHS to efficiently use 3D models to provide personalised care for patients with congenital heart disease. Models could be prepared by the existing workforce and in a timely manner.What might this research lead to?Outside of congenital heart disease, 3D models have also been used to plan complex surgeries in a range of areas. These include orthopaedic, neurosurgery, and transplant surgery. The common challenge amongst these applications is the need to perform image segmentation. The use of machine learning to complete this task is transferrable to different areas of surgical practice. With this in mind, we hope that the combination of medical scanning, machine learning and 3D modelling will support the growth of personalised treatments in the future.

Research activity codes: 4.2 EVALUATION OF MARKERS AND TECHNOLOGIES
Health categories: CARDIOVASCULAR