Title: Optimising diagnostic efficiency in the Emergency Department by using advanced machine learning methods to update and personalise a contemporary clinical prediction model for early identification and exclusion of acute coronary syndromes
Funder: Department of Health and Social Care

Abstract: Research question Can advanced statistical and machine learning methods improve and prevent calibration drift in the Troponin-only Manchester Acute Coronary Syndromes (T-MACS) clinical prediction model (CPM)? Background Chest pain accounts for 6% of all Emergency Department attendances, and less than 20% of those investigated have acute coronary syndromes (ACS). TMACS is a CPM derived to detect ACS and is deploying across fourteen hospitals in Greater Manchester (GM-TMACS) (1), creating a database of 30,000 patients annually. CPMs have been shown to lose accuracy overtime (2) and as static models they struggle to incorporate temporal and geographical heterogenous datasets in their derivation. Advanced machine learning techniques have been effective in updating and improving CPMs by using dynamic logistic regression and meta-modelling (3). This methodology will be applied to GM-TMACS enabling the CPM to continually update, safeguarding it for the future and improving its accuracy. Aims and objectives Our aim is to develop an advanced machine learning-enhanced CPM to 'rule in' and 'rule out' the diagnosis of ACS in the ED setting, which maintains predictive accuracy through time and locality. Specific objectives are: 1) To optimise the methodology for updating the T-MACS, incorporating temporal and geographical factors. 2) Prospectively evaluate the diagnostic accuracy of the enhanced T-MACS model using the optimum methodology identified Methods This will proceed in two stages using real-world data that are contemporaneously collected during the GM-TMACS implementation project. In stage 1 (October 2019 - July 2020), the advanced machine learning methodologies will be examined and the most accurate will be used to update and enhance the CPM. In stage 2 (July 2020 - May 2021), the methodology will be prospectively validated by evaluating the diagnostic accuracy and calibration of the model. In both stages, data collected during routine practice will be used. For stage 1, data from phase 1 of the clinical implementation of TMACS will be analysed, collected at 5 hospitals (approximately 24,000 patients). In stage 2, data collected in a 12-month period at 14 hospitals (approximately 30,000 patients) will be utilised. These data will be automatically linked to regional health records, providing coded diagnoses and outcome data. The primary outcome will be a diagnosis of acute coronary syndrome (ACS), retrieved from diagnostic coding data. In stage 1 (training/optimisation), a comparison will be made of three methodologies (regression coefficient updating, Bayesian state space modelling and meta-modelling) for updating the algorithm. In stage 2 the optimised dynamic algorithm will be applied to prospective data and calculate its impact on diagnostic accuracy and model calibration over time. Impact and dissemination The findings of this project will lead to a continually updating algorithm to optimise diagnostic efficiency and healthcare resource. It is anticipated that the project will achieve rapid regional clinical implementation alongside GM-TMACS with potential to extend further in the UK as well as Estonia and China. A dissemination strategy will be planned upon completion, involving all stakeholders including patients, aiming to achieve high impact publication and international presentations as well as developing an effective strategy for public engagement.

Lay abstract: Chest pain is one of the most common reasons why patients attend the Emergency Department (ED or 'A&E') and is also one of the most common reasons for emergency hospital admission. However, most of these patients are only admitted to hospital to have tests and do not have a condition that needs treatment in hospital. Even with modern tests, healthcare workers often cannot be sure if their patients are having a heart attack. The 'T-MACS' decision aid was developed in previous research to help solve this problem. T-MACS uses a computer to combine test results with details of a patient's symptoms to work out how likely it is that patients in the Emergency Department (ED or 'A&E') have a heart attack. After extensive research, it will now be used at 14 hospitals across Greater Manchester. It could allow patients who do not need to be in hospital to return home sooner, while also quickly enabling faster treatment for patients who do have a heart attack. Unfortunately, decision aids like T-MACS also have some important disadvantages. Over time, they tend to become less accurate because important factors change including the age of patients, the number of long-term medical conditions they have, the tests that are used and the ways in which healthcare workers practice. This means that the research must be repeated, which is inefficient and expensive. In addition, T-MACS is the same for all patients regardless of their age, sex, geographical location and other long-term medical conditions that they may have. Decision aids are likely to work better if they are personalised for each patient based on factors like these. However, this would need data from thousands of patients. Again, taking the usual approach to clinical research, this would be overwhelmingly expensive and time consuming. The research in this PhD will aim to develop a new way of updating and personalising the T-MACS decision aid using advanced machine learning techniques. We will use the information that is routinely collected from up to 30,000 patients per year in Greater Manchester as T-MACS is being used in practice. All the data will be saved on NHS computers. By matching the data with patients' electronic health records, we can find out how accurate T-MACS is in 'real world' practice. There are two stages to this research. First, we will use data from the first 12,000 patients where T-MACS has been used to test different machine learning techniques and identify which techniques lead to the biggest improvements in the accuracy of T-MACS. In stage 2 we will use data from the next 30,000 patients where T-MACS has been used to test how accurate T-MACS would have been if the new machine learning techniques had been used in practice. The research will make sure that T-MACS does not go 'out of date' and will allow decisions to be increasingly personalised for individual patients. The findings will be shared by publications in medical journals, presentations at healthcare conferences, press releases and social media. Because T-MACS is already being implemented across the region, we expect that our findings can rapidly change clinical practice. An integral part of this research is the patient and public involvement strategy. Patient groups have been involved in the inception and development of the project, providing constructive feedback. This feedback will help us to design the process by which patients attending the ED will be given information about the research and an opportunity to 'opt out'. Lay representatives will be included in the trial steering committee and will help us to interpret and share our findings as authors of key publications.

Research activity codes: 4.2 EVALUATION OF MARKERS AND TECHNOLOGIES
Health categories: CARDIOVASCULAR