Title: Stroke Audit Machine Learning (SAMueL-2)
Funder: Department of Health and Social Care

Abstract: AIM_x000D_ _x000D_ To develop a computer-based tool for doctors to help them improve the speed and use of life-changing treatment for stroke. It compares their stroke pathway with other hospitals, and helps to answer the question “What treatment would this patient receive in other hospitals?”_x000D_ _x000D_ BACKGROUND_x000D_ _x000D_ Stroke is a common cause of adult disability. Expert opinion is that about one in five patients should receive clot-busting drugs to break up the blood clot that is causing their stroke, but only one in nine patients actually receive this treatment in the UK. There is a lot of variation between hospitals, which means that the same patient might receive different treatment in different hospitals. _x000D_ _x000D_ Clot-busting drugs are not suitable for everyone. There is a small risk of a bleed in the brain. Doctors must feel confident in their use, and lack of confidence may explain some of the variation in use. In previous work we developed the basic methods for understanding what are the main causes of variation between hospitals: How much difference is due to processes (like how quickly a patient is scanned, an essential step), how much is due to differences in patient populations, and how much difference is due to different decision-making by doctors. This has enabled us to model the ideal number of patients who should be treated in each hospital in the UK._x000D_ _x000D_ METHODS_x000D_ _x000D_ We predict clinical decisions using a computer technique called machine learning. This learns to predict decisions by asking “what happened with similar patients before?” In addition to predicting clinical decisions, which allow us to compare decision making between hospitals, we plan to see how good machine learning is at predicting how well clot-removing treatments reduce how badly people are disabled after a stroke._x000D_ _x000D_ We model hospital processes by replicating the stroke pathway in a computer model, so that we can, for example, see the effect of changing time from arrival at hospital to receiving a brain scan. We will extend our previous computer model by including new information from ambulances. We will also use staffing data to examine how staffing levels affect performance._x000D_ _x000D_ We will explore outcomes for patients measured in “Quality Adjusted Life Years”, which will allow comparison with other treatments for other conditions. This allows us to explore the cost-effectiveness of making changes to the care pathway. This is the system used by NICE to weigh up whether treatments should be provided by the NHS or not._x000D_ _x000D_ We will do research to better understand how doctors would use this tool to improve the care that they give, and how we can make it more engaging for doctors and other staff. We plan to pick a set of doctors we can talk with through the lifetime of the project, sharing knowledge between us throughout the project. They will act as our expert resource._x000D_ _x000D_ PATIENT AND PUBLIC INVOLVEMENT_x000D_ _x000D_ A stroke survivor has been involved in our previous work and is a co-applicant here. We will hold four ‘patient voice’ workshops throughout the project, so that we can be guided by the views of people with stroke in how best to present our findings to the NHS and the public._x000D_ _x000D_ DISSEMINATION_x000D_ _x000D_ The outputs of this work will be built into the quarterly national stroke audit that feeds back results to all stroke teams across the UK. Additionally, we will work with the NHS ‘Stroke Delivery Networks’, present our work at stroke conferences, and publish our results through scientific journals.

Lay abstract: BACKGROUND_x000D_ _x000D_ In England and Wales, about 11% of emergency stroke admission patients receive thrombolysis, against an NHS Long Term Plan target of 20%. Thrombolysis use varies from 2% to 25% between hospitals. In previous modelling and machine learning work [1] we have found that about half of the inter-hospital variation is due to differences in local patient populations, and about half is due to differences in hospital process performance and clinical decision-making, with differences in clinical decision-making being the most important factor. We also found that stroke thrombolysis use could be reasonably expected to reach 18%-19% overall, though there would still be significant inter-hospital variation due to differences in local patient populations._x000D_ _x000D_ AIMS and METHODS_x000D_ _x000D_ The overall aim of SAMUeL-2 is to work with the Sentinel Stroke National Audit Programme (SSNAP) to increase the impact of national audit by providing advanced tools for in-depth comparisons between hospitals, helping to address the gap between actual and achievable thrombolysis use._x000D_ _x000D_ The key objectives and methods are:_x000D_ _x000D_ * Significant extension of the machine learning (random forest and neural networks) and pathway modelling to include:_x000D_ 1) Outcome and adverse event prediction at patient-level_x000D_ 2) Use of ambulance response times in pathway model_x000D_ 3) Use of organizational factors (e.g. staffing) in predicting thrombolysis use _x000D_ 4) Piloting of model for thrombectomy_x000D_ _x000D_ * Incorporation of health economic outcomes (QALYs) into outputs. _x000D_ _x000D_ * Co-production of project outputs with physicians: By alternating between information gathering (through interviews) and intervention refinement (in workshops) we will incrementally modify/improve the style and substance of our modelling tool. Working with PPI group we will also produce some key public-facing output._x000D_ _x000D_ * Promote acceptance of the modelling by increased transparency and explainability, in line with FAT-ML guidelines [2]:_x000D_ 1) Make use of Shapley values to show the contribution of individual features to the prediction that the model is making_x000D_ 2) Improved clustering methods to show patterns of differences in clinical decision-making between hospitals_x000D_ 3) Identification of similar hospitals (by patient population) for inter-hospital peer comparisons_x000D_ 4) Investigation of bias in model (e.g. accuracy by patient subgroups)_x000D_ 5) Generation of dashboards and other interrogative methods_x000D_ _x000D_ * Generation of synthetic data and patient vignettes:_x000D_ 1) Extend pilot work already performed for generating synthetic patient-level SSNAP data that may be shared freely and used for discussion of ‘imagined’ patients_x000D_ 2) Automatic generation of synthetic clinical vignettes from SSNAP data (or synthetic SSNAP data)_x000D_ _x000D_ All work will be conducted in a spirit of Open Science, aligned with The Turing Way of Open Science [3]._x000D_ _x000D_ TIMELINE_x000D_ _x000D_ A 2 year project from Jan 2022_x000D_ _x000D_ IMPACT_x000D_ _x000D_ This work will be conducted with guidance from the National Stroke Audit, so that the models may be implemented in quarterly/annual SSNAP output, which is used widely to drive national, regional, and local QI initiatives. We have also begun to interact with the new Integrated Stroke Delivery Networks (ISDNs)._x000D_ _x000D_ _x000D_ [1] https://samuel-book.github.io/samuel-1_x000D_ [2] https://www.fatml.org/resources/principles-for-accountable-algorithms_x000D_ [3] https://the-turing-way.netlify.app

Research activity codes: 7.3 MANAGEMENT AND DECISION MAKING
Health categories: STROKE