Evaluation of a national surveillance system for mortality alerts: a mixed-methods study
Authors: Aylin Paul , Bottle Alex , Burnett Susan , Cecil Elizabeth , Charles Kathryn L , Dawson Paul , D’Lima Danielle , Esmail Aneez , Vincent Charles , Wilkinson Samantha , Benn Jonathan 
Published: February 2018

Background

Since 2007, Imperial College London has generated monthly mortality alerts, based on statistical process control charts and using routinely collected hospital administrative data, for all English acute NHS hospital trusts. The impact of this system has not yet been studied.

Objectives

To improve understanding of mortality alerts and evaluate their impact as an intervention to reduce mortality.

Design

Mixed methods.

Setting

English NHS acute hospital trusts.

Participants

Eleven trusts were included in the case study. The survey involved 78 alerting trusts.

Main outcome measures

Relative risk of mortality and perceived efficacy of the alerting system.

Data sources

Hospital Episodes Statistics, published indicators on quality and safety, Care Quality Commission (CQC) reports, interviews and documentary evidence from case studies, and a national evaluative survey.

Methods

Descriptive analysis of alerts; association with other measures of quality; associated change in mortality using an interrupted time series approach; in-depth qualitative case studies of institutional response to alerts; and a national cross-sectional evaluative survey administered to describe the organisational structure for mortality governance and perceptions of efficacy of alerts.

Results

A total of 690 mortality alerts generated between April 2007 and December 2014. CQC pursued 75% (154/206) of alerts sent between 2011 and 2013. Patient care was cited as a factor in 70% of all investigations and in 89% of sepsis alerts. Alerts were associated with indicators on bed occupancy, hospital mortality, staffing, financial status, and patient and trainee satisfaction. On average, the risk of death fell by 58% during the 9-month lag following an alert, levelling afterwards and reaching an expected risk within 18 months of the alert. Acute myocardial infarction (AMI) and sepsis alerts instigated institutional responses across all the case study sites, although most sites were undertaking some parallel activities at a more general level to address known problems in care in these and other areas. Responses included case note review and coding improvements, changes in patient pathways, changes in diagnosis of sepsis and AMI, staff training in case note write-up and coding, greater transparency in patient deterioration, and infrastructure changes. Survey data revealed that 86% of responding trusts had a dedicated trust-level lead for mortality reduction and 92% had a dedicated trust-level mortality group or committee in place. Trusts reported that mortality reduction was a high priority and that there was strong senior leadership support for mortality monitoring. The weakest areas reported concerned the accuracy of coding, the quality of specialty-level mortality data and understanding trends in specialty-level mortality data.

Limitations

Owing to the correlational nature of our analysis, we could not ascribe a causal link between mortality alerts and reductions in mortality. The complexity of the institutional context and behaviour hindered our capacity to attribute locally reported changes specifically to the effects of the alerts rather than to ongoing institutional strategy.

Conclusions

The mortality alert surveillance system reflects aspects of quality care and is valued by trusts. Alerts were considered a useful focus for identifying problems and implementing interventions around mortality.

Future work

A further analysis of site visits and survey material, the application of evaluative framework to other interventions, a blinded case note review and the dissemination of findings.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.