Title: Optimising the outputs of National Clinical Audits to support organisations to improve the quality of care and clinical outcomes
Funder: Department of Health and Social Care

Abstract: BACKGROUND. Audit and feedback (A&F) aims to improve patient care by reviewing health care performance against explicit standards. It is widely used to monitor and improve NHS care, including in national clinical audit programmes. Ideally, where a discrepancy is detected, changes are implemented at an individual, team, and/or service level. A&F generally has small to moderate and variable effects on patient care, although potentially substantial population impacts. Yet cumulative meta-analysis of A&F trials indicates that effect sizes stabilised over 10 years ago, suggesting a lack of learning on how to improve effectiveness. We need to improve the effects of A&F. Moreover, how healthcare organisations respond to national audits is highly variable, further limiting the impact of A&F. Building on collaborations with partner national audit programmes, we will develop and evaluate methods to enhance A&F and explore how to routinely embed such work within UK national audits. AIM. To improve patient care by optimising the content, format and delivery of feedback from national clinical audits. OBJECTIVE 1. A consensus panel will prioritise theory and evidence-informed modifications to feedback interventions, drawing upon a state-of-the-science summary of recommendations. 500 individuals drawn from organisations targeted by four national audits will take part in a randomised screening experiment with a fractional factorial design, whereby they receive and respond to different combinations of feedback modifications through a web portal; proxy outcomes include intended enactment, comprehension, user experience, and preferences and engagement. OBJECTIVE 2. A case study will examine how four purposively sampled, linked pairs of healthcare provider and commissioner organisations (two for each of two national audit topics) respond to feedback modifications. We will use both institutional and systems perspectives, the latter guided by a logic model. We will use documentary evidence, observations and interviews with sampled healthcare organisations, board members, clinicians and managers to examine how feedback is delivered, perceived and acted upon. The analysis will exploit diversity sampling, comparing and contrasting findings across localities and audits. Framework and thematic analyses will focus on understanding how and why the intended effects are achieved, or why they are not. OBJECTIVE 3. We will interview 30 purposively sampled national audit staff, A&F researchers, clinicians and managers targeted by feedback to explore understanding, experience and expectations of integrating research within national audit programmes. OUTPUTS. We will (i) identify a set of modifications for national audits which can enhance their impacts and underpin future rigorous evaluations of cost-effectiveness; (ii) develop a training manual, templates and online resources to help organisations optimise the delivery and use of feedback; and (iii) establish the optimal conditions for embedding collaborative trials within national audits. EXPERTISE. Our team combines international expertise in A&F research with experience of designing and delivering national audits.

Lay abstract: Audit and feedback aims to improve patient care by comparing how healthcare is delivered against set standards. It is used by the NHS to work out how well we treat patients. Around 30 national audit programmes presently cover issues such as diabetes, cancer, heart attacks and blood transfusions. Ideally, NHS staff (e.g. doctors, managers) and organisations (e.g. hospitals, general practices) remedy shortfalls in care found by national audits. However, national audits don't improve patient care as much as they should. This is because NHS organisations don’t always act on audit findings. For example, audit reports are written by highly-committed teams keen to improve standards of care but their reports can often be lengthy and difficult for the NHS staff who receive them to digest and act upon. Furthermore, the research evidence about how best to deliver feedback is limited. For example, we are uncertain how different report formats or comparisons affect how NHS staff and organisations act on feedback. We are audit and feedback researchers working in partnership with people responsible for designing and delivering national audits. We plan to test ways to improve the impact of national audits. We have three goals. First, we will draw upon a set of state-of-the-science recommendations for improving feedback (published by members of our group). A Reference Group, made up of people who produce or receive national audit reports, will take part in a structured discussions with us to help decide which ways of delivering feedback we should develop and test. Our PPI Panel will also help us to find ways of including ‘the patient voice’ within feedback. We will then ask around 500 NHS staff (including clinicians and managers) from across the UK to take part in a ‘virtual laboratory’ experiment. We will randomly assign staff (using the play of chance) to see different ways of giving feedback on a website. They will then complete online surveys so that we can measure effects on plans to act on feedback, their understanding of feedback, ease of use and preferences – as well as how long they spend looking at the feedback. Second, we will observe how the most promising ways of giving feedback work under ‘real world’ conditions in four pairs of organisations responsible for commissioning and delivering healthcare. We will add in any ways of giving feedback that we could not test in the ‘virtual laboratory,’ such as varying how often feedback is given. We will gather different kinds of data (e.g. paper trails, interviews, direct observations) to examine how feedback is received, perceived and acted upon. We will compare what actually happens against what should ideally happen. We can then identify how else to strengthen the delivery of feedback. Third, we will explore the opportunities, costs and benefits of doing further research with national audits. We will interview people who produce, receive or evaluate feedback to find out how to make such research a more routine activity. In summary, we will (1) identify better ways of giving feedback to improve the impact of national audits on patient care; (2) provide training and practical support materials for organisations to adopt these modifications; and (3) describe how best to embed future research with national audits.

Research activity codes: 7.3 MANAGEMENT AND DECISION MAKING;8.1 ORGANISATION AND DELIVERY OF SERVICES
Health categories: GENERIC HEALTH RELEVANCE