National Diabetes Audit Data Quality Statement, 2016-2017

Published 10 November 2017

## Contents

| Section | Page | |----------------------------------------------|------| | Introduction | 3 | | Relevance | 4 | | Accuracy and Reliability | 4 | | Testing | 5 | | Validation | 6 | | Timeliness and Punctuality | 6 | | Timeliness | 6 | | Punctuality | 6 | | Accessibility and Clarity | 6 | | Coherence and Comparability | 7 | | Comparability over time | 7 | | Reduction in Participation in 2013-2014 and 2014-2015 | 7 | | Comparable with other sources | 8 | | Assessment of User Needs and Perceptions | 9 | Introduction

The National Diabetes Audit is commissioned by the Healthcare Quality Improvement Partnership (HQIP) and delivered by NHS Digital, formerly the Health and Social Care Information Centre, working in collaboration with Diabetes UK and Public Health England (PHE).

The National Diabetes Audit is a major national clinical audit which measures the effectiveness of diabetes healthcare against NICE Clinical Guidelines and NICE Quality Standards in England and Wales. The NDA collects and analyses data for use by a range of stakeholders to drive changes and improvements in the quality of services and health outcomes for people with diabetes.

The National Diabetes Audit (NDA) answers four key questions:

1. Is everyone with diabetes diagnosed and recorded on a practice diabetes register?
2. What percentage of people registered with diabetes received the nine NICE key processes of diabetes care?
3. What percentage of people registered with diabetes achieved NICE defined treatment targets for glucose control, blood pressure and blood cholesterol?
4. For people with registered diabetes what are the rates of acute and long term complications (disease outcomes)?

The NDA aims to improve the quality of patient care by enabling NHS organisations to:

➢ compare their outcomes of care with similar NHS organisations ➢ identify and share best practice ➢ identify gaps or shortfalls in commissioning services ➢ assess local practice against NICE guidelines and drive service improvement ➢ provide a more comprehensive picture of diabetes care and outcomes in England and Wales.

Through participation in the audit, local services are able to benchmark their performance and identify where they are performing well and also improve the quality of treatment and care they provide. On a national level, wide participation in the audit also provides an overview of the quality of care being provided in England and Wales. Relevance

Data is extracted from general practice clinical systems and specialist service units in secondary care hospitals. The audit is a voluntary audit for GP practices, meaning GP practices have to opt in to be included. For specialist service units, participation in the audit is mandatory. In 2016-2017, the participation rate was 95.3 per cent of all GP practices in England and Wales.

The information collected from GP practices for the audit is individual level data and contains demographic information such as age, sex, ethnicity, diabetes duration and geographic variables such as postcode. As the data collected is identifiable data, it is subject to dissent, meaning patients can dissent from the NDA. This means that the registrations for people with diabetes captured may not be a true reflection of the whole population of people with diabetes for that GP practice.

For the first time this year the audit has collected multiple readings of HbA1c, blood pressure, cholesterol and BMI, and also collected diagnoses of serious mental illness.

Accuracy and Reliability

The 2016-2017 audit covers the majority of England and Wales with a participation rate of 95.3 per cent. Participation is published at Clinical Commissioning Group (CCG) and Local Health Board (LHB) level. Caution should be borne in mind when looking at areas with low participation, and when looking at areas with variable participation over time.

Information is collected from GP clinical systems and equivalent data is collected from secondary care settings.

The collection occurs over a six week period and GP practices and secondary care units provide the information. During this window, data quality is conducted on the key fields contained within the data.

The information has been presented at GP level since the 2013-2014 audit.

The audit report includes only information on eight of the nine care processes. This is due to an issue in the data collection which was identified in the 2011-2012 audit. The issue highlighted inconsistencies in how eye screening was being recorded in GP records, therefore this care process is not covered in the audit.

There are a number of data quality issues the reader needs to be made aware of when looking at the data included in the report:

1. The Urine Albumin/Creatinine Ratio Care Process

Caution should be taken regarding the screening test for early kidney disease (Urine Albumin Creatinine Ratio, UACR) prior to 2013-2014.

Caution should be borne in mind when looking at the variation across CCGs for this care process over time. It was found that some areas were recording the albumin value as a text field and this was not feeding through to the audit via our extractions.

This method of recording is thought to be due to the different ways this test is carried out in localities. Although the values are not brought through to the dataset, the dates for these values now are. This means that even if the value is not extracted, the date that the test took place is, therefore this date is used to determine if the care process has taken place.

Since 2013-2014, the data has been extracted in a consistent way across all service providers. This resulted in better performance for the albumin care process completion in 2013-2014. However, due to the retirement of the Quality Outcomes Framework (QOF) indicator and the potential for the focus of GP practices to shift, the performance in recording this care process has fallen in recent years.

2. Blood Pressure Treatment Target

There is a data quality issue for the blood pressure treatment target for a small number of GP practices in 2015-16. A reading for systolic blood pressure has been recorded however measurements for diastolic blood pressure are incomplete. These patients have been recorded as not meeting the treatment target and have not been included in the treatment target calculation. This issue does not affect the recording of whether the annual check for blood pressure has taken place.

3. The HbA1c Care Process

There is an issue with data supplied to NHS digital for a large number of GP practices in the 2011-2012 audit regarding HbA1c (blood glucose) recording. While this did not materially affect the findings in the national report, the CCG level care process and treatment target reports covering potentially affected practices for the 2011-2012 publication have been removed from our website. The issue is restricted to the 2011-2012 audit data.

4. BMI for Kenyon Medical Centres (M86015) in 2016-17

We are aware of an issue with the BMI data that was received for Kenyon Medical Centres (M86015). An error occurred in transit of the file to NHS Digital. This practice is showing as underperforming when this is not necessarily the case.

Testing

The method of selecting the relevant fields from the GP systems is via read codes. The list of codes is available upon request. The Primary Care Information Service (PRIMIS) developed the extract specifications and these were tested on a number of system types prior to opening the data collection window. Validation

A provisional report is produced for each CCG which provides a comparison to the QOF data on people registered as having diabetes and an overall figure on the care process completion and the treatment target achievement. This provides an opportunity for CCGs to address any data quality issues with the GP practices and re-submit the data.

One file was submitted for Wales by NHS Wales Informatics Service (NWIS) for the 2016-17 audit year and validated.

Internal validation is completed on receipt of the automated extracts to ensure that there are no systematic issues with the data and these are resolved where possible within the collection window.

Any data quality issues with the data are included alongside analysis to make readers aware. For example, “eye screening” - this information was removed from the 2011-2012 publication as the data was not deemed reliable, and the following statement was added to the report.

“To improve alignment with NICE guidelines, a revised read code set of terms describing digital eye screening was used. This identified that variation in the use of terminology and its impact on the consistency of data extraction from electronic clinical records rendered it unreliable as a measure of this care process. The NHS Diabetic Eye Screening Programme (NHS DESP) records every digital eye screening and we believe that its records should now be used as the preferred measure for this annual care process. Presently this is reported only nationally.”

Timeliness and Punctuality

Timeliness

The 2016-17 audit collected data covering the period 1st January 2016 to 31st March 2017. The data was collected between June and August 2017. It was processed and ready to analyse in November 2017 and published on 10 November 2017.

Punctuality

Reports will be produced and data will be presented at National, CCG, LHB and general practice level for England.

The time lag between the end of the audit period and the publication of the data was just over seven months.

Accessibility and Clarity

Key findings and recommendations will be presented in a full report to be published in March 2018. Data is initially provided at English GP, CCG and LHB in Excel format for ease of interrogation and further analysis by users.

Web links to the technical specifications of the data are available through the NHS Digital website and are available here https://digital.nhs.uk/nda

The key elements of the data collection are presented in the methodology document that accompanies this document. Coherence and Comparability

Comparability over time

The NDA has been running since 2003-2004, however there is inconsistency in how the data has been processed prior to 2009-2010. Therefore caution should be taken when looking at earlier years of the audit data.

Due to the nature of the audit, and the fact that it is not mandated, users should also bear in mind the differences in participation over time.

Where the definition of a care process or treatment target has changed, a time series is produced to allow valid comparisons over time.

In the 2012-2013 audit the Blood Pressure Target was amended and a paper was produced to show the impact of this change on the data previously published this can be found here.

Reduction in Participation in 2013-2014 and 2014-2015

There was a drop in participation in the 2013-2014 and 2014-2015 audit collections due to changes in the way the data was collected from GP practices. There was increased complexity to the processes for registrations and submissions due to new Information Governance ‘opt in’ requirements. The collection used to be on an opt out basis. Changes to the Confidential Advisory Group (CAG) requirements meant that from 2013-2014 this changed to an opt in basis. The new governance meant that GP practices had to actively give permission for their data to be extracted or extract the information themselves and provide it directly to the audit.

Participation of GP practices in England is variable across the country. This may be due to the varied levels of support for participation offered to GP practices by CCG’s.

Analysis was completed to ensure that the data collected for 2013-2014 and 2014-2015 was representative both in demographics and in performance:

- There was some change in the age profile of the cohort of patients included in the audit in these collections. Standardising results for this leads to only a very slight change in the national figures, so for simplicity of interpretation and explanation the results have not been standardised.
- With a reduction in participation there was the potential for those practices taking part to be self-selecting towards those that have performed well. Analysis has been carried out on those practices that have participated in the latest collections and also earlier collections. This suggests that there is no bias towards high performing practices taking part. Comparable with other sources

QOF collects information on people registered with diabetes; however this is only broadly comparable as there are differences in the collection period and the definitions of the indicators. More information on the differences can be found here.

QOF collects the number of patients aged 17 years and above with Type 1 and Type 2 diabetes, QOF is an aggregated return and is mandatory for GP practices to participate. The table below compares the number of diabetes registrations in the NDA with the number of diabetes registrations in QOF and shows the case ascertainment based on this.

Diabetes registrations for Type 1 and Type 2 diabetes for patients aged 17 years and over in England and Wales by audit year

| Country | Audit year | NDA registrations | QOF registrations | Percentage of patients recorded in NDA compared with QOF | |---------|------------|-------------------|-------------------|--------------------------------------------------------| | England | 2016-2017 | 2,943,205 | 3,116,399 | 94.4% | | | 2015-2016 | 2,530,561 | 3,033,529 | 83.4% | | | 2014-2015 | 1,702,610 | 2,913,538 | 58.4% | | | 2013-2014 | 1,586,380 | 2,814,004 | 56.4% | | | 2012-2013b | 1,937,705 | 2,703,044 | 71.7% | | | 2011-2012 | 2,269,580 | 2,566,436 | 88.4% | | | 2010-2011 | 2,086,593 | 2,455,937 | 85.0% | | Wales | 2016-2017 | 192,870 | 191,590 | 100.7% | | | 2015-2016 | 190,731 | 188,644 | 101.1% | | | 2014-2015 | 176,472 | 183,348 | 96.2% | | | 2013-2014 | 159,981 | 177,212 | 90.3% | | | 2012-2013b | 120,152 | 173,299 | 69.3% | | | 2011-2012 | 137,768 | 167,537 | 82.2% | | | 2010-2011 | 83,802 | 160,533 | 52.2% |

aNDA data is collected over a 15 month period, between 1st January and 31st March, whereas QOF data is collected over a 12 month period, between 1st April and the 31st March. Therefore, the figures are not directly comparable.

bIn 2012-2013 QOF methodology was updated to include all diabetes (apart from gestational diabetes), not just type 1 and type 2. NDA methodology has been updated in accordance Assessment of User Needs and Perceptions

The NDA advisory group (consisting of patient representatives, Diabetes UK, clinicians, GP representatives, researchers and interested analysts from NHS Digital and Public Health England) provide advice on both analysis and content of the reports as well as the direction and development of the audit.

The NDA team has an active role in the National Cardiovascular Health Intelligence Network (NCVIN) workshops to gain a better understanding of how the CCG’s and localities use the data and how we can improve the publication and supporting information. These workshops are conducted quarterly and are co-ordinated by Public Health England (PHE) and bring together epidemiologists, analysts, clinicians and patient representatives.

NHS Digital is keen to gain a better understanding of the users of this publication and of their needs. Your feedback is welcome and may be sent to enquiries@nhsdigital.nhs.uk (please include ‘National Diabetes Audit’ in the subject line).

Alternatively you can call our enquiries team on 0300 303 5678

Or write to

NHS Digital, 1 Trevelyan Square, Boar Lane, Leeds, LS1 6AE.