National Diabetes Audit Data Quality Statement, 2017-18

Published 13th June 2019

## Contents

| Section | Page | |----------------------------------------------|------| | Introduction | 3 | | Relevance | 4 | | Accuracy and Reliability | 4 | | Testing | 6 | | Validation | 7 | | Timeliness and Punctuality | 7 | | Timeliness | 7 | | Punctuality | 7 | | Accessibility and Clarity | 7 | | Coherence and Comparability | 8 | | Comparability over time | 8 | | Reduction in participation in 2013-14 and 2014-15 | 8 | | Comparability with other sources | 8 | | Assessment of User Needs and Perceptions | 10 | Introduction

The National Diabetes Audit (NDA) is managed by the Healthcare Quality Improvement Partnership (HQIP) on behalf of NHS England and delivered by NHS Digital formerly the Health and Social Care Information Centre (HSCIC), working in collaboration with Diabetes UK and Public Health England (PHE).

The NDA is a major national clinical audit, which measures the effectiveness of diabetes healthcare against NICE Clinical Guidelines and NICE Quality Standards, in England and Wales. The NDA collects and analyses data for use by a range of stakeholders to drive changes and improvements in the quality of services and health outcomes for people with diabetes.

The NDA answers five key questions:

1. Is everyone with diabetes diagnosed and recorded on a practice diabetes register?
2. What percentage of people registered with diabetes received the NICE key processes of diabetes care?
3. What percentage of people registered with diabetes achieved NICE defined treatment targets for glucose control, blood pressure and blood cholesterol?
4. What percentage of people registered with diabetes are offered and attend a structured education course?
5. For people with registered diabetes what are the rates of acute and long term complications (disease outcomes)?

The NDA supports improvement in the quality of diabetes care by enabling participating NHS services and organisations to:

- assess local practice against NICE guidelines
- compare their care and care outcomes with similar services and organisations
- identify gaps or shortfalls that are priorities for improvement
- identify and share best practice
- provide comprehensive national pictures of diabetes care and outcomes in England and Wales

Through participation in the audit, local services are able to benchmark their performance and identify where they are performing well and improve the quality of treatment and care they provide. On a national level, wide participation in the audit also provides an overview of the quality of care being provided in England and Wales. Relevance

Data is extracted from general practice clinical systems and specialist diabetes services in secondary care hospitals. The audit is a voluntary audit for GP practices, meaning GP practices need to opt in to be included. For specialist services, participation in the audit is mandatory. In 2017-18, the participation rate was 98.3 per cent of all GP practices in England and Wales.

The information collected from GP practices for the audit is individual level data and contains demographic information such as age, sex, ethnicity, diabetes duration and geographic variables such as postcode. As the data collected is identifiable data, it is subject to dissent, meaning patients can dissent from the NDA. This means that the registrations for people with diabetes captured may not be a true reflection of the whole population of people with diabetes for that GP practice.

For the first time this year, the audit has gathered data on specified drug types prescribed to people with diabetes in the audit.

Accuracy and Reliability

The 2017-18 audit covers the majority of England and Wales with a participation rate of 98.3 per cent. Participation is published at Clinical Commissioning Group (CCG) and Local Health Board (LHB) level. Caution should be taken when looking at areas with low participation and when looking at areas with variable participation over time.

Primary care data are extracted from GP clinical systems via the General Practice Extraction Service (GPES); the practice must approve the extraction. Specialist diabetes services submit their data manually through the NDA Data Landing Portal.

Data has been reported at GP practice level since the 2013-14 audit.

The audit report includes only information on eight of the nine care processes. This is due to an issue in data collection which was identified in the 2011-12 audit. Examination of the issue highlighted inconsistencies in how eye screening was being recorded in GP records. Therefore, this care process is not covered in the audit.

There are several data quality issues the reader should be aware of when looking at the data included in this report:

1. **Urine albumin/creatinine ratio care process**

   Caution should be taken when using screening test results for early kidney disease (Urine Albumin Creatinine Ratio, UACR) prior to 2013-14 and assessing variation across CCGs for this care process over time.

   Some areas recorded the albumin values in ‘text’ field format that did not carry through to the audit during the extraction process.

   This method of recording is thought to be due to the different ways this test is carried out across localities. Although these values are not brought through to the dataset, the dates for these values are. This means that even if the value is not extracted, the date that the test took place is. Therefore, this date is used to determine if the care process has taken place. Since 2013-14, data have been extracted in a consistent way across all service providers. This resulted in better performance for albumin care process completion in 2013-14. However, due to the retirement of the Quality Outcomes Framework (QOF) indicator and the potential refocusing of GP practices, performance in recording this care process has fallen in recent years.

2. Blood pressure treatment target

A data quality issue for the blood pressure treatment target affected a small number of GP practices in 2015-16. A reading for systolic blood pressure was recorded but measurements of diastolic blood pressure were incomplete. These patients have been recorded as not meeting the treatment target and have not been included in the treatment target calculation. This issue does not affect the recording of whether the annual check for blood pressure has taken place.

3. The HbA1c care process

An issue affected data supplied to NHS Digital for a large number of GP practices in the 2011-12 audit regarding HbA1c (blood glucose) recording. While this did not materially affect the findings in the national report, the CCG level care process and treatment target reports covering potentially affected practices for the 2011-12 publication have been removed from our website. The issue is restricted to the 2011-12 audit data.

4. Possible data file issues in 2016-17 when submitted 14-16th June 2017

We are aware of an issue that may have affected a number of data files collected for the 2016-17 audit that were received between the 14th and 16th June 2017. An error occurred in transit of some files to NHS Digital. Some practices may show as underperforming when this is not necessarily the case. For example, BMI results for Kenyon Medical Centres (M86015) and the HbA1c results for Staithes Surgery (B82046).

5. Foot surveillance data from TPP SystmOne GP clinical system in 2016-17

It has been found that foot surveillance data may have been under-reported in 2016-17 for several GP practices that used the TPP SystmOne clinical system. This may account for some of the fall in overall foot surveillance completion in 2016-17.

6. Incorrect data submitted for Leicester Royal Infirmary (RWEAA) in 2016-17

It has been identified that the data submitted to the audit from Leicester Royal Infirmary was for far fewer patients than was intended. The results for Leicester Royal Infirmary in 2016-17 should be treated with extreme caution.

7. Incorrect HbA1c data submitted for Hull and East Yorkshire Hospitals NHS Trust (RWA) in 2016-17

It has been identified that the HbA1c data submitted to the audit from Hull and East Yorkshire Hospitals NHS Trust was incorrect. The data show a much lower HbA1c reading for patients than was the case. The performance of this trust appears to be much better for 2016-17 than it was, so any relative deterioration in performance should be treated with extreme caution.

8. Diabetes type in 2017-18

Initial data quality checks on the 2017-18 NDA data showed that a higher than expected number of people had a read code which matched to a general diabetes diagnosis, rather than specifically Type 1, Type 2 or one of the smaller Types. Where possible, diabetes Type data collected in previous audits was used to override the general diagnosis with a more specific diabetes Type. This method was also used to complete the diabetes Type for individuals with missing data. Although this method of improving the data worked well, it is still likely that a small number of people with Type 1 diabetes, who have not appeared in a previous audit, will have their data included in the “Type 2 and other” category.

9. Body Mass Index (BMI) care process in 2017-18

GP practices using the Vision clinical system supplier were unable to submit a BMI date or value for people registered with the practice under the new GPES collection process, due to a technical issue. To ensure that BMI data could be included in the NDA for these practices, the 2017-18 collection also extracted height and weight data, where available. For all system suppliers, as well as using submitted BMI dates to indicate that the care process had taken place, dates were used where weights had been taken. This may account for some of the uplift in BMI care process completion in 2017-18.

10. BMI treatment target in 2017-18

As described above, GP practices using the Vision clinical system supplier were unable to submit a BMI date or value for people registered with the practice. Where no BMI value was submitted for a person, but there were values for height and weight, the BMI value was calculated using the standard formula (weight in kg) / (height in metres)^2.

11. Incorrect organisation code and insulin pump data submitted for Manchester Royal Infirmary (RW3MR) in 2017-18

Data for Manchester Royal Infirmary were incorrectly submitted, and subsequently reported in the NDA, under the organisation code for Manchester Children’s Hospital (RW3RM). Also, all patients were incorrectly submitted as using an insulin pump.

Testing

Relevant data is extracted from GP systems via read codes. The list of codes is available upon request. The NHS Digital Primary Care Domain developed the specifications specifically for the NDA extract, and these were verified on several system types prior to collection of the data. Validation Data were automatically extracted from GP systems in England via system supplier-developed queries and GPES. Data submissions were initially checked and certified by GPES using test submissions. Internal validation checks were completed within the NDA team to ensure that data had been received from all eligible participating practices and there were no systematic issues with the data. Comparisons were made with GP practice and CCG level counts from previous years.

One file was submitted for Wales by NHS Wales Informatics Service (NWIS) for the 2017-18 audit year and validated.

Any data quality issues with the data are included alongside analysis to ensure users can interpret the results appropriately. For example, eye screening information was removed from the 2011-12 publication as the data was not deemed reliable, and the following statement was added to the report:

“To improve alignment with NICE guidelines, a revised read code set of terms describing digital eye screening was used. This identified that variation in the use of terminology and its impact on the consistency of data extraction from electronic clinical records rendered it unreliable as a measure of this care process. The NHS Diabetic Eye Screening Programme (NHS DESP) records every digital eye screening and we believe that its records should now be used as the preferred measure for this annual care process. Presently this is reported only nationally.”

Timeliness and Punctuality

Timeliness The 2017-18 audit collected data covering the period 1st January 2017 to 31st March 2018. Primary care data was extracted from GP clinical systems in May 2018. A ‘mop up’ extraction was carried out in June 2018 to capture GP practice data that could not be collected in May. Specialist diabetes services submitted their data during a five-week period between May and June 2018. It was processed and ready to analyse in September 2018 and published on 8th November 2018.

Punctuality Reports are produced and data presented at National, CCG, LHB and General Practice level for England. The time lag between the end of the audit period and the publication of the data was just over seven months.

Accessibility and Clarity Key findings and recommendations are presented in the full NDA report published in June 2019. Data was initially provided in November 2018 for England and Wales at GP, CCG and LHB levels in interactive Excel spreadsheets for ease of interrogation and further analysis by users. Web links to the technical specifications of the data are available through the NHS Digital website here: https://digital.nhs.uk/nda

The key elements of the data collection are presented in the methodology document that accompanies this document. Coherence and Comparability

Comparability over time The NDA has been running since 2003-04; however, there are inconsistencies in how the data has been processed prior to 2009-10. Therefore, caution should be taken when looking at data from earlier years of the audit.

Users should also bear in mind the differences in participation over time.

In the 2012-13 audit, the blood pressure treatment target was amended to be consistent with the Quality and Outcomes Framework (QOF). A paper was produced to show the impact of this change on the previously published data. This can be found here.

Reduction in participation in 2013-14 and 2014-15 There was a drop in participation in the 2013-14 and 2014-15 audit collections due to changes in the way the data was collected from GP practices. There was increased complexity to the processes for registrations and submissions due to new Information Governance ‘opt in’ requirements. The collection used to be on an opt out basis. Changes to the Confidential Advisory Group (CAG) requirements meant that from 2013-14 this changed to an opt in basis. The new governance meant that GP practices had to actively give permission for their data to be extracted or extract the information themselves and provide it directly to the audit.

Participation of GP practices in England is variable across the country and this may be due to the varied levels of support for participation offered to GP practices by CCGs.

Analysis was completed to ensure that the data collected for 2013-14 and 2014-15 were representative both in demographics and in performance:

- There was a change in the age profile of the cohort of patients included in the audit in these collections. Standardising results, to account for this, leads to only a very slight change in the national figures. Therefore, to simplify interpretation and explanation, the results have not been standardised.
- With a reduction in participation there was the potential for those practices taking part to skewed towards those that perform well. Analysis was carried out on practices that participated in the latest collections and those that participated in earlier collections. The findings suggest that there is no bias towards high performing practices taking part.

Comparability with other sources QOF collects information on people registered with diabetes; however, this is only broadly comparable as there are differences in the collection period and the definitions of the indicators. More information on the differences can be found here.

QOF collects the number of patients aged 17 years and above with Type 1 and Type 2 diabetes, QOF is an aggregated return and is mandatory for GP practices to participate. The table below compares the number of diabetes registrations in the NDA with the number of diabetes registrations in QOF and shows the case ascertainment based on this.

## Diabetes registrations for Type 1 and Type 2 diabetes for people aged 17 years and over in England and Wales by audit year

| Country | Audit year | NDA registrations\* | QOF registrations | Percentage of patients recorded in NDA compared with QOF | |---------|------------|--------------------|------------------|--------------------------------------------------------| | England | 2017-18 | 3,131,775 | 3,227,760 | 97.0% | | | 2016-17 | 2,872,565 | 3,139,340 | 91.5% | | | 2015-16 | 2,530,561 | 3,033,529 | 83.4% | | | 2014-15 | 1,702,610 | 2,913,538 | 58.4% | | | 2013-14 | 1,586,380 | 2,814,004 | 56.4% | | | 2012-13b | 1,937,705 | 2,703,044 | 71.7% | | | 2011-12 | 2,269,580 | 2,566,436 | 88.4% | | Wales | 2017-18 | 195,975 | 193,605 | 101.2% | | | 2016-17 | 191,260 | 191,590 | 99.8% | | | 2015-16 | 190,731 | 188,644 | 101.1% | | | 2014-15 | 176,472 | 183,348 | 96.2% | | | 2013-14 | 159,981 | 177,212 | 90.3% | | | 2012-13b | 120,152 | 173,299 | 69.3% | | | 2011-12 | 137,768 | 167,537 | 82.2% |

- NDA data is collected over a 15 month period, between 1st January and 31st March, whereas QOF data is collected over a 12 month period, between 1st April and the 31st March. Therefore, the figures are not directly comparable.

b In 2012-13 QOF methodology was updated to include all diabetes (apart from gestational diabetes), not just type 1 and type 2. NDA methodology has been updated in accordance. Assessment of User Needs and Perceptions

The NDA advisory group (consisting of patient representatives, Diabetes UK, clinicians, GP representatives, researchers and interested analysts from NHS Digital and Public Health England) provide advice on both analysis and content of the reports as well as the direction and development of the audit.

The NDA team has an active role in the National Cardiovascular Health Intelligence Network (NCVIN) workshops to gain a better understanding of how the CCG’s and localities use the data and how we can improve the publication and supporting information. These workshops are conducted quarterly and are co-ordinated by Public Health England (PHE) and bring together epidemiologists, analysts, clinicians and patient representatives.

NHS Digital is keen to gain a better understanding of the users of this publication and of their needs. Your feedback is welcome and may be sent to enquiries@nhsdigital.nhs.uk (please include ‘National Diabetes Audit’ in the subject line).

Alternatively, you can call our enquiries team on 0300 303 5678.

Or write to:

NHS Digital, 1 Trevelyan Square, Boar Lane, Leeds, LS1 6AE