Title: Creating and validating a tool for identifying and addressing unwarranted variation in GP prescribing behaviour
Funder: Department of Health and Social Care

Abstract: Research question We aim to develop new methods to identify CCGs and practices that are outliers in prescribing behaviour; validate these methods; implement them as a tool on the OpenPrescribing site; then further iterate on the tool following clinician and user feedback. Background Our previous research has highlighted substantial variation in prescribing beghaviours, across various different areas of medicine. Variation is sometimes partially explained by demographic changes, or by Clinical Commissioning Group (CCG) policy, but much of the remaining variation is unexplained. OpenPrescribing is an openly accessible online data tool, used by 80,000 people in the past year including clinicians, pharmacists and medicines optimisation teams. Having developed a large body of standard measures driven by clinical evidence and experience, we are now using innovative data science techniques to identify and validate new signals suggestive of unusual prescribing. This innovative approach can help identify problematic prescribing across all drugs and GP practices in England, in order to target work to improve quality of care. Aims and Objectives Our aim is to develop new computational approaches that will identify unwarranted variation in prescribing behaviour, and help clinicians and medicines optimisation teams to explore and address that variation. Specific objectives to achieve this are: Develop and refine new methods of outlier detection Implement these new methods as tools on OpenPrescribing.net Validate and iteratively improve the tool with clinicians and users. Methods Our pilot work using data-driven approaches to discover unusual prescribing has uncovered numerous promising methods for identifying outliers in both cross-sectional and time-series data. For example, we have run analyses looking for prescribing behaviours with unusual distributions, such as those with extreme values of skew and kurtosis, to identify prescribing behaviours exhibited by only a small number of practices; and examined practices that are unusually fast or slow at responding to safety alerts, price changes, or guidance. We will further refine the methods we have piloted, add additional methods as appropriate, then use our software engineers to implement the methods into a freely accessible tool for the use of GPs, medicines optimisation teams, policy makers and other users. Feedback from users will then be used to iteratively refine the tools. Timelines for delivery We will have a working prototype of the tool available within 9 months of the start of the project. This will give us ample time to improve the tool by adding additional methods of outlier detection, improving features in response to feedback, and optimise the tool to ensure it is fast and reliable. Anticipated Impact and Dissemination All tools developed will be implemented on the freely and openly accessible OpenPrescribing site, which had 80,000 unique users last year. We expect user numbers will continue to rise with the inclusion of additional features. We will also seek to actively promote the new tools with site visits at CCGs, engaging with policy makers, conventional academic publications, conference presentations, and our substantial social media following.

Lay abstract: Aims We aim to help GPs prescribe medicines safely and effectively, and safely reduce prescribing costs. We will produce a series of new tools that identify GP practices, and drugs, with unusual patterns of prescribing. We will then make these tools freely available to all GPs, patients, and the public, through our website, which already has very high traffic. Background OpenPrescribing.net is a freely accessible website, developed by our team, that allows anyone to explore monthly prescribing data for every NHS GP Practice in England. Our OpenPrescribing.net had 120,000 users in the last year. NHS clinicians and pharmacists use the tool regularly to compare their prescribing against other practices and regions. Sometimes GPs have unusual prescribing behaviour for good reasons: for example, they may have unusual patients. But often, systematic patterns of unusual prescribing show that there are improvements to be made, to help GPs prescribe more safely, more effectively, and more cost-effectively, as shown in our previous research. It is therefore important that we find new methods to identify potentially problematic prescribing, and help improve prescribing. Design and methods We will develop new methods to identify GP practices with unusual prescribing patterns. These tools will search through all prescribing of all drugs across all practices in England, and find the practices where prescribing is unusual. We will take these new methods and turn them into interactive tools on the OpenPrescribing.net site. We will collect feedback alongside the tools, and use this to further improve them. Patient and public involvement We will recruit a PPI group of approximately 8 people. Engagement with the group would involve face to face meetings and remote phone/video chat. We expect to meet with the group approximately every 6 months throughout the project. Because our site has high traffic, we already get lots of email from patients, clinicians and others. We make it easy for people to contact us: our tools are popular, in part, because we welcome feedback and feature requests from all interested users. Dissemination We will create new tools that will be freely accessible to all through our OpenPrescribing.net website. OpenPrescribing already has a very wide user-base of clinicians, patients, pharmacists and policymakers, with over 120,000 users in the last year. We have strong connections with senior policymakers. All our work is made freely available for re-use by other researchers, patients, and software developers: this includes all our data, software, tools, methods and academic papers. We will present our work at conferences and to individual CCGs, publish academic papers, actively seek new collaborations, and share useful stories about our work via our blog. Our strong social media presence means we can rapidly get information out to interested parties.

Research activity codes: 8.1 ORGANISATION AND DELIVERY OF SERVICES
Health categories: GENERIC HEALTH RELEVANCE