Title: Evaluation of diagnostic imaging test performance: Including interobserver variability and time to diagnosis
Funder: Department of Health and Social Care

Abstract: Research questionHow are interobserver variability and time to diagnosis best evaluated and incorporated into studies evaluating diagnostic imaging tests? BackgroundObtaining an accurate medical diagnosis is fundamental to receiving the correct treatment. Medical imaging is an essential part of diagnostic pathways; over 40 million imaging tests are performed within NHS England each year. Reducing time to diagnosis directly benefits patients by lessening anxiety and allowing earlier treatment, possibly leading to better health outcomes. Decreasing time to diagnosis is a NHS priority, and endorsed by major patient charities such as Cancer Research UK. In addition, diagnostic error in imaging interpretation has been estimated at 3-5%, meaning up to 2 million diagnostic errors could be made each year. Diagnostic errors have the potential for significant patient harm. Studies of imaging technology usually consider basic diagnostic accuracy, despite this being only one aspect of test evaluation. Such studies often neglect many important attributes, notably: Time to diagnosis (incorporating waiting times for performance and interpretation of the test, together with any additional tests required i.e. the time for the full diagnostic test pathway) Interobserver variability (variability in test interpretation between different clinicians)Considering these aspects, over and above simple diagnostic accuracy, will provide more robust evidence to inform implementation, or otherwise, of diagnostic imaging technology in the NHS. Aims and objectivesTo develop and evaluate study designs that best incorporate evaluation of interobserver variability and time to diagnosis into studies of diagnostic medical imaging, and to produce evidence-based guidance on analysis methods and results presentation. Plan of investigation Perform systematic reviews of primary studies, methodology papers and guidelines considering interobserver variability and time to diagnosis, and provide a summary of the study design and methods in current practice. Use case studies and simulation studies to explore different study designs, methods of analysing time to diagnosis, interobserver variability and incorporation of these results into diagnostic test performance study outputs. Three NIHR funded studies of imaging technology: METRIC (MRI and ultrasound in Crohn's disease), STREAMLINE C and STREAMLINE L (whole-body MRI and conventional imaging in colon and lung cancer staging, respectively) will be used as case studies. Use routinely collected data sources to explore the feasibility and validity of estimating time to diagnosis.DisseminationThe systematic reviews will be published, providing a summary of study design and analysis methods currently used. The case studies and simulation studies will be used to produce multiple methods papers making recommendations for appropriate study design, analysis methods, result presentation and incorporation for interobserver variability and time to diagnosis. Case study papers will explore interobserver variability and time to diagnosis. The research results will be presented at relevant conferences, a training course and to specialist NIHR imaging research groups. Anticipated impact To improve methods of measuring diagnostic test performance to deliver optimal care to patients within the NHS, by having more robust evidence for healthcare professionals and stakeholders to make decisions on appropriate use of diagnostic tests.

Lay abstract: Background The correct medical diagnosis is fundamental to a patient receiving the best care. An incorrect diagnosis can lead to the patient receiving the wrong treatment, unnecessary treatment or even no treatment at all. Imaging tests, such as CT and MRI scans, are often used by doctors when making a diagnosis in a patient. There are many types of tests available, with some better at recognising certain conditions than others. When deciding which imaging test to use, the accuracy (how well the test identifies patients with and without the condition), safety and cost are main considerations. It is also important to consider the time from a patient's first symptom to receiving a diagnosis 'time to diagnosis'. A timely diagnosis helps a patient by reducing anxiety, speeding up treatment and improving long-term health. The NHS has a target of diagnosing patients within 6 weeks, but this has not been met since 2013. Cancer Research UK also has faster diagnosis as one of its top priorities. Similarly important is whether performing the same test again at a different time, or having a different doctor interpreting the images, would give a different result `test reliability'. Over 40 million imaging tests are performed each year in NHS England, to diagnose conditions including cancer and heart diseases. From these, up to 2 million mistakes could be made in diagnosis each year, potentially causing significant harm to patients. AimThe aim is to show how choosing an imaging test to diagnose a condition can be improved by having more detail on test accuracy, reliability and time to diagnosis. Design and methods Planned work includes: Review studies that report time to diagnosis and test reliability for imaging tests, to understand how studies are completed and to show the best way to complete studies in the future. Explore the possibility of using routinely collected data to measure time to diagnosis. Explore different ways of designing studies, analysing data and presenting results by using real study data from three ongoing studies in diagnosing Crohn's disease, colon and lung cancer to find the best ways to estimate time to diagnosis and test reliability. To use simulated `mock' data (computer generated to be realistic but not from real patients) to test how new methods will work in different situations.Patient and public involvementMembers of the public have helped choose the most important questions to focus on in my research, and write this summary. They will be involved throughout the project by: Choosing issues to consider in reviews. Deciding on situations to test with mock data. Helping with communicating results to the general public.Output I will publish summaries of the design and analysis methods used in existing time to diagnosis and test reliability studies. I will produce advice based on my research on the best way to design new studies, analyse data, present results and to use these results to look at time to diagnosis and test reliability of diagnostic tests. I will present results at conference and training courses. Potential benefitsThe methods for evaluating diagnostic imaging tests will be improved for future studies. Patients will benefit directly from this research as the results will allow them to get the most appropriate imaging test to diagnose their condition. Getting the correct diagnosis will mean they receive the best possible care. The NHS will also benefit by gaining a better evidence base for choosing which imaging tests to use in hospitals across the UK. Reducing the number of patients waiting for test results and reducing diagnostic errors would lessen the burden on individual hospitals and the NHS as a whole.

Research activity codes: 4.2 EVALUATION OF MARKERS AND TECHNOLOGIES
Health categories: GENERIC HEALTH RELEVANCE