Title: Evaluating test accuracy without assuming a perfect reference test: application to depression
Funder: Department of Health and Social Care

Abstract: Background Healthcare bodies, such as the National Institute of Health and Care Excellence (NICE), determine which tests should be used in clinical practice based on decision models. Crucial parameters in these models are estimates of sensitivity and specificity, which are informed by studies and meta-analysis of test accuracy studies.The data from test accuracy studies is typically analysed by comparing the results of the test(s) under evaluation - the index test(s) -h; to the best test available - the reference or gold standard test. Furthermore, index tests are assumed to be independent of each other conditional on the disease status -h; an assumption called conditional independence. These assumptions are rarely valid in practice.The most commonly used models which do not assume a perfect reference test estimate test accuracy by assuming all tests are measuring the same unobserved (latent) disease, and each individual is modelled as belonging in one of two classes -h; 'diseased' or 'non-diseased'. These models are often referred to as traditional latent class models in the biostatistics literature.These 'non-standard' models have been used and applied; however, these might not be appropriate to use in all cases, especially for complex conditions such as depression. Furthermore, these models are not particularly flexible; for instance, they cannot be extended to model non-dichotomous tests whilst modelling conditional independence. Few more flexible methods have been proposed. PlanThe aim of this project is to develop and compare flexible statistical models that are widely applicable across clinical areas for the analysis of test accuracy data, both for primary studies and meta-analysis, by: Developing, and comparing (using simulation studies), statistical models which can account for: Imperfect reference tests Conditional dependence Individual patient data (IPD) The nature of complex diseases Specifically for meta-analysis, multiple thresholds when IPD is not available for each study. Applying the methods developed in (1) to depression, allowing me to address the following clinically relevant questions: In individuals with persistent unipolar depression, which tests, or test combinations, are best to assess the severity of depression, particularly in individuals whose depression is characterised mostly by anhedonia, who may benefit more from receiving drug treatment? In post-stroke depression, what tests, or test combinations, is best? As suggested by my patient and public involvement (PPI) group - can depression tests be made shorter and less burdensome? Producing tools to communicate my findings to healthcare professionals, researchers, decision-makers and patients, such as an interactive application using the R package Shiny and tutorial papers including model code.Impact and PPI The methods, tutorials and software that I develop will allow researchers to estimate test accuracy whilst making less modelling assumptions, leading to a better understanding of which tests should be used in clinical practice. This will have a positive impact on clinical and patient outcomes, as it will allow for timely and effective detection of conditions. My PPI group, which consists of patients with lived experience of depression, will be involved throughout the project.

Lay abstract: Why are tests important?A test is something that is carried out to detect, diagnose or monitor a condition. For example, a doctor might use a brief questionnaire to detect whether a patient might be depressed; if they test positive, they may be referred for further, more detailed questionnaires and evaluation. It is also possible to combine two or more tests to improve detection of some conditions. Having good tests is important for patients' care, so that we can effectively detect or rule out conditions, allowing healthcare professionals to make informed decisions about the best treatment for their patients. How do researchers decide what tests to use? How well a test can correctly detect people who have a condition and how well it can detect those who do not have the condition is called the accuracy of the test. To find out the accuracy of tests, we can do a research study called a test accuracy study. This process usually involves comparing the test with another test that is known to work well, called a reference test. Sometimes, more than one test accuracy study is done and statistical methods are used to combine their results to give an overall measure of how well the test works. Information on the accuracy and cost-effectiveness of tests is used by healthcare bodies, such as the National Institute of Health and Care Excellence (NICE), to make recommendations about which tests to use. Why is this project needed?The usual methods for looking at how tests work make many assumptions that are rarely valid in clinical practice. Better methods are needed to detect conditions, help doctors and healthcare professionals to make treatment decisions, and to ensure that patients receive the best care possible. PlanI will improve methods used to analyse these studies, such that more appropriate decisions can be made on which tests to use for clinical conditions. I also aim to make advanced methods more accessible to a range of researchers, by Developing new statistical models which, unlike the usual methods used, can account for the fact that: Reference tests are not perfect. Tests may not be independent. The same test might not be appropriate for all patients, similarly to how different patients can respond very differently when given the exact same drug. Different models may be needed depending on the condition. Special care needs to be taken to find out the accuracy of tests used in combination and the best 'cut-off' value to detect whether somebody has a condition. Applying the methods developed in (1) to depression, allowing me to address important questions such as: What tests, or test combinations, are best to assess the severity of depression, particularly in individuals whose depression is characterised mostly by a loss of pleasure, who may benefit more from receiving different treatments? As suggested by my patient and public involvement (PPI) group - can tests be made shorter and less burdensome? Producing tools to communicate my findings to healthcare professionals, researchers, decision-makers and patients, such as interactive web-based software and tutorials.Impact and PPI The methods, tutorials and software that I develop will allow researchers to assess test accuracy without needing to make strict assumptions, leading to a better understanding of which tests should be used in clinical practice. This will have a positive impact on clinical and patient outcomes, because patients will be diagnosed and treated quicker. It will also have a positive impact on healthcare costs, ensuring that they are spent on the best tests. My PPI group, which consists of patients with lived experience of depression, will be involved throughout the project.

Research activity codes: 4.2 EVALUATION OF MARKERS AND TECHNOLOGIES;7.3 MANAGEMENT AND DECISION MAKING
Health categories: MENTAL HEALTH