Why screening tests to predict injury do not work – and probably never will ...

This paper addresses if and how a periodic health examination to screen for risk factors for injury can be used to mitigate injury risk. The key question asked is whether it is possible to use screening tests to identify who is at risk for a sports injury – in order to address the deficit through a targeted intervention program. The paper demonstrates that to validate a screening test to predict and prevent sports injuries, at least three steps are needed. First, a strong relationship needs to be demonstrated in prospective studies between a marker from a screening test and injury risk (Step 1). Second, the test properties need to be examined in relevant populations, using appropriate statistical tools (Step 2). Unfortunately, there is currently no example of a screening test for sports injuries with adequate test properties. Given the nature of potential screening tests (where test performance is usually measured on a continuous scale from low to high), substantial overlap is to be expected between players with high and low risk of injury. Therefore, although there are a number of tests demonstrating a statistically significant association with injury risk, and therefore help the understanding of causative factors, such tests are unlikely to be able to predict injury with sufficient accuracy. The final step needed is to document that an intervention programme targeting athletes identified as being at high risk through a screening program is more beneficial than the same intervention program given to all athletes (Step 3). To date, there is no intervention study providing support for screening for injury risk. Imagine that you are planning an injury prevention program for your team. Chances are that you will consider including a periodic health examination to screen athletes for injury risk as a key component. And, as outlined as the second step in the van Mechelen model[1], the classic approach to sports injury prevention research, it is necessary to understand the risk factors and injury mechanisms that play a part in the occurrence of sports injuries to develop a targeted prevention program. [2, 3] This paper addresses this step in the van Mechelen model, focusing on if and how a periodic health examination (PHE) to screen for risk factors for injury can be used to mitigate injury risk. I will use hamstring and ACL injuries, two of the most common injuries in team sports,[4, 5] to illustrate key issues. The key question is: “Is it possible to use screening tests to identify who is at risk for a hamstring or ACL injury – in order to address the deficit through a targeted intervention program? ” THE PURPOSE OF SCREENING Screening is a strategy used in a population to detect a disease in individuals without signs or symptoms of that disease. The intention is to identify pathological conditions early, thus enabling earlier intervention and management in the hope of reducing future morbidity and mortality. Perhaps the most famous, and successful, example is the infant screening programs instituted world-wide in the early 1960s for phenylketonuria (Følling’s disease). [6] If left untreated, phenylketonuria leads to severe brain function abnormalities. In contrast, patients who follow the prescribed dietary treatment from birth, may have no symptoms at all. More recent screening programs include breast cancer screening with mammography and prostate cancer screening with a blood test measuring prostate-specific antigen. However, it should be noted that although screening may lead to an earlier diagnosis, not all programmes have been shown to be beneficial and the value of current programs for breast and prostate cancer screening are being debated. [7, 8] To ensure that screening programmes confer the benefits intended, the World Health Organization published the Wilson–Jungner criteria for appraising a screening programme. [9] These are the main criteria: (1) that the condition being screened for is an important health problem (depending not just on how serious the condition is, but also how common it is), (2) that there is a detectable early stage, (3) that treatment at an early stage is of more benefit than at a later stage, and (4) that a suitable test is available to detect disease in the early stage. Clearly, injuries represent an important health problem in many sports (criterion 1). However, criteria 2-4 need adaptation when being applied to the case of sports injury prevention. First, while screening for breast cancer involves detecting established disease as early as possible, screening for injury risk usually involves using a performance test to detect impairments which predispose the individual to injury (e.g. hamstring muscle weakness, poor knee alignment). This highlights an important difference between disease detection and injury prediction. When screening for disease the individual is classified as healthy or sick; the outcome is dichotomous (yes/no). When risk factors for injuries are assessed, such as eccentric hamstring strength or knee control in a vertical drop jump test, the outcome is usually continuous. Therefore, one more step is needed to make the test be useful in clinical practice: The continuous variable must be translated to a dichotomous outcome, i.e. whether the athlete is at increased risk or not (yes/no). Second, when screening for disease, the objective is to initiate treatment as early as possible. In sports injury prevention, the objective is early intervention to minimize the risk factor before injury occurs. Examples include a strength training program, targeting players with low hamstring strength, or a balance training to improve knee control, targeting at-risk athletes identified through a vertical drop jump test. Risk factors can be modifiable and non-modifiable, and screening tests typically measure modifiable factors such as strength or knee control since these can be targeted for change, e.g. through specific training programs. However, it should be noted that non-modifiable factors (such as gender or previous injury history) can be used as well, to target intervention measures to the subgroup thought to be at increased risk. DEVELOPING A SCREENING PROGRAM Research on risk factors for injury is advocated for two reasons, to help understand why injuries happen and to predict who is at risk of injury. These two concepts are often, and erroneously, confused. One common misconception is that all it takes to develop a screening test, is to identify a statistically (highly) significant association between the result from a screening test and increased injury risk. Typically, exploratory studies will have a cohort of athletes undergo a series of tests during the pre-season to identify potential risk factors for injury and then injuries are recorded prospectively during the subsequent competitive season. If a significant association is identified between one or more factors and injury risk, it may be tempting to conclude that these can be used to predict who is at risk of injury. However, as illustrated in figure 1, this is only the first step towards a validated screening program. [Figure 1 near here] The next step required is to repeat the same study using the exact same screening test, but this time to use pre-determined cut-off criteria to separate athletes with high risk from the rest. This needs to be done in cohorts representing all potential user groups for the screening test. In this second step, the question is not how strong the association between the test result and injury risk is (e.g. odds ratio, p-value), but how well the test predicts who becomes injured and who does not in a new athlete population, different from the one used to develop the test criteria. Once a test has been developed and validated with acceptable test properties, the final step is to examine the efficacy of a screening program. As noted above, non-modifiable factors may be relevant for stratification purposes, but the ultimate purpose of athlete screening is to identify athletes at risk and reduce their risk by addressing modifiable risk factors. Therefore, a prerequisite for a screening programme to be effective, is that methods exist to modify the risk factor before injury occurs. The final step should be completed as a randomized controlled trial, where the treatment group receives the combined screening and intervention program. The treatment group outcome (injury rate) can be compared to that of a control group, which trains as usual, but should also be compared to that of a control group where all athletes are given the prevention program. This is another issue that separates disease screening from athlete screening. If the disease is breast cancer, treatment is obviously only relevant for those identified with disease (or early stages of disease). However, if the goal is to prevent ACL or hamstring injury, the intervention can be offered to all athletes. The delivery cost is in most cases the same, there is usually no risk associated with the prevention programme per se and the training may even improve sports performance. In other words, for a screening test to be relevant, it needs to capture the majority of athletes with increased injury risk, so they do not miss the opportunity to prevent injury through targeted training programs. Ideally, it should also be able to separate athletes with low risk from the rest of the group, so they do not waste time doing prevention programs they do not need. SCREENING TEST PROPERTIES The ability of a test to predict injury is often described using the same test properties as those used for diagnostic tests, i.e. sensitivity (does the test capture all those with injury), specificity (does it capture only those with injury), positive predictive value (how many with a positive test are injured) and negative predictive value (how many with a negative test are not injured). The following will explain these concepts and examine their relevance in the athle