Does size matter for sports performance researchers?

Many authors who submit manuscripts to the Journal of Sports Sciences now recognize that a change in performance that has been found to be statistically significant might not necessarily mean that the change is of practical importance to sport. Statistical significance indicates that it is unlikely that a change in performance will have been observed if, in reality (in the whole population), there is no change. Another interpretation of statistical significance is that one can be reasonably certain that the change in performance is greater than zero in the population. Nevertheless, given a large enough sample size and a small enough error variance, extremely small and possibly negligible changes in performance might be statistically significant. Therefore, statistical and practical significance are not the same. It is generally accepted that confidence limits help to distinguish between statistical and practical significance in sports performance research (Hopkins et al., 1999; Atkinson and Nevill, 2000). In brief, confidence limits can provide information about the likely ‘true’ size of the change in performance in the population. For example, a researcher may have found 95% confidence limits for an improvement in soccer kicking velocity (after an intervention of some sort) of 0.5 and 1.5 m×s. The researcher can be reasonably certain that the ‘true’ performance change lies somewhere between 0.5 and 1.5 m×s. To judge the likelihood of whether a change in performance is worthwhile, the confidence limits are essentially compared with a minimal change that the researcher deems to have a useful or practical impact. Interested readers can refer to Atkinson and Nevill (2000) for sport-relevant examples of confidence limits interpretation. The salient point is that the use of confidence limits clearly makes the researcher’s delimitation of a ‘minimal worthwhile effect’ important to study conclusions. Hopkins et al. (1999) provided an empirical approach for selecting the minimal worthwhile effect in sports performance research. They suggested that a minimal worthwhile effect could be judged in relation to whether an overall improvement in Olympic medal tally of a country is predicted to occur for a given change in performance. Such predictions have been applied to athletics (Hopkins et al., 1999) and swimming (Stewart and Hopkins, 2000). This work is useful and the concept should be applied to other sports and standards of athlete. Another informative statistic that could be adapted for judging the minimal worthwhile effect in sport performance research is the ‘number needed to treat’ (Dalton and Keating, 2000), which, in a clinical context, is the expected number of patients who must be treated with a therapy to prevent one additional harmful outcome. In a sports performance context, the practical significance of a treatment or intervention might be judged from the expected number of athletes who must be exposed to an intervention to mediate one additional beneficial outcome. Despite the development of these approaches for calculating and comparing practical significance, several questions about the minimal worthwhile effect remain. First, how should one delimit a minimal worthwhile effect when a variable within a construct of performance is being studied as in the above example? What should be the minimal worthwhile effect for an improvement in kicking velocity when the ultimate aim is to improve soccer performance? How would a researcher know that a selected minimal worthwhile effect for kicking velocity of, say, 0.5 m×s would lead to a practical improvement in a player’s overall soccer performance and, ultimately, a change in the league position of a team? It is uncertain whether the generalized effect sizes (Cohen, 1988) that are available would help answer such a question. One might calculate that the generalized effect size for improvement in kicking velocity is, say, 0.9. Although this effect size could be described as ‘large’, the relative contribution of a change in kicking velocity of this size to an overall change in soccer performance might still be small. Secondly, should the complexity and severity of the treatment thought to improve performance be taken into account? For example, should the minimal worthwhile effect for a 3-month regime of plyometric training be larger than for the wearing of a ‘nasal strip’? One might be satisfied that the simple act of wearing a nasal strip might improve performance only slightly on the basis of it being easy to implement. Conversely, one might set a relatively high minimal worthwhile effect for plyometric training to outweigh the considerable investment in time and the potential risk of injury associated with this intervention. In other words, Journal of Sports Sciences, 2003, 21, 73–74