Statistical considerations (or recommendations) for publishing in Science and Medicine in Football

During the last months, the world of sport science has witnessed an important but also hard-nosed debate on a statistical approach commonly used in our area: the Magnitude Based Inferences (MBI; Batterham and Hopkins 2006). While concerns have already been raised about this method a few years ago (Welsh and Knight 2015), the publication by Sainani in 2018 (Sainani 2018) has attracted widespread attention of other statisticians and researchers (Borg et al. 2018). In addition to the recent scientific publications, several discussions have also been posted in social media and blogs, which exemplify the modern way to debate, even in science. The replies from the proponents of MBI (Alan Batterham and Will Hopkins) in response to the points of criticism being made can be found here: www.sportsci. org. Based on the criticism of the method, some journals such as Medicine and Science in Sports and Exercise (see journal’s Instructions for Authors), which incidentally is also the journal which contributed to the popularisation of this method (Hopkins et al. 2009), has decided to ban MBI analyses completely. Irrespective of the decision from other journals, the question inevitably also affects Science and Medicine in Football. As editors we try to offer a reasonable and balanced solution, while following the debate closely and possibly awaiting a conclusion. This editorial summarizes our recommendations on the preferred statistical approaches when submitting a paper to our journal. MBIs were developed to address the drawbacks related to the Null Hypothesis Significance Testing (NHST). A shared view in this debate, however, is that effectively NHST has several shortcomings as already underlined in the literature (Szucs and Ioannidis 2017). Where the positions diverge is in the ‘solution’. In light of this, we believe that the main analysis reported in the studies should provide information about the magnitude of an effect and the precision of the estimate. In other words, we recommend that papers submitted to our journal report the confidence intervals and effect sizes according to other guidelines such as those of the American Psychological Association (2010) or author guidelines like the instructions of the International Journal of Epidemiology (IJE 2018). As commonly used, the confidence level should be 95%. However, in particular cases, the author may give reasons to apply a different level of confidence. Other approaches such as MBI or NHST can be used but only as a complementary analysis. Relying on confidence intervals and effect sizes facilitates researchers to evaluate and interpret the effects also in terms of relevance (practical, physiological or clinical). This was one of the goals of MBI but even without using this approach, we strongly encourage the authors to discuss and comment the relevance of the magnitude of the effects even if in a qualitative way. As an example, in studies on football match analysis: Is a difference of 50 m in high intensity running practically important? Is this difference large enough to justify different training strategies? And what about the uncertainty as reflected by the width of the confidence interval? It is also important to remember that the confidence intervals can be interpreted in the sense of the NHST by observing whether the confidence intervals overlap the value of no effect, i.e., 0 for group differences, or 1 in the case of relative risks, corresponding to the null hypothesis. So, reporting the p values may even be redundant (but we leave the authors freedom to decide whether to report or not). Furthermore, as suggested by Lakens (2017), there are established alternative test procedures that can also be used such as the equivalence tests if we want to stay in the frequentist realm or the region of practical equivalence if we move to a fully Bayesian approach. The latter has been suggested to provide similar results compared to the MBI and uses the smallest effect of interest instead of the smallest worthwhile change of the MBI (Hopkins and Batterham 2018). Recommending the confidence intervals and the effect sizes as primary analysis, we clearly push the authors to go beyond the p values and for this reason, the main interpretation of the study results should be based on confidence intervals, effect sizes, practical significance and uncertainty (width of the confidence interval).