# Inferential Statistics Is an Unfit Tool for Interpreting Data
## 5 When Inferential Statistics Can Be Helpful  
When researchers have applied their domain knowledge to interpret the data in their study sample and are satisfied with understanding what it means, they often also want to make some generalizations based on the result. In the second example on \(f_{\rm o}\) (Figure 2 and Table 2), 38% (3 out 8) of the participants had relevant changes in \(f_{\rm o}\) (reached \(f_{\rm o}\) frequencies around 180 Hz), and in this section, I will shorthand those individual treatment outcomes as those individuals having \"benefitted\" from the training. The study result suggests--_given that we accept the representativeness of the sample, the validity of the outcome measures, the study methodology, my interpretation of the data based on my domain knowledge, and so on--_that among a future group of similar transwomen going through a similar training program some of those individuals will also benefit (reach\(f_{\rm o}\) frequencies of around 180 Hz). However, it is improbable that exactly 38% of future transwomen will benefit. Can we give a reasonable range of possible future outcomes based on this study? Answering _this_ question is where some types of inferential statistics can be helpful. However, before tackling inferential statistics, researchers should think hard about whether their study methodology, recruitment procedure, and sample composition are such that it makes sense to try to generalize the study results. The _default_ in many research fields is to report inferential statistics even when it is obvious that the sample is not representative or that the circumstances will never exist outside of the study. It would be preferable if researchers separate when a study result is interesting for theoretical reasons from when a study is interesting because its result is to some degree generalizable, and only report inferential calculations after having decided that they do contribute meaningfully. A meaningful application of _inferential_ statistics is to give some reasonable ranges of what can be _generalized_ from the study sample, when it is appropriate to do so. A confidence interval (and its cousins likelihood intervals and credible intervals) illustrates what values _outside the sample_ are most compatible with the data, given the statistical assumptions used to compute the interval [28]. To exemplify, we can calculate a confidence interval around the proportion of participants who \"benefited\" in example 2, namely three out of eight participants. This can be achieved with different degrees of sophistication, but for simplicity, we can calculate a Wilson confidence interval to be 95% CI [0.14, 0.69]. This interval (given all assumptions underlying the interval) illustrates that the data are compatible with somewhere between 14% and 69% of similar patients going through similar training will also benefit (reach\(f_{\rm o}\) frequencies around 180 Hz). Researchers should try to discuss the complete interval compatible with their data and accept the uncertainty it contains [28]. Importantly, interpreting confidence intervals does not fundamentally change just because zero happens to be included in the interval. If the 95% CI in example 2 would have been between 0% and 55%, we proceed as if nothing happened and say something to the effect of \"If 0% of patients have 'beneficial' outcomes, the training is ineffective, and if 55% of patients have 'beneficial' outcomes, that is quite promising, _and our data are equally compatible with both scenarios_\". The range of reasonable parameter values does not collapse unto a point, just because the confidence intervals happen to overlap with zero. Using a confidence interval to declare something as \"effective\" or \"significant\" based on whether or not zero is included in the interval faces all the same issues of \"dichotomania\" as relying on \(p\)-values does [9,21]. To return to the main point, simply calculating a confidence interval does not by itself make the data meaningful.