# Inferential Statistics Is an Unfit Tool for Interpreting Data
## 3 Two Examples on Interpretating \(f_{\rm o}\) Data in a Voice Training Program  
To illustrate the difference between obtaining a statistically significant test result and interpreting data, my example will be an intervention study evaluating a voice training program aimed at helping transgender persons assigned male at birth, abbreviated as transwomen, to have a voice more in line with their preferences and gender identity. To make the example clear, participants in this example study are recruited because they explicitly want their voice to have a fundamental frequency (the acoustic correlate of pitch) in line with typically female-sounding voices. To justly evaluate such a training program, we need to take many dimensions into account, not least the participants' expectations of the program and their satisfaction with the outcome. Here, the point is not that many dimensions are needed to comprehensively evaluate an intervention. The point is just to contrast interpreting data in a purely statistical manner with interpreting data using domain knowledge. For this reason, I seek a single, simple outcome measure with which I expect most readers to have some experience. Thus, in this example, I will focus only on evaluating the training program _vis-a-vis_ the fundamental frequency of the patients' voice,\(f_{\rm o}\). Although including a (randomized) control group or at least repeated measures is necessary to have strong internal validity in intervention studies [23], for simplicity, I will compare only a measure taken immediately before training and another measure taken directly after training for one group of participants. The research question is whether the training program is effective at helping transwomen to have a fundamental frequency more in line with typically female-sounding voices. In this made-up example, eight transwomen participated in the training program. The inter-individual variability and distribution of \(f_{\rm o}\) measured before and after training are illustrated in boxplots in Figure 1, panel A. Individual treatment trajectories as they relate to reference values of typically female-sounding voices are illustrated in Figure 1, panel B. Table 1 also describes the individual data in the sample. We can see that the participants, _on average_, increased in fundamental frequency and that seven of the eight participants increased in their fundamental frequency. Seeing all this, the researcher must now interpret whether the data support the training as effective or not. Unfortunately, what some researchers do at this junction is that they apply a statistical significance test to _interpret_ the data. The result of a \(t\)-test applied on the _pre-post_ mean difference is \(t_{7}=3.5\), \(p=0.01\). As the \(p\)-value is below 0.05 (the set \(\alpha\) level), the training effect is statistically significant. In some research projects, the intervention would thus be interpreted as effective _because_ the training effect was statistically significant [17; 18; 19]. However, if we instead apply our domain knowledge--what we know about fundamental frequencies, what we know about typically female- and male-sounding voices [24; 25; 26]--to the data and research context, it is clear that none of the transwomen had any _relevant_ changes in their \(f_{\rm o}\). A relevant change would be if a person's \(f_{\rm o}\) moved toward 180 Hz. The intervention was clearly not effective, whatever the \(p\)-value might be. Figure 1: Example 1. **Panel A (left)**: Boxplots over an example sample’s fundamental frequencies (in Hz) before (_pre_) and after (_post_) training. A boxplot illustrates quartiles; the central black thick line is the median, the upper and lower limit of the box are the 25 th and 75 th percentile, and the lower and upper whiskers indicate minimum and maximum values. **Panel B (right)**: Individual treatment “trajectories”. Each participant’s _pre_ and _post_ measure is color-coded per participant and illustrated with a circle connected by a straight line.