# Inferential Statistics Is an Unfit Tool for Interpreting Data
## 4 An Example on Interpreting Visual Analog Scale Data in Voice Training Program  
Because many readers are familiar with \(f_{\text{o}}\) and could use their domain knowledge to interpret the data in the previous examples, the inappropriateness of relying on the statistical significance tests was hopefully clear. However, and importantly, a statistical significance test is equally inappropriate as a tool for interpreting data in situations where we do not have or cannot use our domain knowledge. To continue with the voice training program example, let us say that the participating transwomen also rated their voice  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Participant** & _Pre (\(\mathbf{f_{\text{o}}}\) Hz)_ & _Post (\(\mathbf{f_{\text{o}}}\) Hz)_ & _Diff (\(\mathbf{f_{\text{o}}}\) Hz) (Post-Pre)_ \\ \hline A & 145 & 175 & 30 \\ B & 135 & 180 & 45 \\ C & 125 & 185 & 60 \\ D & 115 & 116 & 1 \\ E & 130 & 124 & \(-6\) \\ F & 120 & 116 & \(-4\) \\ G & 110 & 113 & 3 \\ H & 140 & 140 & 0 \\ Mean & 127.5 & 143.6 & 16.1 \\ SD & 12.2 & 31.4 & 25.4 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Example 2. Each individual’s average fundamental frequency (in Hz) before (_pre_) and after (_post_) treatment. The right-most column is the _post_ minus _pre_ difference in Hz. Further illustrated are the mean and standard deviation for each column. Because the inter-individual variation in the difference scores is so large as only some participants have large treatment effects, a _t_-test applied to the mean difference is not statistically significant. \(t_{7}=1.8\), \(p=0.12\). The standardized mean difference, [PERSON]’s \(d\), is 0.64.  
Figure 2: Example 2. **Panel A (left)**: Boxplots over the sample’s fundamental frequencies (in Hz) before (_pre_) and after (_post_) training. **Panel B (right)**: Individual treatment “trajectories”. Each participant’s _pre_ and _post_ measure is color-coded per participant and illustrated with a circle connected by a straight line. In both figures, 180 Hz is illustrated as a horizontal dashed line because it has been suggested to be fundamental frequency range relevant for being perceived as a female speaker [24; 25; 26] which is the goal of the participants.  
satisfaction on a Visual Analog scale (VA-scale). The question was \"Are you satisfied with your voice?\" and the answering scheme was to put a mark on a line going from No at 0 to Yes at 100. The participants answered this question before and after training, and the example data are illustrated in Figure 3 and Table 3. As before, the researcher must now interpret the data based on their domain knowledge. Moreover, because of the nature of the outcome measure, it is much more difficult to do so in this example. Looking at the figures and tables, something happened with the ratings, but what? Can we be sure that any of the participants are satisfied with their voice after training? Do we know if a rating of 50 on the VA scale is where responses go from unsatisfied to satisfied? Can we be sure that the different participants use the scale in similar ways, or might different VA scale values mean different things for different raters?In cases such as this, where we are not able to apply our domain knowledge, the risk that we fall into the trap of interpreting data based solely on statistical tests is much greater. For example, interpreting the training as effective _because_ the statistical test applied on the VA data is statistically significant (\(t_{7}=8.5\), \(p<0.0001\)). However, a statistical test does not evaluate how the data meaningfully relates to the research question and cannot be used as alchemy to convert unclear data into certainties [27]. If we cannot apply our domain knowledge to interpret the data because we are too unfamiliar with the measuring procedure (or, perhaps, because the measuring procedure is sub-optimal), that is the first and fundamental problem. No statistical test can plaster over that.