# Whispered Speech Recognition Based on Audio Data Augmentation and Inverse Filtering
## 6 Results
### The Impact of Data Augmentation Strategies  
Of the several augmentation techniques outlined in Section 3, three methods (single) and their combinations (2-way and 3-way) were examined in this study, resulting in a total of seven different augmentation techniques. * Pitch Shift (PS),
* Time Stretch (TST),
* Volume Control (VC),
* PS + TST,
* TST + VC,
* PS + VC, and
* PS + TST + VC. The probability of applying the augmentation method was set to 0.5 (see Figure 2a). The sequential augmentations had the following parameter ranges:  
* PS--Semitone Shift Range: [-2, 2]
* TST--Speedup Factor Range: [0.8, 1.2]
* VC--Volume Gain Range: [-3 dB, 3 dB]  
The values (real numbers) for the augmentation parameters are randomly assigned within these specified ranges. It is worth noting that adding noise and time shifting were excluded from the analysis due to their lack of positive influence on the performance of the ASR system. The average word recognition accuracy for the speech database DBtest can be found in Table 4 (HMM) and Table 5 (CNN). The codes for experiments are accessible in [32]. In these experiments, the speech database used for augmentation is of the same size as the original one (sequential augmentation, as illustrated in Figure 2). Moreover, the analysis included training solely with augmented speech samples (W--whisper augmented and P--pseudo-whisper augmented), as well as the fusion of original and augmented datasets (OW, OP, and OWP raws). \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{
\begin{tabular}{c} **Training** \\ **Scenario** \\ \end{tabular} } & \multirow{2}{*}{**PS**} & \multicolumn{4}{c}{**Augmentation**} \\  & & & **VC** & **PS + TST** & **PS + VC** & **TST + VC** & **PS + TST + VC** \\ \hline W & 90.66 & 90.66 & 91.61 & 89.06 & 90.75 & 90.98 & 89.88 \\ P & 72.16 & 71.32 & 69.51 & 72.11 & 72.06 & 72.26 & 70.84 \\ OW & 91.29 & 91.21 & 91.61 & 91.09 & 91.29 & 91.49 & 91.14 \\ OP & 91.17 & 90.91 & 91.34 & 91.06 & 91.20 & 91.09 & 90.88 \\ OWP & 91.53 & **91.91** & **92.09** & 91.37 & 91.48 & **92.06** & 91.52 \\ ORIGINAL & & & & 91.61 & & & \\ (NO AUG) & & & & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Average recognition accuracy (DBtest database) for seven data augmentation techniques and HMM framework. The accuracies with statistically significant improvements are bolded. The results obtained from training using solely augmented utterances (W and P raw in Tables 4 and 5) indicate the following:  
* When augmentation was applied to whispered utterances (W raw), the performance of the HMM recognizer slightly weakened, while it remained comparable for the CNN recognizer. * When augmentation was applied to pseudo-whisper utterances only (P raw), the performances significantly worsened for both the HMM and CNN recognizer. When training was applied to both the original and augmented datasets (OW, OP, and OWP raw in Tables 4 and 5), the performances were notably improved for the CNN recognizer for all techniques. At the same time, only a particular scenario and augmentation method led to performance improvement for the HMM recognizer. The significance of the inverse filtering algorithm is evident when examining the OWP and OW rows in Tables 4 and 5. In every augmentation scenario, adding pseudo-whisper utterances in training has resulted in a marked improvement in accuracy for both recognizers. In order to evaluate the statistical significance of obtained results in cases when augmented datasets are added to the original, a one-tailed Wilcoxon test for hypothesis H\({}_{\text{A}}\) was performed. * Hypothesis H\({}_{\text{0A}}\): \"Both recognizers (initial recognizer when training was applied using only original samples and the other with augmentation--OW/OP/OWP) produce the same accuracy\". * Hypothesis H\({}_{\text{1A}}\): \"Recognizer with augmentation--OW/OP/OWP gives higher accuracy\". The \(p\)-values for H\({}_{\text{0A}}\)/H\({}_{\text{1A}}\) hypothesis are provided in Table 6 for values of alpha 0.05.