# Whispered Speech Recognition Based on Audio Data Augmentation and Inverse Filtering
###### Abstract
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Ref.** & **Augmentation** & **Metric** & **Speech** \\ \hline
[7] & CycleGAN, SpecAugment, Speed perturbation & Word Error Rate (_WER_) & Children’s \\ \hline
[8] & Glottal source extraction, Silence removal, Impulse response, Noise addition & Unweighted Average Recall (UAR) & Emotional \\ \hline
[9] & GANs, Time Stretch, Pitch Shift, Add Noise & UAR, F-score & Emotional \\ \hline
[10] & Speed perturbation & _WER_ & Continuous \\ \hline
[11] & Simulated Room Impulse Response, Add noise & _WER_ & Continuous \\ \hline
[12] & MP3 and Full rate GSM &
\begin{tabular}{c} _WER_ and Mean \\ Opinion Score (MOS) \\ \end{tabular} & Conversational \\ \hline
[13] & Add noise & _WER_ & Continuous \\ \hline
[14] & Multiplying the region, Replacing the region, Input concatenation & _WER_ & Read \\ \hline
[15] & CycleGAN & _WER_ & Whispered \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The review of augmentation techniques. Increasing the formant bandwidth with a moving average filter. Audio data augmentation utilizes various algorithms to generate new (artificial) audio samples from existing ones (natural) to enlarge the dataset size and mitigate over-fitting issues in ASR systems. In the context of low-resource languages such as Serbian, where only a limited number of publicly available speech databases are accessible, utilization of audio data augmentation can play an important role. The main motivation of this research study is to demonstrate that by applying audio data augmentation techniques to some existing datasets, we can elevate the quality and diversity of training data, ultimately leading to improved performance of ASR. The primary contributions outlined in this paper are:  
* The whispered speech database, which has been annotated and made publicly accessible, was developed for testing purposes. * The efficacy of using inverse filtering as an algorithm for data augmentation in generating pseudo-whisper has been validated in speaker-independent whisper recognition. * The efficacy of typical augmentation techniques (such as pitch shifting, time stretching, and volume adjustment) in artificially enlarging whispered speech databases has been illustrated for an ASR system utilizing Hidden Markov Models (HMM) and Convolutional Neural Networks (CNN). * The beneficial effect of parallel augmentation, where a single natural utterance produces multiple artificial ones, has been verified to enhance the performance of CNN-based recognizers. This study analyzes the performance of the recognition systems that utilize Mel Frequency Cepstrum Coefficients (MFCC) as feature vectors. Additionally, Linear Predictive Coding (LPC), based on the source-filter model of speech signals, is commonly employed in the formant extraction procedure. In this paper, LPC coefficients are utilized for inverse filtering to generate pseudo-whisper utterances from normal speech. Stochastic Gradient Descent (SGD) is an effective algorithm for training neural networks, and the CNN recognizer developed in this study uses SGD with an optimization algorithm based on Adaptive Moment Estimation (Adam). The paper is structured into seven sections as follows. Section 2 provides an overview of whispered speech and compares it with normal speech. Section 3 explores the methodology and theory underlying audio data augmentation techniques. Section 4 explains the rationale behind the usage of the inverse filtering algorithm for artificially expanding the speech database used for training. Section 5 details the experimental preparation, including the speech databases (natural and augmented) used for training and testing, the feature extraction procedure, and the characteristics of ASR systems. Section 6 presents the experimental results and discussion, focusing on the contribution of augmentation strategies and inverse filtering concerning _WER_ reduction. Finally, Section 7 concludes with final remarks and outlines directions for future work.