# Whispered Speech Recognition Based on Audio Data Augmentation and Inverse Filtering
## 3 Audio Data Augmentation  
Data augmentations encompass a variety of effective algorithms developed to generate synthetic data and modify existing datasets to improve recognition systems' capability for generalization. Even though the technique was initially created for image processing, audio data may as well benefit from its usage. In the area of audio-related machine-learning tasks, a number of audio augmentation techniques have been created. Depending on the format of the input audio, data augmentation techniques can be categorized into two groups:  
* Audio data augmentation and
* Spectrogram audio data augmentation.  
Audio Data Augmentation (Audio DA) includes a set of modifications (transformations) performed directly on the raw audio signal (waveform). Commonly used augmentation techniques are pitch shifting, time stretching, volume control, adding noise, and time shifting.  
* Pitch shift  
The voice fundamental frequency (F\({}_{0}\)) is one of the speaker's most significant individual acoustic characteristics. This method modifies the F\({}_{0}\), usually expressed in semitones (one-twelfth of an octave). Pitch shifting is widely recognized as one of the most common augmentation techniques.  
* Time stretch  
This method adjusts the speaking rate or speed of the speech signal while maintaining the pitch. The speedup factor, also known as the stretch ratio, typically ranges from 0.8 (slower) to 1.2 (faster).  
Figure 1: The waveform (**a**) and spectrogram (**b**) of the phrase “Govor sapata”. spoken in Serbian (normal—capital letters; whisper—small letters). The horizontal axis represents time in seconds.  
* Volume control (volume gain) This method applies a gain to the speech signal, typically measured in decibels.
* Add noise This technique adds white noise to the speech signal based on the specified Signal-to-Noise Ratio (\(SNR\)) as defined by Equation (1), where \(P_{s}\) represents the power of the signal and \(P_{n}\) represents the power of the noise. \[SNR\;[\mathrm{dB}]=10 log_{10}\frac{P_{S}}{P_{N}}\] (1)  
Despite improving its robustness in noisy conditions, it may not be effective when the test set is noiseless [21].  
* Time shift This method shifts the audio signal left or right by a specified time in milliseconds. While it is more useful in sound classification tasks, its applicability in ASR tasks is limited [21]. The other audio DA techniques include:
* Wow Resampling
* Clipping
* Add Impulse Response
* Filtering
* MP3 Compression
* Inversion The image and spectrogram DA techniques are described in more detail in [22]. Traditional methods used for data augmentation include flip, zoom range, shift, rotation angle, and brightness range. Spectrogram DA is applied to spectral images and is often used in audio classification tasks. Audio augmentation algorithms can be implemented either in a sequential manner (in series) or independently (in parallel). The architecture of sequential augmentation is illustrated in Figure 2a (single augmented output) and Figure 2b (N augmented output signals) for parallel random sequential augmentations. The probability of applying an augmentation and the value of any parameters that are probabilistically determined are independent.  
Figure 2: The architecture of sequential audio augmentation for one (**a**) and N (**b**) augmentations of the input audio signal.