# ODIN112-AI-Assisted Emergency Services in Romania
## 3 Method
### Automatic Speech Recognition Model  
For this project, we used the Kaldi toolkit to develop our model (see Figure 6) and designed a framework for ingesting our training data. As with any Kaldi recipe, the processing logic happens in several stages, each representing a step of the data preparation process or training a model. The most important steps are extraction of the acoustic features from input recordings, training several HMM-GMM models that become gradually more complex and use more data, and finally training a Deep Neural Network model that uses twelve Time Delay Neural Network Factorization (TDNNF) layers. Time Delay Neural Networks classify patterns with shift-invariance and learn context at each layer of the network. This network does not require explicit segmentation prior to classification.  
The features extracted for the acoustic features are the Mel Frequency Cepstral Coefficients (MFCC; Mermelstein [35]), followed by applying Cepstral Mean and Variance Normalization (CMVN). The MFCCs are extracted from frames of 25 ms with a stride of 10 ms. For the input of the TDNNF-DNN, additional identity vectors (also known as iVectors) are extracted to characterize speaker characteristics.  
Identity vectors (iVectors) [36] are generally used in speaker identification and represent information about the speaker and audio recording characteristics. In our context, iVectors are used as an additional input to help the network adapt to speaker characteristics and usually result in a 1% WER improvement. The iVectors used by [PERSON] are similar in the sense that they are based on similar ideas as Joint Factor Analysis, where a Universal Background Model collects sufficient statistics for iVector extraction, and a Probabilis  
Figure 5: ODIN112 Architecture: Communication and integration of components.  
tic Linear Discriminant Analysis backend computes a similarity score between iVectors. [PERSON] replaced the Universal Background Model based on a GMM with one considering a DNN [37].  
Before the features are extracted, the data is first processed to normalize the audio and label inputs. For example, the audio is transformed to mono-channel, 8000 Hz, 16-bit PCM WAVs, and various automatic corrections are applied to the labels (for example, the punctuation marks are added, \"i\" inside word is replaced with \"a\" and vice-versa), numbers are transformed to words (for example \"42\" is transformed to \"patruzeci si doi\"), and words are transformed to uppercase.  
There are four HMM-GMM models built on top of each other, each with corresponding methods to improve accuracy and to consider larger volumes of data. Each subsequent model is used to align the training data used for the next model (see Figure 7). The first model is a simple, context-independent mono-phone model. The second model is a more complex, triphone context-dependent system trained using first (delta) and second derivatives (delta-delta or double delta) features. The third model applies Linear Discriminant Analysis (LDA) and a diagonalizing transformation known as Maximum Likelihood Linear Transform (MLLT) to the input training. The last HMM model's features are transformed one last time using a feature space maximum likelihood linear regression (fMLLR), which is known to be a Speaker Adaptive Training (SAT) method.  
Figure 6: Model architecture for Automatic Speech Recognition system.  
The last model (i.e., our final ASR model) was trained for 12 epochs using the MFCC and iVectors of the entire training dataset. It was defined using [PERSON]'s XConfig domain-specific language (DSL) and employs the implementation of the neural network named \"nnet3\" from [PERSON] (available at [[https://kaldi-asr.org/doc/dnn3.html](https://kaldi-asr.org/doc/dnn3.html)]([https://kaldi-asr.org/doc/dnn3.html](https://kaldi-asr.org/doc/dnn3.html)), accessed on 1 December 2022; [PERSON] et al. [38]).