# Efficient and Robust Arabic Automotive Speech Command Recognition System
## 3 Methodology
### Classifier Selection
#### 3.3.2 Deep-Learning Models  
For deep-learning models, we tested the performance of four strong models, namely CNN, LSTM, Bi-LSTM, and our proposed architecture, Bi-LSTM-CNN. All of these models use categorical cross-entropy loss with a SoftMax output layer of 20 units, Adam optimizer, 32 batch size, and a learning-rate scheduler that multiplies the learning rate by 0.2 when the training plateaus for three consecutive epochs. These models are trained for 40 epochs.  
Convolutional Neural Networks (CNN)  
CNNs have been recently used widely as they have been shown to improve SR performance [34]. In addition to their simple structure and the use of fewer parameters to train the model (relatively), CNNs are known for their ability to learn complex patterns and recognize local and global input-data characteristics, making them suitable for SR applications [35].  
The architecture of the built 1D CNN model consists of three stacked hidden layers with 512, 256, and 128 units, respectively. Each layer is followed by Batch Normalization and Max-Pooling layers. The 1D CNN processes sequences of features, such as Mel-Frequency Cepstral Coefficients (MFCCs), where each feature represents a 1D array of temporal data. Finally, a fully connected SoftMax layer with 20 units (corresponding to the number of commands) completes the model. This design allows the network to effectively capture and learn temporal patterns in the sequential feature data.  
Long Short-Term Memory Networks (LSTM)  
LSTMs are artificial recurrent neural network architectures proposed to solve the gradient-vanishing problem [36]. They have proven to be effective in processing sequential data due to their ability to capture long-term dependencies, which is crucial in SR where the input data is a sequence of audio frames.  
The architecture of the LSTM model consists of three stacked hidden layers with 512, 256, and 128 units, respectively. Each layer is followed by a dropout layer with a 0.20 omission probability. Finally, a fully connected SoftMax output layer of 20 units is succeeded.  
Bidirectional Long Short-Term Memory Networks (Bi-LSTM)  
Bi-LSTMs are an advanced form of LSTM networks that enhance the capacity to capture contextual information from both preceding and succeeding sequences in sequential data. By analyzing the input data in both forward and reverse directions, Bi-LSTMs are especially useful in applications such as speech recognition, where comprehending the context of audio frames from both directions is essential for precise predictions.  
The architecture of the Bi-LSTM model consists of three stacked hidden layers with 256, 128, and 64 units, respectively. Each layer is followed by a Batch-Normalization layer. Finally, a fully connected SoftMax output layer of 20 units is succeeded.  
BiLSTM-CNN Architecture  
To leverage the complementary advantages of CNN and Bi-LSTM, capturing complex patterns, short-term dependencies, and long-term dependencies, we built a hybrid model that combines CNN and Bi-LSTM layers into one unified architecture. As illustrated in Figure 4, our proposed architecture consists of two stacked hidden Bi-LSTM layers with 128 and 64 units, respectively, which are responsible for extracting long dependency features. Each Bi-LSTM layer is followed by a Batch normalization. These Bi-LSTM layers are followed by two stacked hidden CNN layers with 256 and 256 units, respectively. CNN layers are responsible for extracting complex patterns and local features. Each layer is followed by a Batch normalization, max-Poling layers, and a dropout layer with 0.20 omission probability. This is followed by a fully connected Dence layer of 128 units and a SoftMax output layer of 20 units.