# Enhanced Speech Emotion Recognition Using Conditional-DCGAN-Based Data Augmentation
## 3 Proposed Methodology
### Proposed Model
#### 3.2.2 CNN + Bi-LSTM Architecture  
This study evaluated the impact of the proposed data augmentation technique on the performance of the emotion speech recognition model by using Mel-spectrograms extracted from the original voice data and those generated by the Conditional-DCGAN. The emotion speech recognition model combines CNN and Bi-LSTM. Mel-spectrograms are based on the time axis (\(X\)-axis) and frequency axis (\(Y\)-axis), representing temporal and frequency changes in the speech signal, while the color (\(Z\)-axis) visually displays decibel information. CNN is effective at analyzing the spatial patterns of Mel-spectrograms, enabling it to learn specific frequency bands and temporal features related to emotions in the speech \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Network** & **Layer** & **Input Shape** & **Kernel Shape** & **Output Shape** \\ \hline \multirow{8}{*}{Generator} & Embedding & & - & [6] \\  & Linear & [106] & - & [16,384] \\  & ReLU & [16,384] & - & [16,384] \\  & ConvTranspose2d & [1024, 4, 4] & [4, 4] & [512, 8, 8] \\  & BatchNorm2d & [512, 8, 8] & - & [512, 8, 8] \\  & ReLU & [512, 8, 8] & - & [512, 8, 8] \\  & ConvTranspose2d & [512, 8, 8] & [4, 4] & [256, 16, 16] \\  & BatchNorm2d & [256, 16, 16] & - & [256, 16, 16] \\  & ReLU & [256, 16, 16] & - & [256, 16, 16] \\  & ConvTranspose2d & [256, 16, 16] & [4, 4] & [128, 32, 32] \\  & BatchNorm2d & [128, 32, 32] & - & [128, 32, 32] \\  & ReLU & [128, 32, 32] & - & [128, 32, 32] \\  & ConvTranspose2d & [128, 32, 32] & [4, 4] & [64, 64, 64] \\  & BatchNorm2d & [64, 64, 64] & - & [64, 64, 64] \\  & ReLU & [64, 64, 64] & - & [64, 64, 64] \\  & ConvTranspose2d & [64, 64, 64] & [3, 3] & [1, 64, 64] \\  & Tanh & [1, 64, 64] & - & [1, 64, 64] \\ \hline \multirow{8}{*}{Discriminator} & Embedding & [1] & - & [4096] \\  & Conv2d & [2, 64, 64] & [3, 3] & [64, 64, 64] \\  & LeakyReLU & [64, 64, 64] & - & [64, 64, 64] \\  & Conv2d & [64, 64, 64] & [4, 4] & [128, 32, 32] \\  & BatchNorm2d & [128, 32, 32] & - & [128, 32, 32] \\  & LeakyReLU & [128, 32, 32] & - & [128, 32, 32] \\  & LeakyReLU & [128, 32, 32] & - & [128, 32, 32] \\  & Conv2d & [128, 32, 32] & [4, 4] & [256, 16, 16] \\  & BatchNorm2d & [128, 32, 32] & - & [256, 16, 16] \\  & LeakyReLU & [256, 16, 16] & - & [256, 16, 16] \\  & Conv2d & [256, 16, 16] & [4, 4] & [512, 8, 8] \\  & BatchNorm2d & [256, 16, 16] & - & [512, 8, 8] \\  & LeakyReLU & [512, 8, 8] & - & [512, 8, 8] \\  & Conv2d & [512, 8, 8] & [4, 4] & [1, 4, 4] \\  & AdaptiveAvgPool2d & [1, 4, 4] & - & [1, 1, 1] \\  & Sigmoid & [1] & - & [1] \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Architecture of the generator and discriminator in Conditional-DCGAN for Mel-Spectro gram augmentation. signal. This reflects how emotions are often tied to specific frequency and volume changes. Bi-LSTM analyzes these temporal changes, recognizing long-term patterns of emotional variations in the speech signal. By combining CNN's ability to learn the spatial features of Mel-spectrograms with Bi-LSTM's ability to capture temporal changes, this model enables more precise and accurate analysis of emotion speech recognition. The CNN + Bi-LSTM model used in this study consists of three Convolution Layers, one Bi-LSTM Layer, and two Fully Connected Layers. The size of the Convolution Layers gradually increases as the layers deepen, followed by batch normalization and the application of the ELU activation function, with Max-Pooling used to reduce the dimensionality of the data. The extracted features are then flattened and passed to the Bi-LSTM, ultimately producing the final output using the Softmax function to yield class probabilities. This structure is designed to optimize the accuracy of emotion speech recognition, and Dropout was applied during the training process to randomly drop neurons in the hidden layers, preventing the model from overfitting by becoming too adapted to the training data. This visualizes how the proposed data augmentation technique contributes to the performance improvement of the emotion speech recognition model.