# Enhanced Speech Emotion Recognition Using Conditional-DCGAN-Based Data Augmentation
## 5 Conclusions  
As a new approach to improve the performance of emotional speech recognition, this study proposes a data augmentation technique using a generative model combining CGAN and DCGAN. This method is an attempt to solve the data sparsity problem faced by existing emotion recognition models and is specifically designed to enable generative models to learn the detailed speech patterns required for emotion recognition. The proposed model proposes a new loss function calculation method, which borrows the reward concept of reinforcement learning and uses the probability value of the emotion recognition model as the reward. This is designed to effectively manage the loss of the generator. Through this reward mechanism, by providing Mel-spectrograms with high probability values in the emotion recognition model as rewards, the generative model can generate more sophisticated emotion data, which has a positive impact on the model performance. The generator used Emo-DB data to generate Mel-spectrograms, and we compared the performance of the emotion speech recognition model using the Mel-spectrograms extracted from the original Emo-DB data and the augmented data. We used Weighted Accuracy (WA) and Unweighted Average Recall (UAR) as performance metrics for comparison. WA is the \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline  & & \multicolumn{6}{c}{**Predicted**} \\ \cline{3-8}  & & **Anger** & **Boredom** & **Fear** & **Happiness** & **Sadness** & **Neutral** \\ \hline \multirow{4}{*}{Real} & anger & 0.85 & 0.00 & 0.08 & 0.07 & 0.00 & 0.00 \\  & boredom & 0.00 & 0.83 & 0.00 & 0.00 & 0.17 & 0.00 \\  & fear & 0.20 & 0.00 & 0.80 & 0.00 & 0.00 & 0.00 \\  & happiness & 0.30 & 0.00 & 0.00 & 0.70 & 0.00 & 0.00 \\  & sadness & 0.00 & 0.00 & 0.11 & 0.00 & 0.89 & 0.00 \\  & neutral & 0.00 & 0.38 & 0.00 & 0.00 & 0.00 & 0.62 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Confusion Matrix using original Emo-DB only. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Author** & **Database** & **Features** & **Model** & **WA** & **UAR** \\ \hline This paper & Emo-DB & Mel-spectrogram & CNN + Bi-LSTM & 79.31\% & 78.16\% \\ \hline This paper & Emo-DB & Mel-spectrogram & CNN + Bi-LSTM & 91.46\% & 91.61\% \\ \hline [PERSON] et al. [24] & Emo-DB & 1582 features from + Augmented & DNN & 84.49\% & 83.33\% \\ \hline [PERSON] et al. [28] & Emo-DB & Log Mel-spectrogram & Deep CNN & 84.61\% & 82.27\% \\ \hline [PERSON] et al. [29] & Emo-DB & 32D Low Level Descriptor + 40D High-Level Statistical Functions & CNN + Bi-LSTM & 86.44\% & 84.65\% \\ \hline [PERSON] et al. [30] & Emo-DB & MFCC & LSTM-Transformer & 87.72\% & 85.55\% \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Comparison of Performance Metrics. accuracy calculated by considering the number of samples in each class as a weight and is a performance evaluation metric that takes into account class imbalance. UAR is the recall of all classes averaged equally, ensuring that all classes are evaluated fairly. In our experiments, the WA and UAR values were 91.46% and 91.61%, respectively, when the augmented data was included, compared to 79.31% WA and 78.16% UAR when the original data was used alone. It also outperformed the methods of [PERSON] et al. [24] and [PERSON] et al. [28], which used different data augmentation techniques. This confirms that the proposed data augmentation technique is effective in improving the performance of emotion recognition models. This study is particularly significant in that it demonstrates that effective learning is possible even when the amount of data is limited, and it is applicable not only to emotion speech data but also to various domains that suffer from data imbalance. However, there is a limitation that some emotional states suffered from mode collapse problem, which did not fully reflect the diversity of certain emotional states.