# Enhanced Speech Emotion Recognition Using Conditional-DCGAN-Based Data Augmentation
## 1 Introduction  
The advancement and integration of Internet of Things (IoT) and Artificial Intelligence (AI) technologies have increasingly highlighted the importance of speech recognition technology in modern society. In particular, emotion speech recognition is gaining importance in various fields of application, including personalized services, enhanced security, and the medical field. Research on emotion recognition is actively being conducted and developed alongside the growth of the global market. Early research on emotion speech recognition primarily focused on the Hidden Markov Model (HMM) [1; 2; 3; 4]. HMM was effectively used for recognizing patterns in sequence data by assuming hidden states and inferring them. More recently, with the advancement of deep learning, studies utilizing deep neural networks [5; 6; 7; 8; 9; 10; 11] have become mainstream. Furthermore, research employing graph-based theories to leverage the structural and statistical relationships between speech data has also emerged [12]. Graph-based methods can also extract key structural features from speech data, providing simpler and faster performance than traditional deep learning methods. These studies help in learning the complex patterns of speech data, allowing for more accurate differentiation of varioussubtle emotional states. However, building speech emotion recognition systems still faces many challenges due to the diversity and complexity of speech data. Emotions can vary not only based on acoustic characteristics but also depending on the context, and in certain scenarios such as business conversations, cultural and linguistic differences can significantly influence emotional expression. Therefore, it is important for speech emotion recognition research to use data that minimize these differences and focuses on the auditory characteristics of emotions. This research mainly uses dialog data where actors focus on emotional expressions, which is effective for learning emotional patterns. Commonly used datasets include Emo-DB [13], RAVDESS [14], and IEMOCAP [15], though these datasets are significantly smaller in scale compared to large datasets used in other fields such as image research [16; 17]. In this study, we propose a novel data augmentation technique using Conditional-DCGAN (Deep Convolutional Generative Adversarial Nets), which combines CGAN (Conditional Generative Adversarial Nets) [18] and DCGAN [19], to address these issues and improve the performance of emotion recognition models. In this study, the Mel-spectrogram, which analyzes the characteristics of the voice signal and visually represents the change in frequency of the signal over time, was used as an image. Additionally, instead of the traditional loss function calculation methods used by GAN (Generative Adversarial Nets) [20], we implemented a new loss function calculation method borrowed from reinforcement learning, enabling a more precise model for emotion analysis. Reinforcement learning [21] is a field of machine learning in which an agent learns optimal actions through trial and error. The agent interacts with its environment and receives rewards based on the outcomes of its actions. Using these rewards, the agent develops an optimal strategy for action. In this study, we applied the concept of reinforcement learning to explore a new method for improving the performance of adversarial generative models. Specifically, we designed a reward mechanism using an emotion recognition model to evaluate the generated images, and used this to adjust the training direction of the generative model. This paper aims to demonstrate the impact of the augmented data on the performance improvement of emotion speech recognition models. For this purpose, we used a CNN + BiLSTM model capable of effectively analyzing the unique characteristics of Mel-spectrogram data as an emotion speech recognition model. We used WA (weighted accuracy) and UAR (weighted average recall) as performance metrics. The structure of this paper is as follows.