# Enhanced Speech Emotion Recognition Using Conditional-DCGAN-Based Data Augmentation
## 3 Proposed Methodology
### Proposed Model  
In this study, we propose a new data augmentation technique that combines the traditional Conditional-DCGAN with an emotional speech recognition model. In the traditional Conditional-DCGAN method, a discriminator evaluates the authenticity of the Mel-spectrogram generated by the generator, which is in the range [0, 1]. This approach requires time at the beginning of training until the discriminator is sufficiently trained, and effective training of the discriminator may be limited, especially with a small amount of data. In this study, we further introduce an emotional speech recognition model to evaluate how the generated Mel-spectrogram expresses emotion and implement a new loss optimization method by reflecting it in the loss calculation. The emotional speech recognition model is pre-trained with an accuracy of 0.86, and the Mel-spectrogram generated by the model uses a loss adjustment method where the better the emotion is expressed, the better the reward. While this process borrows the reward concept from reinforcement learning, it is somewhat different from the traditional policy gradient [35] method of reinforcement learning. In the policy gradient method, the agent learns the probability of choosing an action in such a way to maximize the reward. In this study, we learn by minimizing the loss value by evaluating how well the generated Mel-spectrogram expresses emotions. Table 2 summarizes how we applied the concept of reinforcement learning in this study. In this study, the probability values obtained from the emotion recognition model are used as rewards to optimize the generator's network. The more accurately the Mel-spectrogram expresses emotions, the higher the probability values, and these values are used to adjust the loss so that the generator is guided to produce data that better represent emotional expressions. The model is designed such that the higher the reward, the lower the loss value, reinforcing the generator's learning as emotional expression improves. Although the concept of rewards from reinforcement learning is applied, this approach differs from the traditional policy gradient method. While policy gradient methods directly optimize the probability of selecting actions, our approach focuses on minimizing loss based on the probability values provided by the emotion recognition model. Thus, the goal \begin{table}
\begin{tabular}{c c} \hline
**Concept** & **Application in the Proposed Method** \\ \hline Agent & Generator \\ Environment & Pre-trained emotion speech recognition model \\ Action & The process of generating Mel-spectrograms \\  & by the generator \\ State & The information currently held by the generator \\  & (Loss value) \\ Reward & The probability value assigned by the emotion recognition model to the \\  & generated Mel-spectrogram \\ Policy & The generator’s network that determines how to generate \\  & Mel-spectrogram \\ \hline \end{tabular}
\end{table} 
Table 2: Application of Reinforcement Learning Concepts in This Study. Figure 1: Mel-spectrogram. of this study is not to maximize rewards but to minimize loss based on the accuracy of the emotion recognition model. The equation used in this process is shown in Equation (3). \[\
abla_{\theta}J(\theta)=E_{\tau\sim\pi_{\theta}}[\
abla_{\theta}log\pi_{\theta}( a_{t}|s_{t})]R \tag{3}\]  
\(R\) is a value that quantifies the emotion expression accuracy of the generated Mel-spectrogram, and in this study, it was converted to (1-emotion recognition probability value) to minimize the loss. \(log\pi_{\theta}(a_{t}|s_{t})\) denotes the logarithmic probability of selecting action \(a_{t}\) given state \(s_{t}\) at time \(t\), where a higher value means that the probability of choosing the correct behavior in that state increases. This indicates that the generated Mel-spectrogram is an accurate representation of the emotion. The optimization method proposed in this study combines the traditional Conditional-DCGAN loss function with the new loss function based on emotion recognition.