# Arabic Emotional Voice Conversion Using English Pre-Trained StarGANv2-VC-Based Model
## 3 Method
#### Training Objectives  
StarGANv2-VC includes adversarial loss (\(\mathcal{L}_{adv}\)), adversarial source classifier loss (\(\mathcal{L}_{adcels}\)), style reconstruction loss (\(\mathcal{L}_{sty}\)), style diversification loss (\(\mathcal{L}_{ds}\)), F0 consistency loss (\(\mathcal{L}_{f0}\)), speech consistency loss (\(\mathcal{L}_{asr}\)), norm consistency loss (\(\mathcal{L}_{norm}\)), and cycle consistency loss (\(\mathcal{L}_{cyc}\)). Given a mel-spectrogram \(X\in\ \mathcal{X}_{y_{src}}\), the source and target domains \(\mathcal{Y}_{src}\ \in\ y\) and \(\mathcal{Y}_{trg}\ \in\ y\), respectively, in addition to a style code \(s\). We trained our model with the following loss functions:  
**The generator objective** function calculation is as follows:  
\[ \begin{split} min_{G,S,M}\ \mathcal{L}_{adv}+\lambda_{adcels} \mathcal{L}_{adcels}+\lambda_{sty}\mathcal{L}_{sty}-\ \lambda_{ds}\mathcal{L}_{ds}+\lambda_{f0}\mathcal{L}_{f0}+\lambda_{asr} \mathcal{L}_{asr}+\lambda_{norm}\mathcal{L}_{norm}\\ +\lambda_{cyc}\mathcal{L}_{cyc}\\ \end{split}  \tag{1}\]  
**The discriminator's objective** function is calculated as follows:  
\[ \begin{split} min_{C,D}-\mathcal{L}_{adv}+\lambda_{cls} \mathcal{L}_{cls}\\ \end{split}  \tag{2}\]  
Figure 1: StarGAN model framework. \(\mathcal{L}_{adv}\) denoted to the **adversarial loss** that figures out how to generate a new mel-spectrogram \(G\) (\(\mathbf{X},s\)) as in Equation (3):  
\[\mathcal{L}_{adv}=\mathbb{E}_{\mathbf{X},\;\mathcal{Y}_{trg}}[logD(\mathbf{X}, \mathcal{Y}_{src})]+\mathbb{E}_{\mathbf{X},\;\mathcal{Y}_{trg}s}\left[log\big{(}1- D\big{(}G(\mathbf{X},s),\mathcal{Y}_{trg}\big{)}\big{)}\right] \tag{3}\]  
where \(\mathbf{X}\) is the mel-spectrogram that was given as an input and \(s\) is a style vector. \(\mathbf{X}\) and \(s\) are the inputs of the generator \(G\); \(D(.,\mathcal{Y})\) is the real or fake classifier's output for the domain \(\mathcal{Y}\in\;y\;\). \(\mathcal{L}_{advels}\) denotes the **adversarial source classifier loss** obtained by using the cross-entropy loss function \(CE(\cdot)\) with the source classifier as in Equation (4):  
\[\mathcal{L}_{advels}=\mathbb{E}_{\mathbf{X},\;\mathcal{Y}_{trg}s}\left[\mathbf{CE} \big{(}C(G(\mathbf{X},s)),\mathcal{Y}_{trg}\big{)}\right] \tag{4}\]  
\(\mathcal{L}_{sty}\) represents the **style reconstruction loss that is applied** to ensure that the generated samples can be used to reconstruct the style code as in Equation (5):  
\[\mathcal{L}_{sty}=\mathbb{E}_{\mathbf{X},\;\mathcal{Y}_{trg}s}\left[\parallel s- S\big{(}G(X,s),\mathcal{Y}_{trg}\big{)}\parallel_{1}\right] \tag{5}\]  
\(\mathcal{L}_{ds}\) denotes the **style diversification loss** which is used to push the generator to generate a wide variety of samples, each with a unique combination of style codes by maximizing \(\mathcal{L}_{ds}\), which is defined in Equation (6):  
\[\mathcal{L}_{ds}=\mathbb{E}_{\mathbf{X},s1,s2,\;\mathcal{Y}_{trg}}\left[\parallel G (\mathbf{X},s1)-G(\mathbf{X},s2)\parallel_{1}\right]+\mathbb{E}_{\mathbf{X},s1,s2,\; \mathcal{Y}_{trg}}[\parallel F_{conv}(G(\mathbf{X},s1))-\;F_{conv}(G(\mathbf{X},s2)) \parallel_{1}\right] \tag{6}\]  
where \(s1\) and \(s2\) represent two randomly sampled style codes from \(\mathcal{Y}_{trg}\); \(F_{conv}(. )\) denotes the output of convolutional layers of \(F0\) network \(F\). \(\mathcal{L}_{f0}\) denotes the **F0 consistency loss** that is applied to obtain F0-consistent results and is defined in Equation (7):  
\[\mathcal{L}_{f0}=\mathbb{E}_{\mathbf{X},s}\left[\parallel\hat{F}(\mathbf{X})-\hat{F} (G(X,s))\parallel_{1}\right] \tag{7}\]  
where \(\hat{F}(\mathbf{X})\) normalizes the absolute F0 values by its temporal mean. \(\mathcal{L}_{asr}\) denotes **speech consistency loss** that is applied to guarantee that the converted speech contains the same linguistic content as the source. It is defined in Equation (8):  
\[\mathcal{L}_{ssr}=\mathbb{E}_{\mathbf{X},s}\left[\parallel h_{asr}(\mathbf{X})-h_{asr }(G(X,s))\parallel_{1}\right] \tag{8}\]  
where \(h_{asr}(.