# Emotionally Controllable Talking Face Generation from an Arbitrary Emotional Portrait
## 4 Experiment
### Qualitative Evaluation  
Subject evaluation is equally crucial for the visual content generation and emotion generation tasks. As shown in Figure 3, two advanced techniques (listed in Section 4.1) were compared with the proposed approach. Our system requires the additional input of an emotional source portrait (the portrait in the lower left corner) in addition to audio and target portrait inputs. The proposed approach allows the target portrait to produce expressions corresponding to the seen emotional source portrait while ensuring the visual quality of the video and the identity of the target. Specifically, in [PERSON] et al. [13] and [PERSON] et al. [15], lip movements are acceptable, and [PERSON] et al. [15] additionally considers head posture. However, the expressions of both characters are constrained by the input portrait and less attention is accorded to the emotional aspect. In our case, lip movement and head posture are guaranteed, and the input of the emotional source image allows control of the emotional content in the output video. It has been shown that multimodal inputs may provide rich emotional qualities, allowing the creation of emotionally richer videos via the fusion of diverse input types.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Method** & **SSIM\(\uparrow\)** & **PSNR\(\uparrow\)** & **FID\(\downarrow\)** & **L-LD\(\downarrow\)** & **L-LVD\(\downarrow\)** & **F-LD\(\downarrow\)** & **F-LVD\(\downarrow\)** \\ \hline [PERSON] et al. [13] & 0.60 & 28.55 & 67.60 & 3.27 & 2.09 & 3.82 & 1.71 \\ [PERSON] et al. [15] & 0.62 & 28.94 & 17.33 & 2.49 & 1.66 & 3.55 & 1.64 \\ [PERSON] et al. [10] & 0.68 & 28.61 & 22.52 & 2.52 & 2.28 & 3.16 & 2.01 \\ [PERSON] et al. [17] & 0.71 & 29.53 & 7.99 & 2.45 & 1.78 & 3.01 & 1.56 \\ Ours & **0.62** & **28.92** & **16.80** & **2.52** & **1.68** & **2.98** & **1.48** \\ \hline \hline \end{tabular}
\end{table}
Table 1: Quantitative comparison with state-of-the-art methods. Calculation of landmark accuracy and video quality by comparing the results of different methods with real data. L- represents the lip area and F-represents the face area.  
In addition to this, we used our proposed method to perform a comparison between the same target portrait with audio and different emotion-driven sources as input. As shown in Figure 4, we generated seven emotions for the target portrait from top to bottom: anger, contempt, disgust, fear, happiness, sadness, and surprise. Our approach allows for diverse expressions to be generated based on the emotion source. Expressions with more evident traits, such as anger, happiness, and melancholy, provide better outcomes, but the more muddled disdain and disgust, surprise, and sadness produce fewer effective results. In addition to the basic categories of facial emotions, there are also compound emotions, meaning those that can be constructed by combining the basic component categories to create new ones [47]. Theoretically, the proposed method can generate expressions of arbitrary emotional origin, including compound emotions. The experimental results show that this system ensures the visual quality and target identity to a certain extent and enriches the emotional representation ability of the talking face animation. However, it is limited by the identity source portrait and the landmark-to-animation translation network, leading to some confusion in the reenactment of certain expressions.  
Figure 3: Qualitative comparison with the state-of-the-art methods. All the methods in the figure are based on image generation animation. The input images are all the first frames of the target video; the input videos are all the same speech; and our method additionally selects the unseen portrait as the emotion source.