# EmoStyle: Emotion-Aware Semantic Image Manipulation with Audio Guidance
## 4 Experiments
### Qualitative Analysis  
#### 4.3.1 Comparison in Lyric-Based Audio Settings  
In our study, we specifically collect music tracks that featured lyrics rich in emotional content, aiming to analyze how this emotional richness influences image manipulation when used as input. This approach allows us to compare the effectiveness of different methods in translating the emotional nuances of songs into visual representations.  
For the StyleCLIP method, we extract the lyrical content of the songs and use it as textual input. This provides a basis for assessing how well text-based manipulation captures the emotional essence of the lyrics. As shown in Figure 2, we find that image manipulation driven by audio input generally produces better quality results when compared with methods that rely solely on textual input, like StyleCLIP, or methods that use audio in a different way, such as the approach proposed by [PERSON] et al. [8].  
In the context of facial generation within the FFHQ setting, it was observed that StyleCLIP, guided by lyrics text, often struggled with accurately interpreting and representing the emotional dimensions of the songs. This misalignment indicates a limitation in the method's ability to fully grasp and translate the emotional subtleties present in the lyrics into corresponding visual cues.  
Similarly, the [PERSON] et al. [8] method, which utilizes audio input, showed some instability in maintaining image fidelity. The images generated by this method often lacked naturalness, suggesting difficulty in achieving a harmonious balance between the audio input and the visual output.  
The landscape images within the LHQ setting further highlighted these discrepancies. When examining the natural scenery results (shown in the rightmost three columns of Figure 2), both StyleCLIP and the [PERSON] et al. methods seemed to falter in accurately representing the emotional dimensions conveyed by the audio. This inadequacy was evident in their failure to capture the mood and tone suggested by the music's emotional content.  
In stark contrast, our proposed method, EmoStyle, demonstrated a superior ability to effectively handle audio inputs with lyrics. It is noteworthy that, for the third image in Figure 2, our method generated an expression depicting a bitter smile, representing a negative emotional change. This contrasts with images produced by other methods, as our approach excels in capturing and expressing emotional nuances while maintaining facial texture features. Similarly, for the fifth image in Figure 2, which originally had a positive expression, our method was tested for its ability to accurately identify and edit the image to enhance the appropriate emotional tone, ensuring the facial features and visual recognizability were preserved. The slight rosiness added to the cheeks in our generated image subtly amplified the positive emotion, illustrating the effectiveness and sensitivity of our approach in both scenarios. Thus, it is evident that EmoStyle not only accurately represented the emotional content of the songs but also maintained high image fidelity after incorporating this emotional information. This suggests that EmoStyle is more adept at interpreting and visually rendering the complex interplay of emotions conveyed in music, offering a more nuanced and faithful translation of audio-lyrical content into visual imagery.