# MuIm: Analyzing Music-Image Correlations from an
## 5 Dataset Analysis
### Statistical Analysis
#### 5.1.1 Artistic Data  
In the artistic section, we utilized a 28-dimensional (28D) emotion representation and AI-generated captions for semantic information. We then attempted to pair the two based on their emotional and semantic characteristics. To analyze this segment, we first present the mean distribution of emotions for both music and image data in Figure 7. Next, Figure 8 illustrates the cross-emotion correlation, revealing three categories of information:  
* Some emotions demonstrated positive correlation values, such as 'Goose bumps' and 'Scary/Fearful', indicating a relatively strong alignment between the emotions evoked by the music and images. This suggests that similar emotional responses may be consistently induced across both modalities.
* Certain emotions showed negative correlations, such as 'Tender/Longing' and 'Romantic/Loving', implying that music and images can evoke these emotions differently, potentially leading to distinct variations in their emotional impact.
* Emotions with correlations close to zero suggest little to no observable alignment between how music and images evoke these specific emotional responses.  
Next, we present the self-emotion correlation for the artistic music and image datasets to demonstrate the richness of the emotional responses of the two modalities, as depicted in Figure 9. This behavior can be summarized as follows:  
* For images, emotions exhibit a broad distribution of correlations, indicating that each emotion carries distinct and meaningful information. This suggests that a diverse range of emotions are necessary to fully capture the emotional responses elicited by images.
* For music, certain emotions display strong correlations, suggesting that a smaller number of broader emotional categories may be sufficient to capture music-induced emotional experiences. This reflects a more condensed emotional response structure.
* This difference in emotional correlation patterns highlights the modality-specific nature of emotional responses, where images evoke a wider range of distinct emotions, while music tends to elicit a more interconnected or overlapping set of emotional responses.  
Next, for the artistic dataset, we adopted an approach to better analyze and summarize the semantic information, showcasing the effectiveness and diversity of the modalities' semantic labels. First, we obtained embeddings for the semantic information using [59] for both modalities. We then applied k-means clustering to extract the top 10 clusters and plot their frequencies. This analysis allowed us to assess the diversity of the semantic information and ensured a balanced dataset when pairing, as shown in Figure 10.  
Figure 8: Cross music-image correlation coefficient to show the emotional similarity between the two modalities.  
Figure 7: Average emotional representation for (**a**) image and (**b**) music data in the artistic part.  
Finally, to assess the appropriateness of the pairing of the cross music-image dataset, we consider our observation that music-image correlation is multilayered and can have four different categories in terms of semantic and emotional responses. To better analyze this, we made a scatter plot showing the similarity between the two modalities in both semantic and emotional spaces. This scatter plot illustrates the modality pairing, where we set a threshold of 0.2 for negative correlations and 0.6 for positive correlations. Using these thresholds, we found that pairs of music and images with both semantic and emotional similarities above the positive threshold, indicating strong alignment, were more appropriately paired. Conversely, pairs with weak alignment exhibited both semantic and emotional distinctions. Figure 11 illustrates the scatter plot representation of our artistic cross-modal pairing.  
Figure 10: Semantic information for the music and image artistic data.  
Figure 9: Self-emotion correlation for the (**a**) music and (**b**) image modalities to show the significance of emotional response in the datasets.