# MuIm: Analyzing Music-Image Correlations from an
## 5 Dataset Analysis
### User Study
#### 5.2.2 Result Evaluation  
To ensure unbiased and informed feedback from participants, we assessed their understanding of both music and image domains, as visualized in Figure 16. The distribution indicates that the participants typically had a balanced proficiency in both music and art. A significant subset appeared more well versed in art than in music. Nonetheless, the majority possessed foundational knowledge in both domains, ensuring diverse and informed evaluations. In assessing our dataset pairing, we referenced the dimensionality of the dataset, as outlined in Table 7. The assessment was structured as follows: First, the dataset was divided into two segments: artistic and realistic (web-based). Each segment targeted a distinct music-image category and varied in scale, with the artistic segment being relatively smaller compared to the realistic segment. Second, we aligned the modalities on a bi-dimensional plane, focusing on semantics and emotions. Positive pairs were the primary focus, as negative pairs were derived from the positives and can theoretically be infinitely numerous. Third, for the artistic segment, due to its smaller scale, we considered only a single video sample per group. These samples were randomly assigned to represent either similar or contrasting properties of semantics and emotions. Conversely, for the realistic (web-based) segment, which had a larger scale, we created four video samples per group: two showcasing contrasting pairings, and two reflecting similar emotion and semantic content. This structured approach ensured a balanced representation across both segments, while maintaining the diversity and integrity of the dataset. Figure 15: User-study overview showcasing participants’ distribution based on gender, followed by their age group and the corresponding number of participants. Using the established setup, we generated histogram plots for all four questions. The initial three questions focused on the semantic connection between the modalities, while the final one targeted their emotional relation. Figure 17 displays the average responses from six groups of five participants each, scored on a 0-5 Likert scale (with 0 indicating \"disagree\" and 5 as \"strongly agree\"). The concluding plot in Figure 17 aggregates the average semantic ratings from the three questions for both data segments. In the \"average symbol rating\" plot from Figure 17, the artistic data scored below 2, suggesting limited recognition of shared semantics between the modalities. This can be attributed to the constraints of AI-generated semantics for artistic data. Though this pattern was observed in subsequent questions, the scores were slightly higher, possibly due to the participants' ability to associate music with imagery based on prior experiences. For the web data, the \"varied semantics, same emotions\" responses for the first question hovered around neutral. However, for questions 2 and 3, there was a general  
Figure 16: Comparative analysis of participants’ knowledge in music and image domains. Figure 17: Trends in semantic analysis responses. The combined average semantic rating histogram consolidates the results from the other three histograms. disagreement, which was expected. In the \"varied emotions, same semantics\" category, while responses for question 2 were neutral, questions 1 and 3 leaned towards agreement, suggesting consistent semantic interpretations across the modalities. Lastly, columns representing \"similar emotions and semantics\" showed high levels of agreement across all questions. Their close alignment underscored a robust correlation in paired samples, as demonstrated by two distinct videos. In Figure 18, the \"average semantic rating\" plot reveals a general disagreement among participants regarding the semantic similarity between modalities for the artistic data. This could be attributed to the suboptimal representation by the media captioning models.