# MuIm: Analyzing Music-Image Correlations from an
## 4 MuIm Dataset
### Data Acquisition and Labeling  
To address the diverse categories of media in our dataset--artistic and realistic--we first outline the data collection and labeling processes for each segment. Artistic data are isolated because they often encapsulate complex emotions and nuanced meanings. The \begin{table}
\begin{tabular}{|c|c|l|l|c|c|c|} \hline \multirow{2}{*}{**Information Layer**} & \multirow{2}{*}{**Modality**} & \multicolumn{3}{c|}{**Dataset**} & \multirow{2}{*}{**ML Tasks**} \\ \cline{3-4} \cline{6-7}  & & **Name** & **Domain** & **Annotations** & **Quantity** & **ML Tasks** \\ \hline \multirow{8}{*}{Affective} & Visual & [4] & EMOTIC & Human Emotions & BOX + CES (26) + DES (VAD) & 23,571 & DIS \\  & (Image/Art) & [17] & Artifinis + Arcepting & CES (9) + captions & 80,031 & GEN \\ \cline{2-7}  & & [18] & ATEmis 2.0 & Affective Art Captioning & CES (9) + captions + contrast & - & GEN \\ \cline{2-7}  & & [18] & DEMAT & Static Manicon & DES (VA) & 1802 & DIS \\  & & [38] & Soundtrack & Static Manicon & CES (4) & 360 & DIS \\  & & [45] & Bi-Modal & Static Manicon & CES (4) + Lyrics & 162 & DIS \\  & & [39] & CAL100 & Static Manicon & CES (18) & 502 & DIS \\  & & [19] & CAL300 epn & Dynamic Music Emotion & CES (8) & 3223 & DIS \\  & & [46] & EMM & Static Manicon & DES (VA) & 489 & DIS \\  & & [20] & MPF & Static Music Emotion & Perceptual Features + DES (8) & 5000 & DIS \\  & & [21] & EAM & Dynamic Music Emotion & DES (VA) & 744 & DIS \\ \cline{2-7}  & Audio-Visual & [22] & IMEMNet & Music-Image & DIS (VA) & 144,435 pairs & ID & DIS \\  & & [23] & IMAC & Music-Image & CES (5) & 0,01 & DIS \\  & (Music-Image) & [15] & ASD & Music-Image & CES (8) & 250,000 pairs & ED, FCC & DIS \\ \hline \multirow{8}{*}{Semantic} & Visual & [10] & Conceptual Captions & Image captioning & Web Captions & 3.4M & GEN \\  & (Image/Art) & [10] & COCO & Image captioning & Human captions & 328 & DIS \\  & & [11] & VG & Image Captioning & Captions + QA + Seme Graphs & 108 & DIS + GEN \\ \cline{2-7}  & & [22] & ICALS & Text-Wave Retrieval & Tag-Captions & 517,022 & DIS \\  & & [47] & MC & Text to Music & Captions & 5.5 & GEN \\  & [48] & MIT & Music Retrieval & Tag (Semantic + Mood) & 26 & DIS \\  & & [18] & I-Music Caps & Audio Captioning & Tag + Pseudo captions & 514 & GEN \\  & & [49] & WaveCaps & Audio Captioning & Captions & 400 & DIS + GEN \\  & & [41] & ALOxbo-Audio-A60X & Text-Audio Representation & Tag (K2C) + Captions & 633,526 & DIS \\  & & [50] & MulLaMCap & Text to Music & Pseudo Captions & 400 & GEN \\  & & [51] & AudioCaps & Audio captioning & Captions & 52,904 & GEN \\  & & [42] & Clotho & Audio captioning & Captions & 9529 & GEN \\ \cline{2-7}  & & [22] & ITW & Audio-Visual & Fiocentric Videos & 97 & Temporal & GEN \\  & & [23] & CH & Audio-Visual & Videos & 977 & Temporal & DIS + GEN \\  & Audio-Visual & [23] & MCA & Music-Image & Music Cover Art & 78,325 & 1-to-1 & DIS \\  & (Music-Image) & [23] & ITW/200K & Music-Video Retrieval & Micro Video & 2005 & Temporal & DIS \\  & (Music-Image) & [23] & ITW-150K & Music-Video Retrieval & Micro Video & 150 & Temporal & DIS \\  & & [23] & YouTube-SM & Video Classification & Labels & 8M & Temporal & DIS \\  & & [24] & AudioNet & Audio Senset recognition & Audio classes & 12M & Temporal & DIS \\  & & [31] & Sub-UtMP + DIS & Music-Visual & Videos + Labels & 17,555 + 7200 & Temporal & GEN \\ \hline \multirow{2}{*}{\
\begin{tabular}{} \end{tabular} } & Audio/Visual & \multirow{2}{*}{**MuIm**} & \multirow{2}{*}{Music-Image} & \multirow{2}{*}{Artistic 2 BD emotions + captions + epochs} & \multirow{2}{*}{500 pairs} & \multirow{2}{*}{Several Dis.} & \multirow{2}{*}{
\begin{tabular}{} \end{tabular} } \\  & (Music-Image) & & & & & \\ \end{tabular}
\end{table} 
Table 2: Overview of selected datasets for deep learning in audio, visual, and audio-visual modalities: focus on music in audio and images/art in visual domain. Highlighted red part specifies previous audio-visual correlation data, while the green part shows our data.