## 6 Results and Discussion  
### Comparison of Different Modalities of Features  
Table 6 presents classification performances on three modalities of features (i.e., audio-only, physiology-only, and audio + physiology) for arousal and valence prediction. The classification performance (in macro F1-score and AUC) showed a similar pattern across experiments with imbalanced or resampled training data. The combined multimodal feature set achieved the best performance in each experiment condition, while the improvement for valence prediction was more significant than that for arousal prediction. Specifically, for _valence_ classification, the combined feature set consistently outperformed the single-modality feature sets (i.e., audio-only, physiology-only) with statistical significance (at \(p<0.001\)). In comparing the single-modality feature sets, the acoustic features showed slightly better performance than the physiological features. The relative strength of acoustic features was significant in AUC but not in the macro F1-score. For _arousal_ prediction, though the combined feature set was found to be the best-performing, it did not significantly outperform the acoustic features (the best-performing single-modality feature set). Finally, the audio-only feature set showed a statistically significant advantage over the physiology-only feature set. To sum up, with regard to RQ1, our results suggest that: for valence prediction, features extracted from peripheral physiological signals from listeners might be mediocre by themselves, but, when combined with acoustic features extracted from music content, the multimodal feature set showed significant improvement compared to feature sets of single modalities. For arousal prediction, the performances of physiological signals were  
\begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline  & \multicolumn{4}{c}{**Imbalanced Data**} & \multicolumn{4}{c}{**Resampled Data**} \\ \cline{2-9}
**Feature Set** & \multicolumn{2}{c}{**Arousal**} & \multicolumn{2}{c}{**Valence**} & \multicolumn{2}{c}{**Arousal**} & \multicolumn{2}{c}{**Valence**} \\ \cline{2-9}  & **F1** & **AUC** & **F1** & **AUC** & **F1** & **AUC** & **F1** & **AUC** \\ \hline Audio-only & 71.38\% & 81.31\% & 54.74\% & 63.86\% & 72.93\% & 81.41\% & 58.09\% & 63.72\% \\ Physiology-only & 47.64\% & 58.60\% & 54.14\% & 57.44\% & 55.79\% & 58.85\% & 57.07\% & 59.62\% \\ Combined & **71.67\%** & **81.42\%** & **58.92\%** & **70.26\%** & **73.05\%** & **81.46\%** & **62.59\%** & **68.96\%** \\ \hline Aud. vs. phy. & \in line with those on valence prediction, but little contribution was observed from the physiological features in the performance of the combined feature set. In other words, the acoustic modality alone was comparable with the multimodal approach. Results of acoustic features shown in Table 6 are also in line with the findings of the previous research that acoustic features extracted from the music content are effective for arousal classification but less successful for valence prediction [25, 49]. Associations between the dynamical and rhythmic characteristics of music and the induced arousal of listeners have been evidenced by studies in music psychology literature, while listeners' valence was also found to be influenced by some nonmusical factors as well, such as individuals' aesthetic judgement [58]. Hence, the acoustic features might be more comprehensive in modeling listeners' arousal than modeling listeners' valence. On the other hand, for the more challenging valence classification task, neither acoustic nor physiological feature set alone performed well, but the combined feature set significantly improved the classification performance over either single modality, indicating that the acoustic and physiological feature sets compensated for one another in valence classification. In contrast to previous studies that used only acoustic features, these findings reveal the potential values of physiological signals in tasks related to music-induced emotion.