## 4 Feature Extraction
### Acoustic Features  
Five categories of acoustic features corresponding to five major music characteristics were extracted, including loudness, rhythm, timbre, pitch, and harmony. The features and their dimensionalities are summarized in Table 5 and explained as follows. _Loudness_ and dynamics generally represent the intensity of a sound and the variation of it. In this study, the root-mean-square energy (RMSE) was adopted as the loudness feature and was computed from the Short-Time Fourier Transform (STFT) spectrogram. In addition, we also computed the logarithmic compression version of RMSE to simulate human perception [48]. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Category** & **Code** & **Description** \\ \hline \multirow{5}{*}{Time domain} & SDSD & Standard deviation of successive differences between adjacent NN intervals. \\  & RMSSD & The root mean square of successive differences between adjacent NN intervals. \\  & NN50, NN20 & Number of pairs of adjacent NN intervals differing by more than 50 or 20 milliseconds. \\  & pNN50, pNN20 & Percentage of differences between adjacent NN intervals that exceed 50 or 20 milliseconds. \\  & CVSD & The coefficient of variation of successive differences, i.e., RMSSD divided by the mean of \\  & CVNN & The coefficient of variation, i.e., the ratio of SDNN divided by the mean of the NN intervals. \\ \hline \multirow{5}{*}{Frequency domain} & Total power & Total power spectral density. \\  & VLF & Power in the very-low-frequency band (i.e., 0.003–0.04 Hz). \\  & LF & Power in the low-frequency band (i.e., 0.04–0.15 Hz). \\  & HF & Power in the high-frequency band (i.e., 0.15–0.4 Hz). \\  & LF/HF & The ratio of the power in the low-frequency band to that in the high-frequency band. \\  & LFNU & Low-frequency power in normalized units. \\  & HFNU & High-frequency power in normalized units. \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Heart rate variability features. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Category** & **Feature** & **Dimensions** \\ \hline \multirow{5}{*}{Timbbre} & Loudness & Root-mean-square energy & 4 \\  & Rhythm & Tempo & 1 \\  & Rhythm strength & 1 \\  & Global onset autocorrelation & 2 \\  & Average onset frequency & 1 \\  & Mel-frequency cepstral coefficient & 38 \\  & \(\Delta\) MFCC, \(\Delta\Delta\) MFCC & 76 \\  & Spectrum characteristics & 22 \\  & Zero crossing rate & 2 \\  & Harmony & Tonal centroid (tonnetz) & 12 \\  & Pitch & STFT chromagram & 24 \\  & Constant-Q chromagram & 24 \\  & CENS chromagram & 24 \\ \hline Total & & 231 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Extracted Acoustic Features. Rhythm_, which depicts the pulse of music and the pattern of arrangement of musical notes, was characterized by the estimated tempo (i.e., beats per minute), rhythm strength (i.e., the average of the onset strength envelope), global onset autocorrelation (i.e., autocorrelation computed on the onset strength envelope), and the average onset frequency (i.e., the number of notes per second) [49]. Timbre, another music trait widely exploited in content-based MER, refers to the texture of a musical sound. Spectrum characteristics, zero crossing rate of the time domain audio signals, and Mel-frequency cepstral coefficient (MFCC) related features were used for this music trait dimension. Specifically, the spectrum characteristics involved in this study were roll-off frequency, flatness, spectral centroid, spectral contrast, and spectral bandwidth. Furthermore, we took the first 20 MFCCs and excluded the direct current (DC) term. The first and second order derivatives of MFCCs (i.e., \(\Delta\) MFCC and \(\Delta\Delta\) MFCC) were computed as well. Pitch and harmonyThree versions of chromatgrams were computed to represent the energy distribution of the 12 pitch classes (from C, C#,... to B), including a chromagram computed from the STFT spectrogram, a chromagram computed from Constant-Q Transform (CQT) spectrogram, and Chroma Energy Normalized Statistics (CENS).