# Musical Rhythms Affect Heart Rate Variability: Algorithm and Models

Authors: Hui-Min Wang; Sheng-Chieh Huang
Journal: Advances in Electrical Engineering
 (2014)

---

## Abstract

There were a lot of psychological music experiments and models but there were few psychological rhythm experiments and models. There were a lot of physiological music experiments but there were few physiological music models. There were few physiological rhythm experiments but there was no physiological rhythm model. We proposed a physiological rhythm model to fill this gap. Twenty-two participants, 4 drum loops as stimuli, and electrocardiogram (ECG) were employed in this work. We designed an algorithm to map tempo, complexity, and energy into two heart rate variability (HRV) measures, the standard deviation of normal-to-normal heartbeats (SDNN) and the ratio of low- and high-frequency powers (LF/HF); these two measures form the physiological valence/arousal plane. There were four major findings. Initially, simple and loud rhythms enhanced arousal. Secondly, the removal of fast and loud rhythms decreased arousal. Thirdly, fast rhythms increased valence. Finally, the removal of fast and quiet rhythms increased valence. Our work extended the psychological model to the physiological model and deepened the musical model into the rhythmic model. Moreover, this model could be the rules of automatic music generating systems.


---

## Body

## 1. Introduction

The relation of music to emotion has been studied for decades and the literature is fruitful [1]. There exist a lot of psychological models between music and emotion [2], but the physiological models between music and emotion are limited [3]. One of the physiological actions, heart rate variability (HRV), which is controlled by the autonomic nervous system (ANS), is tightly connected with emotions [4]. Previously, we had analyzed the relationship between musical rhythms and HRV [5] and built two heuristic models [6, 7]. In this paper, a systematic algorithm is proposed to construct new models.Musical emotions change with psychophysiological measures and musical features [8], whilst three basic questions are highlighted [9]: how do musical features evoke emotions; how do actions involved in musical emotions progress; and which actions and brain processes are involved in musical emotions. Basically, people feel what music expresses but need not be always; in a simple case, only 61% of 45 participants felt what they perceived [10]. More particularly, a stronger correlation is suggested by a recently developed theory that the aesthetic awe accompanies by being moved (cognitive), emotions (psychological), and thrills (physiological) in the same time [11]. The three associated levels of musical response should be analyzed individually and can be discussed together. In the research about music psychology, sometimes the perceived and felt emotions are examined separately; sometimes the music clips are just labeled artificially; whether they are perceived or felt had not been mentioned yet [12]. The mappings between music space and emotion space [13, 14] employ the following synonyms: music mood detection [15], music emotion measurement [12], characterization [12], recognition [16, 17], classification [12, 18], predicting [19], or modeling [20]; the review articles demonstrate the fruits of experts’ interests [16–20].In a valuable model, four elementary properties are goodness-of-fit [21] (generalization [22]), simplicity [21, 22], predictability [21, 22], and the relation to existing theories [22]. Beyond cognitive reactions, the prediction of emotional reactions is also important in music psychology [22, 23]. In addition to an existing review [24], a survey of the models [25–45] for emotional responses is given in this paper. The acoustic features, emotion spaces, and methods to generate these models are listed in Table 1 with four major observations. Initially, regression analysis [30, 31, 36, 38–40, 42, 43, 45] is widely employed, while some soft computing methods (fuzzy [41]; support vector machine [34, 37]; neural network [35]; K-nearest neighborhood [27]; Gaussian mixture model [25]) are also used. Secondly, beyond the enviable valence and arousal, tension [29, 32, 36, 40, 44] is another common dimension too. Thirdly, the felt emotion [28, 35] is infrequently examined compared to the perceived emotion. Finally, most works employ a wide range of acoustic features, while the interest in a single feature moves from pitch [40, 44] to timbre [28] and rhythm [26]. For the emotion space, Russell [13] proposed the valence/arousal space and Thayer [14] reduced it to four labels; the following synonyms are commonly used: valence as pleasantness [38]; arousal as activity [36, 45]; tension [29, 32, 36, 40, 44] as interest [43], expectancy [40], strength [38], potency [38], and resonance [27].Table 1
Psychological models of musical features.
Reference
Year
Acoustic features
Emotion space
Method
Number
Catalog
Type A: rhythm; B: pitch; C: timbre; D: other
[25]
2012
70
A(1)B(17)C(52)
A: dynamic (1); B: tonal (17); C: spectral (11), timbre (41).
Russell
Perceived
GMM
[26]
2012
11
A(11)
A: rhythm (11).
Thayer
Perceived
C5
[27]
2012
20
A(10)B(6)C(4)
A: rhythm (7), intensity (3); B: pitch and tonality (4), harmony (2); C: timbre (4).
VA-Resonance
Perceived
KNN
[28]
2012
60
C(60)
C: timbre (60).
Thayer
Felt
C4
[29]
2012
9
A(6)B(3)
A: dynamics (1), onset frequency (1), tempo (1), meter (1), rhythmic regularity (1), syncopation (1); B: harmony (1), pitch height (1), melodic expectation (1).
T
Perceived
MPW
[30]
2012
16
A(6)B(10)
A: dynamic (1), crescendo (1), density (1), speed (1), tempo (1), articulation (1); B: direction (1), surprise (1), tendency (1), mode (1), dissonance (1), harmonic surprise (1), harmonic tempo (1), high pitch (1), low pitch (1), closure (1).
D11
Perceived
Regression
[31]
2011
46
A(16)B(10)C(10)D(10)
A: temporal (6), rhythmic (10); B: melody/harmony (10); C: spectral (10); D: lyrics (10).
Russell
Perceived
Regression
[32]
2011
4
A(2)B(2)
A:tempo (1), dynamics (1); B: register (1), contour (1).
T
Perceived
ANOVA
[33]
2011
32
C(32)
C: spectral contrast and MFCCs (20), echo nest timbre (12).
Russell
Perceived
CRF
[34]
2010
292
A(87)B(22)C(18)D(165)
A: rhythm (87); B: chords (22); C: spectral (18); D: lyrics (12), metadata (153).
Thayer
Perceived
SVM
[35]
2009
13
A(3)B(6)C(4)
A: dynamics (2), tempo (1); B: mean pitch (2), pitch variation (3), texture (1); C: timbre (4).
Russell
Felt
NN
[36]
2009
18
A(7)B(8)C(3)
A: dynamics (1), rhythm (3), articulation (3); B: harmony (5), register (3); C: timbre (3).
VAT
Perceived
Regression
[37]
2008
29
B(M)C(M)D(F)
B: pitch (many); C: timbre (many); D: other (few).
Russell
Perceived
SVM
[38]
2008
9
A(4)B(3)D(2)
A: temporal surface (2), dynamics (2); B: register (2), tonality (1); D: other (2).
VA-Strength
Perceived
Regression
[39]
2008
15
A(2)B(7)C(6)
A: loudness (1), volume (1); B: tonal dissonance (2), pure tonal (1), complex tonal (1), multiplicity (1), tonality (1), chord (1); C: spectral centroid (1), sharpness (2), timbral width (1), spectral dissonance (2).
Thayer
Perceived
Regression
[40]
2007
1
B(1)
B: pitch (1).
T
Perceived
Regression
[41]
2006
15
A(7)B(8)
A: loudness (1), duration (6); B: pitch (8).
Thayer
Perceived
Fuzzy
[42]
2006
18
A(3)B(11)C(4)
A: dynamics (2), tempo (1); B: mean pitch (2), pitch variation (3), harmony (5), texture (1); C: timbre (4).
Russell
Perceived
Regression
[43]
2007
8
A(3)B(4)C(1)
A: tempo (1), loudness (1), articulation (1); B: roughness (1), melody (1), ambitus (1), register (1); C: brightness (1).
VA-Interest
Perceived
Regression
[44]
2005
4
B(4)
B: pitch (stability, proximity, direction, mobility).
T
Perceived
Formula
[45]
2004
6
A(3)B(3)
A: loudness (1), spectral centroid (1), tempo (1); B: melodic (1), contour (1), texture (1).
Russell
Perceived
Regression
Emotion space: V: valence; A: arousal; T: tension; D11: 11 dimensions.
Method: GMM: Gaussian mixture model; C5: classifiers 5; KNN: K-nearest neighborhood; C4: classifiers 4; MPW: moving perceptual window; ANOVA: analysis of variance; CRF: conditional random field; SVM: support vector machine; NN: neural network.Whether perceived or felt emotion, the three commonly used methods, including valence/arousal dimensions, lists of basic emotions, and diverse emotion inventories, are not complete in the musical emotion study [46]. To have a more comprehensive overview [4], the underlying mechanisms of the central nervous system (CNS) [46, 47] and the ANS [48–50] should be also deliberated. In the most essential sense, the acoustic properties can be perceived by the nervous system and evoke psychophysiological responses. For example, fast and loud voices usually cause emotional arousal and increased respiratory and heart rates through the auditory and limbic systems [46, 47]. Some related physiological reactions coincided with the emotion expressed in music [48] and rated valence/arousal level [49]. Therefore a valence/arousal model [50] of musical emotion had been built based on the associated physiological measures, including electromyogram, electrocardiogram, skin conductivity, and respiration changes. Extended linear discriminant analysis (pLDA) was employed in the classification of musical emotions.Since emotion correlates with physiological responses [48–50], too, the models of musical emotions were developed by integrating the acoustic features and physiological measures [51–53]. Despite combining with physiological measures, these models [51–53] tend to be similar to the psychological models mentioned above [25–45] in spite of the same perceived or felt emotion space. In addition, using acoustic features to model [3] the physiological responses seems more radical if the underlying mechanisms [46, 47] are considered. The model [3] employs 11 musical characteristics and the conclusion is that rhythmic features are the major factors of the physiological responses (respiration, skin conductance, and heart rate) to music. It also points out a limitation that some acoustic features would correlate with each other. This makes it difficult to discriminate their relative contributions to the detected relationships.The rest of this paper is organized as follows. In Section2, the rhythmic features for modeling and the HRV features for the physiological emotion space are introduced. In Section 3, the experiment and developed algorithm are presented. In Section 4, the models of the relationships between the rhythmic and the HRV features are illustrated in figures, and the equations and tables of statistics are provided. In Section 5, how tempo, complexity, and energy work in the psychological experiments, psychological models, physiological experiments, and physiological models are reviewed and compared. Finally, in Section 6, we summarize the contributions made in this study and suggest the directions for the further research.


## 2. Preliminary

### 2.1. Why Are Physiological Models Necessary?

To study the relation of musical features to emotion, most psychological models focus on the perceived emotion, as Table1 demonstrated. However, the perceived emotion need not be equal to the felt emotion [10, 54], and the felt emotion may not have the related physiological responses neither [55]. If the music clips are selected by the participants themselves, the perceived emotion has no statistical significance with the felt emotion in the ranks. Otherwise the differences are statistically significant [54]. Moreover, some patterns of the physiological responses appear while there is no related self-report [55]. Since the perceived emotion, felt emotion, and physiological actions play different roles, a physiological model is necessary beyond the psychological models.


### 2.2. Valence/Arousal Model

Although four dimensions are necessary for emotion spaces [56], it is difficult to realize a four-dimensional model. Three dimensions are also employed in the psychological models [27, 36, 38, 43]. Some work [57] announced that three dimensions could be reduced into two, without significant loss of goodness-of-fit; it also shows that the dimensional model is better than the discrete (categorical) model in resolution. Thus Russell’s two-dimensional valence/arousal model [13] states its merit, within the ebb and flow of relevant woks, in the discipline of psychology.


### 2.3. A Physiological Valence/Arousal Model

SDNN and LF/HF, two measures of HRV [58], are employed as two dimensions in our physiological valence/arousal model. SDNN is the standard deviation of normal-to-normal heartbeat intervals in time domain and LF/HF is the ratio of low- and high-frequency powers after the fast Fourier transform (FFT); SDNN presents the variation of the circulatory system and LF/HF presents the balance of the sympathetic and parasympathetic nervous systems [59]. HRV is highly correlated with emotion [60–64] and some evidence [65–70] reveals that SDNN is a good indicator of valence in the physiological perspective. In general cases, the normal subjects’ SDNNs are higher than the depression subjects’ [65–67]. For the normal subjects’ case, the SDNN levels of subjects with positive mood are higher than the negative mood [68]. For the depression subjects’ case, the SDNN levels of low-depression patients’ are higher than the high-depression patients’ [69, 70]. All of these findings reach a statistical significance level. Hence SDNN could be a proper indicator as the physiological valence. In addition, increased LF/HF values denote that the sympathetic nerve activity tended to be strong; thus LF/HF could be used as a physiological indicator of arousal levels [59].


### 2.4. Musical Features Employed in the Model

The three most important features of music are rhythm, pitch, and timbre [71–76]. Although pitch [77] and timbre [78] have been employed to recognize emotion in music, the role of rhythm seems more rudimentary. In fact, the rhythmic features [26] had been used in modeling the emotional responses and acquired a reasonable performance.An all-encompassing representation of the rhythm may be the temporal organization of sound, tightly connected with meter, which refers to the periodic composition in music [23]. Musical rhythm may have its roots in the motor rhythms controlling heart rate, breathing, and locomotion [79], dominated by brain stem the ancient structure [47]; fast, loud sounds produce an increased activation of the brain [47] and some evidence suggests that the musical rhythm can regulate emotion and motivational states [80].Rhythm is less tractable than pitch and timbre in localizing its specific neural substances [81]. However, many cultures emphasize the rhythm, with two others playing a less crucial role [82]. To study the neural basis of rhythm, brain imaging, psychophysical, and computational approaches have been employed [83]. Beat and meter induction are the fundamental elements of cognitive mechanisms [84], while the representations of metric structure and neural markers of expectation of the beat have been found in both electroencephalography (EEG) and magnetoencephalography (MEG) [85]. Beat perception is innate; newborn infants expect downbeats (onsets of rhythmic cycles), even unmarked by stress or the spectral features [86]. Infants also engage in the rhythmic movement to rhythmically regular sounds and the faster movement tempo is associated with the faster auditory tempo [87]. Although the beat perception is innate, the ability to detect rhythmic changes is more developed in adults than infants [88] and the trained musicians than the untrained individuals [89].Of all rhythmic features, experts suggest that tempo, complexity (regularity), and energy (intensity, strength, dynamic, loudness, and volume) [15, 27, 90, 91] are the most significant.


## 2.1. Why Are Physiological Models Necessary?

To study the relation of musical features to emotion, most psychological models focus on the perceived emotion, as Table1 demonstrated. However, the perceived emotion need not be equal to the felt emotion [10, 54], and the felt emotion may not have the related physiological responses neither [55]. If the music clips are selected by the participants themselves, the perceived emotion has no statistical significance with the felt emotion in the ranks. Otherwise the differences are statistically significant [54]. Moreover, some patterns of the physiological responses appear while there is no related self-report [55]. Since the perceived emotion, felt emotion, and physiological actions play different roles, a physiological model is necessary beyond the psychological models.


## 2.2. Valence/Arousal Model

Although four dimensions are necessary for emotion spaces [56], it is difficult to realize a four-dimensional model. Three dimensions are also employed in the psychological models [27, 36, 38, 43]. Some work [57] announced that three dimensions could be reduced into two, without significant loss of goodness-of-fit; it also shows that the dimensional model is better than the discrete (categorical) model in resolution. Thus Russell’s two-dimensional valence/arousal model [13] states its merit, within the ebb and flow of relevant woks, in the discipline of psychology.


## 2.3. A Physiological Valence/Arousal Model

SDNN and LF/HF, two measures of HRV [58], are employed as two dimensions in our physiological valence/arousal model. SDNN is the standard deviation of normal-to-normal heartbeat intervals in time domain and LF/HF is the ratio of low- and high-frequency powers after the fast Fourier transform (FFT); SDNN presents the variation of the circulatory system and LF/HF presents the balance of the sympathetic and parasympathetic nervous systems [59]. HRV is highly correlated with emotion [60–64] and some evidence [65–70] reveals that SDNN is a good indicator of valence in the physiological perspective. In general cases, the normal subjects’ SDNNs are higher than the depression subjects’ [65–67]. For the normal subjects’ case, the SDNN levels of subjects with positive mood are higher than the negative mood [68]. For the depression subjects’ case, the SDNN levels of low-depression patients’ are higher than the high-depression patients’ [69, 70]. All of these findings reach a statistical significance level. Hence SDNN could be a proper indicator as the physiological valence. In addition, increased LF/HF values denote that the sympathetic nerve activity tended to be strong; thus LF/HF could be used as a physiological indicator of arousal levels [59].


## 2.4. Musical Features Employed in the Model

The three most important features of music are rhythm, pitch, and timbre [71–76]. Although pitch [77] and timbre [78] have been employed to recognize emotion in music, the role of rhythm seems more rudimentary. In fact, the rhythmic features [26] had been used in modeling the emotional responses and acquired a reasonable performance.An all-encompassing representation of the rhythm may be the temporal organization of sound, tightly connected with meter, which refers to the periodic composition in music [23]. Musical rhythm may have its roots in the motor rhythms controlling heart rate, breathing, and locomotion [79], dominated by brain stem the ancient structure [47]; fast, loud sounds produce an increased activation of the brain [47] and some evidence suggests that the musical rhythm can regulate emotion and motivational states [80].Rhythm is less tractable than pitch and timbre in localizing its specific neural substances [81]. However, many cultures emphasize the rhythm, with two others playing a less crucial role [82]. To study the neural basis of rhythm, brain imaging, psychophysical, and computational approaches have been employed [83]. Beat and meter induction are the fundamental elements of cognitive mechanisms [84], while the representations of metric structure and neural markers of expectation of the beat have been found in both electroencephalography (EEG) and magnetoencephalography (MEG) [85]. Beat perception is innate; newborn infants expect downbeats (onsets of rhythmic cycles), even unmarked by stress or the spectral features [86]. Infants also engage in the rhythmic movement to rhythmically regular sounds and the faster movement tempo is associated with the faster auditory tempo [87]. Although the beat perception is innate, the ability to detect rhythmic changes is more developed in adults than infants [88] and the trained musicians than the untrained individuals [89].Of all rhythmic features, experts suggest that tempo, complexity (regularity), and energy (intensity, strength, dynamic, loudness, and volume) [15, 27, 90, 91] are the most significant.


## 3. Methods

### 3.1. Experiment

#### 3.1.1. Participants

There were 22 healthy subjects, 15 males and 7 females, engaged in the experiment. The average age was 23. None of them had been professionally trained in music.


#### 3.1.2. Musical Stimuli

There were four drum loops in this study (L1 to L4). The parameters (tempo, complexity, and energy) are listed in Table2 and the rankings are illustrated in Figure 1.Table 2
Rhythmic features, model factors, and HRV data collection of four drum loops (L1~L4).
Definition
L1
L2
L3
L4
Tp
Tempo
89.60
105.30
139.50
176.50
Cp
Perceptual complexity
2.73
2.27
1.14
3.86
Eg
Energy,∑(k2)/(1013)
1.99
1.02
3.83
1.56
Valence (C1)
−1/Tp
−4.18
−2.23
0.50
2.26
Valence (C2)
Tp/Eg
−2.33
4.48
−3.34
5.64
Arousal (C1)
Eg/Cp
−0.02
−0.05
0.27
−0.05
Arousal (C2)
-
Tp
×
Eg
−0.15
−0.10
−0.42
−0.23
SDNN (C1)
Standard deviation of all RR intervals of C1
−5.01
±6.95
−0.81
±10.57
−0.10
±11.63
2.27
±12.06
SDNN (C2)
Standard deviation of all RR intervals of C2
−2.66
±7.31
5.17
±9.66
−3.14
±15.52
5.08
±10.29
LF/HF (C1)
Ratio of low frequency and high frequency of C1
−0.04
±1.67
−0.02
±0.81
0.28
±0.89
−0.06
±0.66
LF/HF (C2)
Ratio of low frequency and high frequency of C2
−0.10
±0.76
−0.10
±0.68
−0.41
±0.59
−0.29
±1.05Figure 1
Four drum loop patterns (L1~L4) employed in this study with ranked tempo (L4 > L3 > L2 > L1), complexity (L4 > L1 > L2 > L3), and energy (L3 > L1 > L4 > L2).


#### 3.1.3. Apparatus

The ECG signal was captured by a 3-channel portable device (MSI E3-80, FDA 510(k) K071085) at 500 Hz sampling rate from the chest surface of the body. Only the channel-1 data was taken to be analyzed.


#### 3.1.4. Procedures

All experiments were carried out in moderate temperature, humidity, and light with subjects sitting and wearing headphones (eyes closed) in a quiet room. Each subject accepted 4 rounds of experiment in different days. Each round took 15 minutes, separated as 5 stages, as Figure2 illustrated. Stage 1 had 5 minutes to let the subjects calm down. Stage 2 had 3 minutes of rest, as the baseline for the responses of the drum loops. Stage 3 had 2 minutes, as the baseline for the responses after the drum loops. Stage 4 had 3 minutes of stimulus of some drum loop. Stage 5 had 2 minutes of rest. The ECG signal from stage 2 to stage 5 was recorded and separated as epoch 1 (E1) to epoch 4 (E4). Comparison 1 (C1) was the difference of E1 and E3 and comparison 2 (C2) was the difference of E2 and E4.Figure 2
The procedure of each experiment: the epoch of 5 min to 8 min is the baseline of the physiological responses during music listening and the epoch of 8 min to 10 min is the baseline of the physiological responses after music listening.


### 3.2. Signal Processing

#### 3.2.1. Musical Features

Musical rhythm is expressed by the successive notes that record the relating temporal information. Tempo (Tp) could be decided as the unit, beats per minute (bpm) [92].The other characteristic, perceptual complexity (Cp), was obtained by asking the human subjects to judge the complexity of the rhythms they had listened to. It was assessed by the subjects using a subjective rating of 1 to 4 on a Likert Scale (4 being the most complex) [93].The energy parameter (Eg) was defined as Σk2, the summation of square of k, where k is the amplitude of the signal [94, 95].


#### 3.2.2. HRV Features

To acquire the measures of HRV features, QRS detection was the first step [96–98], where R denotes the peak in a heartbeat signal. After the abnormal beats were rejected [99], the mean of R-R intervals (MRR), standard deviation of normal-to-normal R-R intervals (SDNN), and root of mean of sum of square of differences of adjacent R-R intervals (RMSSD) were measured in time domain [59]. After interpolation [100] (prepared for FFT) and detrending [101] (to filter the respiratory signal), the FFT [102] was applied to calculate the low- (LF) and high-frequency (HF) powers and their ratio (LF/HF). The results of SDNN and LF/HF are listed in Table 2. Four groups of data, SDNN C1, SDNN C2, LF/HF C1, and LF/HF C2, were observed in our analysis.


### 3.3. Algorithm

For modeling the responses of some HRV measure and the related musical rhythms, our algorithm included 3 steps. Initially, all possible combinations of the rhythmic features were explored. Secondly, the values of the combinations were linearly transformed. Finally, the coefficients of the linear transformations were calculated such that the Euclidian metric between the results of step 2 and the related HRV responses is the minimum.The stimuli are 4 drum loops (rhythm), named asR1 to R4 here:
(1)Rri,ri∈{1,2,3,4}.Each rhythm relates to some HRV measureHri:
(2)Hri,ri∈{1,2,3,4}.For each rhythm, there are three rhythmic features,Tp, Cp, and Eg, named as F1 to F3:
(3)Ffi,fi∈{1,2,3}.Since a musical stimulus contains multiple combinations and interactions of various features, it is difficult to realize which feature is contributing to the perceived emotion [45], or physiological responses. Our solution is considering all possible combinations: the influence of each feature Ffi is linear (order 1), of no effect (order 0), or inverse (order −1). Although the higher orders are plausible, order one is still the most suitable to construct a simplified model for understanding the relation of the musical rhythms to their related HRV measures:
(4)Eri=∏fi=13(Rri.Ffi)fi.p,fi.p∈{-1,0,1}.Linear transformation is necessary because the units of rhythmic and HRV features are not uniform. All we need are the related correspondences. Consider(5)Tri=x(Eri)+y,x∈R,y∈R.Thus for some subject’s HRV responsesH1 to H4 (e.g., SDNN) to rhythms R1 to R4, some combination of rhythmic features can model the relation if the metric between Tri and Hri is the minimum. Euclidean metric [27, 103] is employed in this work:
(6)D=∑ri=14(Tri-Hri)2.After squaring each side of (6), we can acquire the coefficients of the linear transformation to get the minimum metric if the partial derivatives of D2 (with respect to x and y) are both zero:
(7)D2=∑ri=14(Tri-Hri)2.


### 3.4. Statistics

#### 3.4.1. Outliers

The judgment and removal of outliers are fundamental in the processing of experimental data [104]. In our study, the experimental data was separated into three groups by the minimum metric mentioned in Section 3.3 and the group with larger metric was eliminated.The basic idea is to rank a group of data by the algorithm we proposed and make the data with larger metric obsolete. There were four groups of data, SDNN (C1), SDNN (C2), LF/HF (C1), and LF/HF (C2). Each group had 22 records of the 22 participants. Each record had four subrecords of drum loops L1 to L4. First, we used the algorithm in each group and ranked the 22 records as 1 to 22 by their minimum metric. After summing the ranks of the four groups of these 22 participants, these participants were partitioned into three levels: small, medium, and large metric (7, 7, and 8 participants). We defined the predictable class with the small and medium metrics and the nonpredictable class with the large metric. The nonpredictable class was excluded from our experimental data. After removing the nonpredictable class, the rhythmic features and the four averages of these four groups of predictable class were modeled by our algorithm again.


#### 3.4.2. ANOVA andt-Test

Repeated measurements have four major advantages: obtaining the individual patterns of change, less subjects, serving as subjects’ own controls, and reliability; the disadvantages are the complication by the dependence among repeated observations and less control of the circumstances [105]. In our experiment, the repeated measurements were employed.Two basic methods of ANOVA are one-way between-groups and one-way within-groups. A more complicated method is two-way factorial ANOVA with one between-groups and one within-groups factor [106]. The repeated-measures ANOVA can be considered as a special case of two-way ANOVA [107]. The formula of repeated measures ANOVA with one within-groups factor (W) and one between-groups factor (B) is listed as [106]
(8)y~B*W+Error(SubjectW).For each HRV measure, there are two ANOVA tables.W is the participant. If the epoch is fixed, B is the rhythm. If the rhythm is fixed, B is the epoch. Then the Tukey honest significant difference test was used to acquire pair-by-pair comparisons [108] for each between-groups pair.Finally, the pairwiset-tests [109, 110] were applied for C1 and C2 to realize whether the HRV responses are influenced by some particular rhythm.


## 3.1. Experiment

### 3.1.1. Participants

There were 22 healthy subjects, 15 males and 7 females, engaged in the experiment. The average age was 23. None of them had been professionally trained in music.


### 3.1.2. Musical Stimuli

There were four drum loops in this study (L1 to L4). The parameters (tempo, complexity, and energy) are listed in Table2 and the rankings are illustrated in Figure 1.Table 2
Rhythmic features, model factors, and HRV data collection of four drum loops (L1~L4).
Definition
L1
L2
L3
L4
Tp
Tempo
89.60
105.30
139.50
176.50
Cp
Perceptual complexity
2.73
2.27
1.14
3.86
Eg
Energy,∑(k2)/(1013)
1.99
1.02
3.83
1.56
Valence (C1)
−1/Tp
−4.18
−2.23
0.50
2.26
Valence (C2)
Tp/Eg
−2.33
4.48
−3.34
5.64
Arousal (C1)
Eg/Cp
−0.02
−0.05
0.27
−0.05
Arousal (C2)
-
Tp
×
Eg
−0.15
−0.10
−0.42
−0.23
SDNN (C1)
Standard deviation of all RR intervals of C1
−5.01
±6.95
−0.81
±10.57
−0.10
±11.63
2.27
±12.06
SDNN (C2)
Standard deviation of all RR intervals of C2
−2.66
±7.31
5.17
±9.66
−3.14
±15.52
5.08
±10.29
LF/HF (C1)
Ratio of low frequency and high frequency of C1
−0.04
±1.67
−0.02
±0.81
0.28
±0.89
−0.06
±0.66
LF/HF (C2)
Ratio of low frequency and high frequency of C2
−0.10
±0.76
−0.10
±0.68
−0.41
±0.59
−0.29
±1.05Figure 1
Four drum loop patterns (L1~L4) employed in this study with ranked tempo (L4 > L3 > L2 > L1), complexity (L4 > L1 > L2 > L3), and energy (L3 > L1 > L4 > L2).


### 3.1.3. Apparatus

The ECG signal was captured by a 3-channel portable device (MSI E3-80, FDA 510(k) K071085) at 500 Hz sampling rate from the chest surface of the body. Only the channel-1 data was taken to be analyzed.


### 3.1.4. Procedures

All experiments were carried out in moderate temperature, humidity, and light with subjects sitting and wearing headphones (eyes closed) in a quiet room. Each subject accepted 4 rounds of experiment in different days. Each round took 15 minutes, separated as 5 stages, as Figure2 illustrated. Stage 1 had 5 minutes to let the subjects calm down. Stage 2 had 3 minutes of rest, as the baseline for the responses of the drum loops. Stage 3 had 2 minutes, as the baseline for the responses after the drum loops. Stage 4 had 3 minutes of stimulus of some drum loop. Stage 5 had 2 minutes of rest. The ECG signal from stage 2 to stage 5 was recorded and separated as epoch 1 (E1) to epoch 4 (E4). Comparison 1 (C1) was the difference of E1 and E3 and comparison 2 (C2) was the difference of E2 and E4.Figure 2
The procedure of each experiment: the epoch of 5 min to 8 min is the baseline of the physiological responses during music listening and the epoch of 8 min to 10 min is the baseline of the physiological responses after music listening.


## 3.1.1. Participants

There were 22 healthy subjects, 15 males and 7 females, engaged in the experiment. The average age was 23. None of them had been professionally trained in music.


## 3.1.2. Musical Stimuli

There were four drum loops in this study (L1 to L4). The parameters (tempo, complexity, and energy) are listed in Table2 and the rankings are illustrated in Figure 1.Table 2
Rhythmic features, model factors, and HRV data collection of four drum loops (L1~L4).
Definition
L1
L2
L3
L4
Tp
Tempo
89.60
105.30
139.50
176.50
Cp
Perceptual complexity
2.73
2.27
1.14
3.86
Eg
Energy,∑(k2)/(1013)
1.99
1.02
3.83
1.56
Valence (C1)
−1/Tp
−4.18
−2.23
0.50
2.26
Valence (C2)
Tp/Eg
−2.33
4.48
−3.34
5.64
Arousal (C1)
Eg/Cp
−0.02
−0.05
0.27
−0.05
Arousal (C2)
-
Tp
×
Eg
−0.15
−0.10
−0.42
−0.23
SDNN (C1)
Standard deviation of all RR intervals of C1
−5.01
±6.95
−0.81
±10.57
−0.10
±11.63
2.27
±12.06
SDNN (C2)
Standard deviation of all RR intervals of C2
−2.66
±7.31
5.17
±9.66
−3.14
±15.52
5.08
±10.29
LF/HF (C1)
Ratio of low frequency and high frequency of C1
−0.04
±1.67
−0.02
±0.81
0.28
±0.89
−0.06
±0.66
LF/HF (C2)
Ratio of low frequency and high frequency of C2
−0.10
±0.76
−0.10
±0.68
−0.41
±0.59
−0.29
±1.05Figure 1
Four drum loop patterns (L1~L4) employed in this study with ranked tempo (L4 > L3 > L2 > L1), complexity (L4 > L1 > L2 > L3), and energy (L3 > L1 > L4 > L2).


## 3.1.3. Apparatus

The ECG signal was captured by a 3-channel portable device (MSI E3-80, FDA 510(k) K071085) at 500 Hz sampling rate from the chest surface of the body. Only the channel-1 data was taken to be analyzed.


## 3.1.4. Procedures

All experiments were carried out in moderate temperature, humidity, and light with subjects sitting and wearing headphones (eyes closed) in a quiet room. Each subject accepted 4 rounds of experiment in different days. Each round took 15 minutes, separated as 5 stages, as Figure2 illustrated. Stage 1 had 5 minutes to let the subjects calm down. Stage 2 had 3 minutes of rest, as the baseline for the responses of the drum loops. Stage 3 had 2 minutes, as the baseline for the responses after the drum loops. Stage 4 had 3 minutes of stimulus of some drum loop. Stage 5 had 2 minutes of rest. The ECG signal from stage 2 to stage 5 was recorded and separated as epoch 1 (E1) to epoch 4 (E4). Comparison 1 (C1) was the difference of E1 and E3 and comparison 2 (C2) was the difference of E2 and E4.Figure 2
The procedure of each experiment: the epoch of 5 min to 8 min is the baseline of the physiological responses during music listening and the epoch of 8 min to 10 min is the baseline of the physiological responses after music listening.


## 3.2. Signal Processing

### 3.2.1. Musical Features

Musical rhythm is expressed by the successive notes that record the relating temporal information. Tempo (Tp) could be decided as the unit, beats per minute (bpm) [92].The other characteristic, perceptual complexity (Cp), was obtained by asking the human subjects to judge the complexity of the rhythms they had listened to. It was assessed by the subjects using a subjective rating of 1 to 4 on a Likert Scale (4 being the most complex) [93].The energy parameter (Eg) was defined as Σk2, the summation of square of k, where k is the amplitude of the signal [94, 95].


### 3.2.2. HRV Features

To acquire the measures of HRV features, QRS detection was the first step [96–98], where R denotes the peak in a heartbeat signal. After the abnormal beats were rejected [99], the mean of R-R intervals (MRR), standard deviation of normal-to-normal R-R intervals (SDNN), and root of mean of sum of square of differences of adjacent R-R intervals (RMSSD) were measured in time domain [59]. After interpolation [100] (prepared for FFT) and detrending [101] (to filter the respiratory signal), the FFT [102] was applied to calculate the low- (LF) and high-frequency (HF) powers and their ratio (LF/HF). The results of SDNN and LF/HF are listed in Table 2. Four groups of data, SDNN C1, SDNN C2, LF/HF C1, and LF/HF C2, were observed in our analysis.


## 3.2.1. Musical Features

Musical rhythm is expressed by the successive notes that record the relating temporal information. Tempo (Tp) could be decided as the unit, beats per minute (bpm) [92].The other characteristic, perceptual complexity (Cp), was obtained by asking the human subjects to judge the complexity of the rhythms they had listened to. It was assessed by the subjects using a subjective rating of 1 to 4 on a Likert Scale (4 being the most complex) [93].The energy parameter (Eg) was defined as Σk2, the summation of square of k, where k is the amplitude of the signal [94, 95].


## 3.2.2. HRV Features

To acquire the measures of HRV features, QRS detection was the first step [96–98], where R denotes the peak in a heartbeat signal. After the abnormal beats were rejected [99], the mean of R-R intervals (MRR), standard deviation of normal-to-normal R-R intervals (SDNN), and root of mean of sum of square of differences of adjacent R-R intervals (RMSSD) were measured in time domain [59]. After interpolation [100] (prepared for FFT) and detrending [101] (to filter the respiratory signal), the FFT [102] was applied to calculate the low- (LF) and high-frequency (HF) powers and their ratio (LF/HF). The results of SDNN and LF/HF are listed in Table 2. Four groups of data, SDNN C1, SDNN C2, LF/HF C1, and LF/HF C2, were observed in our analysis.


## 3.3. Algorithm

For modeling the responses of some HRV measure and the related musical rhythms, our algorithm included 3 steps. Initially, all possible combinations of the rhythmic features were explored. Secondly, the values of the combinations were linearly transformed. Finally, the coefficients of the linear transformations were calculated such that the Euclidian metric between the results of step 2 and the related HRV responses is the minimum.The stimuli are 4 drum loops (rhythm), named asR1 to R4 here:
(1)Rri,ri∈{1,2,3,4}.Each rhythm relates to some HRV measureHri:
(2)Hri,ri∈{1,2,3,4}.For each rhythm, there are three rhythmic features,Tp, Cp, and Eg, named as F1 to F3:
(3)Ffi,fi∈{1,2,3}.Since a musical stimulus contains multiple combinations and interactions of various features, it is difficult to realize which feature is contributing to the perceived emotion [45], or physiological responses. Our solution is considering all possible combinations: the influence of each feature Ffi is linear (order 1), of no effect (order 0), or inverse (order −1). Although the higher orders are plausible, order one is still the most suitable to construct a simplified model for understanding the relation of the musical rhythms to their related HRV measures:
(4)Eri=∏fi=13(Rri.Ffi)fi.p,fi.p∈{-1,0,1}.Linear transformation is necessary because the units of rhythmic and HRV features are not uniform. All we need are the related correspondences. Consider(5)Tri=x(Eri)+y,x∈R,y∈R.Thus for some subject’s HRV responsesH1 to H4 (e.g., SDNN) to rhythms R1 to R4, some combination of rhythmic features can model the relation if the metric between Tri and Hri is the minimum. Euclidean metric [27, 103] is employed in this work:
(6)D=∑ri=14(Tri-Hri)2.After squaring each side of (6), we can acquire the coefficients of the linear transformation to get the minimum metric if the partial derivatives of D2 (with respect to x and y) are both zero:
(7)D2=∑ri=14(Tri-Hri)2.


## 3.4. Statistics

### 3.4.1. Outliers

The judgment and removal of outliers are fundamental in the processing of experimental data [104]. In our study, the experimental data was separated into three groups by the minimum metric mentioned in Section 3.3 and the group with larger metric was eliminated.The basic idea is to rank a group of data by the algorithm we proposed and make the data with larger metric obsolete. There were four groups of data, SDNN (C1), SDNN (C2), LF/HF (C1), and LF/HF (C2). Each group had 22 records of the 22 participants. Each record had four subrecords of drum loops L1 to L4. First, we used the algorithm in each group and ranked the 22 records as 1 to 22 by their minimum metric. After summing the ranks of the four groups of these 22 participants, these participants were partitioned into three levels: small, medium, and large metric (7, 7, and 8 participants). We defined the predictable class with the small and medium metrics and the nonpredictable class with the large metric. The nonpredictable class was excluded from our experimental data. After removing the nonpredictable class, the rhythmic features and the four averages of these four groups of predictable class were modeled by our algorithm again.


### 3.4.2. ANOVA andt-Test

Repeated measurements have four major advantages: obtaining the individual patterns of change, less subjects, serving as subjects’ own controls, and reliability; the disadvantages are the complication by the dependence among repeated observations and less control of the circumstances [105]. In our experiment, the repeated measurements were employed.Two basic methods of ANOVA are one-way between-groups and one-way within-groups. A more complicated method is two-way factorial ANOVA with one between-groups and one within-groups factor [106]. The repeated-measures ANOVA can be considered as a special case of two-way ANOVA [107]. The formula of repeated measures ANOVA with one within-groups factor (W) and one between-groups factor (B) is listed as [106]
(8)y~B*W+Error(SubjectW).For each HRV measure, there are two ANOVA tables.W is the participant. If the epoch is fixed, B is the rhythm. If the rhythm is fixed, B is the epoch. Then the Tukey honest significant difference test was used to acquire pair-by-pair comparisons [108] for each between-groups pair.Finally, the pairwiset-tests [109, 110] were applied for C1 and C2 to realize whether the HRV responses are influenced by some particular rhythm.


## 3.4.1. Outliers

The judgment and removal of outliers are fundamental in the processing of experimental data [104]. In our study, the experimental data was separated into three groups by the minimum metric mentioned in Section 3.3 and the group with larger metric was eliminated.The basic idea is to rank a group of data by the algorithm we proposed and make the data with larger metric obsolete. There were four groups of data, SDNN (C1), SDNN (C2), LF/HF (C1), and LF/HF (C2). Each group had 22 records of the 22 participants. Each record had four subrecords of drum loops L1 to L4. First, we used the algorithm in each group and ranked the 22 records as 1 to 22 by their minimum metric. After summing the ranks of the four groups of these 22 participants, these participants were partitioned into three levels: small, medium, and large metric (7, 7, and 8 participants). We defined the predictable class with the small and medium metrics and the nonpredictable class with the large metric. The nonpredictable class was excluded from our experimental data. After removing the nonpredictable class, the rhythmic features and the four averages of these four groups of predictable class were modeled by our algorithm again.


## 3.4.2. ANOVA andt-Test

Repeated measurements have four major advantages: obtaining the individual patterns of change, less subjects, serving as subjects’ own controls, and reliability; the disadvantages are the complication by the dependence among repeated observations and less control of the circumstances [105]. In our experiment, the repeated measurements were employed.Two basic methods of ANOVA are one-way between-groups and one-way within-groups. A more complicated method is two-way factorial ANOVA with one between-groups and one within-groups factor [106]. The repeated-measures ANOVA can be considered as a special case of two-way ANOVA [107]. The formula of repeated measures ANOVA with one within-groups factor (W) and one between-groups factor (B) is listed as [106]
(8)y~B*W+Error(SubjectW).For each HRV measure, there are two ANOVA tables.W is the participant. If the epoch is fixed, B is the rhythm. If the rhythm is fixed, B is the epoch. Then the Tukey honest significant difference test was used to acquire pair-by-pair comparisons [108] for each between-groups pair.Finally, the pairwiset-tests [109, 110] were applied for C1 and C2 to realize whether the HRV responses are influenced by some particular rhythm.


## 4. Results

Table2 collected the musical features, factors of models, and HRV data in this study. Valence C1 and arousal C1 reveal how rhythmic features influenced the HRV responses while listening to music, and valence C2 and arousal C2 reveal how rhythmic features influenced the HRV responses after listening to music. The values of modified combinations of the rhythmic and HRV features were illustrated in Figures 3(a) and 3(b). Furthermore, (9) to (12) demonstrated the relationships.(a) The gray bars represent the averages of SDNN and the white bars represent the approached values of the model. Items 1 to 4 represent drum loops 1 to 4 in C1 and items 6 to 9 represent drum loops 1 to 4 in C2; (b) gray bars represent the averages of LF/HF and white bars represent the approached values of the model. Items 1 to 4 represent drum loops 1 to 4 in C1 and items 6 to 9 represent drum loops 1 to 4 in C2 (note: related numeric data is presented in Table2).
(a)
(b)Fast tempo enhanced SDNN; that is, people prefer faster tempi:(9)SDNN(C1)∝-1Tp.High intensity and low complexity enhanced LF/HF, the physiological arousal:(10)LFHF(C1)∝EgCp.People prefer faster tempi. However, the sound should not be too loud if we hope the effect can remain after the music stops:(11)SDNN(C2)∝TpEg.
If fast and loud music stops, people feel relaxed:
(12)LFHF(C2)∝-Tp×Eg.Table3 is the statistical results. The SDNN (C2) of the four rhythms had a significance value 0.01. The LF/HF of L3 had a significance value 0.03 while and after listening to music.Table 3
StatisticalP values for epochs and rhythms.
(a)
SDNN
Epoch
Pairwise
L1-L2
L2-L3
L3-L4
L1–L3
L2–L4
L1–L4
L1~L4
E1
1.00
1.00
1.00
1.00
1.00
1.00
0.57
E2
0.40
0.40
0.63
1.00
1.00
1.00
0.15
E3
1.00
1.00
1.00
0.96
1.00
1.00
0.47
E4
1.00
1.00
1.00
1.00
1.00
1.00
0.77
C1
0.64
1.00
1.00
1.00
1.00
0.55
0.26
C2
0.14
0.14
0.14
1.00
1.00
0.14
0.01
**
Rhythm
Pairwise
E1-E2
E2-E3
E3-E4
E1–E3
E2–E4
E1–E4
E1~E4
C1-C2
L1
0.75
0.75
0.94
0.09
0.75
0.07
0.02
*
0.34
L2
0.02
0.19
1.00
1.00
0.27
1.00
0.07
.
0.12
L3
1.00
1.00
1.00
1.00
1.00
1.00
0.60
0.41
L4
1.00
1.00
1.00
1.00
0.53
1.00
0.40
0.40
(b)
LF/HF
Epoch
Pairwise
L1-L2
L2-L3
L3-L4
L1–L3
L2–L4
L1–L4
L1~L4
E1
1.00
0.09
0.36
0.97
1.00
1.00
0.41
E2
1.00
1.00
1.00
1.00
1.00
1.00
0.77
E3
1.00
1.00
1.00
1.00
1.00
1.00
0.78
E4
1.00
1.00
1.00
1.00
1.00
1.00
0.44
C1
1.00
1.00
1.00
1.00
1.00
1.00
0.80
C2
1.00
0.66
1.00
1.00
1.00
1.00
0.54
Rhythm
Pairwise
E1-E2
E2-E3
E3-E4
E1–E3
E2–E4
E1–E4
E1~E4
C1-C2
L1
1.00
1.00
1.00
1.00
1.00
1.00
0.87
0.90
L2
0.27
0.74
0.07
1.00
1.00
0.27
0.09
.
0.55
L3
0.42
1.00
0.45
0.82
0.14
1.00
0.12
0.03
*
L4
1.00
1.00
0.92
1.00
1.00
0.92
0.23
0.50
The values of HRV measures in C1 are the comparisons (differences) of values of HRV measures in epochs E1 and E3.
The values of HRV measures in C2 are the comparisons (differences) of values of HRV measures in epochs E2 and E4.
L1-L2: drum loops 1 and 2; L1~L4: drum loops 1 to 4.
Significant codes: 0 “***” 0.001 “**” 0.01 “*” 0.05 “.” 0.1 “  ” 1.


## 5. Discussion

### 5.1. Advantages of the Proposed Model

There are four major advantages in this work. Initially, the psychological music models are many [2, 25–45], but the physiological music models are rare [3]. Secondly, the psychological music models [2, 25–45] and physiological musical experiments [1, 111] are many, but studies of the psychological rhythm model [26] and the physiological rhythmic experiment [112] are rare. Thirdly, almost all studies concern the responses during the music, but discussions about the responses after the music are limited [113]. Finally, there are many music studies about HRV [1, 111], but there is no model. Our work is the first one.


### 5.2. Comparison of the Models

There are six levels of musical model illustrated in Figure4: (a) the model from the results of the psychological experiments, (b) the model of the perceived emotion, (c) the model of the felt emotion, (d) the model from the physiological results, (e) the proposed physiological valence/arousal model while listening to the musical rhythms, and (f) the proposed physiological valence/arousal model after listening to the musical rhythms.Six levels of the relationships between the rhythmic features and the valence/arousal model; the horizontal axis is valence and the vertical axis is arousal. (a) Model from the results of the psychological experiments, (b) model of the perceived emotion, (c) model of the felt emotion, (d) model from the results of the physiological experiments, (e) proposed physiological valence/arousal model while listening to the musical rhythm, and (f) proposed physiological valence/arousal model after listening to the musical rhythm.
(a)
(b)
(c)
(d)
(e)
(f)The six levels of model need not reach a consensus. There are four reasons. Initially, the perceived emotion is not always the same as the felt emotion [10, 54]. These two emotions are closer if the music clips are chosen by the participants themselves instead of others. Secondly, some measured physiological responses might not have been reported by self-report [55]. Thirdly, if the music clips are simplified to the form of rhythm or tempo, the responses are changed [112]. Finally, the responses while and after listening to music need not be the same obviously.Figure4(a) shows that Tp and Eg dominate arousal and Cp dominates valence in a survey of psychological experiments (pp. 383–392) [1]. The effects of Tp and Eg on arousal are clear. As for valence, the regular and varied rhythms derive positive emotions and the irregular rhythms derive negative emotions (pp. 383–392) [1]. Hence valence is negatively correlated to Cp.Figure4(b) shows that Tp [26, 31, 45] and Eg [26, 27, 31, 45] dominate arousal and Tp [27] and Cp [27] dominate valence in the perceived models. The results of arousal for the perceived models are the same as the psychological experiments. But Tp and Cp are viewed as the valence-based features. Whether they are positively correlated to valence or not has not been highlighted in the paper [27].Figure4(c) shows that Tp [35] and Eg [35] dominate arousal and Tp [35] dominates valence in the felt model. The results of felt and perceived models are similar. But the valence-based feature is limited at Tp in the felt model [35].Figure4(d) shows that Tp [55, 113, 114] and Eg [114] dominate arousal and Tp [55] dominates valence in the physiological experiments. Both psychological studies (Figures 4(a), 4(b), and 4(c)) and physiological studies (Figure 4(d)) have the same results about arousal. But the faster tempi are found to decrease SDNN, the physiological valence [55].Figure4(e) shows that Eg and Cp dominate arousal and Tp dominates valence. Eg is directly proportional to arousal and Cp is inversely proportional to arousal. As for valence, faster tempi increase SDNN, the physiological valence. This relation is also illustrated in the left part of Figure 3(a).Figure4(f) shows that Tp and Eg dominate both arousal and valence after the stimuli. The arousal is negatively correlated to Tp×Eg. Hence if a fast and loud rhythm is removed, the subjects will feel relaxed. Fast tempi increase valence in Figure 4(e), the responses during music. This effect remains after the music stops if the intensity of the music is low.In the valence perspective,Tp and Cp are considered as two major rhythmic parameters [27]. As for Tp, there exist opposite comments. Tp is positively correlated with valence in the felt models [35] but Tp is negatively correlated with physiological valence since the faster tempo decreased SDNN [55]. The proper conclusion is that the role of Tp is dependent on the context: both slow and fast tempi can derive different emotions (pp. 383–392) [1]. For example, both happy and angry music have fast tempi. As for Cp, its contribution is not defined in the perceived model [27]. Although Cp is negatively correlated with valence in the psychological experiments (pp. 383–392) [1], the same definition has opposite results: the firm rhythm derives positive valence (pp. 383–392) [1] and the firm rhythm derives negative valence [31]. Instead of music, if only the rhythm is considered, higher SDNN is observed with faster tempi in our work, as Figure 4(e) illustrated; and Eg affects the responses of physiological valence after the music stops, as Figure 4(f) illustrated.In the arousal perspective,Tp and Eg are two of the most dominant features. Our findings also show that if a fast, loud sound (drum loop 3) was removed, LF/HF would decrease, as illustrated in the right part of Figure 3(b). Although Tp is usually considered as the most important factor in all features (pp. 383–392) [1], another research indicates Eg is more important than Tp in arousal [45]; our findings support this opinion, as illustrated in Figure 4(e). Cp is considered as a valence parameter in Figures 4(a) and 4(b). But our results show that simple rhythms enhanced the physiological arousal.


### 5.3. Statistical Results

Table3 collects all P values and there are two significant results: the ANOVA of SDNN (C2) for L1~L4 and the t-test of LF/HF (C1, C2) for L3. Although the significant value is more than 0.05, the left part of Figure 3(a) still shows that SDNN, the physiological valence, is positively correlated with the tempi of the drum loops. The right part of Figure 3(a) shows that SDNN is positively correlated with Tp/Eg (P<0.01). Both L2 and L4 have faster tempi and smaller energies. The left part of Figure 3(b) shows that LF/HF, the physiological arousal, is positively correlated with Eg/Cp. L3, a fast drum loop with a very simple complexity, enhances arousal in the largest degree. The right part of Figure 3(b) shows that after the music stops, the fast and loud rhythms make the participants relaxed. Although there is no statistical significance in Figure 3(b), the t-test result of L3, while and after listening to the music clip, reaches statistical significance (P<0.05).


### 5.4. Limitations

#### 5.4.1. The Three Rhythmic Parameters

Of the three elementary rhythmic features, tempo is the most definite [92]. Tp is positively correlated with SDNN, the physiological valence (Figure 4(e)). In the other way, Tp is negatively correlated with SDNN [55]. To integrate these two opposite findings, the psychological perspectives (Figures 4(a), 4(b), and 4(c)) are more suitable to explain this dilemma. That is, Tp is a feature of arousal; both positive and negative valence can be measured; it depends on the context (pp. 383–392) [1].To measure the complexity of a music or rhythm clip, mathematical quantities are better than adjectives. For example, the firm rhythms can derive both positive (pp. 383–392) [1] and negative [31] valence. We can acquire a number from Likert Scale [93]; however, it is not proper as an engineering reference. Of the mathematical studies about rhythmic complexity [115–117], oddity [117] is the closest method for measuring the perceptual complexity. There is a more fundamental issue; two rhythms with different responses may have the same complexity measure.The last parameter, the total energy of a waveform, is defined asΣk2, the summation of square of k, where k is the amplitude of the signal [94, 95]. But the intensity within the waveform may not be fixed: large or small variations suggest fear or happiness; the rapid or few changes in intensity may be associated with pleading or sadness (pp. 383–392) [1]. If the waveform is partitioned into small segments [118], the energy feature will be not only an arousal, but also a valence parameter (pp. 383–392) [1]. And the analytic resolution of the musical emotion will increase.


#### 5.4.2. Nonlinearity of the Responses

The linear and inverse functions are simple, intuitive, and useful in a general overview. However, some studies assume that the liking (or valence) of music is an inverted-U curve, dependent on arousal (tempo and energy) [119–121] and complexity [119, 120]. The inverted-U phenomenon may disappear with the professional musical expertise [122]. There is also a plausible transform of the inverted-U curve: although 120 bpm may be a preferred tempo for common people [123], the twin-peak curve was found in both a popular music database and physiological experiments [124]. Finally, the more complex curves are possible [3].


### 5.5. The Modified Model

T
p and Eg correlate with arousal, as Figures 4(a)–4(d) and Figure 4(f) illustrated, but Eg dominates the arousal [45], as Figure 4(e) showed. To integrate these models, let Eg′ be the average energy of all beats. Thus Eg is the product of Tp and Eg′, and (9)–(12) can be modified as (13)–(16). This provides us with an advanced aspect.The valence of musical responses keeps the same:(13)SDNN(C1)∝-1Tp.Both tempo and energy are arousal parameters now. Moreover, our model demonstrates that the complexity of rhythm is also an arousal parameter:(14)LFHF(C1)∝Tp×Eg′Cp.The valence after the musical stimuli is influenced by the energy:(15)SDNN(C2)∝1Eg′.Finally, both tempo and energy influence the arousal after musical stimuli, but tempo dominates the effect:(16)LFHF(C2)∝-Tp2×Eg′.To integrate all psychological and physiological music theories, we summarize briefly that tempo, energy, and complexity are both valence and arousal parameters. The inverted-U curve [119–121] is suggested for valence. Considering the valence after music, people prefer the music of low intensity. For the responses after the musical stimuli, the arousal correlates negatively with tempo and energy whereas tempo is the dominant parameter.


### 5.6. Criteria for Good Models

There are four criteria for a valuable model [22]. They are generalization (goodness-of-fit [21]) and predictive power, simplicity, and its relation to existing theories. In plain words, a model should be able to explain experimental data used in the model and not used in the model for verification. The parameters should be small relative to the amount of the data. The model should be able to unify two formerly unrelated theories. It will help to understand the underlying phenomenon instead of an input-output routine.Our proposed model satisfies generalization, simplicity, and the relation to other theories. In our study, the selected model has the least metric (error) among all possible models. It has only three parameters (Tp, Cp, and Eg). And it fills the gap among the psychological and physiological experiments and the models of music and rhythm. The proposed model does not have enough predictive power. It was built for the subjects whose responses are able to be modeled; the nonpredictable class, one-third of all subjects, was excluded. There was no test data in this study. However, the model derived from the experimental data can be integrated well with the literature in this field, as Section 5.5 mentioned.


## 5.1. Advantages of the Proposed Model

There are four major advantages in this work. Initially, the psychological music models are many [2, 25–45], but the physiological music models are rare [3]. Secondly, the psychological music models [2, 25–45] and physiological musical experiments [1, 111] are many, but studies of the psychological rhythm model [26] and the physiological rhythmic experiment [112] are rare. Thirdly, almost all studies concern the responses during the music, but discussions about the responses after the music are limited [113]. Finally, there are many music studies about HRV [1, 111], but there is no model. Our work is the first one.


## 5.2. Comparison of the Models

There are six levels of musical model illustrated in Figure4: (a) the model from the results of the psychological experiments, (b) the model of the perceived emotion, (c) the model of the felt emotion, (d) the model from the physiological results, (e) the proposed physiological valence/arousal model while listening to the musical rhythms, and (f) the proposed physiological valence/arousal model after listening to the musical rhythms.Six levels of the relationships between the rhythmic features and the valence/arousal model; the horizontal axis is valence and the vertical axis is arousal. (a) Model from the results of the psychological experiments, (b) model of the perceived emotion, (c) model of the felt emotion, (d) model from the results of the physiological experiments, (e) proposed physiological valence/arousal model while listening to the musical rhythm, and (f) proposed physiological valence/arousal model after listening to the musical rhythm.
(a)
(b)
(c)
(d)
(e)
(f)The six levels of model need not reach a consensus. There are four reasons. Initially, the perceived emotion is not always the same as the felt emotion [10, 54]. These two emotions are closer if the music clips are chosen by the participants themselves instead of others. Secondly, some measured physiological responses might not have been reported by self-report [55]. Thirdly, if the music clips are simplified to the form of rhythm or tempo, the responses are changed [112]. Finally, the responses while and after listening to music need not be the same obviously.Figure4(a) shows that Tp and Eg dominate arousal and Cp dominates valence in a survey of psychological experiments (pp. 383–392) [1]. The effects of Tp and Eg on arousal are clear. As for valence, the regular and varied rhythms derive positive emotions and the irregular rhythms derive negative emotions (pp. 383–392) [1]. Hence valence is negatively correlated to Cp.Figure4(b) shows that Tp [26, 31, 45] and Eg [26, 27, 31, 45] dominate arousal and Tp [27] and Cp [27] dominate valence in the perceived models. The results of arousal for the perceived models are the same as the psychological experiments. But Tp and Cp are viewed as the valence-based features. Whether they are positively correlated to valence or not has not been highlighted in the paper [27].Figure4(c) shows that Tp [35] and Eg [35] dominate arousal and Tp [35] dominates valence in the felt model. The results of felt and perceived models are similar. But the valence-based feature is limited at Tp in the felt model [35].Figure4(d) shows that Tp [55, 113, 114] and Eg [114] dominate arousal and Tp [55] dominates valence in the physiological experiments. Both psychological studies (Figures 4(a), 4(b), and 4(c)) and physiological studies (Figure 4(d)) have the same results about arousal. But the faster tempi are found to decrease SDNN, the physiological valence [55].Figure4(e) shows that Eg and Cp dominate arousal and Tp dominates valence. Eg is directly proportional to arousal and Cp is inversely proportional to arousal. As for valence, faster tempi increase SDNN, the physiological valence. This relation is also illustrated in the left part of Figure 3(a).Figure4(f) shows that Tp and Eg dominate both arousal and valence after the stimuli. The arousal is negatively correlated to Tp×Eg. Hence if a fast and loud rhythm is removed, the subjects will feel relaxed. Fast tempi increase valence in Figure 4(e), the responses during music. This effect remains after the music stops if the intensity of the music is low.In the valence perspective,Tp and Cp are considered as two major rhythmic parameters [27]. As for Tp, there exist opposite comments. Tp is positively correlated with valence in the felt models [35] but Tp is negatively correlated with physiological valence since the faster tempo decreased SDNN [55]. The proper conclusion is that the role of Tp is dependent on the context: both slow and fast tempi can derive different emotions (pp. 383–392) [1]. For example, both happy and angry music have fast tempi. As for Cp, its contribution is not defined in the perceived model [27]. Although Cp is negatively correlated with valence in the psychological experiments (pp. 383–392) [1], the same definition has opposite results: the firm rhythm derives positive valence (pp. 383–392) [1] and the firm rhythm derives negative valence [31]. Instead of music, if only the rhythm is considered, higher SDNN is observed with faster tempi in our work, as Figure 4(e) illustrated; and Eg affects the responses of physiological valence after the music stops, as Figure 4(f) illustrated.In the arousal perspective,Tp and Eg are two of the most dominant features. Our findings also show that if a fast, loud sound (drum loop 3) was removed, LF/HF would decrease, as illustrated in the right part of Figure 3(b). Although Tp is usually considered as the most important factor in all features (pp. 383–392) [1], another research indicates Eg is more important than Tp in arousal [45]; our findings support this opinion, as illustrated in Figure 4(e). Cp is considered as a valence parameter in Figures 4(a) and 4(b). But our results show that simple rhythms enhanced the physiological arousal.


## 5.3. Statistical Results

Table3 collects all P values and there are two significant results: the ANOVA of SDNN (C2) for L1~L4 and the t-test of LF/HF (C1, C2) for L3. Although the significant value is more than 0.05, the left part of Figure 3(a) still shows that SDNN, the physiological valence, is positively correlated with the tempi of the drum loops. The right part of Figure 3(a) shows that SDNN is positively correlated with Tp/Eg (P<0.01). Both L2 and L4 have faster tempi and smaller energies. The left part of Figure 3(b) shows that LF/HF, the physiological arousal, is positively correlated with Eg/Cp. L3, a fast drum loop with a very simple complexity, enhances arousal in the largest degree. The right part of Figure 3(b) shows that after the music stops, the fast and loud rhythms make the participants relaxed. Although there is no statistical significance in Figure 3(b), the t-test result of L3, while and after listening to the music clip, reaches statistical significance (P<0.05).


## 5.4. Limitations

### 5.4.1. The Three Rhythmic Parameters

Of the three elementary rhythmic features, tempo is the most definite [92]. Tp is positively correlated with SDNN, the physiological valence (Figure 4(e)). In the other way, Tp is negatively correlated with SDNN [55]. To integrate these two opposite findings, the psychological perspectives (Figures 4(a), 4(b), and 4(c)) are more suitable to explain this dilemma. That is, Tp is a feature of arousal; both positive and negative valence can be measured; it depends on the context (pp. 383–392) [1].To measure the complexity of a music or rhythm clip, mathematical quantities are better than adjectives. For example, the firm rhythms can derive both positive (pp. 383–392) [1] and negative [31] valence. We can acquire a number from Likert Scale [93]; however, it is not proper as an engineering reference. Of the mathematical studies about rhythmic complexity [115–117], oddity [117] is the closest method for measuring the perceptual complexity. There is a more fundamental issue; two rhythms with different responses may have the same complexity measure.The last parameter, the total energy of a waveform, is defined asΣk2, the summation of square of k, where k is the amplitude of the signal [94, 95]. But the intensity within the waveform may not be fixed: large or small variations suggest fear or happiness; the rapid or few changes in intensity may be associated with pleading or sadness (pp. 383–392) [1]. If the waveform is partitioned into small segments [118], the energy feature will be not only an arousal, but also a valence parameter (pp. 383–392) [1]. And the analytic resolution of the musical emotion will increase.


### 5.4.2. Nonlinearity of the Responses

The linear and inverse functions are simple, intuitive, and useful in a general overview. However, some studies assume that the liking (or valence) of music is an inverted-U curve, dependent on arousal (tempo and energy) [119–121] and complexity [119, 120]. The inverted-U phenomenon may disappear with the professional musical expertise [122]. There is also a plausible transform of the inverted-U curve: although 120 bpm may be a preferred tempo for common people [123], the twin-peak curve was found in both a popular music database and physiological experiments [124]. Finally, the more complex curves are possible [3].


## 5.4.1. The Three Rhythmic Parameters

Of the three elementary rhythmic features, tempo is the most definite [92]. Tp is positively correlated with SDNN, the physiological valence (Figure 4(e)). In the other way, Tp is negatively correlated with SDNN [55]. To integrate these two opposite findings, the psychological perspectives (Figures 4(a), 4(b), and 4(c)) are more suitable to explain this dilemma. That is, Tp is a feature of arousal; both positive and negative valence can be measured; it depends on the context (pp. 383–392) [1].To measure the complexity of a music or rhythm clip, mathematical quantities are better than adjectives. For example, the firm rhythms can derive both positive (pp. 383–392) [1] and negative [31] valence. We can acquire a number from Likert Scale [93]; however, it is not proper as an engineering reference. Of the mathematical studies about rhythmic complexity [115–117], oddity [117] is the closest method for measuring the perceptual complexity. There is a more fundamental issue; two rhythms with different responses may have the same complexity measure.The last parameter, the total energy of a waveform, is defined asΣk2, the summation of square of k, where k is the amplitude of the signal [94, 95]. But the intensity within the waveform may not be fixed: large or small variations suggest fear or happiness; the rapid or few changes in intensity may be associated with pleading or sadness (pp. 383–392) [1]. If the waveform is partitioned into small segments [118], the energy feature will be not only an arousal, but also a valence parameter (pp. 383–392) [1]. And the analytic resolution of the musical emotion will increase.


## 5.4.2. Nonlinearity of the Responses

The linear and inverse functions are simple, intuitive, and useful in a general overview. However, some studies assume that the liking (or valence) of music is an inverted-U curve, dependent on arousal (tempo and energy) [119–121] and complexity [119, 120]. The inverted-U phenomenon may disappear with the professional musical expertise [122]. There is also a plausible transform of the inverted-U curve: although 120 bpm may be a preferred tempo for common people [123], the twin-peak curve was found in both a popular music database and physiological experiments [124]. Finally, the more complex curves are possible [3].


## 5.5. The Modified Model

T
p and Eg correlate with arousal, as Figures 4(a)–4(d) and Figure 4(f) illustrated, but Eg dominates the arousal [45], as Figure 4(e) showed. To integrate these models, let Eg′ be the average energy of all beats. Thus Eg is the product of Tp and Eg′, and (9)–(12) can be modified as (13)–(16). This provides us with an advanced aspect.The valence of musical responses keeps the same:(13)SDNN(C1)∝-1Tp.Both tempo and energy are arousal parameters now. Moreover, our model demonstrates that the complexity of rhythm is also an arousal parameter:(14)LFHF(C1)∝Tp×Eg′Cp.The valence after the musical stimuli is influenced by the energy:(15)SDNN(C2)∝1Eg′.Finally, both tempo and energy influence the arousal after musical stimuli, but tempo dominates the effect:(16)LFHF(C2)∝-Tp2×Eg′.To integrate all psychological and physiological music theories, we summarize briefly that tempo, energy, and complexity are both valence and arousal parameters. The inverted-U curve [119–121] is suggested for valence. Considering the valence after music, people prefer the music of low intensity. For the responses after the musical stimuli, the arousal correlates negatively with tempo and energy whereas tempo is the dominant parameter.


## 5.6. Criteria for Good Models

There are four criteria for a valuable model [22]. They are generalization (goodness-of-fit [21]) and predictive power, simplicity, and its relation to existing theories. In plain words, a model should be able to explain experimental data used in the model and not used in the model for verification. The parameters should be small relative to the amount of the data. The model should be able to unify two formerly unrelated theories. It will help to understand the underlying phenomenon instead of an input-output routine.Our proposed model satisfies generalization, simplicity, and the relation to other theories. In our study, the selected model has the least metric (error) among all possible models. It has only three parameters (Tp, Cp, and Eg). And it fills the gap among the psychological and physiological experiments and the models of music and rhythm. The proposed model does not have enough predictive power. It was built for the subjects whose responses are able to be modeled; the nonpredictable class, one-third of all subjects, was excluded. There was no test data in this study. However, the model derived from the experimental data can be integrated well with the literature in this field, as Section 5.5 mentioned.


## 6. Conclusion and Future Work

A novel experiment, a novel algorithm, and a novel model have been proposed in this study. The experiment explored how rhythm, the fundamental feature of music, influenced the ANS and HRV. And the relationships between musical features and physiological features were derived from the algorithm. Moreover, the model demonstrated how the rhythmic features were mapped to the physiological valence/arousal plane.The equations in our model could be the rules for music generating systems [125–127]. The model could also be fine-tuned if the rhythmic oddity [117] (for complexity), small segments [118] (for energy), and various curves [3, 119–121, 124] are considered.



---

*Source: 851796-2014-09-17.xml*