## 4 Analysis of Modulation Spectral Features  
To extract MSFs from emotional speech signals, it is first necessary to calculate the modulation spectrogram using a modulation filterbank. Figure 6 shows the modulation process we used. Emotional speech signals \(s\) were divided into several frequency bands by using an auditory-based band-pass filterbank:  
\[s_{k}(n)=s(n)*h_{k}(n), \tag{3}\]  
where * denotes the convolution operator, \(h_{k}(n)\) is the impulse response of the \(k\)th channel, and \(n\) is the sample number in the time domain. The bandwidth and boundary frequencies of the 6 th-order Butterworth IIR BPFs were the same as those of the auditory filterbank mentioned in Section 2.1. The boundary frequencies of the BPFs were defined as the ERB\({}_{\text{N}}\)-number from 3 to 35 Cam with an 8 ERB\({}_{\text{N}}\) bandwidth, and the number of channels was four. The temporal envelope of the output signal from each BPF \(s_{k}(n)\) was extracted using the Hilbert transform, and a 2 nd-order Butterworth IIR LPF (cut-off frequency 64 Hz) was used as follows:  
\[e_{k}(n)=\text{LPF}[|s_{k}(n)+j\mathcal{H}[s_{k}(n)]|], \tag{4}\]  
where \(\mathcal{H}\) denotes the Hilbert transform. The next step involved decomposing the temporal envelope into several modulation-frequency bands by using a modulation filterbank:  
\[E_{k,m}(n)=g_{m}(n)*(e_{k}(n)-\overline{e_{k}(n)}), \tag{5}\]  
where \(m\) is the channel number of the modulation filter, \(g_{m}(n)\) is the impulse response of the modulation filterbank, and \(\overline{e_{k}(n)}\) is the time-averaged amplitude of \(e_{k}(n)\). The modulation filterbank consisted of six filters (one LPF and five BPFs). The boundary frequencies of the filters were spaced on an octave frequency band from 2 to 64 Hz. Figure 5: Process of calculating the discrimination index of experimental results. The root-mean square (RMS) of \(E_{k,m}(n)\) was calculated as the modulation spectrogram,  
\[\bar{E}_{k,m}=\sqrt{\frac{1}{N}\sum_{n=1}^{N}E_{k,m}^{2}(n)}, \tag{6}\]  
where \(N\) is the length of the speech signal \(s(n)\). The \(\bar{E}_{k,m}\) was then used to calculate modulation spectral features later. In the above analysis, the modulation spectrogram of modulation speech signals and the confusion matrix of the vocal-emotion-recognition results could provide qualitative evidence to clarify the contribution of modulation information of TAE to vocal emotion recognition. In the subsequent analysis, the similarity between the discriminability index of the vocal-emotion-recognition results (\(d^{\prime}_{\text{P}}\)) and MSFs (\(d^{\prime}_{\text{MSF}}\)) was then calculated to provide quantitative evidence. Ten MSFs should be extracted from the modulation spectrograms. Ten are those in the acoustic-frequency domain (the subscript is \(m\)) and in the modulation-frequency domain (the subscript is \(k\)): the modulation spectral centroid (MSC\({}_{m/k}\)), modulation spectral spread (MSSP\({}_{m/k}\)), modulation spectral skewness (MSSK\({}_{m/k}\)), and modulation spectral kurtosis (MSKT\({}_{m/k}\)), which are defined as follows:  
\[\text{MSCR}_{\text{m}}=\frac{\sum_{k=1}^{K}k\bar{E}_{k,m}}{\sum_{k=1}^{K}\bar {E}_{k,m}} \tag{7}\]  
\[\text{MSSP}_{\text{m}}=\frac{\sum_{k=1}^{K}[k-\text{MSCR}_{\text{m}}]^{2}\bar {E}_{k,m}}{\sum_{k=1}^{K}\bar{E}_{k,m}} \tag{8}\]  
\[\text{MSSK}_{\text{m}}=\frac{\sum_{k=1}^{K}[k-\text{MSCR}_{\text{m}}]^{3}\bar {E}_{k,m}}{\sum_{k=1}^{K}\bar{E}_{k,m}} \tag{9}\]  
\[\text{MSKT}_{\text{m}}=\frac{\sum_{k=1}^{K}[k-\text{MSCR}_{\text{m}}]^{4} \bar{E}_{k,m}}{\sum_{k=1}^{K}\bar{E}_{k,m}} \tag{10}\]  
\[\text{MSCR}_{\text{k}}=\frac{\sum_{m=1}^{M}m\bar{E}_{k,m}}{\sum_{m=1}^{M} \bar{E}_{k,m}} \tag{11}\]  
\[e_{k}(n)=\text{LPF}[|s_{k}(n)+j\mathcal{H}[s_{k}(n)]|], \tag{4}\]0  
\[e_{k}(n)=\text{LPF}[|s_{k}(n)+j\mathcal{H}[s_{k}(n)]|], \tag{4}\]1  
\[e_{k}(n)=\text{LPF}[|s_{k}(n)+j\mathcal{H}[s_{k}(n)]|], \tag{4}\]2  
\[e_{k}(n)=\text{LPF}[|s_{k}(n)+j\mathcal{H}[s_{k}(n)]|], \tag{4}\]3 Process of calculating the discrimination index of MSFs [15].