# Emotion Recognition Method for Call/Contact Centre Systems
## 4 Methods
### Speech Signal Descriptors  
Different types of descriptors can be used when considering the issue of recognizing emotions occurring in audio conversations. They usually belong to one of three groups, namely: temporal, frequency, or cepstral descriptors. When examining a speech signal in the time domain, it can be parameterized using classical statistical parameters such as mean, variance, and standard deviation. Sometimes, signal energy, power, and RMS (root mean square) values are also determined [67]. In speech analysis systems, the average fundamental frequency denoted as F0, and the jitter and shimmer coefficients are also determined [68]. For the frequency representation of a speech signal, the parameters of the first and second formant frequency (formants F1 and F2) can be considered. Increased formant values may indicate a signal with a strong energy charge (e.g., a raised tone of voice). The audio signal can also be parameterized with different variants, cepstral coefficients. The most commonly used coefficients in the literature on emotion recognition problems [69; 70] are: mel-frequency cepstral coefficients (MFCC), linear predictive cepstral coefficients (LPCC), and linear predictive coding (LPC). These coefficients reflect the acoustic nature of the signal.  
The DBV1 database, described in detail in Section 2, was used to assess the relevance of each descriptor to the works presented in this paper. Based on the analyses performed, the following speech signal descriptors were finally selected: fundamental frequency F0, MFCC, jitter, and shimmer coefficients. As a result of literature analysis [71; 72; 73; 74; 75], it was determined that the first 13 MFCC coefficients describing the frequency parameters of the speech signal would be used for emotion recognition. These features are extended  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Word** & **Group of Words** \\ \hline wish & nice, hours, day \\ \hline works & slow, messenger, mostly \\ \hline unfortunately & my, telephone, allows, important \\ \hline very & Mr, thank \\ \hline where & information, stated, important \\ \hline please & prompt, find, can \\ \hline help & can, somehow, something \\ \hline unacceptable & disconnected, can, number \\ \hline client & service, 24, disconnected, headquarters \\ \hline case & weeks, last, 2 \\ \hline better & contact, attention, was \\ \hline want & by, given, indicated \\ \hline waiting & once, help, case \\ \hline favour & short, window, mark \\ \hline best & wishes, end, day, nice \\ \hline \hline \end{tabular}
\end{table}
Table 7: Collocations of words.  
to include the jitter parameter that determines changes in voice frequency during speech and the shimmer parameter that determines changes in amplitude. [PERSON]'s correlation coefficients between individual characteristics and emotional states were determined. For this purpose, for each example in the dataset, a value of 1 was taken if it corresponded to the selected emotional state and a value of 0 when the emotional state differed from the selected one.  
The values of the relevant Pearson correlation coefficients are summarized in Table 8. A significant linear relationship can be observed between the emotional state ANGER and the descriptors MFCC0, MFCC2, MFCC5 and MFCC8 (absolute value of [PERSON]'s coefficient > 0.3). The SADNESS emotional state is moderately inversely correlated with the MFCC0 descriptor. In other cases, more complex, nonlinear relationships between traits and emotional states can be inferred from small Pearson coefficient values. Therefore, advanced classification tools, such as convolutional neural networks, should be used.