# Review of Automatic Estimation of Emotions in Speech
## 5 Task Variability and Evaluation of SER  
Applications for SER vary considerably, unlike for many other speech tasks. In speech coding, the aim is direct, i.e., efficient reduction of data sequences while also retaining quality (both intelligibility and naturalness). For enhancement of speech, the objective is reducing distortion (echo and noise) while also retaining quality. Text-to-speech also needs speech quality, while having varied voices and emotions are useful. ASR criteria are also straightforward, i.e., speech understanding.  
For SER, on the other hand, while determining which emotion to which speech corresponds seems like a simple aim, SER applications have a wide span. SER factors include: (1) the range of outputs (e.g., a fixed set of a small number of \"classic\" emotion labels), (2) the audio quality, (3) the length of speech used to test and that used to train the SER models, (4) whether emotion changes during an utterance, and (5) whether emotion is natural or simulated.  
For ASR or ASV, one typically compares automatic recognition to how well native listeners comprehend speech or recognize speakers. Nonetheless, most current SER research uses tests for closed sets using several emotion labels, which is often difficult for human listeners, especially if one uses large numbers of overlapping emotion labels, e.g., beyond a few basic emotions. One can develop algorithms using different amounts of speech for training, and may obtain better performance than humans (which is also found for some ASV tasks when using unfamiliar speakers) [65].  
As SER tasks are variable, so also are evaluation measures. SER using closed-set answers can be estimated with basic accuracy (percentage of test utterances with emotions with correct identification). Some SERs can be a binary task (i.e., does given speech correspond to a specific emotion?); in this case, equal error rate (EER) is often used: the percentage where false rejections (incorrect estimation of a sample that does not have that emotion) equal false acceptances (wrong estimation that a sample has a specified emotion).  
A common way to evaluate SER performance is the weighted-F1 score [66]. This measure combines recall and precision of a classifier into a single metric by use of their harmonic mean. Accuracy, precision, and recall are common ways to measure success in pattern recognition, and are defined below in terms of the numbers of recognition successes and failures in tests with data examples (Equations (1)-(4)).  
\[\text{Accuracy}=\text{(true positives + true negatives)}/\text{total number of examples} \tag{1}\]  
\[\text{Precision}=\text{true positives}/\text{(true positives + false positives)} \tag{2}\]  
\[\text{Recall}=\text{true positives}/\text{(true positives + false negatives)} \tag{3}\]  
\[\text{F1 score}=2\times\text{Recall }\times\text{Precision}/\text{(Recall + Precision)}. \tag{4}\]  
(The weighted-F1 score takes account of the proportion of true instances for each label).  
Unweighted accuracy (UA) is often a primary metric, but some reports use weighted accuracy (WA) [67]. WA considers label imbalance in computing accuracy, while UA does not. The concordance correlation coefficient provides an estimate of how well a predicted distribution matches the actual one [68], and is a typical measure for evaluating SER.