# Review of Automatic Estimation of Emotions in Speech
## 3 Acoustic Analysis of Speech
### Basic Digital Signal Processing for SER  
Like most physical data of interest (e.g., diverse signals such as images, temperature, odors), speech is always changing. For use in digital computers, signals are periodically sampled at a high rate (exceeding the Nyquist rate--double the highest signal frequency), to retain information for a pertinent frequency range in each application (e.g., in speech by telephone, up to 3.2 kHz [36]). Then, processes, including neural networks, convert the data sample sequence to representations that are more relevant to perform classification, e.g., into estimated text or classes of emotion.  
Despite the large set of objectives among different applications for speech, very similar analysis methods are regularly used to convert high-dimensional speech data for classification of much lower dimensions, for many speech applications: ASR, automatic speaker verification (ASV), language identification, and SER. It is generally assumed that most useful information in speech comes from intentional speaker behavior in VT motion. However, this assumption may not work for medical diagnosis using speech, for physiological aspects of ASV, or for emotions. Nonetheless, such general processing is often applied in all speech domains, sometimes for a lack of suitable alternatives. A major aim for speech processing has been to derive a compact representation with few dimensions that highlights relevant features. However, defining such features is complex (which has led instead to general machine learning methods; see Section 7.3). To accommodate the many variations across speakers and acoustical environments, different adaptation approaches have also been used [39].  
For SER, intentions of the speaker are not a primary objective, as speakers focus on articulating a textual message (the lexical content of the speech), and emotional aspects are generally secondary, if at all intentional, for speakers. When speaking with emotion, a person is generally aware of emotions (and may indeed alter VT articulation to further emphasize emotional aspects, with intent), but is primarily focused on conveying the textual content. Thus, SER need not focus on acoustic aspects that mainly deal with VT shape (despite the fact that many SER systems indeed do). Instead, much research has shown that intonation (which has little to do with VT shape) is likely the major acoustical factor for SER [40].  
Some modern SER systems (often termed \"end-to-end\" or E2E) avoid any direct intonational or spectral analysis of speech, assuming that all specific processing not directly for the task objective risks discarding relevant information. Indeed, recent SER has focused heavily on methods that tend to derive general speech features helpful for understanding and coding the phonetic information in speech, rather than for emotions. Nonetheless, as most SER does some form of analysis, we examine acoustic processing in this section.  
Common spectral features used for SER include: MFCCs, linear prediction cepstral coefficients, formant frequencies (VT resonances), modulation spectral features, and bandwidths of formant frequencies [41]. Voice quality features for SER include shimmer, jitter, and NAQ, as they are related to characteristics of glottal excitation [42; 43; 44]. These are detailed below.