## 2 Literature Review and Hypotheses Development
### Combination of Speech and Physiological Measures
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Study** & **Contribution** & **Contribution** \\ \hline [PERSON] et al. (2019) & Improved the recognition of human arousal level & EDA \\  & during the pronunciation of single affective words. & Speech Features (F0 and MFCC) \\ \hline [PERSON] et al. (2020) & Developed a speech activity detection system which & EDA \\  & can perform in noisy environment and compensate & Speech Features (MFCC) \\  & for the presence of emotional conditions. & \\ \hline [PERSON] et al. (2006) & Proposed a framework to model affective states in & Speech Features (prosody related to pitch \\  & naturalistic video sequences. & and rhythm) \\  & & Bodily Expression (excluded in the fusion \\ \hline [PERSON] et al. (2008) & Presented framework of multimodal automatic & Facial Expression \\  & emotion recognition system during a & Speech Features (MFCC) \\  & speech-based interaction. & Bodily Expression \\ \hline [PERSON] et al. (2019) & Proposed a framework consisting of mobile phone & Facial Expression \\  & technology backed by cloud computing to recognize & Speech Features (MFCC) \\  & emotion in speech and facial expression in real-time. & \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Summary of the multi-method studies utilizing speech and physiological measures in relation to emotion recognition. **Hypothesis 2b (H2b). **_There is a relationship between the amplitude of the extracted AFE-based valence feature and the emotional intensity of users during voice user interface interactions._  
Similarly to physiology, speech features are linked to the dimensions of valence and arousal. Indeed, the emotional arousal of a speaker is accompanied by physiological changes, consequently affecting respiration, phonation, and articulation, resulting in emotion-specific patterns of acoustic parameters [63]. As suggested by [PERSON] [63], F0, energy, and rate are considered the most indicative of arousal. More precisely, high arousal is associated with high mean F0, F0 variability, fast speech rate, short pauses, increased voice intensity, and increased high frequency energy [64; 65; 66; 67; 68; 69; 70; 71; 72]. Indeed, emotions associated with high levels of physiological arousal, such as anger, fear, joy, and anxiety, have depicted increases in mean F0 and F0 variability, in addition to vocal intensity [30]. For example, put into context, it is not uncommon for one to speak with a loud voice when feeling gleeful. In contrast, emotions associated with low arousal levels, such as sadness, tend to have lower mean F0, F0 variability, and vocal intensity [30]. With this said, vocal aspects can covary with emotional attributes, which reflect and communicate arousal levels associated to emotional reactions [63]. Across studies, results regarding arousal and speech remain consistent [73]. On the contrary, results regarding the relationship of speech and valence are noticeably inconsistent. In some studies, positive valence has been linked to low mean F0, fast speech rate, F0 variability, and little high-frequency energy [68; 69; 72; 74; 75]. In others, valence is not associated to specific patterns of vocal cues [65; 67; 70]. Moreover, research has suggested that valence values are better assessed using facial features in comparison to acoustic features [76; 77]. In other words, the relationship between speech and valence appears to be weaker in comparison to the physiological measure of facial expression. Considering the inconsistencies and suggested weakness of the relationship between speech features and valence, in addition to the predictive capabilities of physiological measures in relation to both valence and arousal dimensions, we hypothesize the following. **Hypothesis 3 (H3). **_Physiological features are more explicative of emotional voice interaction events in comparison to speech features._