# How Much Does the Dynamic F0 Curve Affect the Expression of Emotion in Utterances?
## 1 Introduction
To address these challenges, more sophisticated statistical techniques are needed, particularly those capable of identifying non-linear patterns in dynamic speech data. Generalized Additive Mixed Models (GAMMs) offer such capabilities, allowing for the modeling of non-linear trajectories and interactions [21]. While several speech emotion recognition frameworks combine different feature types, the direct incorporation of dynamic F0 contours into emotion classification systems using GAMMs has not been extensively explored [8]. This study aims to classify emotional states by extracting and analyzing dynamic F0 contours using GAMMs, which have proven effective in capturing non-linear trajectory patterns in speech data [22; 23]. By leveraging GAMMs, we seek to uncover the underlying patterns of vocal expression associated with different emotional states, thereby refining emotion classification in speech and advancing our understanding of the vocal expression of emotions. To achieve this, a dataset with classified emotion types is necessary. Additionally, the results of modeling based on the F0 curve should be compared to the state of the art achieved using the same data to assess the level of explanatory power. For reasons like this, the RAVDESS dataset was chosen. The RAVDESS dataset consists of only two sentences per emotion, which imposes certain limitations on the generalizability of the findings from this study. While the dataset is valuable for controlled experimentation, the limited number of sentences may not capture the full variability present in natural speech, thereby restricting the extent to which these results can be applied to broader contexts. Nevertheless, the dataset has a few strengths: Professional actors and controlled recording conditions ensure consistent audio quality. The dataset covers a wide range of emotions, which is essential for comprehensive emotion classification. Equal numbers of male and female actors facilitate gender-related analyses. Using the same sentences across emotions and actors controls for linguistic variability, allowing us to focus on acoustic features related to emotion. Several prior studies have attempted emotion classification using the RAVDESS dataset with varying degrees of success. For instance, [PERSON]. [24] utilized Support Vector Machines (SVMs) with features selected via Continuous Wavelet Transform (CWT) and achieved an accuracy of 60.1%. Additionally, Ref. [25] focused on a subset of four emotions and obtained an accuracy of 57.14% using group multi-task feature selection. Deep learning approaches have reported higher accuracies. For example, Ref. [26] implemented a Deep Neural Network (DNN) using spectrograms and achieved a 65.9% accuracy. Also, Ref. [27] fine-tuned a pre-existing DNN and, using VGG-16, reported an accuracy of 71%. Similarly, Ref. [28] employed a one-dimensional deep CNN with a combination of acoustic features like MFCCs, mel-scaled spectrograms, chromograms, spectral contrast features, and tonnetz (e.g., tone network) representations, reaching an accuracy of 71.61%. While these deep learning models outperform the human accuracy rates of 67% reported by [29], they often function as black boxes, offering limited insight into which features drive the classification. The use of multiple acoustic features in these models combines various sound characteristics, potentially leading to improved performance. However, this complexity makes it challenging to discern the contribution of individual features to the overall classification, which can hinder the interpretability and explainability of the results. Our approach offers the following contributions: First, we utilize the full temporal dynamics of F0 contours rather than static or simplified representations, capturing intricate pitch patterns associated with specific emotions. Second, we apply GAMMs to model non-linear relationships and individual variability in emotional speech, accommodating the complex interplay between time, emotion, and speaker characteristics.