# How Much Does the Dynamic F0 Curve Affect the Expression of Emotion in Utterances?
## 2 Materials and Methods
### Gamm  
To capture the complex, non-linear relationships inherent in emotional speech data, we employed Generalized Additive Mixed Models (GAMMs). A Generalized Additive Mixed Model (GAMM) is an extension of the generalized linear mixed model (GLMM) that incorporates non-linear relationships in the data through smooth functions. GAMMs are adept at revealing complex patterns that are not apparent under linear analysis, making them particularly powerful for nuanced analyses where relationships between variables are  
Figure 1: **Left panel**: Sample of F0 values in semitones extracted from the audio file “03-01-08-01-01-03.wav.” **Right panel**: Resampled pitch contours using the resampy function. The pitch values in semitone, which are randomly generated for an illustrative purpose, are represented by the solid line, and the resampled pitch values are represented by the dotted line. The \(X\)-axis on both panels represents frames.  
not strictly linear or change across different levels of another variable. The basic formula for a Generalized Additive Mixed Model (GAMM) can be expressed as follows:  
\[y=\beta 0+\sum_{j}s_{j}\big{(}x_{j}\big{)}+\sum_{k}b_{k}z_{k}+\epsilon\]  
in which \(\beta_{0}\) is the intercept term, and the term \(\sum_{j}s_{j}\big{(}x_{j}\big{)}\) represents the sum of smooth functions \(f_{j}\) applied to predictor variable \(x_{j}\). The term \(\sum\limits_{k}b_{k}z_{k}\) is for the sum of random effects \(b_{k}\) for grouping variables \(z_{k}\), capturing variability in \(y\) due to random factors. \(\epsilon\) indicates the error term. In summary, the GAMM combined fixed effects and random effects to model complex data with both non-linear patterns and hierarchical measures [19; 20].  
In our study, we embraced the flexibility of GAMMs to interpret the intricate dynamics within our emotional speech dataset. We constructed our GAMMs using the mgcv package in R (version 1.8-36) [21], a robust environment for statistical computing and graphics. The mgcv package offers comprehensive tools for building and evaluating GAMMs, facilitating the modeling of complex data structures and relationships. It enables the modeling of complex, non-linear, and non-fixed relationships between predictors and the response variable: in our case, the various speech features influencing emotional expression. The visualization and interpretation of the models were enhanced using the itssudap package (version 2.4) in R [35], which provides utilities for plotting smooths, assessing model diagnostics, and interpreting interaction effects. This combination allowed us to effectively communicate the subtleties and strengths of our models through graphical representations.  
The use of GAMMs in phonetic research is well established, with studies demonstrating their efficacy in analyzing dynamic speech patterns [36; 37; 38; 39; 40]: Ref. [36] applied GAMMs to investigate phonetic variations and changes in Scottish English, capturing complex interactions between social factors and speech acoustics. Ref. [37] utilized GAMMs to analyze articulatory trajectories in tongue movement data, revealing non-linear patterns associated with language proficiency. Ref. [38] employed GAMMs to model the time-varying nature of electrophysiological responses in psycholinguistic experiments, showcasing the models' capacity to handle time-series data.  
These precedents highlight the suitability of GAMMs for our study, which involves modeling the non-linear F0 contours associated with different emotional expressions over time. GAMMs have also been instrumental in investigating the temporal dynamics of speech and the interplay between articulatory movements and acoustic outcomes [22; 23; 32], further evidencing their utility in phonetics.