# Advancing Spanish Speech Emotion Recognition: A Comprehensive Benchmark of Pre-Trained Models
## 1 Introduction
Models like wav2 vec 2.0 [44], HuBERT [45], WavLM [46], and Whisper [47] employ large volumes of unlabeled data to learn speech representations. These models can be fine-tuned for specific tasks, reducing the need for extensive labeled datasets [48; 49; 50]. By making use of such models, SER has achieved state-of-the-art (SOTA) performance, particularly enhancing recognition accuracy in cross-lingual and out-of-domain scenarios [51; 52; 53]. The integration of PTMs like wav2 vec 2.0 with fine-tuning approaches, such as task-adaptive pre-training and multi-task learning, has consistently outperformed traditional methods, highlighting the potential of transfer learning to effectively capture emotional signals [54; 55; 56]. Furthermore, models like Whisper have challenged the conventional notion that automatic speech recognition models are suboptimal for SER, demonstrating that ASR models can perform exceptionally well in emotion recognition tasks [51]. Despite these achievements, SER still faces significant challenges, particularly in underrepresented languages like Spanish. Variability in emotional expression across speakers, languages, and cultures complicates model generalization and robustness [4; 20; 33]. The scarcity of large, diverse, and emotionally annotated speech corpora hinders the training of deep learning models, often resulting in overfitting and poor performance in real-world scenarios [20; 51; 57; 58]. Cultural and linguistic differences further require the development of language-specific resources, as features like native language, accent, and pronunciation, significantly influence model performance, complicating cross-lingual generalization [59]. Additionally, capturing the subtle nuances of emotions and addressing the ambiguity and subjectivity inherent in emotional expression remain complex tasks [60]. The lack of consensus on optimal acoustic features and the significant differences in performance between acted and natural emotional speech databases complicate the development of universal SER systems [21; 61]. Research on the performance and generalization of SER models and methodologies that use databases underrepresented in the current SOTA is therefore necessary. While recent advances have been made in Spanish SER, several methodological gaps remain. Many systems still rely on handcrafted features such as MFCCs and spectral descriptors, limiting their ability to generalize across speakers and contexts [62; 63; 64; 65; 66]. Others integrate deep or multimodal models but conduct limited evaluations, often omitting speaker-independent protocols or cross-corpus validation [67; 68]. Furthermore, while some proposals incorporate pretrained models, they do not fully exploit their representational capacity, overlooking layer-wise analysis and targeted fine-tuning [67; 69; 70; 71]. These limitations highlight the need for broader benchmarking across Spanish emotional speech corpora and a deeper investigation of PTMs to build more robust and transferable SER systems. This work focuses on six databases: EmoMatchSpanishDB [64], Mexican Emotional Speech Database [72], Spanish MEACorpus [68], EmoWisconsin [66], INTERISP [73], and EmoFilm [74]. These databases were selected for their emphasis on categorical annotations spanning the full range of emotional representation elicitation, including acted, induced, and natural expressions collected from various demographics such as adults and children, professional actors, and non-actors. The diversity in data sources allows for a thorough benchmarking of model generalizability across varied real-world scenarios. A wide range of advanced PTMs in the field of audio processing and SER were chosen, such as Wav2 Vec 2.0 [44], Whisper [47], HuBERT [45], WavLM [46], TRILLsson [75], and CLAP [76]. These models include self-supervised architectures, sequence-to-sequence frameworks, and multimodal contrastive learning approaches.