GenAI4ELab/papercli-papers huggingface dataset:
Evaluating Progress of CALL System Users on Accentedness and Comprehensibility: An Acoustic and ASR-Based Approach
Authors: Wenwei Dong; Catia Cucchiarini; Roeland van Hout; Helmer Strik
Venue: Interspeech 2025 (main)

Abstract: Accentedness and comprehensibility scales are widely used to evaluate the effectiveness of Computer-Assisted Language Learning (CALL) tools. Such evaluations mainly rely on subjective expert assessment to measure accentedness and comprehensibility. In this study, we applied Automatic Speech Recognition (ASR) measures and used acoustic features to investigate the importance of objective features and measures that may underlie accentedness and comprehensibility. Furthermore, we combined the practice length of CALL tool users with the pre- and post-test data to gain insight into how practice affects progress. The experimental results showed that ASR measures, loudness, and formant-related features were most relevant for accentedness and comprehensibility. Additionally, practice time was positively correlated with progress in accentedness and comprehensibility. We also found that longer practice time contributes more to the changes in the acoustic features selected by the Lasso regression.
Does English fish sound like French fiche? Perceptual similarity judgments versus acoustic similarity
Authors: Rory Turnbull; Elisa Kiefer; Sharon Peperkamp
Venue: Interspeech 2025 (main)

Abstract: We examine the relationship between phonological word similarity judgments from listeners and acoustic measures of similarity. Native speakers of English or French with varying degrees of proficiency in the other language listened to pairs of words, one in French and the other in English. The words were highly phonologically similar but did not overlap in meaning. Participants judged how similar the items sounded. Each item pair was acoustically analyzed with six different acoustic distance metrics. The results demonstrate a weak correlation between the similarity judgments and five of the six acoustic distance measures. Within the experiment, we found an order of presentation effect: when the first word of a pair was in a participant's L1, the pair was rated more similar than if the first word was in the L2. This effect diminished in magnitude with increasing L2 proficiency. We discuss the implications of our results in light of theories of speech representation and processing.
Acoustic Features of Mandarin Tone Production in Noise: A Comparison Between Chinese Native Speakers and Korean L2 Learners
Authors: Jinxin Ji; Yiying Hu; Xiaohu Yang; Gang Peng
Venue: Interspeech 2025 (main)

Abstract: Research on Lombard speech produced by Chinese learners remains underdeveloped. This study investigates how native Chinese Mandarin speakers and Korean second language (L2) learners adjust the production of lexical tones under noise conditions. Fifty-two participants (26 Chinese native speakers and 26 higher-level Korean L2 learners) produced 28 syllables under three conditions: babble noise 80 dB SPL, white noise 80 dB SPL, and quiet. Acoustic features, including intensity, duration and F0 contours were analysed. Results indicated that Korean L2 learners significantly increased both intensity and duration across all four tones in the two noise conditions, whereas Chinese native speakers significantly increased durations only for Tone 3 and Tone 4 under white noise. Both groups adjusted F0 contours for all tones in noise, with a notable group difference in the F0 contour of Tone 3. These findings were discussed in association with feedback control theory in second language learners, showing that L2 learners rely more heavily on feedback control and are more detrimentally affected by noise conditions than native speakers.
The Role of Contextual Variation in Learning Cantonese Tones from Naturalistic Speech
Authors: Fengyue Lisa Zhao; Jennifer Kuo
Venue: Interspeech 2025 (main)

Abstract: A central question in speech acquisition is how infants learn contrastive sound categories from variable acoustic cues. The Distributional Learning Across Contexts (DLAC) proposal suggests that infants may identify contrastive sound dimensions by tracking distributions across contexts, with greater contextual variation signaling contrastive categories. Most research on acquisition of sound categories has focused on simple two-way vowel and consonant contrasts, leaving open how infants learn more complex or tonal contrasts. This study addresses these gaps by analyzing naturalistic speech to examine whether the ease of learning Cantoneseâs six tonal contrasts is linked to their variation across different contexts. By comparing our findings with existing acquisition data, we show that this proposal could predict which contrasts are easier or harder to learn, suggesting that in absence of invariant acoustic cues, infants may rely on contextual variation to acquire complex sound categories.
Pitch Target Realization in Putonghua Tone Production of Children from Dialect-Speaking Regions
Authors: Mengxue Cao; Tianxin Zheng; Jiewen Zheng
Venue: Interspeech 2025 (main)

Abstract: Tonal production can be understood as a target realization process. In a hybrid linguistic environment, tonal targets may interfere with one another. This study examines the production of Putonghua tones from both on- and off-target perspectives, using data from 139 children (aged 35â71 months) with Changli dialect exposure, drawn from the CL-CHILD corpus. Results reveal that: (1) children's pitch target realization is universally constrained by physiological limitations, exhibiting cross-linguistic commonalities; (2) dialect exposure causes persistent pitch target interference over an extended period; (3) off-target pronunciations stem from phonetic similarities across tonal categories and emerge through mutual interference between multiple targets. These findings underscore that children's tonal development is a dynamic process of target approximation, shaped by both physiological constraints and language experience.
The Development of Speech Rhythm in Putonghua-Learning Preschool Children in South Xinjiang Uyghur Autonomous Region of China
Authors: Aijun LI; Zhiwei Wang; Jun Gao; Xin Zhou
Venue: Interspeech 2025 (main)

Abstract: This study investigates Putonghua rhythm development among Uyghur preschoolers in South Xinjiang, China. We examined 40 children (ages 5-7) grouped by Putonghua learning time: mid-class (12 months, mean age 6;1) and senior-class (24 months, mean age 7;1). Speech samples were elicited through picture-guessing tasks and analyzed using rhythmic metrics. Our analyses reveal distinct developmental patterns compared to Putonghua-speaking children, with neither group achieving metrics comparable to age-matched Putonghua-speaking 6-year-olds. Consonant-related metrics emerged as increasingly salient markers in later developmental stages. The findings demonstrate that rhythm acquisition in this bilingual context is modulated by cross-linguistic influence from Uyghur. These results advance our understanding of L2 rhythm acquisition in bilingual settings and have implications for developing automated speech assessment tools and optimizing speech recognition systems for multilingual contexts.
PARROT: Synergizing Mamba and Attention-based SSL Pre-Trained Models via Parallel Branch Hadamard Optimal Transport for Speech Emotion Recognition
Authors: Orchid Chetia Phukan; Mohd Mujtaba Akhtar; Girish; Swarup Ranjan Behera; Jaya Sai Kiran Patibandla; Arun Balaji Buduru; Rajesh Sharma
Venue: Interspeech 2025 (main)

Abstract: The emergence of Mamba as an alternative to attention-based architectures has led to the development of Mamba-based self-supervised learning (SSL) pre-trained models (PTMs) for speech and audio processing. Recent studies suggest that these models achieve comparable or superior performance to state-of-the-art (SOTA) attention-based PTMs for speech emotion recognition (SER). Motivated by prior work demonstrating the benefits of PTM fusion across different speech processing tasks, we hypothesize that leveraging the complementary strengths of Mamba-based and attention-based PTMs will enhance SER performance beyond the fusion of homogenous attention-based PTMs. To this end, we introduce a novel framework, PARROT that integrates parallel branch fusion with Optimal Transport and Hadamard Product. Our approach achieves SOTA results against individual PTMs, homogeneous PTMs fusion, and baseline fusion techniques, thus, highlighting the potential of heterogeneous PTM fusion for SER.
Towards Machine Unlearning for Paralinguistic Speech Processing
Authors: Orchid Chetia Phukan; Girish; Mohd Mujtaba Akhtar; Shubham Singh; Swarup Ranjan Behera; Vandana Rajan; Muskaan Singh; Arun Balaji Buduru; Rajesh Sharma
Venue: Interspeech 2025 (main)

Abstract: In this work, we pioneer the study of Machine Unlearning (MU) for Paralinguistic Speech Processing (PSP). We focus on two key PSP tasks: Speech Emotion Recognition (SER) and Depression Detection (DD). To this end, we propose, SISA++, a novel extension to previous state-of-the-art (SOTA) MU method, SISA by merging models trained on different shards with weight-averaging. With such modifications, we show that SISA++ preserves performance more in comparison to SISA after unlearning in benchmark SER (CREMA-D) and DD (E-DAIC) datasets. Also, to guide future research for easier adoption of MU for PSP, we present âcookbook recipesâ - actionable recommendations for selecting optimal feature representations and downstream architectures that can mitigate performance degradation after the unlearning process.
Infant Cry Emotion Recognition Using Improved ECAPA-TDNN with Multi-scale Feature Fusion and Attention Enhancement
Authors: Junyu Zhou; Yanxiong Li; Haolin Yu
Venue: Interspeech 2025 (main)

Abstract: Infant cry emotion recognition is crucial for parenting and medical applications. It faces many challenges, such as subtle emotional variations, noise interference, and limited data. The existing methods lack the ability to effectively integrate multi-scale features and temporal-frequency relationships. In this study, we propose a method for infant cry emotion recognition using an improved Emphasized Channel Attention, Propagation and Aggregation in Time Delay Neural Network (ECAPA-TDNN) with both multi-scale feature fusion and attention enhancement. Experiments on a public dataset show that the proposed method achieves accuracy of 82.20%, number of parameters of 1.43 MB and FLOPs of 0.32 Giga. Moreover, our method has advantage over the baseline methods in terms of accuracy. The code is at https://github.com/kkpretend/IETMA.
Speech Mutil-label Emotion Recognition Using Asymmetric Class Loss Function Based on Effective Samples
Authors: Shanshan Xiang; Hankiz Yilahun; Askar Hamdulla
Venue: Interspeech 2025 (main)

Abstract: Although significant progress has been made in Speech Emotion Recognition (SER), most current research focuses on single-label emotion classification, while multi-label SER faces low classification accuracy due to the long-tail data distribution (e.g., many happiness samples and few fear samples). To address this, we propose an Asymmetric Category Loss Function based on Effective Sample Selection (ER-ASCL) to alleviate the data imbalance. This method adjusts the asymmetric loss by calculating class weights from effective samples and introduces a label-guided matrix to enhance the relationships between emotion categories, improving classification accuracy. Experimental results show that on the CMU-MOSEI multimodal dataset, using the audio modality, the model achieved 53.5% accuracy, a 2.1% improvement over the best existing method. On the Chinese multi-label emotion dataset CNSCED, the model achieved 69.23% accuracy, outperforming existing multi-label classification loss functions.
EmoDB 2.0: A Database of Emotional Speech in a World that is not Black or White but Grey
Authors: Felix Burkhardt; Oliver SchrÃ¼fer; Uwe Reichel; Hagen Wierstorf; Anna Derington; Florian Eyben; BjÃ¶rn W. Schuller
Venue: Interspeech 2025 (main)

Abstract: In 2004, the Berlin Database of Emotional Speech (Berlin EmoDB) has made publicly available, and since then, it has been utilized in numerous studies on emotional speech, with over 3,000 citations. We now extend this database with original material that was previously absent from the distributed versions within the research community. This includes ambiguous samples that were not agreed upon by at least 80 % of the raters, the addition of glottograms for all samples, and a new rater label indicating the perceived naturalness of the samples. The paper provides detailed descriptions and reports on preliminary studies conducted using this extended data. For instance, incorporating glottogram information has been shown to improve the average recall of an SVM classifier by 8.1 % UAR.
Cross-corpus open-set Speech Emotion Recognition Method Based on Spatiotemporal Features with Inverse-Entropy Regularization
Authors: ZhaoHui Zhou; Hui Luo
Venue: Interspeech 2025 (main)

Abstract: We propose a method to address the performance degradation due to the distribution shift and unknown emotion categories in cross-corpus open-set speech emotion recognition. The method combines spatiotemporal feature extraction and inverse-entropy regularization. First, the long-range spatiotemporal dependencies are extracted from emotional audio sequences using a deep fusion network. To further align distributions from the source and target corpora, the MMD regularization is applied to minimize the distance between their joint distributions. Moreover, we propose an inverse-entropy regularization to learn the discriminative information used to reject known classes, which can balance the classification confidence of samples from the known or unknown categories in the open-set setting, allowing the model to predict unknown classes while preventing over-prediction. Experimental results show that our method outperforms baseline models across four cross-corpus speech emotion datasets.
CLEP-DG: Contrastive Learning for Speech Emotion Domain Generalization via Soft Prompt Tuning
Authors: Jiacheng Shi; Yanfu Zhang; Ye Gao
Venue: Interspeech 2025 (main)

Abstract: Speech Emotion Recognition (SER) is fundamental to affective computing and human-computer interaction, yet existing models struggle to generalize across diverse acoustic conditions. While Contrastive Language-Audio Pretraining (CLAP) provides strong multimodal alignment, it lacks dedicated mechanisms for capturing emotional cues, making it suboptimal for SER. To address this, we propose CLEP-DG, a framework that enhances CLAPâs robustness in emotion recognition. First, we fine-tune CLAP to obtain CLEP, adapting it on large-scale emotional speech datasets to better encode emotion-relevant features. Then, we introduce Acoustic Context Prompt Tuning (ACPT), a text-driven augmentation strategy that optimizes learnable prompt vectors to model diverse acoustic environments without additional labeled audio. Finally, leveraging cross-modal transferability, we train a classifier on text-derived embeddings and apply it to the audio encoder during inference, mitigating domain shifts between textual supervision and audio-based emotion recognition. Experiments across five benchmark datasets show that CLEP-DG outperforms prior CLAP-based approaches, achieving state-of-the-art performance in both supervised and domain generalization settings.
Leveraging Unlabeled Audio-Visual Data in Speech Emotion Recognition using Knowledge Distillation
Authors: Varsha Pendyala; Pedro Morgado; William Sethares
Venue: Interspeech 2025 (main)

Abstract: Voice interfaces integral to the human-computer interaction systems can benefit from speech emotion recognition (SER) to customize responses based on user emotions. Since humans convey emotions through multi-modal audio-visual cues, developing SER systems using both the modalities is beneficial. However, collecting a vast amount of labeled data for their development is expensive. This paper proposes a knowledge distillation framework called LightweightSER (LiSER) that leverages unlabeled audio-visual data for SER, using large teacher models built on advanced speech and face representation models. LiSER transfers knowledge regarding speech emotions and facial expressions from the teacher models to lightweight student models. Experiments conducted on two benchmark datasets, RAVDESS and CREMA-D, demonstrate that LiSER can reduce the dependence on extensive labeled datasets for SER tasks.
Who knows best? Effects of speech disfluencies on incentivized decision-making
Authors: Ambika Kirkland; Jens Edlund
Venue: Interspeech 2025 (main)

Abstract: Previous work has shown that speech disfluencies can negatively impact judgments about a speaker's competence and confidence. However, these effects have primarily been examined with Likert-type rating scales, which are not informative about how judgments might translate to behavior. Does the presence of disfluencies actually guide decision-making when listeners stand to gain concretely from making the correct choice? We sought to address this question with a web-based decision task in which participants were asked to choose between two conflicting sources of information. Our results suggest that listeners do take speech fluency into account when deciding who or what to believe.
Enhancing Transcripts of Open-Source Automatic Speech Recognition Models Through Fine-Tuning with Laughter and Speech-Laugh
Authors: Phuoc Hoang Ho; DragoÈ Alexandru BÄlan; Dirk K. J. Heylen; Khiet P. Truong
Venue: Interspeech 2025 (main)

Abstract: Non-lexical sounds such as laughter are considered important discourse markers in conversation as these sounds complement the lexical information in shaping interpersonal relations, managing the conversation, and in expressing attitudes and affect. However, general purpose open-source automatic speech recognition (ASR) systems are typically not focusing on these sounds. Laughter transcription is an understudied task in ASR: laughter is often not modelled as a token and is discarded in the evaluation of ASR. In our study, we investigate how current open-source ASR models are handling laughter and speech-laugh (speech interspersed with laughter). Using Switchboard and BuckEye as conversational speech corpora, we fine-tuned and evaluated Whisper and Wav2Vec2. Our results show that laughter can be integrated in ASR transcriptions without substantially degrading word error rate.
Investigating the Reasoning Abilities of Large Language Models for Understanding Spoken Language in Interpersonal Interactions
Authors: Pranjal Aggarwal; Ghritachi Mahajani; Pavan Kumar Malasani; Vaibhav Jamadagni; Caroline J. Wendt; Ehsanul Haque Nirjhar; Theodora Chaspari
Venue: Interspeech 2025 (main)

Abstract: This study evaluates large language modelsâ (LLMs) reasoning capabilities in spoken language understanding (SLU) during interpersonal interactions. We incorporated several factors into LLM prompts: instructing via examples (IE), integrating domain knowledge (DK), and including context (IC). Experiments with Gemini-1.5-pro, GPT-3.5-turbo, and GPT-4o were conducted on an SLU task that classifies the degree of explanation (i.e., under-explained, succinct, comprehensive, over-explained) in job interview responsesâan important step toward developing automatic interview training systems. Results demonstrate the feasibility of few-shot (1- to 4-shot) learning, with ablation studies confirming that modifications to prompts, especially when combining IE, DK, and IC, lead to further performance improvements.
A Naturally Elicited Multimodal Stress Database and Speech Breathing Based Stress Detection
Authors: Karumannil Mohamed Ismail Yasar Arafath; Mohammed Abeer K. C.; Aurobinda Routray
Venue: Interspeech 2025 (main)

Abstract: Stress is a natural physiological and psychological response to demanding situations, but accurate detection remains challenging in real-world scenarios. Existing stress databases are limited and often recorded under extreme conditions or acted, which makes real-world applications difficult. This study introduces a multi-modal stress database, naturally elicited through student evaluations, incorporating normal and thermal video alongside speech recordings annotated by psychologists. While physiological signals provide robust stress detection, speech and facial expressions are more accessible. We extract physiological parameters from speech to bridge this gap by automatically detecting breath locations. These breath indices are then used to derive speech breathing parameters. Machine learning techniques classify these parameters for stress detection. Our proposed method marginally outperforms conventional speech-based features, demonstrating a promising approach to stress analysis.
From Context to Code-switching: Examining the Interplay of Language Proficiency and Multilingualism in Speech
Authors: Debasmita Bhattacharya; Aanya Tolat; Julia Hirschberg
Venue: Interspeech 2025 (main)

Abstract: Multilingual speakers are known to code-switch across language pairs and in association with various paralinguistic aspects of conversation. We build on prior work by asking: how is a multilingual speaker's language proficiency, as shaped by their linguistic background, related to their code-switching (CSW) behavior? To answer this question, we examine the Bangor Miami corpus of spontaneous Spanish-English speech and analyze the linguistic and demographic profiles of its speakers alongside features of their conversational language production. We find that a speaker's parents' primary language, medium of secondary schooling, and self-reported ability are strongly associated with the quantity of CSW and language distribution in code-switched speech. Our work is the first to empirically show a relationship between language proficiency and multilingual speech production and calls for the inclusion of linguistic background features in future paralinguistic studies of CSW.
Extending the Fongbe to French Speech Translation Corpus: resources, models and benchmark
Authors: D. FortunÃ© Kponou; Salima Mdhaffar; FrÃ©jus A. A. Laleye; EugÃ¨ne C. Ezin; Yannick EstÃ¨ve
Venue: Interspeech 2025 (main)

Abstract: This paper introduces FFSTC 2, an expanded version of the existing Fongbe-to-French speech translation corpus, addressing the critical need for resources in African dialects for speech recognition and translation tasks. We extended the dataset by adding 36 hours of transcribed audio, bringing the total to 61 hours, thereby enhancing its utility for both automatic speech recognition (ASR) and speech translation (ST) in Fongbe, a low-resource language. Using corpus, we develop both cascade and end-to-end speech translation systems. Our models employ AfriHuBERT and HuBERT147 as encoders, and NLLB and mBART as decoders. We introduce a diacritic-substitution technique for both ASR and Machine Translation (MT) which, yields a BLEU score of 37.23 compared to 39.60 for the fully diacritized configuration. Among the evaluated end-to-end architectures, AfriHuBERT-NLLB with data augmentation attains the highest BLEU score of 26.32.