GenAI4ELab/papercli-papers huggingface dataset:
A Study on The Impact of Foundation Models on Automatic Depression Detection from Speech Signals
Authors: Bubai Maji; Monorama Swain; Shazia Nasreen; Debabrata Majumdar; Rajlakshmi Guha; Aurobinda Routray; Anders SÃ¸gaard
Venue: Interspeech 2025 (main)

Abstract: An automatic depression detection (ADD) system using spoken language offers the opportunity to develop practical, low-cost tools to detect symptoms early. However, limited data availability, privacy concerns, and transcription efforts pose significant challenges. Recent advancements in foundational models, capable of understanding and processing multimodal inputs, present opportunities for enhancing ADD systems. This study explores various speech foundation models to investigate their impact on ADD. We leverage Whisper and MMS for automatic transcription and integrate speech and text embeddings into a language model optimized with low-rank adaptation (LoRA). In addition, we examine the effects of fine-tuning strategies and prompt formats on model performance. We used English and Bengali datasets to demonstrate the potential of our method in ADD, even with moderate-quality transcriptions. The best speech and language foundation models outperform baseline models on both datasets.
Identifying Vocal and Facial Biomarkers of Depression in Large-Scale Remote Recordings: A Multimodal Study Using Mixed-Effects Modeling
Authors: Nelson Hidalgo Julia; Robert Lewis; Craig Ferguson; Simon Goldberg; Wendy Lau; Caroline Swords; Gabriela Valdivia; Christine Wilson-Mendenhall; Raquel Tartar; Rosalind Picard; Richard Davidson
Venue: Interspeech 2025 (main)

Abstract: We examine vocal and facial data from a new study with n=954 depressed participants, each characterized by six time points of the eight-item Patient Health Questionnaire survey (PHQ-8). Patients interacted with a smartphone app over four weeks, with a 3-month follow-up. The app's animated character asked participants to describe, for 90 seconds, an emotional experience from the past 24 hours. We obtained 4,875 audio-video recordings, and applied linear mixed-effects models to examine associations between depression severity and 30 acoustic, linguistic and facial action unit features. Significant associations were found with speech timing and prosody, voice quality, linguistic sentiment, the use of self-referential pronouns, and facial action units related to smiling. We also show that these features allow accurate estimation of depression severity in multimodal mixed-effects machine learning models.
M3L: A Multi-Modal and Multi-Lingual Depression Detection Framework
Authors: Jiajun You; Shuai Wang; Xun Gong; Xiang Wan
Venue: Interspeech 2025 (main)

Abstract: Early diagnosis are essential to reduce costs and improve treatment efficiency. Recently, automatic depression detection (ADD) based on audio and textual features from participant interviews has emerged as a promising approach, attracting significant attention. However, existing models are constrained to monolingual depression datasets, with limited exploration of multi-lingual scenarios. To investigate the effectiveness of multi-lingual data for the ADD task and its transferability in low-resource scenarios, in this paper, we propose a Multi-Modal Multi-Lingual (M3L) depression detection framework and an effective language adaptive fine-tuning (LAFT) to further boost the performance on the target language. M3L utilizes the pretrained speech model Whisper and the text model XLM-RoBERTa to enhance the encoding of multilingual information. Evaluations on the DAIC-WOZ (English) and EATD (Chinese) datasets demonstrate that M3L effectively integrates multi-lingual and multi-modal information, while the proposed LAFT consistently boosts performance across both datasets.
Using and comprehending language in face-to-face conversation
Authors: Judith Holler
Venue: Interspeech 2025 (main)

Abstract: AbstractFace-to-face conversational interaction is at the very heart of human sociality and the natural ecological niche in which language has evolved and is acquired. Yet, we still know rather little about how utterances are produced and comprehended in this environment. In this talk, I will focus on how hand gestures, facial and head movements are organised to convey semantic and pragmatic meaning in conversation, as well as on how the presence and timing of these signals impacts utterance comprehension and responding. Specifically, I will present studies based on complementary approaches, which feed into and inform one another. This includes qualitative and quantitative multimodal corpus studies showing that visual signals indeed often occur early, and experimental comprehension studies, which are based on and inspired by the corpus results, implementing controlled manipulations to test for causal effects between visual bodily signals and comprehension processes and mechanisms. These experiments include behavioural and EEG studies, most of them using multimodally animated virtual characters. Together, the findings provide evidence for the hypothesis that visual bodily signals form an integral part of semantic and pragmatic meaning communication in conversational interaction, and that they facilitate language processing, especially due to their timing and the predictive potential they gain through their temporal orchestration.BiographyJudith Holler is Associate Professor at the Donders Institute for Brain, Cognition & Behaviour, Radboud University where she leads the research group Communication in Social Interaction, and senior investigator at the Max Planck Institute for Psycholinguistics. Her research program investigates human language in the very environment in which it has evolved, is acquired, and used most: face-to-face interaction. Within this context, Judith focuses on the semantics and pragmatics of human communication from a multimodal perspective considering spoken language within the rich, visual infrastructure that embeds it, such as manual gestures, head movements, facial signals, and gaze. She uses a combination of methods from different fields to investigate human multimodal communication, including quantitative conversational corpus analyses, in-situ eyetracking, behavioural and neurocognitive experimentation using multimodal language stimuli involving virtual animations. Her research has been supported by a range of prestigious research grants from funders including the European Research Council (EU), The Dutch Science Foundation (NWO), Marie Curie Fellowships (EU), Economic & Social Research Council (UK), Parkinson UK, The Leverhulme Trust (UK), the British Academy (UK), Volkswagen Stiftung (Germany) and the German Science Foundation (DFG, Mercator Fellowships).
On the Relevance of Clinical Assessment Tasks for the Automatic Detection of Parkinsonâs Disease Medication State from Speech
Authors: David Gimeno-GÃ³mez; RubÃ©n Solera-UreÃ±a; Anna Pompili; Carlos-D. MartÃ­nez-Hinarejos; Rita Cardoso; Isabel GuimarÃ£es; Joaquim J. Ferreira; Alberto Abad
Venue: Interspeech 2025 (main)

Abstract: The automatic identification of medication states of Parkinson's disease (PD) patients can assist clinicians in monitoring and scheduling personalized treatments, as well as studying the effects of medication in alleviating the motor symptoms that characterize the disease. This paper explores speech as a non-invasive and accessible biomarker for identifying PD medication states, introducing a novel approach that addresses this task from a speaker-independent perspective. While traditional machine learning models achieve competitive results, self-supervised speech representations prove essential for optimal performance, significantly surpassing knowledge-based acoustic descriptors. Experiments across diverse speech assessment tasks highlight the relevance of prosody and continuous speech in distinguishing medication states, reaching an F1-score of 88.2%. These findings may streamline clinicians' work and reduce patient effort in voice recordings.
Speech power spectra: a window into neural oscillations in Parkinson's disease
Authors: Sevada Hovsepyan; Mathew Magimai Doss
Venue: Interspeech 2025 (main)

Abstract: This study analyzes the power spectral density (PSD) of speech in healthy controls (HC) and Parkinson's disease (PD) patients, focusing on the 0-100 Hz range. These low frequency components are below the fundamental frequency and may reflect both motor and neural mechanisms in speech production. We hypothesize that neural oscillations (NOs) involved in speech perception and production - theta (4-8 Hz), beta (15-35 Hz), and gamma (36-80 Hz) - shape the low-frequency PSD. Since NOs are linked to motor control and cognition, and are altered in PD, we expect systematic differences between HC and PD speech. Using multitaper estimation, we found significant differences in beta power, in line with research on beta oscillations and motor dysfunction in PD. Beyond distinguishing HC from PD speech, our results suggest that sub-fundamental frequency information may reflect neural dynamics in speech production, offering new perspectives for speech pathology and neural oscillation research.
Synchronous analysis of abnormal acoustic and linguistic production in Parkinson's speech
Authors: Daniel Escobar-Grisales; Cristian David RÃ­os-Urrego; Sabato Marco Siniscalchi; Adolfo M. Garcia; Yamile Bocanegra; Leonardo Moreno; Elmar NÃ¶th; Juan Rafael Orozco-Arroyave
Venue: Interspeech 2025 (main)

Abstract: Parkinson's disease is a neurodegenerative disorder involving speech and language deficits. Often, these are separately studied as proxies of motor and non-motor (e.g., cognitive) symptoms, respectively. Conversely, links between both dimensions remain virtually uncharted. This paper introduces a methodology that enables the synchronous study of acoustic and linguistic patterns in Parkinson's speech. Our findings show that verbs and nouns provided relevant acoustic and linguistic information not only to model motor impairments but also to understand non-motor symptoms like those that appear when Parkinson's disease patients develop mild cognitive impairment.
Automatic Detection and Sub-typing of Primary Progressive Aphasia from Speech: Integrating Task-Specific Features and Spatio-Semantic Graphs
Authors: Fritz Peters; W Richard Bevan-Jones; Grace Threlfall; Jenny M Harris; Julie S Snowden; Matthew Jones; Jennifer C Thompson; Daniel J Blackburn; Heidi Christensen
Venue: Interspeech 2025 (main)

Abstract: Primary progressive aphasia (PPA) describes a group of neurodegenerative diseases that predominantly affect language abilities. Its diagnostic process typically requires experienced clinicians, often available only in specialised hospital departments. Patients with PPA frequently display changes in speech and language early in the disease progression. In this study, we extracted acoustic, linguistic, and task-specific features from audio recordings and evaluated their utility for PPA classification. Using a subset of task-specific features, we detected PPA with 97% accuracy. For sub-typing, models trained on the full feature set achieved 74% accuracy in a three-way classification of PPA variants. Our results highlight the added value of task-specific features, which complement traditional approaches. Additionally, their visualisation offers an intuitive representation of task execution, improving clinical interpretability and potential diagnostic utility.
Towards Classification of Typical and Atypical Disfluencies: A Self Supervised Representation Approach
Authors: Priyanka Kommagouni; Pragya Khanna; Vamshiraghusimha Narasinga; Anirudh Bocha; Anil Kumar Vuppala
Venue: Interspeech 2025 (main)

Abstract: This paper investigates the nuanced distinctions between typical and atypical speech disfluencies, focusing on features captured by self-supervised models. Typical disfluencies are natural, non-pathological irregularities in speech, while atypical disfluencies are linked to speech disorders like stuttering, characterized by more frequent and severe disruptions. Despite progress in automatic disfluency detection, limited research addresses the direct classification of these two types. This study leverages intermediate representations from four pretrained models Wav2Vec2.0, HuBERT, WavLM, and TERA, to analyze and classify typical and stuttered disfluencies. The experiments utilize two novel Indian English datasets, IIITH-IEDE and IIITH-TISA, enabling a comprehensive analysis of disfluency patterns in a linguistically diverse context. Classification experiments with support vector machines (SVM) and convolutional neural networks (CNN) reveal that features from HuBERTâs 5th layerâbalancing low-level acoustic and high-level semantic informationâachieve a peak F1 score of 0.97. These findings highlight the importance of intermediate layer representations of self-supervised models in distinguishing nuanced speech variations and contribute to robust and interpretable systems for speech disfluency classification.
Stuttering Detection Based on Self-Attention Weights of Temporal Acoustic Vector Sequence
Authors: Genzo Miyahara; Tsuneo Kato; Akihiro Tamura
Venue: Interspeech 2025 (main)

Abstract: Stuttering detection is gaining attention to enable automatic monitoring of the condition of persons who stutter and to develop inclusive automatic speech recognition. Stuttering has various symptoms, i.e., repetitions of sounds, syllables, or words, prolongation of sounds, blocks, and interjections, and the acoustic properties are highly individual. Although detection performances have been improved by introducing deep learning techniques and creating larger datasets of stuttered speech, the datasets are still much smaller than those of normal speech, which leads to overfitting of the models. We propose a self-attention weight feature of a temporal acoustic vector sequence. This feature can efficiently extract the temporal structure specific to stuttered speech with less regard for the varied acoustic properties. Experimental results showed that a model utilizing multi-layer self-attention weight features of wav2vec 2.0 outperformed a previous attention-based model using wav2vec 2.0.
Speech-Based Automatic Chronic Kidney Disease Diagnosis via Transformer Fusion of Glottal and Spectrogram Features
Authors: Jihyun Mun; Minhwa Chung; Sunhee Kim
Venue: Interspeech 2025 (main)

Abstract: Chronic kidney disease (CKD) is a global health concern characterized by a gradual and irreversible decline in kidney function. Early diagnosis and timely intervention are crucial, yet current methods rely primarily on invasive blood and urine tests. Since CKD affects the respiratory system and alters speech production, vocal characteristics may serve as biomarkers for disease detection. This study proposes a deep learning-based approach that integrates spectrogram and glottal features for CKD diagnosis. Spectrograms capture broad acoustic characteristics, whereas glottal features, known to be influenced by CKD, provide complementary phonatory information. To effectively fuse these features, we employ a transformer-like architecture. The proposed method achieves an accuracy and a macro F1 score of 0.96, demonstrating its potential as an objective, non-invasive diagnostic tool. In addition, we analyze attention weights and gradient-based saliency maps to enhance model interpretability.
Influence of Room Acoustics on Objective Voice Assessment Methods in the Context of Speech and Language Therapy
Authors: Sven Franz; Tanja Grewe; Bernd T. Meyer; JÃ¶rg Bitzer
Venue: Interspeech 2025 (main)

Abstract: Voice and speech disorders can be assessed through subjective perceptual measures or objective indices, such as the Acoustic Voice Quality Index (AVQI) or the Acoustic Breathiness Index (ABI). These objective measures reduce diagnostic variability by eliminating subjective evaluations. However, the impact of room acoustics on their robustness for therapy monitoring remains unclear. In this contribution, reverberation times, impulse responses, and background noise were recorded and analyzed in conjunction with speech samples using statistical models to evaluate their influence. Results indicate that room acoustics significantly affect AVQI and ABI, particularly for non-pathological voices and with distant microphones. For reliable therapeutic use, standardized measurement environments or robust analysis methods are essential. Optimized recording conditions with source-proximate microphones enhance accuracy, advancing objective voice quality assessments and evidence-based speech therapy.
Multimodal Speech-Based Biomarkers Outperform the ALS Functional Rating Scale in Predicting Individual Disease Progression in ALS
Authors: Hardik Kothare; Michael Neumann; Vikram Ramanarayanan
Venue: Interspeech 2025 (main)

Abstract: Disease progression in ALS is heterogeneous due to the varying presentation of clinical symptoms. This heterogeneity makes it difficult to accurately quantify longitudinal disease severity in people with ALS (pALS), making it difficult to determine the efficacy of therapeutic interventions. In this work, we explore a Bayesian Logistic Mixed-Effects model that can help predict individual trajectories in pALS. We used metrics extracted from 143 pALS who interacted with a cloud-based multimodal assessment platform comprising standard speaking exercises. We found that multimodal biomarkers can be predicted more accurately than the ALSFRS-R, the clinical gold standard to measure disease state, with dense and sparse training data. Such non-linear models have the potential to help with stratification of pALS into fast and slow progressors and thus inform treatment approaches. Patient stratification is also a key factor in designing clinical trials to test drug efficacy in slowing progression.
Naturalness-Aware Curriculum Learning with Dynamic Temperature for Speech Deepfake Detection
Authors: Taewoo Kim; Guisik Kim; Choongsang Cho; Young Han Lee
Venue: Interspeech 2025 (main)

Abstract: Recent advances in speech deepfake detection (SDD) have significantly improved artifacts-based detection in spoofed speech. However, most models overlook speech naturalness, a crucial cue for distinguishing bona fide speech from spoofed speech. This study proposes naturalness-aware curriculum learning, a novel training framework that leverages speech naturalness to enhance the robustness and generalization of SDD. This approach measures sample difficulty using both ground-truth labels and mean opinion scores, and adjusts the training schedule to progressively introduce more challenging samples. To further improve generalization, a dynamic temperature scaling method based on speech naturalness is incorporated into the training process. A 23% relative reduction in the EER was achieved in the experiments on the ASVspoof 2021 DF dataset, without modifying the model architecture. Ablation studies confirmed the effectiveness of naturalness-aware training strategies for SDD tasks.
Leveraging SSL Speech Features and Mamba for Enhanced DeepFake Detection
Authors: Hoan My Tran; Damien Lolive; David Guennec; Aghilas Sini; Arnaud Delhay; Pierre-FranÃ§ois Marteau
Venue: Interspeech 2025 (main)

Abstract: Large-scale self-supervised learning models have proven highly effective in extracting robust features for detecting both genuine and spoofed speech. However, leveraging these features remains challenging, particularly in balancing in-domain specialization with out-of-domain generalization. In this work, we propose an effective approach for automatically selecting features using self-gated mechanism, and aggregation of speech representations from a pretrained foundation model to enhance deepfake detection. Our approach integrates a multi-kernel gated convolution module to improve feature learning and facilitate the fusion of features. Additionally, we employ Mamba to effectively capture both short and long-range discriminative patterns in speech. The proposed method achieves strong performance in audio deepfake detection, demonstrating improved generalization across diverse datasets. The source code will be made available on Github.
A Comparative Study on Proactive and Passive Detection of Deepfake Speech
Authors: Chia-Hua Wu; Wanying Ge; Xin Wang; Junichi Yamagishi; Yu Tsao; Hsin-Min Wang
Venue: Interspeech 2025 (main)

Abstract: Solutions for defending against deepfake speech fall into two categories: proactive watermarking models and passive conventional deepfake detectors. While both address common threats, their differences in training, optimization, and evaluation prevent a unified protocol for joint evaluation and selecting the best solutions for different cases. This work proposes a framework to evaluate both model types in deepfake speech detection. To ensure fair comparison and minimize discrepancies, all models were trained and tested on common datasets, with performance evaluated using a shared metric. We also analyze their robustness against various adversarial attacks, showing that different models exhibit distinct vulnerabilities to different speech attribute distortions. Our training and evaluation code is available at Github.
PhonemeFake: Redefining Deepfake Realism with Language-Driven Segmental Manipulation and Adaptive Bilevel Detection
Authors: Oguzhan Baser; Ahmet Ege Tanriverdi; Sriram Vishwanath; Sandeep Chinchali
Venue: Interspeech 2025 (main)

Abstract: Deepfake (DF) attacks pose a growing threat as generative models become increasingly advanced. However, our study reveals that existing DF datasets fail to deceive human perception, unlike real DF attacks that influence public discourse. It highlights the need for more realistic DF attack vectors. We introduce PhonemeFake (PF), a DF attack that manipulates critical speech segments using language reasoning, significantly reducing human perception by up to 42% and benchmark accuracies by up to 94%. We release an easy-to-use PF dataset on HuggingFace and open-source bilevel DF segment detection model that adaptively prioritizes compute on manipulated regions. Our extensive experiments across three known DF datasets reveal that our detection model reduces EER by 91% while achieving up to 90% speed-up, with minimal compute overhead and precise localization beyond existing models as a scalable solution.
From Sharpness to Better Generalization for Speech Deepfake Detection
Authors: Wen Huang; Xuechen Liu; Xin Wang; Junichi Yamagishi; Yanmin Qian
Venue: Interspeech 2025 (main)

Abstract: Generalization remains a critical challenge in speech deepfake detection (SDD). While various approaches aim to improve robustness, generalization is typically assessed through performance metrics like equal error rate without a theoretical framework to explain model performance. This work investigates sharpness as a theoretical proxy for generalization in SDD. We analyze how sharpness responds to domain shifts and find it increases in unseen conditions, indicating higher model sensitivity. Based on this, we apply Sharpness-Aware Minimization (SAM) to reduce sharpness explicitly, leading to better and more stable performance across diverse unseen test sets. Furthermore, correlation analysis confirms a statistically significant relationship between sharpness and generalization in most test settings. These findings suggest that sharpness can serve as a theoretical indicator for generalization in SDD and that sharpness-aware training offers a promising strategy for improving robustness.