GenAI4ELab/papercli-papers huggingface dataset:
Synthetic Data Generation for Phrase Break Prediction with Large Language Model
Authors: Hoyeon Lee; Sejung Son; Ye-Eun Kang; Jong-Hwan Kim
Venue: Interspeech 2025 (main)

Abstract: Current approaches to phrase break prediction address crucial prosodic aspects of text-to-speech systems but heavily rely on vast human annotations from audio or text, incurring significant manual effort and cost. Inherent variability in the speech domain, driven by phonetic factors, further complicates acquiring consistent, high-quality data. Recently, large language models (LLMs) have shown success in addressing data challenges in NLP by generating tailored synthetic data while reducing manual annotation needs. Motivated by this, we explore leveraging LLM to generate synthetic phrase break annotations, addressing the challenges of both manual annotation and speech-related tasks by comparing with traditional annotations and assessing effectiveness across multiple languages. Our findings suggest that LLM-based synthetic data generation effectively mitigates data challenges in phrase break prediction and highlights the potential of LLMs as a viable solution for the speech domain.
Speech Reference Intervals: An Assessment of Feasibility in Depression Symptom Severity Prediction
Authors: Lauren White; Ewan Carr; Judith Dineley; Catarina Botelho; Pauline Conde; Faith Matcham; Carolin Oetzmann; Amos Folarin; George Fairs; Agnes Norbury; Stefano Goria; Srinivasan Vairavan; Til Wykes; Richard Dobson; Vaibhav Naraya; Matthew Hotopf; Alberto Abad; Isabel Trancoso; Nicholas Cummins
Venue: Interspeech 2025 (main)

Abstract: Major Depressive Disorder (MDD) is a prevalent mental disorder. Combining speech features and machine learning has promise for predicting MDD, but interpretability is crucial for clinical applications. Reference intervals (RIs) represent a typical range for a speech feature in a population. RIs could increase interpretability and help clinicians identify deviations from norms. They could also replace conventional speech features in machine learning models. However, no work has yet assessed the feasibility of speech RIs in MDD. We generated and compared RIs from three reference datasets varying in size, elicitation prompt, and health information. We then calculated deviations from each RI set for people with MDD to compare performance on a depression symptom severity prediction task. Our RI-based models trained with demographic data performed similarly to each other and equivalent models using conventional features or demographics only, demonstrating the value of RI-derived features.
DepressGEN: Synthetic Data Generation Framework for Depression Detection
Authors: Wenrui Liang; Rong Zhang; Xuezhen Zhang; Ying Ma; Wei-Qiang Zhang
Venue: Interspeech 2025 (main)

Abstract: Automated depression detection is vital for early diagnosis, but ethical and privacy concerns often limit the availability of sufficient training data, hindering research in depression screening. To address this, we introduce DepressGEN, a novel framework that generates synthetic interview dialogue texts and speech simulating depressed patients to improve training for detection models. By inputting linguistic features associated with depression into a large language model, we create dialogue texts and use a TTS system to generate corresponding speech. We also developed a depression modulation module to modify the synthesized speech, as well as a speech verification module to bridge the gap between synthetic and real data distributions. Our results demonstrate that a GRU/BiLSTM-based model trained with additional synthetic data improves F1 scores by 9.9% compared to the same model trained only on original data, outperforming existing methods on the EATD dataset.
Emotion-Guided Graph Attention Networks for Speech-Based Depression Detection under Emotion-Inducting Tasks
Authors: Yuqiu Zhou; Yongjie Zhou; Yudong Yang; Yang Liu; Jun Huang; Shuzhi Zhao; Rongfeng Su; Lan Wang; Nan Yan
Venue: Interspeech 2025 (main)

Abstract: Depression affects emotional expression and perception. As a non-invasive and privacy-preserving method, speech is widely used for automatic depression detection. However, existing models often focus only on depressive features in speech, ignoring the differential emotion expression patterns across different emotion-inducing tasks. To address this, we propose an emotion-guided graph attention network (emoGAT) for depression detection. By collecting speech-text data from depressed individuals and healthy controls during emotion-inducing tasks, we construct graph embeddings using sentiment cues from both speech and text. Experimental results show our method reduces the standard deviation by 1.8% and improves accuracy by 4.36%. Graph attention visualization also reveals depression-specific characteristics, such as flattened prosody in neutral picture description tasks and cognitive biases toward negative information, offering deeper insights into emotional relational expressions.
Explainable Depression Detection using Masked Hard Instance Mining
Authors: Patawee Prakrankamanant; Shinji Watanabe; Ekapol Chuangsuwanich
Venue: Interspeech 2025 (main)

Abstract: This paper addresses the critical need for improved explainability in text-based depression detection. While offering predictive outcomes, current solutions often overlook the understanding of model predictions which can hinder trust in the system. We propose the use of Masked Hard Instance Mining (MHIM) to enhance the explainability in the depression detection task. MHIM strategically masks attention weights within the model, compelling it to distribute attention across a wider range of salient features. We evaluate MHIM on two datasets representing distinct languages: Thai (Thai-Maywe) and English (DAIC-WOZ). Our results demonstrate that MHIM significantly improves performance in terms of both prediction accuracy and explainability metrics.
Test-Time Training for Speech-based Depression Detection
Authors: Sri Harsha Dumpala; Chandramouli S. Sastry; Rudolf Uher; Sageev Oore
Venue: Interspeech 2025 (main)

Abstract: Previous works on speech-based depression detection typically use datasets collected in similar environments for both training and testing the models. However, in practice, the training and testing distributions often differ. Distributional shifts in speech can result from various factors, such as differences in recording environments (e.g., background noise) and demographic attributes (e.g., gender, age). These shifts can significantly degrade the performance of depression detection models. In this paper, we analyze the application of test-time training (TTT) to improve the robustness of depression detection models against such shifts. Our results demonstrate that TTT can substantially enhance model performance under various distributional shifts, including those caused by (a) background noise, (b) gender bias, and (c) differences in data collection and curation procedures, where training and testing samples originate from different datasets.
Leveraging Ordinal Information for Speech-based Depression Classification
Authors: Lishi Zuo; Man-Wai Mak
Venue: Interspeech 2025 (main)

Abstract: While depression is inherently ordinal, much of the previous work in depression detection oversimplifies the problem by treating it as a binary classification problem, ignoring the subtle variations and the order in depression severity. We propose creating a latent space that contains ordinal information via an ordinal loss to benefit the learning of depression classification. Specifically, we define K thresholds for the depression scores, thereby creating a series of binary classification tasks on different levels of depression (e.g., mild vs. non-mild). The ordinal loss allows the model to capture the relationships between these levels on top of the binary classification task. Our approach outperforms current state-of-the-art depression detection methods, highlighting the importance of considering the inherent ordinal nature of depression severity.
Zero-Shot Speech-Based Depression and Anxiety Assessment with LLMs
Authors: Erfan Loweimi; Sofia de la Fuente Garcia; Saturnino Luz
Venue: Interspeech 2025 (main)

Abstract: The use of Large Language Models (LLMs) for psychological state assessment from speech has gained significant interest, particularly in analysing and predicting mental health. In this paper, we explore the potential of eight instruct-tuned LLMs (Llama-3.1-8B, Ministral, Gemma-2-9B, Phi-4, Mistral, DeepSeek-Qwen, QwQ-Preview and Llama-3.3-70B) in a zero-shot setting to predict Hospital Anxiety and Depression Scale (HADS) depression and anxiety scores from one-to-two minute spontaneous speech recordings from the PsyVoiD database. We evaluate how transcript quality affects LLM responses by comparing performance using ground-truth transcriptions versus transcripts generated by Whisper models of different sizes. Spearman correlation coefficients and statistical analysis demonstrate significant and notable potential of the LLMs to predict psychological states in a zero-shot setting.
Towards the Objective Characterisation of Major Depressive Disorder Using Speech Data from a 12-week Observational Study with Daily Measurements
Authors: Robert Lewis; Szymon Fedor; Nelson Hidalgo Julia; Joshua Curtiss; Jiyeon Kim; Noah Jones; David Mischoulon; Thomas F Quatieri; Nicholas Cummins; Paola Pedrelli; Rosalind Picard
Venue: Interspeech 2025 (main)

Abstract: Our analysis focuses on identifying relations between properties of the voice and depression symptom severity. On a novel corpus of 3,374 longitudinal speech recordings from 71 patients clinically diagnosed with major depressive disorder (MDD), we use a statistical modelling approach to identify associations between depression symptom severity and 38 acoustic and cognitive features. Significant negative associations with daily within-individual fluctuations of depression include speaking rate and articulation rate. Furthermore, when analysing how the changes in speech-derived features covary over time with the change in depression severity, we find that the standard deviation of the pitch has a significant negative association, as well as the speaking and articulation rate. We also discover that several performance metrics derived from the cognitive tasks (digit-span and Stroop) have significant associations with fluctuations or changes in depression symptom severity.
Can Speech Accurately Detect Depression in Patients With Comorbid Dementia? An Approach for Mitigating Confounding Effects of Depression and Dementia
Authors: Sophie Young; Fuxiang Tao; Bahman Mirheidari; Madhurananda Pahar; Markus Reuber; Heidi Christensen
Venue: Interspeech 2025 (main)

Abstract: Approximately 15.9% of people living with dementia experience co-occurring major depressive disorder. Both disorders cause similar early clinical symptoms in older people but treatment options and patient outcomes differ. While it is challenging, it is therefore critical for clinicians to be able to distinguish between them. We build on existing research into objective markers of depression in speech, testing their generalizability to a more complex population. On a novel, comorbidity dataset, we demonstrate that existing depression classification methods perform worse for participants with dementia than they do for those with no cognitive decline. We also propose a method of applying Wasserstein distance-based weight vectors to emphasize depression-related information which is robust against the effect of dementia. This improves performance for users with dementia, without requiring changes to the model architectures. Our best performing model achieves an overall F1-score of 81.0%.
Temporal Convolutional Network with Smoothed and Weighted Losses for Distant Voice Activity and Overlapped Speech Detection
Authors: Shaojie Li; Qintuya Si; De Hu
Venue: Interspeech 2025 (main)

Abstract: Voice Activity Detection (VAD) and Overlapped Speech Detection (OSD) are key steps in various audio/speech processing tasks. Recent advances in VAD or OSD are moving toward using Temporal Convolutional Networks (TCNs) with frame-independent cross-entropy loss, which may be unable to cope with transient errors or boundary errors (caused by weak recordings at speech boundaries). In this paper, we formulate two novel losses, namely smoothed loss and weighted loss, in which the former copes with transient errors while the latter deals with boundary errors. In addition, we adopt Mel Frequency Cepstral Coefficients (MFCCs) and Instantaneous Correlation Coefficients (ICCs) as the acoustic and spatial features to drive the model. To improve computing efficiency, we also propose a spatial feature extraction module by selecting those frequencies with information-rich ICCs, which delivers good lightweight nature. Numerical experiments validate the efficacy of the proposed method.
Attention Is Not Always the Answer: Optimizing Voice Activity Detection with Simple Feature Fusion
Authors: Kumud Tripathi; Chowdam Venkata Kumar; Pankaj Wasnik
Venue: Interspeech 2025 (main)

Abstract: Voice Activity Detection (VAD) plays a vital role in speech processing, often relying on hand-crafted or neural features. This study examines the effectiveness of Mel-Frequency Cepstral Coefficients (MFCCs) and pre-trained model (PTM) features, including wav2vec 2.0, HuBERT, WavLM, UniSpeech, MMS, and Whisper. We propose FusionVAD, a unified framework that combines both feature types using three fusion strategies: concatenation, addition, and cross-attention (CA). Experimental results reveal that simple fusion techniques, particularly addition, outperform CA in both accuracy and efficiency. Fusion-based models consistently surpass single-feature models, highlighting the complementary nature of MFCCs and PTM features. Notably, our best fusion model outperforms state-of-the-art Pyannote VAD model across multiple datasets, achieving an absolute average improvement of 2.04%. These results confirm that simple feature fusion enhances VAD robustness while maintaining computational efficiency.
SpeechMLC: Speech Multi-label Classification
Authors: Miseul Kim; Seyun Um; Hyeonjin Cha; Hong-Goo Kang
Venue: Interspeech 2025 (main)

Abstract: In this paper, we propose a multi-label classification framework to detect multiple speaking styles in a speech sample. Unlike previous studies that have primarily focused on identifying a single target style, our framework effectively captures various speaker characteristics within a unified structure, making it suitable for generalized human-computer interaction applications. The proposed framework integrates cross-attention mechanisms within a transformer decoder to extract salient features associated with each target label from the input speech. To mitigate the data imbalance inherent in multi-label speech datasets, we employ a data augmentation technique based on a speech generation model. We validate our model's effectiveness through multiple objective evaluations on seen and unseen corpora. In addition, we provide an analysis of the influence of human perception on classification accuracy by considering the impact of human labeling agreement on model performance.
Fully End-to-end Streaming Open-vocabulary Keyword Spotting with W-CTC Forced Alignment
Authors: Dohyun Kim; Jiwook Hwang
Venue: Interspeech 2025 (main)

Abstract: In open-vocabulary keyword spotting, an acoustic encoder pre-trained with Connectionist Temporal Classification (CTC) loss is typically used to train a text encoder by aligning audio embedding space with text embedding space. In previous work, word-aligned datasets were created by forced alignment algorithms such as the Montreal Forced Aligner (MFA) to train text encoder and verifier models. In this paper, we propose a new training pipeline for open-vocabulary keyword spotting using the W-CTC forced alignment algorithm, a simple modification of the practical CTC algorithm. Our approach eliminates the need for creating word-aligned datasets, operates in a fully end-to-end manner, and demonstrates superior performance on the Libriphrase hard dataset.
Comparative Evaluation of Acoustic Feature Extraction Tools for Clinical Speech Analysis
Authors: Anna Seo Gyeong Choi; Alexander Richardson; Ryan Partlan; Sunny X. Tang; Sunghye Cho
Venue: Interspeech 2025 (main)

Abstract: This study compares three acoustic feature extraction toolkitsâOpenSMILE, Praat, and Librosaâapplied to clinical speech data from individuals with schizophrenia spectrum disorders (SSD) and healthy controls (HC). By standardizing extraction parameters across the toolkits, we analyzed speech samples from 77 SSD and 87 HC participants and found significant toolkit-dependent variations. While F0 percentiles showed high cross-toolkit correlation (r=0.962â0.999), measures like F0 standard deviation and formant values often had poor, even negative, agreement. Additionally, correlation patterns differed between SSD and HC groups. Classification analysis identified F0 mean, HNR, and MFCC1 (AUC > 0.70) as promising discriminators. These findings underscore reproducibility concerns and advocate for standardized protocols, multi-toolkit cross-validation, and transparent reporting.
Can We Trust Machine Learning? The Reliability of Features from Open-Source Speech Analysis Tools for Speech Modeling
Authors: Tahiya Chowdhury; Veronica Romero
Venue: Interspeech 2025 (main)

Abstract: Machine learning-based behavioral models rely on features extracted from audio-visual recordings. The recordings are processed using open-source tools to extract speech features for classification models. These tools often lack validation to ensure reliability in capturing behaviorally relevant information. This gap raises concerns about reproducibility and fairness across diverse populations and contexts. Speech processing tools, when used outside of their design context, can fail to capture behavioral variations equitably and can then contribute to bias. We evaluate speech features extracted from two widely used speech analysis tools, OpenSMILE and Praat, to assess their reliability when considering adolescents with autism. We observed considerable variation in features across tools, which influenced model performance across context and demographic groups. We encourage domain-relevant verification to enhance the reliability of machine learning models in clinical applications.
HK-GenSpeech: A Generative AI Scene Creation Framework for Speech Based Cognitive Assessment
Authors: Vi Jun Sean Yong; Serkan Kumyol; Pau Le Lisa Low; Suk Wai Winnie Leung; Tristan Braud
Venue: Interspeech 2025 (main)

Abstract: Current methods of automated speech-based cognitive assessment often rely on fixed-picture descriptions in major languages, limiting repeatability, engagement, and locality. This paper introduces HK-GenSpeech (HKGS), a framework using generative AI to create pictures that present similar features to those used in cognitive assessment, augmented with descriptors reflecting the local context. We demonstrate HKGS through a dataset of 423 Cantonese speech samples collected in Hong Kong from 141 participants, with HK-MoCA scores ranging from 11 to 30. Each participant described the cookie theft picture, an HKGS fixed image, and an HKGS dynamic image. Regression experiments show comparable accuracy for all image types, indicating HKGS' adequacy in generating relevant assessment images. Lexical analysis further suggests that HKGS images elicit richer speech. By mitigating learning effects and improving engagement, HKGS supports broader data collection, particularly in low-resource settings.
Leveraging AM and FM Rhythm Spectrograms for Dementia Classification and Assessment
Authors: Parismita Gogoi; Vishwanath Pratap Singh; Seema Khadirnaikar; Soma Siddhartha; Sishir Kalita; Jagabandhu Mishra; Md Sahidullah; Priyankoo Sarmah; S. R. M. Prasanna
Venue: Interspeech 2025 (main)

Abstract: This study explores the potential of Rhythm Formant Analysis (RFA) to capture long-term temporal modulations in dementia speech. Specifically, we introduce RFA-derived rhythm spectrograms as novel features for dementia classification and regression tasks. We propose two methodologies: (1) handcrafted features derived from rhythm spectrograms, and (2) a data-driven fusion approach, integrating proposed  RFA-derived rhythm spectrograms with vision transformer (ViT) for acoustic representations along with BERT-based linguistic embeddings. We compare these with existing features. Notably, our handcrafted features outperform eGeMAPs with a relative improvement of 14.2% in classification accuracy and comparable performance in the regression task. The fusion approach also shows improvement, with RFA spectrograms surpassing Mel spectrograms in classification by around a relative improvement of 13.1% and a comparable regression score with the baselines. All codes are available in GitHub repo.
Leveraging Cascaded Binary Classification and Multimodal Fusion for Dementia Detection through Spontaneous Speech
Authors: Yin-Long Liu; Yuanchao Li; Rui Feng; Liu He; Jia-Xin Chen; Yi-Ming Wang; Yu-Ang Chen; Yan-Han Peng; Jia-Hong Yuan; Zhen-Hua Ling
Venue: Interspeech 2025 (main)

Abstract: This paper presents our submission to the PROCESS Challenge 2025, focusing on spontaneous speech analysis for early dementia detection. For the three-class classification task (Healthy Control, Mild Cognitive Impairment, and Dementia), we propose a cascaded binary classification framework that fine-tunes pre-trained language models and incorporates pause encoding to better capture disfluencies. This design streamlines multi-class classification and addresses class imbalance by restructuring the decision process. For the Mini-Mental State Examination score regression task, we develop an enhanced multimodal fusion system that combines diverse acoustic and linguistic features. Separate regression models are trained on individual feature sets, with ensemble learning applied through score averaging. Experimental results on the test set outperform the baselines provided by the organizers in both tasks, demonstrating the robustness and effectiveness of our approach.
Whisper-Based Multilingual Alzheimer's Disease Detection and Improvements for Low-Resource Language
Authors: Kaichen Jia; Jinpeng Li; Ke Li; Wei-Qiang Zhang
Venue: Interspeech 2025 (main)

Abstract: Alzheimer's Disease (AD) poses a growing global health challenge due to population aging. Using spontaneous speech for the early diagnosis of AD has emerged as a notable area of research. In response to the global trend of AD, our study proposes a speech-based multilingual AD detection method. In our study, we utilize Whisper for transfer learning to build a multilingual pre-trained AD diagnostic model that achieves 81.38% accuracy on a test set comprising multiple languages. To enhance low-resource language performance, we fine-tune the pre-trained model with multilingual data and full transcripts as prompts, achieving a 4-7% accuracy improvement. Additionally, we incorporate the speaker's background information, enhancing the accuracy of low-resource languages by 11-13%. The results demonstrate the validity of our work in multilingual Alzheimer's detection tasks and also illustrate the feasibility of our approach in addressing the global need for Alzheimer's detection.