GenAI4ELab/papercli-papers huggingface dataset:
LRBA: Stealthy Backdoor Attacks on Speech Classification via Latent Rearrangement in VITS
Authors: Zexin Li; Wenhan Yao; Ye Xiao; Jinsu Yang; Fen Xiao; Weiping Wen
Venue: Interspeech 2025 (main)

Abstract: Speech classification systems based on deep learning are vulnerable to backdoor attacks, causing the model's predictions to deviate from normal behavior. Existing speech backdoor methods often produce poisoned samples by perceptible modifications, which reduce the stealthiness of the attack and make it easier to detect. To improve stealthiness, this paper proposed the Latent Rearrangement Backdoor Attack (LRBA), a novel backdoor attack framework utilizing the latent space in a pre-trained VITS model to achieve an imperceptible attack. Explicitly, we manipulate the latent representations by utilizing the normalizing flow of VITS to generate rearranged utterances, where the rearranged semantics can be associated with the attacker's specific target label, achieving a backdoor attack. Results show that our method achieves an excellent attack success rate with a very low poisoning rate and maintains a high mean opinion score, outperforming existing methods in effectiveness and stealthiness.
LitMAS: A Lightweight and Generalized Multi-Modal Anti-Spoofing Framework for Biometric Security
Authors: Nidheesh Gorthi; Kartik Thakral; Rishabh Ranjan; Richa Singh; Mayank Vatsa
Venue: Interspeech 2025 (main)

Abstract: Biometric authentication systems are increasingly being deployed in critical applications, but they remain susceptible to spoofing. Since most of the research efforts focus on modality-specific anti-spoofing techniques, building a unified, resource-efficient solution across multiple biometric modalities remains a challenge. To address this, we propose LitMAS, a Lightweight and generalizable Multi-modal Anti-Spoofing framework designed to detect spoofing attacks in speech, face, iris, and fingerprint-based biometric systems. At the core of LitMAS is a Modality-Aligned Concentration Loss, which enhances interclass separability while preserving cross-modal consistency and enabling robust spoof detection across diverse biometric traits. With just 6M parameters, LitMAS surpasses state-of-the-art methods by 1.36% in average EER across seven datasets, demonstrating high efficiency, strong generalizability, and suitability for edge deployment. Code and trained models are available.
Pitfalls and Limits in Automatic Dementia Assessment
Authors: Franziska Braun; Christopher Witzl; Andreas Erzigkeit; Hartmut Lehfeld; Thomas Hillemacher; Tobias Bocklet; Korbinian Riedhammer
Venue: Interspeech 2025 (main)

Abstract: Current work on speech-based dementia assessment focuses on either feature extraction to predict assessment scales, or on the automation of existing test procedures. Most research uses public data unquestioningly and rarely performs a detailed error analysis, focusing primarily on numerical performance. We perform an in-depth analysis of an automated standardized dementia assessment, the Syndrom-Kurz-Test. We find that while there is a high overall correlation with human annotators, due to certain artifacts, we observe high correlations for the severely impaired individuals, which is less true for the healthy or mildly impaired ones. Speech production decreases with cognitive decline, leading to overoptimistic correlations when test scoring relies on word naming. Depending on the test design, fallback handling introduces further biases that favor certain groups. These pitfalls remain independent of group distributions in datasets and require differentiated analysis of target groups.
On the Within-class Variation Issue in Alzheimer's Disease Detection
Authors: Jiawen Kang; Dongrui Han; Lingwei Meng; Jingyan Zhou; Jinchao Li; Xixin Wu; Helen Meng
Venue: Interspeech 2025 (main)

Abstract: Alzheimer's Disease (AD) detection employs machine learning classification models to distinguish between individuals with AD and those without. Different from conventional classification tasks, we identify within-class variation as a critical challenge in AD detection: individuals with AD exhibit a spectrum of cognitive impairments. Therefore, simplistic binary AD classification may overlook two crucial aspects: within-class heterogeneity and instance-level imbalance. In this work, we found using a sample score estimator can generate sample-specific soft scores aligning with cognitive scores. We subsequently propose two simple yet effective methods: Soft Target Distillation (SoTD) and Instance-level Re-balancing (InRe), targeting two problems respectively. Based on the ADReSS and CU-MARVEL corpora, we demonstrated and analyzed the advantages of the proposed approaches in detection performance. These findings provide insights for developing robust and reliable AD detection models.
Alzheimerâs Disease Detection Using Co-Attention Mechanism for Acoustic and ASR-Transcribed Text Features
Authors: Yongqi Shao; Tao Fang
Venue: Interspeech 2025 (main)

Abstract: Alzheimer's disease (AD) is a progressive disorder that gradually affects memory, language, and reasoning, making early detection crucial for timely intervention. Traditional methods, like medical imaging and clinical evaluations, are costly and limit accessibility. To address this, we propose a speech-based AD detection method that leverages a co-attention mechanism to integrate multilevel acoustic and transcribed text features. These acoustic features include spectrograms, MFCC features, and wav2vec2 embeddings. The mechanism dynamically assigns weights to different features, enhancing their interaction and improving fusion. We also compare early and late fusion strategies to optimize integration. Tested on the ADReSSo dataset, our model achieves 83.15% accuracy, demonstrating the effectiveness of a well-structured integration of acoustic and transcribed text features for accessible and cost-efficient AD detection.
Beyond Manual Transcripts: The Potential of Automated Speech Recognition Errors in Improving Alzheimerâs Disease Detection
Authors: Yin-Long Liu; Rui Feng; Jia-Xin Chen; Yi-Ming Wang; Jia-Hong Yuan; Zhen-Hua Ling
Venue: Interspeech 2025 (main)

Abstract: Recent breakthroughs in Automatic Speech Recognition (ASR) have enabled fully automated Alzheimerâs Disease (AD) detection using ASR transcripts. Nonetheless, the impact of ASR errors on AD detection remains poorly understood. This paper fills the gap. We conduct a comprehensive study on AD detection using transcripts from various ASR models and their synthesized speech on the ADReSS dataset. Experimental results reveal that certain ASR transcripts (ASR-synthesized speech) outperform manual transcripts (manual-synthesized speech) in detection accuracy, suggesting that ASR errors may provide valuable cues for improving AD detection. Additionally, we propose a cross-attention-based interpretability model that not only identifies these cues but also achieves superior or comparable performance to the baseline. Furthermore, we utilize this model to unveil AD-related patterns within pre-trained embeddings. Our study offers novel insights into the potential of ASR models for AD detection.
Voice-Based Dysphagia Detection: Leveraging Self-Supervised Speech Representation
Authors: Injune Hwang; Jung-Min Kim; Ju Seok Ryu; Kyogu Lee
Venue: Interspeech 2025 (main)

Abstract: This study introduces a framework for diagnosing dysphagia using self-supervised speech representation learning (SSL) models. Previously reported methods typically rely on mel spectrograms; however, due to the limited amount of medical data, they struggle to accurately diagnose dysphagia from low-dimensional features. However, SSL models, trained on large-scale speech data, are well suited for tasks with smaller dataset. Employing SSL features significantly enhances model performance, allowing for the modelâs size reduction while outperforming larger models based on mel spectrograms. Although a decrease in specificity was observed, recall, a crucial metric for disease diagnosis, showed a marked improvement, leading to a general improvement in diagnostic accuracy. Among the SSL models evaluated, the features of the 10th layer of WavLM had the highest performance. Additionally, increasing the size of the filter in the convolutional layers does not contribute to performance gains.
ADCeleb: A Longitudinal Speech Dataset from Public Figures for Early Detection of Alzheimerâs Disease
Authors: Kunxiao Gao; Anna Favaro; Najim Dehak; Laureano Moro Velazquez
Venue: Interspeech 2025 (main)

Abstract: Previous studies on early Alzheimerâs disease (AD) detection using speech have been limited by small sample sizes, scarce prodromal phase recordings, and minimal longitudinal data. Many also rely on standardized tasks that do not reflect natural language use. To address these gaps, we introduce ADCeleb, a longitudinal speech dataset comprising publicly available recordings from individuals who later disclosed an AD diagnosis. It includes samples from 40 individuals with prodromal AD and 40 cognitively normal controls (CNs), matched by age and sex, spanning 1 to 10 years before diagnosis. Classification experiments using multimodal models integrating speech and text achieved 0.72 accuracy in distinguishing AD from CNs in the 6- to 10-year pre-diagnosis window and 0.80 in the 1- to 5-year pre-diagnosis window. These results highlight the potential of speech-based technologies as non-invasive tools for early AD detection in real-world settings or for triage improvement in clinical trials.
Anne Rowling Neurological Speech Corpus: clinically annotated longitudinal dataset for developing speech biomarkers in neurodegenerative disorders
Authors: Johnny Tam; Christine Weaver; Oliver Watts; Siddharthan Chandran; Suvankar Pal; Rowling Speech Consortium
Venue: Interspeech 2025 (main)

Abstract: There is urgent need for scalable, non-invasive and quantifiable biomarkers in neurodegenerative disorders. Speech is an attractive candidate with potential for remote and cheap assessments. Progress is limited by a lack of high quality clinically annotated speech data. We present a longitudinal speech corpus including speakers with dementia, motor neuron disease, Parkinsonâs disease, progressive multiple sclerosis, and healthy individuals. Participants complete standardised recordings on an app co-produced with patients, aligned to contemporaneous phenotyping (clinical rating scales, cognitive tests and blood-based biomarkers). 780 participants have provided 5169 recordings in 1033 assessments. Benchmark classification and regression models show promising performance, and predictions on non-speech segments demonstrate limited bias from recording conditions. We continue to upscale data collection and analysis across larger diverse populations to accelerate clinical translation.
Multitask Learning with Fused Attention for Improved ASR and Mispronunciation Detection in Children's Speech Sound Disorders
Authors: Selina S. Sung; Seunghee Ha; Tae-Jin Yoon; Jungmin So
Venue: Interspeech 2025 (main)

Abstract: This study proposes a multitask learning framework with fused attention to enhance automatic speech recognition (ASR) for pronunciation-based transcription and mispronunciation detection (MPD) in speech sound disorders (SSD). Our approach leverages multitask learning by carrying out ASR and classification tasks concurrently. To further improve performance, we propose a fused attention mechanism that refines hidden states by weighting features relevant to mispronunciations. The classification head and the attention mechanism work synergistically, jointly optimizing transcription and detection performance. Evaluated on a Korean children SSD dataset, our approach outperforms the baseline, achieving lower Character Error Rates (CER) and higher Unweighted Average Recall (UAR), demonstrating the effectiveness of multitask learning with fused attention for mispronunciation detection.
Multimodal Speech, Language and Orofacial Analysis for Remote Assessment of Positive, Negative and Cognitive Symptoms in Schizophrenia
Authors: Michael Neumann; Hardik Kothare; Beverly Insel; Anzalee Khan; Danyah Nadim; Jean-Pierre Lindenmayer; Vikram Ramanarayanan
Venue: Interspeech 2025 (main)

Abstract: This study provides a comprehensive analysis of digital speech, orofacial and linguistic features for the assessment of schizophrenia. We recorded audio and video from 94 people with schizophrenia (pSCZ) and 100 healthy controls (HC) and extracted features automatically. Clinical rating scales were administered to assess positive, negative, and cognitive symptoms. We show that pSCZ exhibit significant alterations in speech timing, orofacial dynamics, and lexical richness, as compared to HC. A multimodal classification approach achieved high accuracy (96% AUC, 87% UAR), with speech features contributing most to discrimination. Correlation analysis revealed that speech timing and lip velocity measures are correlated with blunted affect and alogia. Linguistic features correlate well with positive symptoms, particularly conceptual disorganization and excitement. Cognitive abilities are most strongly associated with speech timing and specific linguistic features.
Aligning ASR Evaluation with Human and LLM Judgments: Intelligibility Metrics Using Phonetic, Semantic, and NLI Approaches
Authors: Bornali Phukon; Xiuwen Zheng; Mark Hasegawa-Johnson
Venue: Interspeech 2025 (main)

Abstract: Traditional ASR metrics like WER and CER fail to capture intelligibility, especially for dysarthric and dysphonic speech, where semantic alignment matters more than exact word matches. ASR systems struggle with these speech types, often producing errors like phoneme repetitions and imprecise consonants, yet the meaning remains clear to human listeners. We identify two key challenges: (1) Existing metrics do not adequately reflect intelligibility, and (2) while LLMs can refine ASR output, their effectiveness in correcting ASR transcripts of dysarthric speech remains underexplored. To address this, we propose a novel metric integrating Natural Language Inference (NLI) scores, semantic similarity, and phonetic similarity. Our ASR evaluation metric achieves a 0.890 correlation with human judgments on Speech Accessibility Project data, surpassing traditional methods and emphasizing the need to prioritize intelligibility over error-based measures.
SOVA-Bench: Benchmarking the Speech Conversation Ability for LLM-based Voice Assistant
Authors: Yixuan Hou; Heyang Liu; Yuhao Wang; Ziyang Cheng; Ronghua Wu; Qunshan Gu; Yanfeng Wang; Yu Wang
Venue: Interspeech 2025 (main)

Abstract: Thanks to the steady progress of large language models (LLMs), speech encoding algorithms and vocoder structure, recent advancements have enabled generating speech response directly from a user instruction. However, benchmarking the generated speech quality has been a neglected but critical issue, considering the shift from the pursuit of semantic accuracy to vivid and spontaneous speech flow. Previous evaluation focused on the speech-understanding ability, lacking a quantification of acoustic quality. In this paper, we propose Speech cOnversational Voice Assistant Benchmark (SOVA-Bench), providing a comprehension comparison of the general knowledge, speech recognition and understanding, along with both semantic and acoustic generative ability between available speech LLMs. To the best of our knowledge, SOVA-Bench is one of the most systematic evaluation frameworks for speech LLMs, inspiring the direction of voice interaction systems.
An approach to measuring the performance of Automatic Speech Recognition(ASR) models in the context of Large Language Model(LLM) powered applications
Authors: Sujith Pulikodan; Sahapthan K; Prasanta Kumar Ghosh; Visruth Sanka; Nihar Desai
Venue: Interspeech 2025 (main)

Abstract: Automatic Speech Recognition (ASR) plays a crucial role in human-machine interaction and serves as an interface for a wide range of applications. Traditionally, ASR performance has been evaluated using Word Error Rate (WER), a metric that quantifies the number of insertions, deletions, and substitutions in the generated transcriptions. However, with the increasing adoption of large and powerful Large Language Models (LLMs) as the core processing component in various applications, the significance of different types of ASR errors in downstream tasks warrants further exploration. In this work, we analyze the capabilities of LLMs to correct errors introduced by ASRs and propose a new measure to evaluate ASR performance for LLM-powered applications.
DC-Spin: A Speaker-invariant Speech Tokenizer for Spoken Language Models
Authors: Heng-Jui Chang; Hongyu Gong; Changhan Wang; James Glass; Yu-An Chung
Venue: Interspeech 2025 (main)

Abstract: Spoken language models (SLMs) have gained increasing attention with advancements in text-based, decoder-only language models. SLMs process text and speech, enabling simultaneous speech understanding and generation. This paper presents SpinHuBERT and Double-Codebook Speaker-invariant Clustering (DC-Spin) to improve speech tokenization for bridging audio signals and SLM tokens. DC-Spin extracts speaker-invariant tokens rich in phonetic information and resilient to input variations, enhancing zero-shot SLM tasks and speech resynthesis. Comparisons of tokenization methods and downstream task proxies show that tokens easily modeled by an n-gram LM or aligned with phonemes offer strong performance, offering insights for designing speech tokenizers for SLMs.
Exploring the Effect of Segmentation and Vocabulary Size on Speech Tokenization for Speech Language Models
Authors: Shunsuke Kando; Yusuke Miyao; Shinnosuke Takamichi
Venue: Interspeech 2025 (main)

Abstract: The purpose of speech tokenization is to transform a speech signal into a sequence of discrete representations, serving as the foundation for speech language models (SLMs). While speech tokenization has many options, their effect on the performance of SLMs remains unclear. This paper investigates two key aspects of speech tokenization: the segmentation width and the cluster size of discrete units. First, we segment speech signals into fixed/variable widths and pooled representations. We then train K-means models in multiple cluster sizes. Through the evaluation on zero-shot spoken language understanding benchmarks, we find the positive effect of moderately coarse segmentation and bigger cluster size. Notably, among the best-performing models, the most efficient one achieves a 50% reduction in training data and a 70% decrease in training runtime. Our analysis highlights the importance of combining multiple tokens to enhance fine-grained spoken language understanding.
Hearing deficits of transformer-based ASR for anechoic and spatial signals
Authors: Dirk Hoffner; Simon Weihe; Thomas Brand; Bernd T. Meyer
Venue: Interspeech 2025 (main)

Abstract: We compare automatic speech recognition (ASR) with human speech recognition (HSR) based on speech material that is traditionally used for diagnosing hearing deficits. Specifically, we quantify the human-machine gap with sentences in noise for different model sizes and two languages supported by Whisper, e.g., German and English. For German speech, we also determine the gap in different rooms and the presence of reverberation and localized noise. Results are put in context of audiological diagnosis using the speech reception threshold (SRT) (i.e., the SNR with 50% word recognition rate). We find that the largest ASR system is mildly hearing impaired when exposed to non-spatial, unpredictable US-English sentences and that using German speech degrades the SRT by 4.9 dB. In reverberant rooms, the gap reaches at least 5.9 dB. Based on the language bias, we estimate that same model can achieve better performance than normal-hearing listeners in anechoic conditions for US-English.
TELVID: A Multilingual Multi-modal Corpus for Speaker Recognition
Authors: Karen Jones; Kevin Walker; Christopher Caruso; Elliot Singer; Trang Nguyen; Robert Dunn; Stephanie Strassel
Venue: Interspeech 2025 (main)

Abstract: The TELVID corpus is a new multi-language, multi-modal resource for speaker recognition, comprising multiple conversational telephone speech and video recordings from each of 300 multilingual speakers. Consented subjects contributed recordings in a wide variety of recording conditions, with a minimum of 11 calls, 10 videos and one selfie image per person. Every speaker made recordings in Tunisian Arabic, North African French and/or English, along with two âfreestyleâ recordings that utilize the speakerâs choice of any language, dialect or mix of varieties. Recordings were audited to verify quality and speaker identity and portions of the data were selected for test data for the NIST 2024 Speaker Recognition Evaluation. We developed audio and visual baseline systems and measured baseline system performance. The TELVID corpus will be published in the Linguistic Data Consortium Catalog, making it broadly available for language-related research, education and technology development.
EmoSpeechAuth: Emotion-Aware Speaker Verification
Authors: Magdalena GoÅÄbiowska; Piotr Syga
Venue: Interspeech 2025 (main)

Abstract: Speaker verification is a process of verifying the identity of a user. Research has shown that emotional variability in speech degrades the performance of speaker verification tasks. Prior approaches were more computationally expensive and did not focus on the state-of-the-art speaker representations. In this paper, we propose a novel framework for constructing emotional speaker embeddings. Our framework utilizes pre-trained state-of-the-art feature extractors for speaker and emotion recognition, including both speaker and emotional information in the final embeddings. We present results of speaker verification on emotional speech datasets. We show that fusing ECAPA2 speaker representations and emotional features from emotion2vec with a cross-attention module improves EER by 8.29 percentage points compared to the baseline.
The 2024 NIST Speaker Recognition Evaluation
Authors: Craig Greenberg; Lukas Diduch; Audrey Tong; Elliot Singer; Trang Nguyen; Robert Dunn; Lisa Mason; Beth Matys
Venue: Interspeech 2025 (main)

Abstract: The 2024 U.S. National Institute of Standards and Technology (NIST) Speaker Recognition Evaluation (SRE) is the latest in a series of SREs conducted by NIST since 1996. The SRE24 evaluation task was automated person detection and had three evaluation tracks: audio, visual, and audio-visual. New SRE24 features included: variable duration enrollment segments, shorter duration test segments, segments containing multiple persons, and updated cost function parameters. 11 teams consisting of 26 sites participated in SRE24. Evaluation results indicate audio-visual fusion produces significant performance improvements over audio or visual systems. Further audio results analyses indicate: use of unrestricted training data produced significant performance gains; some systems performed better using three 10s segments than three 60s segments for enrollment data, with overall differences being less than expected; and, single model systems performed competitively with the fused multi-model systems.