GenAI4ELab/papercli-papers huggingface dataset:
On the Relationship between Accent Strength and Articulatory Features
Authors: Kevin Huang; Sean Foley; Jihwan Lee; Yoonjeong Lee; Dani Byrd; Shrikanth Narayanan
Venue: Interspeech 2025 (main)

Abstract: This paper explores the relationship between accent strength and articulatory features inferred from acoustic speech. To quantify accent strength, we compare phonetic transcriptions with transcriptions based on dictionary-based references, computing phoneme-level difference as a measure of accent strength. The proposed framework leverages recent self-supervised learning articulatory inversion techniques to estimate articulatory features. Analyzing a corpus of read speech from American and British English speakers, this study examines correlations between derived articulatory parameters and accent strength proxies, associating systematic articulatory differences with indexed accent strength. Results indicate that tongue positioning patterns distinguish the two dialects, with notable differences inter-dialects in rhotic and low back vowels. These findings contribute to automated accent analysis and articulatory modeling for speech processing applications.
A Multi-Stream Framework Utilizing 3D Human Reconstruction for Cued Speech Recognition
Authors: Katerina Papadimitriou; Gerasimos Potamianos
Venue: Interspeech 2025 (main)

Abstract: In this paper, we propose a novel multi-stream framework for automatic cued speech recognition (ACSR) that directly processes the upper-body region, addressing hand-lip asynchrony without requiring explicit segmentation or synchronization. Our model integrates two distinct modalities: (i) an appearance-based stream leveraging the ResNet18 for feature extraction and (ii) a skeletal stream based on a modulated graph convolutional network (GCN). For graph construction, we incorporate, for the first time in ACSR, 3D pose parameters inferred from the PIXIE model. Both modalities are coupled with temporal convolution for short-range dynamics learning and a BiGRU encoder for long-term sequence modeling. In addition, we introduce an alignment module that combines CTC with two auxiliary losses, improving each modality performance and enabling effective late fusion during inference. Our model achieves state-of-the-art performance across three benchmark datasets, demonstrating its effectiveness.
On the cross-modal makeup of charisma: Insights from a field-data analysis
Authors: Oliver Niebuhr
Venue: Interspeech 2025 (main)

Abstract: This study uses expert ratings and prosodic analyses of the most popular 2025 video speeches of the German DAX-40 CEOs to investigate the interplay of visual, verbal, and prosodic factors in perceiving speaker charisma. Results show strong correlations between all factors, point to a special role of prosody, and provide initial evidence for industry-specific speaking styles.
Generalizable Audio Spoofing Detection using Non-Semantic Representations
Authors: Arnab Das; Yassine El Kheir; Carlos Franzreb; Tim Herzig; Tim Polzehl; Sebastian MÃ¶ller
Venue: Interspeech 2025 (main)

Abstract: Rapid advancements in generative modeling have made synthetic audio generation easy, making speech-based services vulnerable to spoofing attacks. Consequently, there is a dire need for robust countermeasures more than ever. Existing solutions for deepfake detection are often criticized for lacking generalizability and fail drastically when applied to real-world data. This study proposes a novel method for generalizable spoofing detection leveraging non-semantic universal audio representations. Extensive experiments have been performed to find suitable non-semantic features using TRILL and TRILLsson models. The results indicate that the proposed method achieves comparable performance on the in-domain test set while significantly outperforming state-of-the-art approaches on out-of-domain test sets. Notably, it demonstrates superior generalization on public-domain data, surpassing methods based on hand-crafted features, semantic embeddings, and end-to-end architectures.
Adversarial Attacks on Text-dependent Speaker Verification System
Authors: Sreekanth Sankala; Venkatesh Parvathala; Ramesh Gundluru; K. Sri Rama Murty
Venue: Interspeech 2025 (main)

Abstract: Adversarial attacks against text-independent speaker verification (TI-SV) systems assume access to genuine speaker's enrollment speech (e[n]). This assumption is self-defeating because if an attacker has e[n], they can bypass the system directly, making adversarial examples unnecessary. In contrast, we observe that the text-dependent SV (TD-SV) system, where the genuine speaker must say a password, offers a more practically relevant attack scenario. In reality, the attacker may not have access to the password spoken by a genuine speaker, but they can likely obtain normal speech from the genuine speaker. Therefore, generating adversarial noise that, when added to the genuine speaker's normal speech, can bypass the password requirement of a TD-SV system constitutes a potential realistic attack. This work investigates the feasibility of such a practical attack and shows that even the state-of-the-art TD-SV system is vulnerable with an attack success rate of 64.28 %.
Beyond Attacks: Advancing Fake Speech Detection with Attack-Agnostic Methods
Authors: Shilpa Chandra; Akansha Tyagi; Shiven Patel; Padmanabhan Rajan
Venue: Interspeech 2025 (main)

Abstract: Detectors of fake speech are prone to poor performance when presented with data unseen during training. Unseen data can include not only new methods for generating fake speech and various transformations applied to fake speech, but also fake speech in new languages. This study investigates two methods for improving generalization capability of detectors that use a self supervised model as a front-end. The first method uses an encoder-decoder architecture to learn representations robust to different types of fake speech. The second method uses a subspace-based decomposition and learns common information present in various types of fake speech.  The proposed methods are evaluated on the standard ASVspoof 2021 dataset, as well as on a multilingual speech dataset of Indic languages. Experiments reveal that state-of-the-art detectors struggle with speech in unseen languages, and the proposed generalization methods help in reducing the performance gap.
ASVspoof2019 vs. ASVspoof5: Assessment and Comparison
Authors: Avishai Weizman; Yehuda Ben-Shimol; Itshak Lapidot
Venue: Interspeech 2025 (main)

Abstract: ASVspoof challenges are designed to advance the understanding of spoofing speech attacks and encourage the development of robust countermeasure systems. These challenges provide a standardized database for assessing and comparing spoofing-robust automatic speaker verification solutions. The ASVspoof5 challenge introduces a shift in database conditions compared to ASVspoof2019. While ASVspoof2019 has mismatched conditions only in spoofing attacks in the evaluation set, ASVspoof5 incorporates mismatches in both bonafide and spoofed speech statistics. This paper examines the impact of these mismatches, presenting qualitative and quantitative comparisons within and between the two databases. We show the increased difficulty for genuine and spoofed speech and demonstrate that in ASVspoof5, not only are the attacks more challenging, but the genuine speech also shifts toward spoofed speech compared to ASVspoof2019.
Evaluating Parameter Sharing for Spoofing-Aware Speaker Verification: A Case Study on the ASVspoof 5 Dataset
Authors: Aykut BÃ¼ker; OÄuzhan Kurnaz; Åule BekiryazÄ±cÄ±; Selim Can DemirtaÅ; Cemal HanilÃ§i
Venue: Interspeech 2025 (main)

Abstract: Spoofing-aware speaker verification (SASV) is an important but challenging task and has been a primary focus of the recently organized ASVspoof 5 challenge. As SASV integrates automatic speaker verification (ASV) and countermeasure (CM) systems, its performance depends on the effectiveness of each system. This study systematically examines the impact of different parameter-sharing (PS) strategies, which facilitate joint optimization, on SASV performance using the ASVspoof 5 dataset. Experimental results indicate that PS enhances performance for specific attack types and codec conditions. For example, the baseline system achieves a min a-DCF of 0.329 on the A26 attack, which improves to 0.233 with PS. Similarly, for AMR-compressed signals, PS yields a 14.09% performance gain. These observations show that PS techniques are effective in mitigating certain spoofing attacks and improving robustness to degraded audio conditions in SASV systems.
Can Quantized Audio Language Models Perform Zero-Shot Spoofing Detection?
Authors: Bikash Dutta; Rishabh Ranjan; Shyam Sathvik; Mayank Vatsa; Richa Singh
Venue: Interspeech 2025 (main)

Abstract: Quantization is essential for deploying large audio language models (LALMs) efficiently in resource-constrained environments. However, its impact on complex tasks, such as zero-shot audio spoofing detection, remains underexplored. This study evaluates the zero-shot capabilities of five LALMs, GAMA, LTU-AS, MERaLiON, Qwen-Audio, and SALMONN, across three distinct datasets: ASVspoof2019, In-the-Wild, and WaveFake, and investigates their robustness to quantization (FP32, FP16, INT8). Despite high initial spoof detection accuracy, our analysis demonstrates severe predictive biases toward spoof classification across all models, rendering their practical performance equivalent to random classification. Interestingly, quantization to FP16 precision resulted in negligible performance degradation compared to FP32, effectively halving memory and computational requirements without materially impacting accuracy. However, INT8 quantization intensified model biases, significantly degrading balanced accuracy. These findings highlight critical architectural limitations and emphasize FP16 quantization as an optimal trade-off, providing guidelines for practical deployment and future model refinement.
ClapFM-EVC: High-Fidelity and Flexible Emotional Voice Conversion with Dual Control from Natural Language and Speech
Authors: Yu Pan; Yanni Hu; Yuguang Yang; Jixun Yao; Jianhao Ye; Hongbin Zhou; Lei Ma; Jianjun Zhao
Venue: Interspeech 2025 (main)

Abstract: Despite great advances, achieving high-fidelity emotional voice conversion (EVC) with flexible and interpretable control remains challenging. This paper introduces ClapFM-EVC, a novel EVC framework capable of generating high-quality converted speech driven by natural language prompts or reference speech with adjustable emotion intensity. We first propose EVC-CLAP, an emotional contrastive language-audio pre-training model, guided by natural language prompts and categorical labels, to extract and align fine-grained emotional elements across speech and text modalities. Then, a FuEncoder with an adaptive intensity gate is presented to seamless fuse emotional features with Phonetic PosteriorGrams from a pre-trained ASR model. To further improve emotion expressiveness and speech naturalness, we propose a flow matching model conditioned on these captured features to reconstruct Mel-spectrogram of source speech. Subjective and objective evaluations validate the effectiveness of ClapFM-EVC.
PromptEVC: Controllable Emotional Voice Conversion with Natural Language Prompts
Authors: Tianhua Qi; Shiyan Wang; Cheng Lu; Tengfei Song; Hao Yang; Zhanglin Wu; Wenming Zheng
Venue: Interspeech 2025 (main)

Abstract: Controllable emotional voice conversion (EVC) aims to manipulate emotional expressions to increase the diversity of synthesized speech. Existing methods typically rely on predefined labels, reference audios, or prespecified factor values, often overlooking individual differences in emotion perception and expression. In this paper, we introduce PromptEVC that utilizes natural language prompts for precise and flexible emotion control. To bridge text descriptions with emotional speech, we propose emotion descriptor and prompt mapper to generate fine-grained emotion embeddings, trained jointly with reference embeddings. To enhance naturalness, we present a prosody modeling and control pipeline that adjusts the rhythm based on linguistic content and emotional cues. Additionally, a speaker encoder is incorporated to preserve identity. Experimental results demonstrate that PromptEVC outperforms state-of-the-art controllable EVC methods in emotion conversion, intensity control, mixed emotion synthesis, and prosody manipulation.
StarVC: A Unified Auto-Regressive Framework for Joint Text and Speech Generation in Voice Conversion
Authors: Fengjin Li; Jie Wang; Yadong Niu; Yongqing Wang; Meng Meng; Jian Luan; Zhiyong Wu
Venue: Interspeech 2025 (main)

Abstract: Voice Conversion (VC) modifies speech to match a target speaker while preserving linguistic content. Traditional methods usually extract speaker information directly from speech while neglecting the explicit utilization of linguistic content. Since VC fundamentally involves disentangling speaker identity from linguistic content, leveraging structured semantic features could enhance conversion performance. However, previous attempts to incorporate semantic features into VC have shown limited effectiveness, motivating the integration of explicit text modeling. We propose StarVC, a unified autoregressive VC framework that first predicts text tokens before synthesizing acoustic features. The experiments demonstrate that StarVC outperforms conventional VC methods in preserving both linguistic content (i.e., WER and CER) and speaker characteristics (i.e., SECS and MOS).
FasterVoiceGrad: Faster One-step Diffusion-Based Voice Conversion with Adversarial Diffusion Conversion Distillation
Authors: Takuhiro Kaneko; Hirokazu Kameoka; Kou Tanaka; Yuto Kondo
Venue: Interspeech 2025 (main)

Abstract: A diffusion-based voice conversion (VC) model (e.g., VoiceGrad) can achieve high speech quality and speaker similarity; however, its conversion process is slow owing to iterative sampling. FastVoiceGrad overcomes this limitation by distilling VoiceGrad into a one-step diffusion model. However, it still requires a computationally intensive content encoder to disentangle the speaker's identity and content, which slows conversion. Therefore, we propose FasterVoiceGrad, a novel one-step diffusion-based VC model obtained by simultaneously distilling a diffusion model and content encoder using adversarial diffusion conversion distillation (ADCD), where distillation is performed in the conversion process while leveraging adversarial and score distillation training. Experimental evaluations of one-shot VC demonstrated that FasterVoiceGrad achieves competitive VC performance compared to FastVoiceGrad, with 6.6-6.9 and 1.8 times faster speed on a GPU and CPU, respectively.
Evaluating the Effectiveness of Pre-Trained Audio Embeddings for Classification of Parkinson's Disease Speech Data
Authors: Emmy Postma; Cristian Tejedor-Garcia
Venue: Interspeech 2025 (main)

Abstract: Speech impairments are prevalent biomarkers for Parkinson's Disease (PD), motivating the development of diagnostic techniques using speech data for clinical applications. Although deep acoustic features have shown promise for PD classification, their effectiveness often varies due to individual speaker differences, a factor that has not been thoroughly explored in the existing literature. This study investigates the effectiveness of three pre-trained audio embeddings (OpenL3, VGGish and Wav2Vec2.0 models) for PD classification. Using the NeuroVoz dataset, OpenL3 outperforms others in diadochokinesis (DDK) and listen and repeat (LR) tasks, capturing critical acoustic features for PD detection. Only Wav2Vec2.0 shows significant gender bias, achieving more favorable results for male speakers, in DDK tasks. The misclassified cases reveal challenges with atypical speech patterns, highlighting the need for improved feature extraction and model robustness in PD detection.
On-the-fly Routing for Zero-shot MoE Speaker Adaptation of Speech Foundation Models for Dysarthric Speech Recognition
Authors: Shujie Hu; Xurong Xie; Mengzhe Geng; Jiajun Deng; Huimeng Wang; Guinan Li; Chengxi Deng; Tianzi Wang; Mingyu Cui; Helen Meng; Xunying Liu
Venue: Interspeech 2025 (main)

Abstract: This paper proposes a novel MoE-based speaker adaptation framework for foundation models based dysarthric speech recognition. This approach enables zero-shot adaptation and real-time processing while incorporating domain knowledge. Speech impairment severity and gender conditioned adapter experts are dynamically combined using on-the-fly predicted speaker-dependent routing parameters. KL-divergence is used to further enforce diversity among experts and their generalization to unseen speakers. Experimental results on the UASpeech corpus suggest that on-the-fly MoE-based adaptation produces statistically significant WER reductions of up to 1.34% absolute (6.36% relative) over the unadapted baseline HuBERT/WavLM models. Consistent WER reductions of up to 2.55% absolute (11.44% relative) and RTF speedups of up to 7 times are obtained over batch-mode adaptation across varying speaker-level data quantities. The lowest published WER of 16.35% (46.77% on very low intelligibility) is obtained.
Lightweight Speech Enhancement for Mandarin Esophageal Speech
Authors: Jia-Jyu Su; Yen-Ting Lin; Wu-Hao Li; Chao-Kai Chang; Yan-Zhi Chen; Chen-Yu Chiang
Venue: Interspeech 2025 (main)

Abstract: Esophageal speech is an alternative speech production method for people who have undergone laryngectomy and often suffer from reduced intelligibility. This paper proposes three lightweight speech enhancement methods trained with the loss given by end-to-end automatic speech recognition models. The proposed methods are based on frame mask (FM), ideal ratio mask (IRM), and voice conversion (VC) techniques. The evaluations show that the enhanced speech produced by the proposed methods led to an improvement over the original esophagus speech, specifically in terms of speech recognition rates by automatic speech recognition systems and human evaluators, naturalness as assessed by mean opinion scores, and more detected voicing segments.
VocalAgent: Large Language Models for Vocal Health Diagnostics with Safety-Aware Evaluation
Authors: Yubin Kim; Taehan Kim; Wonjune Kang; Eugene Park; Joonsik Yoon; Dongjae Lee; Xin Liu; Daniel McDuff; Hyeonhoon Lee; Cynthia Breazeal; Hae Won Park
Venue: Interspeech 2025 (main)

Abstract: Vocal health plays a crucial role in peoples' lives, significantly impacting their communicative abilities and interactions. However, despite the global prevalence of voice disorders, many lack access to convenient diagnosis and treatment. This paper introduces VocalAgent, an audio large language model (LLM) to address these challenges through vocal health diagnosis. We leverage Qwen-Audio-Chat fine-tuned on three datasets collected in-situ from hospital patients, and present a multifaceted evaluation framework encompassing a safety assessment to mitigate diagnostic biases, cross-lingual performance analysis, and modality ablation studies. VocalAgent demonstrates superior accuracy on voice disorder classification compared to state-of-the-art baselines. Its LLM-based  method offers a scalable solution for broader adoption of health diagnostics, while underscoring the importance of ethical and technical validation.
A Cookbook for Community-driven Data Collection of Impaired Speech in Low-Resource Languages
Authors: Sumaya Ahmed Salihs; Isaac Wiafe; Jamal-Deen Abdulai; Elikem Doe Atsakpo; Gifty Ayoka; Richard Cave; Akon Obu Ekpezu; Catherine Holloway; Katrin Tomanek; Fiifi Baffoe Payin Winful
Venue: Interspeech 2025 (main)

Abstract: This study presents an approach for collecting speech samples to build Automatic Speech Recognition (ASR) models for impaired speech, particularly, low-resource languages. The project aims to democratize ASR technology and data collection by developing a "cookbook" of best practices and training for community-driven data collection and ASR model building. As a proof-of-concept, this project created the first open-source dataset of impaired speech in Akan: a widely spoken indigenous language in Ghana. The project involved participants from diverse backgrounds with speech impairments. The resulting dataset, along with the cookbook and open-source tools, are publicly available to enable researchers and practitioners to create inclusive ASR technologies tailored to the unique needs of speech impaired individuals. In addition, this study presents the initial results of tuning open-source ASR models to better recognize impaired speech in Akan.
Voice Quality Dimensions as Interpretable Primitives for Speaking Style for Atypical Speech and Affect
Authors: Jaya Narain; Vasudha Kowtha; Colin Lea; Lauren Tooley; Dianna Yee; Vikramjit Mitra; Zifang Huang; Miquel Espi Marques; Jon Huang; Carlos Avendano; Shirley Ren
Venue: Interspeech 2025 (main)

Abstract: Perceptual voice quality dimensions describe key characteristics of atypical speech and other speech modulations. Here we develop and evaluate voice quality models for seven voice and speech dimensions (intelligibility, imprecise consonants, harsh voice, naturalness, monoloudness, monopitch, and breathiness). Probes were trained on the public Speech Accessibility (SAP) project dataset with 11,184 samples from 434 speakers, using embeddings from frozen pre-trained models as features. We found that our probes had both strong performance and strong generalization across speech elicitation categories in the SAP dataset. We further validated zero-shot performance on additional datasets, encompassing unseen languages and tasks: Italian atypical speech, English atypical speech, and affective speech. The strong zero-shot performance and the interpretability of results across an array of evaluations suggests the utility of using voice quality dimensions in speaking style-related tasks.
Towards LLM-Empowered Fine-Grained Speech Descriptors for Explainable Emotion Recognition
Authors: Youjun Chen; Xurong Xie; Haoning Xu; Mengzhe Geng; Guinan Li; Chengxi Deng; Huimeng Wang; Shujie Hu; Xunying Liu
Venue: Interspeech 2025 (main)

Abstract: This paper presents a novel end-to-end LLM-empowered explainable speech emotion recognition (SER) approach. Fine-grained speech emotion descriptor (SED) features, e.g., pitch, tone and emphasis, are disentangled from HuBERT SSL representations via alternating LLM fine-tuning to joint SER-SED prediction and ASR tasks. VAE compressed HuBERT features derived via Information Bottleneck (IB) are used to adjust feature granularity. Experiments on the IEMOCAP and MELD benchmarks demonstrate that our approach consistently out performs comparable LLaMA-based SER baselines, including those using either (a) alternating multi-task fine-tuning alone or (b) feature disentanglement only. Statistically significant increase of SER unweighted accuracy by up to 4.0% and 3.7% absolute (5.4% and 6.6% relative) are obtained. More importantly, emotion descriptors offer further explainability for SER.
From Pretraining to Performance: Benchmarking Self-Supervised Speech Models for Interspeech-25 SER Challenge
Authors: Drishya Uniyal; Vinayak Abrol
Venue: Interspeech 2025 (main)

Abstract: Speech Emotion Recognition (SER) in naturalistic conditions remains a challenging task due to the variability of emotional expression and class imbalances in the real world. As part of the Interspeech-25 SER challenge, we benchmark state-of-the-art large-scale self-supervised speech models on the MSP-Podcast corpus. To extract rich and expressive representations, we systematically investigate fine-tuning strategies, loss functions tailored to mitigate class imbalance, and pre-trained encoder layer freezing techniques to optimize performance. Our findings highlight the impact of these design choices on model robustness and generalization, offering practical guidance for developing SER systems that excel in real-world scenarios.