GenAI4ELab/papercli-papers huggingface dataset:
Dysarthric Speech Recognition Using Curriculum Learning and Multi-stream Architecture
Authors: I-Ting Hsieh; Chung-Hsien Wu
Venue: Interspeech 2025 (main)

Abstract: Dysarthric speech recognition presents significant challenges due to limited resources and the diverse attributes of such speech. Current solutions often involve techniques like data augmentation or domain adaptation. However, applying these methods to a single model frequently results in inconsistent improvements across different speech intelligibility groups. To address this, we propose a novel approach that combines curriculum learning with a multi-stream architecture. Experimental results show the effectiveness of the proposed method, achieving a 10.47% improvement over the baseline.
DYNAC: Dynamic Vocabulary-based Non-Autoregressive Contextualization for Speech Recognition
Authors: Yui Sudo; Yosuke Fukumoto; Muhammad Shakeel; Yifan Peng; Chyi-Jiunn Lin; Shinji Watanabe
Venue: Interspeech 2025 (main)

Abstract: Contextual biasing (CB) improves automatic speech recognition for rare and unseen phrases. Recent studies have introduced dynamic vocabulary, which represents context phrases as expandable tokens in autoregressive (AR) models. This method improves CB accuracy but with slow inference speed. While dynamic vocabulary can be applied to non-autoregressive (NAR) models, such as connectionist temporal classification (CTC), the conditional independence assumption fails to capture dependencies between static and dynamic tokens. This paper proposes DYNAC (Dynamic Vocabulary-based NAR Contextualization), a self-conditioned CTC method that integrates dynamic vocabulary into intermediate layers. Conditioning the encoder on dynamic vocabulary, DYNAC effectively captures dependencies between static and dynamic tokens while reducing the real-time factor (RTF). Experimental results show that DYNAC reduces RTF by 81% with a 0.1-point degradation in word error rate on the LibriSpeech 960 test-clean set.
Beyond Hard Sharing: Efficient Multi-Task Speech-to-Text Modeling with Supervised Mixture of Experts
Authors: Hojun Jin; Eunsoo Hong; Ziwon Hyung; Sungjun Lim; Seungjin Lee; Keunseok Cho
Venue: Interspeech 2025 (main)

Abstract: Hard-parameter sharing is a common strategy to train a single model jointly across diverse tasks. However, this often leads to task interference, impeding overall model performance. To address the issue, we propose a simple yet effective Supervised Mixture of Experts (S-MoE). Unlike traditional Mixture of Experts models, S-MoE eliminates the need for training gating functions by utilizing special guiding tokens to route each task to its designated expert. By assigning each task to a separate feedforward network, S-MoE overcomes the limitations of hard-parameter sharing. We further apply S-MoE to a speech-to-text model, enabling the model to process mixed-bandwidth input while jointly performing automatic speech recognition (ASR) and speech translation (ST). Experimental results demonstrate the effectiveness of the proposed S-MoE, achieving a 6.35% relative improvement in Word Error Rate (WER) when applied to both the encoder and decoder.
OWSM v4: Improving Open Whisper-Style Speech Models via Data Scaling and Cleaning
Authors: Yifan Peng; Muhammad Shakeel; Yui Sudo; William Chen; Jinchuan Tian; Chyi-Jiunn Lin; Shinji Watanabe
Venue: Interspeech 2025 (main)

Abstract: The Open Whisper-style Speech Models (OWSM) project has developed a series of fully open speech foundation models using academic-scale resources, but their training data remains insufficient. This work enhances OWSM by integrating YODAS, a large-scale web-crawled dataset with a Creative Commons license. However, incorporating YODAS is nontrivial due to its wild nature, which introduces challenges such as incorrect language labels and audio-text misalignments. To address this, we develop a scalable data-cleaning pipeline using public toolkits, yielding a dataset with 166,000 hours of speech across 75 languages. Our new series of OWSM v4 models, trained on this curated dataset alongside existing OWSM data, significantly outperform previous versions on multilingual benchmarks. Our models even match or surpass frontier industrial models like Whisper and MMS in multiple scenarios. We will publicly release the cleaned YODAS data, pre-trained models, and all associated scripts via the ESPnet toolkit.
Bona fide Cross Testing Reveals Weak Spot in Audio Deepfake Detection Systems
Authors: Chin Yuen Kwok; Jia Qi Yip; Zhen Qiu; Chi Hung Chi; Kwok Yan Lam
Venue: Interspeech 2025 (main)

Abstract: Audio deepfake detection (ADD) models are commonly evaluated using datasets that combine multiple synthesizers, with performance reported as a single Equal Error Rate (EER). However, this approach disproportionately weights synthesizers with more samples, underrepresenting others and reducing the overall reliability of EER. Additionally, most ADD datasets lack diversity in bona fide speech, often featuring a single environment and speech style (e.g., clean read speech), limiting their ability to simulate real-world conditions. To address these challenges, we propose bona fide cross-testing, a novel evaluation framework that incorporates diverse bona fide datasets and aggregates EERs for more balanced assessments. Our approach improves robustness and interpretability compared to traditional evaluation methods. We benchmark over 150 synthesizers across nine bona fide speech types and release a new dataset to facilitate further research.
BiCrossMamba-ST: Speech Deepfake Detection with Bidirectional Mamba Spectro-Temporal Cross-Attention
Authors: Yassine El Kheir; Tim Polzehl; Sebastian MÃ¶ller
Venue: Interspeech 2025 (main)

Abstract: We propose BiCrossMamba-ST, a robust framework for speech deepfake detection that leverages a dual-branch spectro-temporal architecture powered by bidirectional Mamba blocks and mutual cross-attention. By processing spectral sub-bands and temporal intervals separately and then integrating their representations, BiCrossMamba-ST effectively captures the subtle cues of synthetic speech. In addition, our proposed framework leverages a convolution-based 2D attention map to focus on specific spectro-temporal regions, enabling robust deepfake detection. Operating directly on raw features, BiCrossMamba-ST achieves significant performance improvements, a 67.74% and 26.3% relative gain over state-of-the-art AASIST on ASVSpoof LA21 and ASVSpoof DF21 benchmarks, respectively, and a 6.80% improvement over RawBMamba on ASVSpoof DF21. Code and models is made publicly available.
Few-Shot Speech Deepfake Detection Adaptation with Gaussian Processes
Authors: Neta Glazer; David Chernin; Idan Achituve; Sharon Gannot; Ethan Fetaya
Venue: Interspeech 2025 (main)

Abstract: Recent advancements in Text-to-Speech (TTS) models, particularly in voice cloning, have intensified the demand for adaptable and efficient deepfake detection methods. As TTS systems continue to evolve, detection models must be able to efficiently adapt to previously unseen generation models with minimal data. This paper introduces AADD-GP, a few-shot adaptive framework based on a Gaussian Process (GP) classifier for Audio Deepfake Detection (ADD). We show how the combination of a powerful deep embedding model with the Gaussian processes flexibility can achieve strong performance and adaptability. Additionally, we show this approach can also be used for personalized detection, with greater robustness to new TTS models and one-shot adaptability. To support our evaluation, a benchmark dataset is constructed for this task using new state-of-the-art voice cloning models.
Replay Attacks Against Audio Deepfake Detection
Authors: Nicolas MÃ¼ller; Piotr Kawa; Wei-Herng Choong; Adriana Stan; Aditya Tirumala Bukkapatnam; Karla Pizzi; Alexander Wagner; Philip Sperl
Venue: Interspeech 2025 (main)

Abstract: We show how replay attacks undermine audio deepfake detection: By playing and re-recording deepfake audio through various speakers and microphones, we make spoofed samples appear authentic to the detection model.To study this phenomenon in more detail, we introduce ReplayDF, a dataset of recordings derived from M-AILABS and MLAAD, featuring 109 speaker-microphone combinations across six languages and four TTS models. It includes diverse acoustic conditions, some highly challenging for detection.Our analysis of six open-source detection models across five datasets reveals significant vulnerability, with the top-performing W2V2-AASIST model's Equal Error Rate (EER) surging from 4.7% to 18.2%. Even with adaptive Room Impulse Response (RIR) retraining, performance remains compromised with an 11.0% EER. We release ReplayDF for non-commercial research use.
Enhancing Audio Deepfake Detection by Improving Representation Similarity of Bonafide Speech
Authors: Seung-bin Kim; Hyun-seo Shin; Jungwoo Heo; Chan-yeong Lim; Kyo-Won Koo; Jisoo Son; Sanghyun Hong; Souhwan Jung; Ha-Jin Yu
Venue: Interspeech 2025 (main)

Abstract: The key to audio deepfake detection is distinguishing bonafide speech from carefully generated spoofed speech. The more distinguishable they are, the better and more generalizable the detection becomes. In this work, we propose a novel approach to enhance this distinguishability in the latent space. Inspired by one-class classification, we formulate an objective function that encourages the contraction of bonafide samples while dispersing fake speech samples during training. Our objective consists of two key components: Bonafide-Pair Learning (BPL) loss and an Extended One-Class Softmax (EOC-S) loss. The BPL reduces intra-class variance by aligning the embeddings of augmented bonafide pairs, while the EOC-S leverages Adam-based centroid updates and margin constraints to reinforce separability from spoofed data. Experimental results on ASVspoof datasets demonstrate that our proposed approach enhances detection performance across diverse attack scenarios.
Generalizable Audio Deepfake Detection via Hierarchical Structure Learning and Feature Whitening in PoincarÃ© sphere
Authors: Mingru Yang; Yanmei Gu; Qianhua He; Yanxiong Li; Peirong Zhang; Yongqiang Chen; Zhiming Wang; Huijia Zhu; Jian Liu; Weiqiang Wang
Venue: Interspeech 2025 (main)

Abstract: Audio deepfake detection (ADD) faces critical generalization challenges due to diverse real-world spoofing attacks and domain variations. However, existing methods primarily rely on Euclidean distances, failing to adequately capture the intrinsic hierarchical structures associated with attack categories and domain factors. To address these issues, we design a novel framework Poin-HierNet to construct domain-invariant hierarchical representations in the PoincarÂ´e sphere. Poin-HierNet includes three key components: 1) PoincarÂ´e Prototype Learning (PPL) with several data prototypes aligning sample features and capturing multilevel hierarchies beyond human labels; 2) Hierarchical Structure Learning (HSL) leverages top prototypes to establish a tree-like hierarchical structure from data prototypes; and 3) PoincarÂ´e Feature Whitening (PFW) enhances domain invariance by applying feature whitening to suppress domainsensitive features. We evaluate our approach on four datasets: ASVspoof 2019 LA, ASVspoof 2021 LA, ASVspoof 2021 DF, and In The-Wild. Experimental results demonstrate that Poin-HierNet exceeds state-of-the-art methods in Equal Error Rate.
VoiceNet: Multilingual On-Device Phoneme-To-Audio Alignment
Authors: Kun Jin; Siva Penke; Srinivasa Algubelli
Venue: Interspeech 2025 (main)

Abstract: Phoneme-to-Audio Alignment has many applications and is generally considered as an important task in the lip-sync system where an avatar's lip shape is synchronized with the corresponding speech signal. In this work, we propose such a novel end-to-end on-device multilingual model, VoiceNet, which learns both phoneme recognition and text-independent forced alignment. VoiceNet supports on-device inference in Real-Time as well as in Non Real-time. Moreover, in the Non-RealTime scenario, we show that the performance can be further enhanced when text is given. Our experiments demonstrate competitive performance of VoiceNet compared with state-of-the-art Phoneme Recognition and Forced Alignment results on LibriSpeech and multilingual dataset. Benchmarked on a set of Galaxy phone devices, VoiceNet achieves the average phoneme inference latency of 6ms on CPU, demonstrating the high computational efficiency. Furthermore, VoiceNet can achieve 2x speedup on GPU and 10x speedup on NPU.
Nosey: Open-Source Hardware for Acoustic Nasalance
Authors: Maya Dewhurst; Jack Collins; Justin J. H. Lo; Roy Alderton; Sam Kirkham
Venue: Interspeech 2025 (main)

Abstract: We introduce Nosey (Nasalance Open Source Estimation sYstem), a low-cost, customizable, 3D-printed system for recording acoustic nasalance data that we have made available as open-source hardware (http://github.com/phoneticslab/nosey). We first outline the motivations and design principles behind our hardware nasalance system, and then present a comparison between Nosey and a commercial nasalance device. Nosey shows consistently higher nasalance scores than the commercial device, but the magnitude of contrast between phonological environments is comparable between systems. We also review ways of customizing the hardware to facilitate testing, such as comparison of microphones and different construction materials. We conclude that Nosey is a flexible and cost-effective alternative to commercial nasometry devices and propose some methodological considerations for its use in data collection.
Automatic classification of stop realisation with wav2vec2.0
Authors: James Tanner; Morgan Sonderegger; Jane Stuart-Smith; Jeff Mielke; Tyler Kendall
Venue: Interspeech 2025 (main)

Abstract: Modern phonetic research regularly makes use of automatic tools for the annotation of speech data, however few tools exist for the annotation of many variable phonetic phenomena. At the same time, pre-trained self-supervised models, such as wav2vec2.0, have been shown to perform well at speech classification tasks and latently encode fine-grained phonetic information. We demonstrate that wav2vec2.0 models can be trained to automatically classify stop burst presence with high accuracy in both English and Japanese, robust across both finely-curated and unprepared speech corpora. Patterns of variability in stop realisation are replicated with the automatic annotations, and closely follow those of manual annotations. These results demonstrate the potential of pre-trained speech models as tools for the automatic annotation and processing of speech corpus data, enabling researchers to 'scale-up' the scope of phonetic research with relative ease.
Acquiring Pronunciation from Speech Audio via Multi-task Learning
Authors: Siqi Sun; Korin Richmond
Venue: Interspeech 2025 (main)

Abstract: Recent work has shown the benefit of bootstrapping an integrated sequence-to-sequence (Seq2Seq) linguistic frontend from a traditional pipeline-based frontend for text-to-speech (TTS). To overcome the fixed lexical coverage of bootstrapping training data, previous work has proposed to leverage easily accessible transcribed speech audio as an additional training source for acquiring novel pronunciation knowledge for uncovered words, which relies on cumbersome auxiliary acoustic model training and explicit pronunciation decoding. In this work, we propose an alternative method to leverage transcribed speech audio as an additional training source, based on multi-task learning (MTL). Experiments show that, compared to a baseline Seq2Seq frontend, the proposed MTL-based method improves overall accuracy by 3.7% absolute for those word types covered exclusively in transcribed speech audio, achieving a similar performance to the previous method but with a much simpler implementation flow.
Intelligibility of Text-to-Speech Systems for Mathematical Expressions
Authors: Sujoy Roychowdhury; Ranjani H.G.; Sumit Soman; Nishtha Paul; Subhadip Bandyopadhyay; Siddhanth Iyengar
Venue: Interspeech 2025 (main)

Abstract: There has been limited evaluation of advanced Text-to-Speech (TTS) models with Mathematical eXpressions (MX) as inputs. In this work, we design experiments to evaluate quality and intelligibility of five TTS models through listening and transcribing tests for various categories of MX. We use two Large Language Models (LLMs) to generate English pronunciation from LaTeX MX as TTS models cannot process LaTeX directly. We use Mean Opinion Score from user ratings and quantify intelligibility through transcription correctness using three metrics. We also compare listener preference of TTS outputs with respect to human expert rendition of same MX. Results establish that output of TTS models for MX is not necessarily intelligible, the gap in intelligibility varies across TTS models and MX category. For most categories, performance of TTS models is significantly worse than that of expert rendition. The effect of choice of LLM is limited. This establishes the need to improve TTS models for MX.
The State Of TTS: A Case Study with Human Fooling Rates
Authors: Praveen Srinivasa Varadhan; Sherry Thomas; Sai Teja M S; Suvrat Bhooshan; Mitesh M. Khapra
Venue: Interspeech 2025 (main)

Abstract: While subjective evaluations in recent years indicate rapid progress in TTS, can current TTS systems truly pass a human deception test in a Turing-like evaluation? We introduce Human Fooling Rate (HFR), a metric that directly measures how often machine-generated speech is mistaken for human. Our large-scale evaluation of open-source and commercial TTS models reveals critical insights: (i) CMOS-based claims of human parity often fail under deception testing, (ii) TTS progress should be benchmarked on datasets where human speech achieves high HFRs, as evaluating against monotonous or less expressive reference samples sets a low bar, (iii) Commercial models approach human deception in zero-shot settings, while open-source systems still struggle with natural conversational speech; (iv) Fine-tuning on high-quality data improves realism but does not fully bridge the gap. Our findings underscore the need for more realistic, human-centric evaluations alongside existing subjective tests.
Pairwise Evaluation of Accent Similarity in Speech Synthesis
Authors: Jinzuomu Zhong; Suyuan Liu; Dan Wells; Korin Richmond
Venue: Interspeech 2025 (main)

Abstract: Despite growing interest in generating high-fidelity accents, evaluating accent similarity in speech synthesis has been underexplored. We aim to enhance both subjective and objective evaluation methods for accent similarity. Subjectively, we refine the XAB listening test by adding components that achieve higher statistical significance with fewer listeners and lower costs. Our method involves providing listeners with transcriptions, having them highlight perceived accent differences, and implementing meticulous screening for reliability. Objectively, we utilise pronunciation-related metrics, based on distances between vowel formants and phonetic posteriorgrams, to evaluate accent generation. Comparative experiments reveal that these metrics, alongside accent similarity, speaker similarity, and Mel Cepstral Distortion, can be used. Moreover, our findings underscore significant limitations of common metrics like Word Error Rate in assessing underrepresented accents.
VoiceQualityVC: A Voice Conversion System for Studying the Perceptual Effects of Voice Quality in Speech
Authors: Harm Lameris; Joakim Gustafsson; Ãva SzÃ©kely
Venue: Interspeech 2025 (main)

Abstract: Voice quality is an often overlooked aspect of speech with many communicative functions. Voice quality conveys both paralinguistic and pragmatic information, such as signalling speaker stance and aids in grounding. In this paper, we present VoiceQualityVC, a tool that can manipulate the voice quality of both natural and synthesized speech using voice quality features including CPPS, H1-H2, and H1-A3. VoiceQualityVC is a research tool for perceptual experiments into voice quality and UX experiments for voice design. We perform an objective evaluation demonstrating the control of these features as well as subjective listening tests of the paralinguistic attributes of intimacy, valence, and investment. In these listening tests breathy voice was rated as more intimate and more invested than modal voice and creaky voice was rated as less intimate and less positive. The code and models can be found at https://github.com/Hfkml/VQVC.
Towards Frame-level Quality Predictions of Synthetic Speech
Authors: Michael Kuhlmann; Fritz Seebauer; Petra Wagner; Reinhold Haeb-Umbach
Venue: Interspeech 2025 (main)

Abstract: While automatic subjective speech quality assessment has witnessed much progress, an open question is whether an automatic quality assessment at frame resolution is possible. This would be highly desirable, as it adds explainability to the assessment of speech synthesis systems. Here, we take first steps towards this goal by identifying issues of existing quality predictors that prevent sensible frame-level prediction. Further, we define criteria that a frame-level predictor should fulfill. We also suggest a chunk-based processing that avoids the impact of a localized distortion on the score of neighboring frames. Finally, we measure in experiments with localized artificial distortions the localization performance of a set of frame-level quality predictors and show that they can outperform detection performance of human annotations obtained from a crowd-sourced perception experiment.
Perception of Long and Short Vowel Contrast in Te Reo MÄori in Clean and Everyday Listening Environments
Authors: C. T. Justine Hui; Jenice Kuzhikombil; Isabella Shields; Hiraia Haami-Wells; Catherine I. Watson; Peter [J.] Keegan
Venue: Interspeech 2025 (main)

Abstract: Te reo MÄori (the MÄori language) is the language of the indigenous people in New Zealand and has a long-short vowel contrast. This study first investigates the cues used to perceive vowel length for MÄori fluent users and learners. Secondly, it explores the effect of an everyday listening environment in the form of a traditional meeting house (wharenui) on cue-weighting between duration and stress for identifying long and short vowels when the stimuli are rendered with the wharenui room acoustics.  An identification test was carried out with three pairs of words that differed in stress location and were manipulated in vowel duration. We found both groups to mainly use duration as a cue, even though the advanced listeners commented that they were listening for 'intonation'. For the wharenui acoustics, there were no differences observed between the groups and only a small categorical shift for the /a:/ vowel for the learners.
The function of creaky voice in South Korean: A perception study
Authors: Patrik HrabÃ¡nek; Michaela Watkins; Silke Hamann
Venue: Interspeech 2025 (main)

Abstract: This study examines whether creaky voice serves as a perceptual cue for the fortis-lenis word-initial stop distinction in Standard Seoul Korean. Twenty-nine native speakers completed an online forced-choice ABX task, where they determined whether a manipulated, artificially creaky lenis token (Sound X) with ambiguous values of VOT and F0 resembled more closely a natural fortis (Sound A) or a natural lenis token (Sound B). Results showed considerable inter-speaker variation: some rarely categorized creaky lenis tokens as fortis, while others did so up to 15% of the time. Statistical analysis showed that presence of creak in Sound X did not change the perceptual classification significantly, suggesting creak is not a primary cue alongside VOT and F0. Word-specific effects also emerged, highlighting the complexity of perceptual cues in Korean stop contrasts.