GenAI4ELab/papercli-papers huggingface dataset:
LASPA: Language Agnostic Speaker Disentanglement with Prefix-Tuned Cross-Attention
Authors: Aditya Srinivas Menon; Raj Prakash Gohil; Kumud Tripathi; Pankaj Wasnik
Venue: Interspeech 2025 (main)

Abstract: Speaker recognition models face challenges in multilingual settings due to the entanglement of linguistic information within speaker embeddings. The overlap between vocal traits such as accent, vocal anatomy, and a languageâs phonetic structure complicates separating linguistic and speaker information. Disentangling these components can significantly improve speaker recognition accuracy. To this end, we propose a novel disentanglement learning strategy that integrates joint learning through prefix-tuned cross-attention. This approach is particularly effective when speakers switch between languages. Experimental results show the model generalizes across monolingual and multi-lingual settings, including unseen languages. Notably, the proposed model improves the equal error rate across multiple datasets, highlighting its ability to separate language information from speaker embeddings and enhance recognition in diverse linguistic conditions.
SCD-Conformer: Semantic Content Disentanglement for Text-Independent Speaker Verification
Authors: Shanshan Yao; Dianlong Liu; Tian Li
Venue: Interspeech 2025 (main)

Abstract: Text-independent speaker verification (TISV) identifies a specific speaker without relying on any particular semantic content. In order to eliminate the influence of semantic content in utterances on speaker feature, we propose a SCD-Conformer for semantic content disentanglement. Firstly, a dual-branch Conformer is used to extract the speaker feature and semantic content feature respectively, in which the content feature is directly extracted by a pre-trained model without increasing training parameters and computational complexity. Then, both the frame-level and utterance-level disentanglement methods are used for disentangling the speaker feature and content feature, including dimension matching module, aggregation module and similarity module. Experimental results show that disentangling at utterance level is more effective than that at frame level, whereas the combination of the two is the best, which averagely improved the performance by 11% compared to the best baseline.
Universal Semantic Disentangled Privacy-preserving Speech Representation Learning
Authors: Biel Tura-Vecino; Subhadeep Maji; Aravind Varier; Antonio Bonafonte; Ivan Valles; Michael Owen; Constantinos Papayiannis; Leif Radel; Grant Strimel; Oluwaseyi Feyisetan; Roberto Barra-Chicote; Ariya Rastrow; Volker Leutnant; Trevor Wood
Venue: Interspeech 2025 (main)

Abstract: The use of human speech to train LLMs poses privacy concerns due to these models' ability to generate samples that closely resemble artifacts in the training data. We propose a speaker privacy-preserving representation learning method through the Universal Speech Codec (USC), a computationally efficient codec that disentangles speech into: (i) privacy-preserving semantically rich representations, capturing content and speech paralinguistics, and (ii) residual acoustic and speaker representations that enable high-fidelity reconstruction. Evaluations show that USC's semantic representation preserves content, prosody, and sentiment, while removing identifiable traits. Additionally, we present an evaluation methodology for measuring privacy-preserving properties. We compare USC against other speech codecs and demonstrate its effectiveness on privacy-preserving representation learning, showcasing the trade-offs between speaker anonymization and paralinguistics retention.1
Using gender, phonation and age to interpret automatically discovered speech attributes for explainable speaker recognition
Authors: Carole Millot; Clara Ponchard; CÃ©dric Gendrot; Jean-FranÃ§ois Bonastre; Orane Dufour
Venue: Interspeech 2025 (main)

Abstract: Explainability is particularly necessary for speaker recognition because of its potential forensic applications. A recent approach, BA-LR, offers a new level of explainability. It represents a speech utterance by a binary vector where each coefficient indicates the presence or absence of a given speech attribute. However, the interpretation of these attributes remains unclear, as they are found automatically. The aim of this work is to develop a methodology for interpreting these attributes in terms of comprehensible explicit voice traits such as gender, age and perceived phonation type. Our methodology is based on the assumption that if an attribute is useful for a classification task of an explicit voice trait, this means that this attribute encodes all or part of this trait. Our results show that BA-LR attributes encode, at least partially, gender, age and perceived phonation type. These results pave the way for a comprehensive interpretation of BA-LR speech attributes.
Do you read me? - flow of speech effect on speaker recognition systems
Authors: Alicja Martinek; Joanna Gajewska; Ewelina Bartuzi-Trokielewicz
Venue: Interspeech 2025 (main)

Abstract: Comparing two types of speech â read and spontaneous â poses a significant challenge for speaker verification models. This study examines the impact of these differences on the performance of advanced biometric systems. We conducted tests using two baseline speaker verification models and two spoof-aware approaches to assess their ability to handle variations between read and spontaneous speech. Additionally, we generated synthetic speech using two state-of-the-art Text-to-Speech methods, training the models either on spontaneous or read speech. The results indicate that mixing spontaneous and read speech compared with uniform type of speech yields higher error rates in biometric verification. The situation is similar in both testing scenarios, comparing genuines with impostors and genuines with synthetic speech, regardless of the type of speaker recognition model â base or spoof-aware.
VoxAging: Continuously Tracking Speaker Aging with a Large-Scale Longitudinal Dataset in English and Mandarin
Authors: Zhiqi Ai; Meixuan Bao; Zhiyong Chen; Zhi Yang; Xinnuo Li; Shugong Xu
Venue: Interspeech 2025 (main)

Abstract: The performance of speaker verification systems is adversely affected by speaker aging. However, due to challenges in data collection, particularly the lack of sustained and large-scale longitudinal data for individuals, research on speaker aging remains difficult. In this paper, we present VoxAging, a large-scale longitudinal dataset collected from 293 speakers (226 English speakers and 67 Mandarin speakers) over several years, with the longest time span reaching 17 years (approximately 900 weeks). For each speaker, the data were recorded at weekly intervals. We studied the phenomenon of speaker aging and its effects on advanced speaker verification systems, analyzed individual speaker aging processes, and explored the impact of factors such as age group and gender on speaker aging research.
LLM-based Generative Error Correction for Rare Words with Synthetic Data and Phonetic Context
Authors: Natsuo Yamashita; Masaaki Yamamoto; Hiroaki Kokubo; Yohei Kawaguchi
Venue: Interspeech 2025 (main)

Abstract: Generative error correction (GER) with large language models (LLMs) has emerged as an effective post-processing approach to improve automatic speech recognition (ASR) performance. However, it often struggles with rare or domain-specific words due to limited training data. Furthermore, existing LLM-based GER approaches primarily rely on textual information, neglecting phonetic cues, which leads to over-correction. To address these issues, we propose a novel LLM-based GER approach that targets rare words and incorporates phonetic information. First, we generate synthetic data to contain rare words for fine-tuning the GER model. Second, we integrate ASR's N-best hypotheses along with phonetic context to mitigate over-correction. Experimental results show that our method not only improves the correction of rare words but also reduces the WER and CER across both English and Japanese datasets.
ASR Confidence Estimation using True Class Lexical Similarity Score
Authors: Nagarathna Ravi; Thishyan Raj T; Ravi Teja Chaganti; Vipul Arora
Venue: Interspeech 2025 (main)

Abstract: Deep Neural Networks (DNN) often exhibit overconfidence, leading to poor confidence calibration in Automatic Speech Recognition (ASR) models. State-Of-The-Art (SOTA) approaches to estimate confidence are based on statistical measures or auxiliary models trained in supervised way using binary target scores, which however, fail to capture the degree of errors in substituted outputs. Continuous target score uses temporal alignment between predictions and ground truth, but are prone to inaccurate temporal alignment. To address these limitations, we propose a novel target score, True Class Lexical Similarity (TruCLeS), to train the auxiliary Confidence Estimation Model (CEM). TruCLeS is based on true class probability and lexical similarity between the prediction and ground truth. Experiments with CTC and RNN-Transducer based ASR models support its superiority against SOTA approaches. The codes are available.
Semi-Supervised Learning for Automatic Speech Recognition with Word Error Rate Estimation and Targeted Domain Data Selection
Authors: Chanho Park; Thomas Hain
Venue: Interspeech 2025 (main)

Abstract: There is a growing demand for leveraging untranscribed multi-domain data in semi-supervised learning (SSL) for automatic speech recognition (ASR) to broaden its applications. However, domain mismatch between source and target data can limit SSLâs performance gains, even when transcript accuracy for training is high. While word error rate (WER) estimation (WE) methods for automatic transcription have advanced, they remain insufficient for handling multi-domain data.This paper proposes a novel data selection method for SSL in ASR that integrates WE and acoustic domain similarity (ADS). For WE, multi-target regression for error rate prediction (MTR-ER) is introduced, while ADS is incorporated as a selection criterion, measured using noise-contrastive estimation. The effectiveness of this approach is demonstrated through comparisons with a confidence-based method. Results show that combining WE and ADS achieves 26.66% of the expected performance improvement of fully supervised learning.
Voice Activity-based Text Segmentation for ASR Text Denormalization
Authors: Sashi Novitasari; Takashi Fukuda; Gakuto Kurata
Venue: Interspeech 2025 (main)

Abstract: We introduce a novel technique for text capitalization and punctuation recovery (CP) systems that learn from voice-activity cues to effectively enhance the output readability of E2E ASR. Commonly E2E ASR systems produce uncapitalized text with no punctuation marks. In such situations, CP systems are introduced as external modules to denormalize the ASR output; however, they suffer from performance degradation due to the difference between the text segmentation used to construct them and those resulting from ASR. ASR systems generally produce decoded text of input speech segments determined by a VAD algorithm, while CP systems are often constructed on grammatically well-segmented full-sentence text. To reduce this gap, we construct a CP system by using pseudo VAD-segmented text given by a text segmentation model designed using voice activity cues. Our method reduces false predictions by 4.5%-18.9% compared with the baseline while appropriately formatting the ASR texts.
Phonetically-Augmented Discriminative Rescoring for Voice Search Error Correction
Authors: Christophe Van Gysel; Maggie Wu; Lyan Verwimp; Caglar Tirkaz; Marco Bertola; Zhihong Lei; Youssef Oualil
Venue: Interspeech 2025 (main)

Abstract: End-to-end (E2E) Automatic Speech Recognition (ASR) models are trained using paired audio-text samples that are expensive to obtain, since high-quality ground-truth data requires human annotators. Voice search applications, such as digital media players, leverage ASR to allow users to search by voice as opposed to an on-screen keyboard. However, recent or infrequent movie titles may not be sufficiently represented in the E2E ASR system's training data, and hence, may suffer poor recognition.In this paper, we propose a phonetic correction system that consists of (a) a phonetic search based on the ASR model's output that generates phonetic alternatives that may not be considered by the E2E system, and (b) a rescorer component that combines the ASR model recognition and the phonetic alternatives, and select a final system output.We find that our approach improves word error rate between 4.4 and 7.6% relative on benchmarks of popular movie titles over a series of competitive baselines.
From Weak Labels to Strong Results: Utilizing 5,000 Hours of Noisy Classroom Transcripts with Minimal Accurate Data
Authors: Ahmed Attia; Dorottya Demszky; Jing Liu; Carol Espy-Wilson
Venue: Interspeech 2025 (main)

Abstract: Recent progress in speech recognition has relied on models trained on vast amounts of labeled data. However, classroom Automatic Speech Recognition (ASR) faces the real-world challenge of abundant weak transcripts paired with only a small amount of accurate, gold-standard data. In such low-resource settings, high transcription costs make re-transcription impractical. To address this, we ask: what is the best approach when abundant inexpensive weak transcripts coexist with limited gold-standard data, as is the case for classroom speech data? We propose Weakly Supervised Pretraining (WSP), a two-step process where models are first pretrained on weak transcripts in a supervised manner, and then fine-tuned on accurate data. Our results, based on both synthetic and real weak transcripts, show that WSP outperforms alternative methods, establishing it as an effective training methodology for low-resource ASR in real-world scenarios.
Boundary-Conscious Pruning: Hard Set-Aware Model Compression for Efficient Speaker Recognition
Authors: Seongkyu Mun; Jubum Han
Venue: Interspeech 2025 (main)

Abstract: While modern CNN-based speaker recognition systems have achieved impressive accuracy, their computational and memory demands continue to pose challenges for on-device deployments, such as in smart home devices and consumer electronics. We propose an iterative channel pruning approach that is Hard set-aware, ensuring that channels essential for discriminating near-boundary cases are preserved. Our framework employs an angular medoid algorithm to dynamically partition samples into a hard set and a normal set. Channel importance is computed by integrating gradient-based measures, SE attention, and embedding perturbation, with the contributions of these metrics experimentally optimized. Coupled with iterative fine-tuning and pruning ratio adjustments, our method efficiently reduces model parameters while maintaining robust performance on critical hard set samples, thereby mitigating the severe degradation typically observed in conventional one-shot pruning approaches.
Pushing the Frontiers of Self-Distillation Prototypes Network with Dimension Regularization and Score Normalization
Authors: Yafeng Chen; Chong Deng; Hui Wang; Yiheng Jiang; Han Yin; Qian Chen; Wen Wang
Venue: Interspeech 2025 (main)

Abstract: Developing robust speaker verification (SV) systems without speaker labels has been a longstanding challenge. Earlier research has highlighted a considerable performance gap between self-supervised and fully supervised approaches. In this paper, we enhance the non-contrastive self-supervised framework, Self-Distillation Prototypes Network (SDPN), by introducing dimension regularization that explicitly addresses the collapse problem through the application of regularization terms to speaker embeddings. Moreover, we integrate score normalization techniques from fully supervised SV to further bridge the gap toward supervised verification performance. SDPN with dimension regularization and score normalization sets a new state-of-the-art on the VoxCeleb1 speaker verification evaluation benchmark, achieving Equal Error Rate 1.29%, 1.60%, and 2.80% for trial VoxCeleb1-{O,E,H} respectively. These results demonstrate relative improvements of 28.3%, 19.6%, and 22.6% over the current best self-supervised methods, thereby advancing the frontiers of SV technology.
A Domain Robust Pre-Training Method with Local Prototypes for Speaker Verification
Authors: Qing Gu; Yan Song; Haoyu Song; Nan Jiang; Lirong Dai; Ian McLoughlin
Venue: Interspeech 2025 (main)

Abstract: Existing self-supervised methods for speaker verification (SV) have demonstrated strong potential by training on large-scale unlabeled speech data to learn effective speaker embeddings. However, most rely on utterance-level contrastive learning or self-distillation, which fails to adequately account for domain shifts caused by different styles and languages. In this paper, we propose a novel domain-robust pre-training method with local prototypes for SV. Specifically, we employ a transformer-based encoder to introduce a self-distillation framework for local feature learning. Domain-agnostic pre-training is used to derive local prototypes through online clustering. Furthermore, domain-aware alignment is applied to learn domain-robust local features. Fine-tuning with utterance-level supervision demonstrates the effectiveness of our proposed method on the CNCeleb and VoxCeleb benchmarks.
Clustering-based Hard Negative Sampling for Supervised Contrastive Speaker Verification
Authors: Piotr Masztalski; MichaÅ Romaniuk; Jakub Å»ak; Mateusz Matuszewski; Konrad Kowalczyk
Venue: Interspeech 2025 (main)

Abstract: In speaker verification, contrastive learning is gaining popularity as an alternative to the traditionally used classification-based approaches. Contrastive methods can benefit from an effective use of hard negative pairs, which are different-class samples particularly challenging for a verification model due to their similarity. In this paper, we propose CHNS - a clustering-based hard negative sampling method, dedicated for supervised contrastive speaker representation learning. Our approach clusters embeddings of similar speakers, and adjusts batch composition to obtain an optimal ratio of hard and easy negatives during contrastive loss calculation. Experimental evaluation shows that CHNS outperforms a baseline supervised contrastive approach with and without loss-based hard negative sampling, as well as a state-of-the-art classification-based approach to speaker verification by as much as 18 % relative EER and minDCF on the VoxCeleb dataset using two lightweight model architectures.
MASV: Speaker Verification with Global and Local Context Mamba
Authors: Yang Liu; Li Wan; Yiteng Huang; Ming Sun; Xinhao Mei; Xubo Liu; Yangyang Shi; Florian Metze
Venue: Interspeech 2025 (main)

Abstract: Deep learning models like Convolutional Neural Networks and transformers have shown impressive capabilities in speech verification, gaining considerable attention in the research community. However, CNN-based approaches struggle with modeling long-sequence audio effectively, resulting in suboptimal verification performance. On the other hand, transformer-based methods are often hindered by high computational demands, limiting their practicality. This paper presents the MASV model, a novel architecture that integrates the Mamba module into the ECAPA-TDNN framework. By introducing the Local Context Bidirectional Mamba and Tri-Mamba block, the model effectively captures both global and local context within audio sequences. Experimental results demonstrate that the MASV model substantially enhances verification performance, surpassing existing models in both accuracy and efficiency.
ATMM-SAGA: Alternating Training for Multi-Module with Score-Aware Gated Attention SASV system
Authors: Amro Asali; Yehuda Ben-Shimol; Itshak Lapidot
Venue: Interspeech 2025 (main)

Abstract: The objective of automatic speaker verification (ASV) systems is to determine whether a given test speech utterance corresponds to a claimed enrolled speaker. These systems have a wide range of applications, and ensuring their reliability is crucial. In this paper, we propose a spoofing-robust automatic speaker verification (SASV) system employing a score-aware gated attention (SAGA) fusion scheme, integrating scores from a pre-trained countermeasure (CM) with speaker embeddings from a pre-trained ASV. Specifically, we employ the AASIST and ECAPA-TDNN models. SAGA acts as an adaptive gating mechanism, where the CM score determines how strongly ASV embeddings influence the final SASV decision. Experiments on the ASVspoof2019 logical access dataset demonstrate that the proposed SASV system achieves an SASV equal error rate (SASV-EER) and agnostic detection cost function (a-DCF) of 2.31%, 0.0603 for the development set and 2.18%, 0.0480 for the evaluation set.
Rethinking Leveraging Pre-Trained Multi-Layer Representations for Speaker Verification
Authors: Jin Sob Kim; Hyun Joon Park; Wooseok Shin; Sung Won Han
Venue: Interspeech 2025 (main)

Abstract: Recent speaker verification studies have achieved notable success by leveraging layer-wise output from pre-trained Transformer models. However, few have explored the advancements in aggregating these multi-level features beyond the static weighted average. We present Layer Attentive Pooling (LAP), a novel strategy for aggregating inter-layer representations from pre-trained speech models for speaker verification. LAP assesses the significance of each layer from multiple perspectives time-dynamically, and employs max pooling instead of averaging. Additionally, we propose a lightweight backend speaker model comprising LAP and Attentive Statistical Temporal Pooling (ASTP) to extract speaker embeddings from pre-trained model output. Experiments on the VoxCeleb benchmark reveal that our compact architecture achieves state-of-the-art performance while greatly reducing the training time. We further analyzed LAP design and its dynamic weighting mechanism for capturing speaker characteristics.
SEED: Speaker Embedding Enhancement Diffusion Model
Authors: Kihyun Nam; Jungwoo Heo; Jee-weon Jung; Gangin Park; Chaeyoung Jung; Ha-Jin Yu; Joon Son Chung
Venue: Interspeech 2025 (main)

Abstract: A primary challenge when deploying speaker recognition systems in real-world applications is performance degradation caused by environmental mismatch. We propose a diffusion-based method that takes speaker embeddings extracted from a pre-trained speaker recognition model and generates refined embeddings. For training, our approach progressively adds Gaussian noise to both clean and noisy speaker embeddings extracted from clean and noisy speech, respectively, via forward process of a diffusion model, and then reconstructs them to clean embeddings in the reverse process. While inferencing, all embeddings are regenerated via diffusion process. Our method needs neither speaker label nor any modification to the existing speaker recognition pipeline. Experiments on evaluation sets simulating environment mismatch scenarios show that our method can improve recognition accuracy by up to 19.6% over baseline models while retaining performance on conventional scenarios. We publish our code here.