GenAI4ELab/papercli-papers huggingface dataset:
A Study of Speech Embedding Similarities Between Australian Aboriginal and High-Resource Languages
Authors: Eliathamby Ambikairajah; Jingyao Wu; Ting Dang; Vidhyasaharan Sethu
Venue: Interspeech 2025 (main)

Abstract: Low-resource languages, such as Australian Aboriginal Languages, are underrepresented in the AI landscape due to limited availability of digital data, which in turn hinders speech processing model development. Leveraging sufficiently similar high-resource languages may help bridge this gap. This study examines the similarities between speech embeddings of aboriginal languages and 107 high-resource languages, including English, Spanish, and Mandarin, using Wav2Vec2 and VoxLingua107-ECAPA-TDNN. Through three language identification tasks, we analyze Warlpiri, Dalabon, and Light Warlpiri alongside 107 other languages. Our results reveal that aboriginal languages are most frequently identified as MÄori, suggesting phonetic or structural similarities, while showing significant differences from globally dominant languages. Additionally, we also observe that Warlpiri and Dalabon exhibited closer matches with Hindi and Malayalam, than with other languages.
An Investigative Study on Recent Sharpness- and Flatness-Based Optimizers for Enhanced Self-Supervised Speaker Verification
Authors: Abderrahim Fathan; Jahangir Alam; Xiaolin Zhu
Venue: Interspeech 2025 (main)

Abstract: Flat minima have recently proven effective for improving generalization and robustness against label noise. While Adam and Stochastic gradient descent (SGD) remain the most popular optimizers in deep learning, other optimizers such as Sharpness-Aware Minimization (SAM), simultaneously minimizing loss value and minimizing loss sharpness and/or improving flatness, have proven to be more effective across a variety of tasks. Motivated by this, we present the first comprehensive study of general-purpose optimizers (ADOPT, AdEMAMix) and sharpness-aware variants (SAM, ASAM, GAM, GSAM) for the supervised and self-supervised speaker verification tasks. We also explore different regularization methods such as weight decay, EMA, and SWITCH EMA to enhance robustness and learn flat optima without extra optimization costs. Our experimental results highlight the significant impact of the optimizer choice on generalization, achieving state-of-the-art performance in self-supervised speaker verification.
Privacy-Preserving Speaker Verification via End-to-End Secure Representation Learning
Authors: Chenguang Hu; Yaqian Hao; Fulin Zhang; Xiaoxue Luo; Yao Shen; Yingying Gao; Chao Deng; Shilei Zhang; Junlan Feng
Venue: Interspeech 2025 (main)

Abstract: The widespread adoption of cloud-based speaker verification raises privacy concerns, as unauthorized access to speaker voice can be exploited for fraudulent purposes. In response, privacy preserving speaker verification (PPSV) schemes have emerged to safeguard speaker information and prevent unauthorized access. However, existing PPSV methods often compromise accuracy in their efforts to protect privacy. To enhance both privacy and accuracy, we propose an end-to-end framework for speaker template encryption that simultaneously optimizes verification and encryption tasks. Our approach enhances the security and accuracy of speaker verification by introducing quintuple loss and sharpness-aware minimization, respectively, eliminating texture details that could be exploited by attackers. We distill our methodologies into a PPSV method, Secure-RawNet. Experimental results demonstrate that Secure-RawNet achieves high accuracy and robust privacy protection.
Novel Loss-Enhanced Universal Adversarial Patches for Sustainable Speaker Privacy
Authors: Elvir Karimov; Alexander Varlamov; Danil Ivanov; Dmitrii Korzh; Oleg Rogov
Venue: Interspeech 2025 (main)

Abstract: Deep learning voice models are commonly used nowadays, but the safety of processing of personal data, such as human identity and speech content, remains suspicious. To prevent malicious user identification, speaker obfuscation methods were proposed. Current methods, particularly based on universal adversarial patch (UAP) applications, have drawbacks such as significant degradation of audio quality, decreased speech recognition quality, low transferability across different voice biometrics models, and performance dependence on the input audio length. To mitigate these drawbacks, in this work, we introduce and leverage the novel Exponential Total Variance (TV) loss function and provide experimental evidence that it positively affects UAP strength and imperceptibility. Moreover, we present a novel scalable UAP insertion procedure and demonstrate its uniformly high performance for various audio lengths.
Federated Learning with Feature Space Separation for Speaker Recognition
Authors: Ying Meng; Zhihua Fang; Liang He
Venue: Interspeech 2025 (main)

Abstract: The performance of deep speaker models relies on large-scale, high-quality datasets. To improve the generalization of speaker models, federated learning can be used to share the knowledge learned by local models in private data. However, traditional federated learning methods have mapping conflict problems between local models. In this paper, we propose a federal speaker recognition method with feature space separation. Specifically, we introduce feature anchors for each speaker and transmit them when the client and server communicate. Local models use anchor loss to constrain the feature distribution of private data to avoid mapping conflict. In addition, we introduce a dynamic weight aggregation method when the server aggregates the local models, which amplifies the weights of well-performed local models to achieve a better aggregation effect. Extensive experiments and analyses in VoxCeleb and CN-Celeb demonstrate the effectiveness of our proposed method.
Audio Deepfake Source Tracing using Multi-Attribute Open-Set Identification and Verification
Authors: Pierre Falez; Tony Marteau; Damien Lolive; Arnaud Delhay
Venue: Interspeech 2025 (main)

Abstract: Audio deepfake detection has advanced significantly, in particular, thanks to the ASVSpoof challenge.  However, existing approaches primarily rely on binary classification, which does not provide information about the origin of manipulated audio. In this paper, we address the problem of source tracing and propose two protocols to evaluate model performance in an open-set setting: (1) a few-shot identification protocol, where K reference audios are provided, and (2) a verification protocol inspired by speaker verification. We classify either the entire generation system or its components, such as the acoustic model or vocoder. Our models are trained both on an internal dataset and on the MLAAD source tracing dataset. Evaluation is done on five public datasets: three ASVSpoof sets, MLAAD and Blizzard23. Results show promising discrimination of unseen class attributes. Finally, we emphasize the need for a standardized ontology for source tracing in audio deepfake detection.
Unveiling Audio Deepfake Origins: A Deep Metric learning And Conformer Network Approach With Ensemble Fusion
Authors: Ajinkya Kulkarni; Sandipana Dowerah; Tanel AlumÃ¤e; Mathew Magimai Doss
Venue: Interspeech 2025 (main)

Abstract: Audio deepfakes are acquiring an unprecedented level of realism with advanced AI. While current research focuses on discerning real speech from spoofed speech, tracing the source system is equally crucial. This work proposes a novel audio source tracing system combining deep metric multi-class N-pair loss with Real Emphasis and Fake Dispersion framework, a Conformer classification network, and ensemble score-embedding fusion. The N-pair loss improves discriminative ability, while Real Emphasis and Fake Dispersion enhance robustness by focusing on differentiating real and fake speech patterns. The Conformer network captures both global and local dependencies in the audio signal, crucial for source tracing. The proposed ensemble score-embedding fusion shows an optimal trade-off between in-domain and out-of-domain source tracing scenarios. We evaluate our method using Frechet Distance and standard metrics, demonstrating superior performance in source tracing over the baseline system.
Codec-Based Deepfake Source Tracing via Neural Audio Codec Taxonomy
Authors: Xuanjun Chen; I-Ming Lin; Lin Zhang; Jiawei Du; Haibin Wu; Hung-yi Lee; Jyh-Shing Roger Jang
Venue: Interspeech 2025 (main)

Abstract: Recent advances in neural audio codec-based speech generation (CoSG) models have produced remarkably realistic audio deepfakes. We refer to deepfake speech generated by CoSG systems as codec-based deepfake, or CodecFake. Although existing anti-spoofing research on CodecFake predominantly focuses on verifying the authenticity of audio samples, almost no attention was given to tracing the CoSG used in generating these deepfakes. In CodecFake generation, processes such as speech-to-unit encoding, discrete unit modeling, and unit-to-speech decoding are fundamentally based on neural audio codecs. Motivated by this, we introduce source tracing for CodecFake via neural audio codec taxonomy, which dissects neural audio codecs to trace CoSG. Our experimental results on the CodecFake+ dataset provide promising initial evidence for the feasibility of CodecFake source tracing while also highlighting several challenges that warrant further investigation.
TADA: Training-free Attribution and Out-of-Domain Detection of Audio Deepfakes
Authors: Adriana Stan; David Combei; Dan Oneata; Horia Cucu
Venue: Interspeech 2025 (main)

Abstract: Deepfake detection has gained significant attention across audio, text, and image modalities, with high accuracy in distinguishing real from fake. However, identifying the exact sourceâsuch as the system or model behind a deepfakeâremains a less studied problem. In this paper, we take a significant step forward in audio deepfake model attribution or source tracing by proposing a training-free, green AI approach based entirely on k-Nearest Neighbors (kNN). Leveraging a pre-trained self-supervised learning (SSL) model, we show that grouping samples from the same generator is straightforwardâwe obtain an 0.93 F1-score across five deepfake datasets. The method also demonstrates strong out-of-domain (OOD) detection, effectively identifying samples from unseen models at an F1-score of 0.84. We further analyse these results in a multi-dimensional approach and provide additional insights. All code and data protocols used in this work are available in our open repository.
Source Verification for Speech Deepfakes
Authors: Viola Negroni; Davide Salvi; Paolo Bestagini; Stefano Tubaro
Venue: Interspeech 2025 (main)

Abstract: With the proliferation of speech deepfake generators, it becomes crucial not only to assess the authenticity of synthetic audio but also to trace its origin. While source attribution models attempt to address this challenge, they often struggle in open-set conditions against unseen generators. In this paper, we introduce the source verification task, which, inspired by speaker verification, determines whether a test track was produced using the same model as a set of reference signals. Our approach leverages embeddings from a classifier trained for source attribution, computing distance scores between tracks to assess whether they originate from the same source. We evaluate multiple models across diverse scenarios, analyzing the impact of speaker diversity, language mismatch, and post-processing operations. This work provides the first exploration of source verification, highlighting its potential and vulnerabilities, and offers insights for real-world forensic applications.
STOPA: A Dataset of Systematic VariaTion Of DeePfake Audio for Open-Set Source Tracing and Attribution
Authors: Anton Firc; Manasi Chhibber; Jagabandhu Mishra; Vishwanath Pratap Singh; Tomi Kinnunen; Kamil Malinka
Venue: Interspeech 2025 (main)

Abstract: A key research area in deepfake speech detection is source tracing â determining the origin of synthesised utterances. The approaches may involve identifying the acoustic model (AM), vocoder model (VM), or other generation-specific parameters. However, progress is limited by the lack of a dedicated, systematically curated dataset. To address this, we introduce STOPA, a systematically varied and metadata-rich dataset for deepfake speech source tracing, covering 8 AMs, 6 VMs, and diverse parameter settings across 700k samples from 13 distinct synthesisers. Unlike existing datasets, which often feature limited variation or sparse metadata, STOPA provides a systematically controlled framework covering a broader range of generative factors, such as the choice of the vocoder model, acoustic model, or pretrained weights, ensuring higher attribution reliability. This control improves attribution accuracy, aiding forensic analysis, deepfake detection, and generative model transparency.
Synthetic Speech Source Tracing using Metric Learning
Authors: Dimitrios Koutsianos; Stavros Zacharopoulos; Yannis Panagakis; Themos Stafylakis
Venue: Interspeech 2025 (main)

Abstract: This paper addresses source tracing in synthetic speech-identifying generative systems behind manipulated audio via speaker recognition-inspired pipelines. While prior work focuses on spoofing detection, source tracing lacks robust solutions. We evaluate two approaches: classification-based and metric learning. We tested our methods on the MLAADv5 benchmark using ResNet and self-supervised learning (SSL) backbones. The results show that ResNet achieves competitive performance with the metric learning approach, matching and even exceeding SSL-based systems. Our work demonstrates ResNet's viability for source tracing while underscoring the need to optimize SSL representations for this task. Our work bridges speaker recognition methodologies with audio forensic challenges, offering new directions for combating synthetic media manipulation.
Listen, Analyze, and Adapt to Learn New Attacks: An Exemplar-Free Class Incremental Learning Method for Audio Deepfake Source Tracing
Authors: Yang Xiao; Rohan Kumar Das
Venue: Interspeech 2025 (main)

Abstract: As deepfake speech becomes common and hard to detect, it is vital to trace its source. Recent work on audio deepfake source tracing (ST) aims to find the origins of synthetic or manipulated speech. However, ST models must adapt to learn new deepfake attacks while retaining knowledge of the previous ones. A major challenge is catastrophic forgetting, where models lose the ability to recognize previously learned attacks. Some continual learning methods help with deepfake detection, but multi-class tasks such as ST introduce additional challenges as the number of classes grows. To address this, we propose an analytic class incremental learning method called AnaST. When new attacks appear, the feature extractor remains fixed, and the classifier is updated with a closed-form analytical solution in one epoch. This approach ensures data privacy, optimizes memory usage, and is suitable for online training. The experiments carried out in this work show that our method outperforms the baselines.
VIB-based Real Pre-emphasis Audio Deepfake Source Tracing
Authors: Thien-Phuc Doan; Kihun Hong; Souhwan Jung
Venue: Interspeech 2025 (main)

Abstract: The rise of audio deepfakes presents significant challenges, necessitating effective detection and attribution mechanisms for fabricated speech. This paper addresses the need to identify not only fake speech but also its origins. We develop the audio deepfake source tracing method as follows: First, we establish a protocol using the MLAAD v5 dataset to analyze the acoustic and vocoder processing involved in generating fake speech. Second, we introduce a novel real pre-emphasis method based on the VIB architecture for audio deepfake source tracing. Our evaluation demonstrates that our model outperforms baseline systems, achieving enhancements of 10%. Our approach significantly improves model generalization and reduces complexity while preserving the real pre-emphasis effect established by the baseline.
Defending Unauthorized Voice Cloning with Watermark-Aware Codecs
Authors: Jiankun Zhao; Lingwei Meng; Chengxi Deng; Helen Meng; Xixin Wu
Venue: Interspeech 2025 (main)

Abstract: The proliferation of zero-shot TTS models increases the risk of malicious voice cloning using copyrighted speech prompts. Although audio watermarking provides an effective way for encoding copyright information, attackers may still use watermarked speech as prompts to synthesize unwatermarked speech with the same speaker identity. To protect copyrighted voices from being cloned, this study introduces a method to train open-source TTS models to reject watermarked speech prompts for cloning. We observe that mainstream zero-shot TTS models typically rely on pre-trained codec encoders to process speech prompts. By training the codec to ``mute" when encountering watermarked audio, the quality of generated speech will degrade. In this way, we implicitly prevent zero-shot TTS models from cloning watermarked voices. Experiments show that our approach is robust against various attacks while maintaining high-quality TTS ability given unwatermarked speech prompts.
Open-Set Source Tracing of Audio Deepfake Systems
Authors: Nicholas Klein; Hemlata Tak; Elie Khoury
Venue: Interspeech 2025 (main)

Abstract: Existing research on source tracing of audio deepfake systems has focused primarily on the closed-set scenario, while studies that evaluate open-set performance are limited to a small number of unseen systems. Due to the large number of emerging audio deepfake systems, robust open-set source tracing is critical. We leverage the protocol of the Interspeech 2025 special session on source tracing to evaluate methods for improving open-set source tracing performance. We introduce a novel adaptation to the energy score for out-of-distribution (OOD) detection, softmax energy (SME). We find that replacing the typical temperature-scaled energy score with SME provides a relative average improvement of 31% in the standard FPR95 (false positive rate at true positive rate of 95%) measure. We further explore SME-guided training as well as copy synthesis, codec, and reverberation augmentations, yielding an FPR95 of 8.3%.
Fine-tune Before Structured Pruning: Towards Compact and Accurate Self-Supervised Models for Speaker Diarization
Authors: Jiangyu Han; Federico Landini; Johan Rohdin; Anna Silnova; Mireia Diez; Jan ÄernockÃ½; LukÃ¡Å¡ Burget
Venue: Interspeech 2025 (main)

Abstract: Self-supervised learning (SSL) models like WavLM can be effectively utilized when building speaker diarization systems but are often large and slow, limiting their use in resource-constrained scenarios. Previous studies have explored compression techniques, but usually for the price of degraded performance at high pruning ratios. In this work, we propose to compress SSL models through structured pruning by introducing knowledge distillation. Different from the existing works, we emphasize the importance of fine-tuning SSL models before pruning. Experiments on far-field single-channel AMI, AISHELL-4, and AliMeeting datasets show that our method can remove redundant parameters of WavLM Base+ and WavLM Large by up to 80% without any performance degradation. After pruning, the inference speeds on a single GPU for the Base+ and Large models are 4.0 and 2.6 times faster, respectively. Our source code is publicly available.
Count Your Speakers! Multitask Learning for Multimodal Speaker Diarization
Authors: Prabhav Singh; Jesus Villalba; Najim Dehak
Venue: Interspeech 2025 (main)

Abstract: Recent advances in speaker diarization have explored diverse clustering methods, particularly in multimodal frameworks. However, a critical limitation lies in the clustering stage, where heuristic-based methods often fail to leverage the full potential of multimodal data. For example, threshold-based clustering frequently leads to over-clustering, causing incorrect speaker assignments and elevated DER. To address this, we propose CYS-MSD, a novel framework that fuses audio-visual modalities via a trainable cross-modal attention mechanism. The embeddings are fine-tuned with a multitask objective to jointly predict speaker counts and assign speaker labels, enabling data-driven clustering that adapts to varying speaker scenarios. Additionally, a modality-masking mechanism ensures robustness to missing inputs in real-world conditions. We evaluate CYS-MSD on the AVA-AVD corpus, reporting a 5% reduction in DER over the baseline and an average 2% reduction compared to various SOTA systems.
End-to-End Diarization utilizing Attractor Deep Clustering
Authors: David Palzer; Matthew Maciejewski; Eric Fosler-Lussier
Venue: Interspeech 2025 (main)

Abstract: Speaker diarization remains challenging due to the need for structured speaker representations, efficient modeling, and robustness to varying conditions. We propose a performant, compact diarization framework that integrates conformer decoders, transformer-updated attractors, and a deep clustering style angle loss. Our approach refines speaker representations with an enhanced conformer structure, incorporating cross-attention to attractors and an additional convolution module. To enforce structured embeddings, we extend deep clustering by constructing labelâattractor vectors, aligning their directional structure with audio embeddings. We also impose orthogonality constraints on active attractors for better speaker separation while suppressing non-active attractors to prevent false activations. Finally, a permutation invariant training binary cross-entropy loss refines speaker detection. Experiments show that our method achieves low diarization error while maintaining parameter count.
SDBench: A Comprehensive Benchmark Suite for Speaker Diarization
Authors: Berkin Durmus; Blaise Munyampirwa; Eduardo Pacheco; Atila Orhon; Andrey Leonov
Venue: Interspeech 2025 (main)

Abstract: Even state-of-the-art speaker diarization systems exhibit high variance in error rates across different datasets, representing numerous use cases and domains. Furthermore, comparing across systems requires careful application of best practices such as dataset splits and metric definitions to allow for apples-to-apples comparison. We propose SDBench (Speaker Diarization Benchmark), an open-source benchmark suite that integrates 13 diverse datasets with built-in tooling for consistent and fine-grained analysis of speaker diarization performance for various on-device and server-side systems. SDBench enables reproducible evaluation and easy integration of new systems over time. To demonstrate the efficacy of SDBench, we built SpeakerKit, an inference efficiency-focused system built on top of Pyannote v3. SDBench enabled rapid execution of ablation studies that led to SpeakerKit being 9.6x faster than Pyannote v3 while achieving comparable error rates.
Enhancing Serialized Output Training for Multi-Talker ASR with Soft Monotonic Alignment and Utterance-level Timestamp
Authors: Fengyun Tan; Tao Wei; Kun Zou; Ning Cheng; Shaojun Wang; Jing Xiao
Venue: Interspeech 2025 (main)

Abstract: Multi-talker ASR has gained significant attention due to its broad applications in conference settings. The previously proposed SOT stands out among many approaches. However, when multiple speakers talk simultaneously, predicting the speaker change symbol becomes more challenging. Boundary-Aware SOT (BA-SOT) uses multi-task to learn speaker change point, increasing complexity and training cost, without timestamp prediction. Therefore, we propose enhanced SOT, called Soft Monotonic Alignment SOT (SMA-SOT), which introduces SMA Loss and utterance-level timestamps. These two components complement each other, not only utilizing timestamps to promote monotonic alignment constraint learning but also, in turn, making timestamp prediction more accurate through the SMA Loss. Experimental results on AliMeeting test set show that SMA-SOT achieves a 5.4% and 0.35% relative CER reduction compared to the SOT and BA-SOT respectively and achieves a Speaker Change Accuracy (SCA) of 81.3%.