GenAI4ELab/papercli-papers huggingface dataset:
Language and Accent Familiarity Effects on the Use of Acoustic Cues in Talker Identification
Authors: Shengyue Xiong; Zhe-chen Guo; Bharath Chandrasekaran
Venue: Interspeech 2025 (main)

Abstract: Talkers are identified more accurately when the language or accent is familiar to the listener. This is presumably due to access to linguistically relevant cues on top of lower-level acoustic informationâan explanation implying that reliance on acoustic cues should decrease as language or accent familiarity increases. We tested this prediction by training Mandarin-speaking listeners to identify talkers while listening to Mandarin-accented English (MAE) and Mandarin (NM) and English (NE) speech produced by their respective native speakers. Using representational similarity analysis, we compared the listenersâ responses with the talkersâ acoustic features (e.g., F0, jitter) to assess acoustic-cue reliance. Results showed greater reliance on acoustic cues in less familiar contexts, supporting the prediction. Notably, in MAE, listeners initially relied more on acoustic cues but later shifted to reduced reliance, highlighting the dynamic nature of talker identification strategies.
Characterization of voice cue sensitivity and vocal emotion recognition across the adult lifespan
Authors: Laura Rachman; Deniz BaÅkent
Venue: Interspeech 2025 (main)

Abstract: Studies show that vocal emotion recognition declines with age, but whether this is due to reduced sensitivity to acoustic cues (e.g., pitch-based cues) or cognitive changes is unclear. We assessed sensitivity to two important voice cues, fundamental frequency (F0) and vocal-tract length (VTL), which may also be relevant for emotion perception in speech. Using an adaptive 3I-3AFC paradigm, we measured just-noticeable differences to evaluate sensitivity to F0 and VTL. In a second task, we assessed vocal emotion recognition for happy, angry, and sad utterances in non-language-specific pseudo-speech. Participants ranged in age from 24 to 70 years, allowing us to examine how both lower-level sensory processing and higher-level emotion processing are affected by age, and the predictive value of F0 and VTL sensitivity in emotion recognition tasks.
Creaky Voice Facilitates More Efficient Phonological Processing of Mandarin Tone 3
Authors: Zixia Fan; Ronny Ibrahim; Joshua Penney; Felicity Cox
Venue: Interspeech 2025 (main)

Abstract: Though tone recognition is primarily influenced by pitch cues, phonation type may also play a role. For example, creaky voice, a non-modal phonation, can enhance Mandarin Tone 3 (T3) recognition. While most studies explore creaky voice in tone recognition behaviorally, its neural processing remains unclear. This study aims to investigate tone processing using Electroencephalography to examine how the brain integrates pitch and creaky voice to process T3. We measured Mismatch Negativity (MMN) responses of 10 native Mandarin speakers exposed to an oddball paradigm with real-word single syllable stimuli in two conditions: creak and non-creak. Results showed stronger MMN responses in the creak condition, with larger mean MMN amplitude, shorter peak MMN latency, and larger negative peak MMN amplitude. These findings suggest that creaky voice enhances T3 distinctiveness and facilitates more efficient phonological processing, contributing to a deeper understanding of tone processing mechanisms.
Training Onset-and-Offset-Aware Sound Event Detection on a Heterogeneous Dataset via Probabilistic Sequential Modeling
Authors: Tomoya Yoshinaga; Yoshiaki Bando; Keitaro Tanaka; Keisuke Imoto; Masaki Onishi; Shigeo Morishima
Venue: Interspeech 2025 (main)

Abstract: This paper presents a neural method to train onset-and-offset-aware sound event detection (SED) using heterogeneously labeled data. SED models are typically trained to predict frame-wise event activities, which have temporal fluctuations, resulting in unstable event boundaries. An end-to-end (E2E) method based on a hidden semi-Markov model (HSMM) has been proposed to improve performance by converting frame-wise predictions into event boundaries. This method, however, relies on temporal (strong) labels, which are costly to annotate. To overcome this limitation, we propose an E2E method to train an HSMM-based model from clip-level labels and unlabeled data. While the strong supervision was formulated to maximize event-wise posterior probabilities, we derive probabilistic objectives for such incompletely labeled data. Experimental results on the DESED dataset show that our method outperforms standard frame-wise methods.
Multi-view Fusion and Parameter Perturbation for Few-Shot Class-Incremental Audio Classification
Authors: Yulu Fang; Mingyue He; Qisheng Xu; Jianqiao Zhao; Cheng Yang; Kele Xu; Yong Dou
Venue: Interspeech 2025 (main)

Abstract: Audio classification tasks typically assume a fixed number of classes, which is often unrealistic in real-world applications where the target class vocabulary is dynamic or unknown in advance. A significant challenge arises when models must adapt to new classes incrementally, as this process is prone to catastrophic forgettingâa sharp decline in performance on previously learned classes, especially in data-scarce scenarios. While dynamic network-based methods and prototype refinementbased methods have been proposed to address these challenges, they overlook two critical issues: (1) inadequate representation of raw audio samples, which limits generalization, and (2) the risk of overfitting, which limits adaptivity. In this paper, we propose Multi-View Fusion and Parameter Perturbation (MVF2P), a novel framework that leverages the complementary learning system to enhance generalizability and adaptivity within a unified incremental learning framework. MVF2P addresses the limitations of existing methods by integrating multi-view learning to enrich feature representation and a parameter perturbation mechanism to reduce overfitting. Extensive evaluations on two widely-used audio datasets, NS-100 and LS-100, demonstrate that MVF2P outperforms state-of-the-art methods in terms of average accuracy and performance drop rate. Notably, MVF2P not only mitigates catastrophic forgetting more effectively but also enhances the modelâs adaptability to new classes, making it a robust solution for dynamic audio classification tasks.
Fully Few-shot Class-incremental Audio Classification Using Multi-level Embedding Extractor and Ridge Regression Classifier
Authors: Yongjie Si; Yanxiong Li; Jiaxin Tan; Qianhua He; Il-Youp Kwak
Venue: Interspeech 2025 (main)

Abstract: In the task of Few-shot Class-incremental Audio Classification (FCAC), training samples of each base class are required to be abundant to train model. However, it is not easy to collect abundant training samples for many base classes due to data scarcity and high collection cost. We discuss a more realistic issue, Fully FCAC (FFCAC), in which training samples of both base and incremental classes are only a few. Furthermore, we propose a FFCAC method using a model which is decoupled into a multi-level embedding extractor and a ridge regression classifier. The embedding extractor consists of an encoder of audio spectrogram Transformer and a fusion module, and is trained in the base session but frozen in all incremental sessions. The classifier is updated continually in each incremental session. Results on three public datasets show that our method exceeds current methods in accuracy, and has advantage over most of them in complexity.
Beyond Conventional Metrics: using Entropic Triangles to Explain Balancing Methods in Acoustic Scene Classification
Authors: Claudia Montero-RamÃ­rez; Alba MartÃ­nez-Serrano; Jorge GarcelÃ¡n-GÃ³mez; Francisco J. Valverde-Albacete; Carmen PelÃ¡ez-Moreno
Venue: Interspeech 2025 (main)

Abstract: Understanding soundscapes is essential for making sense of real-world scenarios in complex environments. However, real-life conditions present significant challenges for AI-based methods, particularly due to the highly imbalanced nature of Audio Tagging problems. In this work, we investigate the impact of data imbalance on the training dynamics of state-of-the-art models for Acoustic Scene Classification. Using the DCASE TAU Urban Acoustic Scenes 2022 dataset and the CP-Mobile model, we introduce controlled imbalance scenarios and analyze their effect through the Entropic Triangle framework. Our findings reveal that the training dynamics are strongly influenced by the chosen balancing approach. It also suggests longer training periods with conventional optimizer for re-balanced classification metrics. In this way, this study provides new insights into the role of entropy-based analysis in developing robust Acoustic Scene Classification systems for real-world applications.
Domain Adaptation Method and Modality Gap Impact in Audio-Text Models for Prototypical Sound Classification
Authors: Emiliano Acevedo; MartÃ­n Rocamora; Magdalena Fuentes
Venue: Interspeech 2025 (main)

Abstract: Audio-text models are widely used in zero-shot environmental sound classification as they alleviate the need for annotated data. However, we show that their performance severely drops in the presence of background sound sources. Our analysis reveals that this degradation is primarily driven by SNR levels of background soundscapes, and independent of background type. To address this, we propose a novel method that quantifies and integrates the contribution of background sources into the classification process, improving performance without requiring model retraining. Our domain adaptation technique enhances accuracy across various backgrounds and SNR conditions. Moreover, we analyze the modality gap between audio and text embeddings, showing that narrowing this gap improves classification performance. The method generalizes effectively across state-of-the-art prototypical approaches, showcasing its scalability and robustness for diverse environments.
Unified Microphone Conversion: Many-to-Many Device Mapping via Feature-wise Linear Modulation
Authors: Myeonghoon Ryu; Hongseok Oh; Suji Lee; Han Park
Venue: Interspeech 2025 (main)

Abstract: We present Unified Microphone Conversion, a unified generative framework designed to bolster sound event classification (SEC) systems against device variability. While our prior CycleGAN-based methods effectively simulate device characteristics, they require separate models for each device pair, limiting scalability. Our approach overcomes this constraint by conditioning the generator on frequency response data, enabling many-to-many device mappings through unpaired training. We integrate frequency-response information via Feature-wise Linear Modulation, further enhancing scalability. Additionally, incorporating synthetic frequency response differences improves the applicability of our framework for real-world application. Experimental results show that our method outperforms the state-of-the-art by 2.6% and reduces variability by 0.8% in macro-average F1 score.
The NaijaVoices Dataset: Cultivating Large-Scale, High-Quality, Culturally-Rich Speech Data for African Languages
Authors: Chris Emezue; The NaijaVoices Community; Busayo Awobade; Abraham Toluwase Owodunni; Handel Emezue; Gloria Monica Tobechukwu Emezue; Nefertiti Nneoma Emezue; Sewade Ogun; Bunmi Akinremi; David Ifeoluwa Adelani; Chris Pal
Venue: Interspeech 2025 (main)

Abstract: The development of high-performing, robust, and reliable speech technologies depends on large, high-quality datasets. However, African languages -- including our focus, Igbo, Hausa, and Yoruba -- remain under-represented due to insufficient data. Popular voice-enabled technologies do not support any of the 2000+ African languages, limiting accessibility for circa one billion people. While previous dataset efforts exist for the target languages, they lack the scale and diversity needed for robust speech models. To bridge this gap, we introduce the NaijaVoices dataset, a 1,800-hour speech-text dataset with 5,000+ speakers. We outline our unique data collection approach, analyze its acoustic diversity, and demonstrate its impact through finetuning experiments on automatic speech recognition, averagely achieving 75.86% (Whisper), 52.06% (MMS), and 42.33% (XLSR) WER improvements. These results highlight NaijaVoices' potential to advance multilingual speech processing for African languages.
FaiST: A Benchmark Dataset for Fairness in Speech Technology
Authors: Maliha Jahan; Yinglun Sun; Priyam Mazumdar; Zsuzsanna Fagyal; Thomas Thebaud; Jesus Villalba; Mark Hasegawa-Johnson; Najim Dehak; Laureano Moro Velazquez
Venue: Interspeech 2025 (main)

Abstract: Fairness in speech processing systems is a critical challenge, especially regarding performance disparities based on speakers' backgrounds. To help combat this problem, we are introducing FaiST (Fairness in Speech Technology), a novel speech dataset from American English speakers of various racial, ethnic, and national origin groups. The goal is to evaluate and mitigate possible biases across speech technologies using conversational speech from podcasts and interviews online. In FaiST's current version, speakers self-identified as Asian American and African American, and future iterations will include other groups. White American speakers' speech was extracted from VoxCeleb, and their demographic labels were obtained online. In addition to identifiers of race, ethnicity, and national origins, FaiST is also marked for the exact instances in the conversation where self-identifications occurred. We experimented with FaiST and found racial bias in eighteen Automatic Speech Recognition systems.
On the Language and Gender Biases in PSTN, VoIP and Neural Audio Codecs
Authors: Kemal Altwlkany; Amar Kuric; Emanuel Lacic
Venue: Interspeech 2025 (main)

Abstract: In recent years, there has been a growing focus on fairness and inclusivity within speech technology, particularly in areas such as automatic speech recognition and speech sentiment analysis. When audio is transcoded prior to processing, as is the case in streaming or real-time applications, any inherent bias in the coding mechanism may result in disparities. This not only affects user experience but can also have broader societal implications by perpetuating stereotypes and exclusion. Thus, it is important that audio coding mechanisms are unbiased. In this work, we contribute towards the scarce research with respect to language and gender biases of audio codecs. By analyzing the speech quality of over 2 million multilingual audio files after transcoding through a representative subset of codecs (PSTN, VoIP and neural), our results indicate that PSTN codecs are strongly biased in terms of gender and that neural codecs introduce language biases.
Evaluating Speech Enhancement Performance Across Demographics and Language
Authors: Jose Giraldo; Alex PeirÃ³-Lilja; Carme Armentano-Oller; Rodolfo Zevallos; Cristina EspaÃ±a-Bonet
Venue: Interspeech 2025 (main)

Abstract: Speech enhancement models have traditionally relied on VoiceBank-DEMAND for training and evaluation. However, this dataset presents significant limitations due to its limited diversity and simulated noise conditions. As an alternative, we propose and demonstrate the usefulness of evaluating the generalization capabilities of recent speech enhancement models using CommonPhone, a multilingual and crowdsourced dataset. Since CommonPhone is derived from CommonVoice, it allows to analyze enhancement performance based on demographic variables such as age and gender. Our experiments reveal significant performance variations across these variables. We also introduce a new benchmark dataset designed to challenge enhancement models with difficult and diverse speech samples, facilitating future research in universal speech enhancement.
Towards Better Disentanglement in Non-Autoregressive Zero-Shot Expressive Voice Conversion
Authors: Åeymanur Akti; Tuan-Nam Nguyen; Alexander Waibel
Venue: Interspeech 2025 (main)

Abstract: Expressive voice conversion aims to transfer both speaker identity and expressive attributes from a target speech to a given source speech. In this work, we improve over a self-supervised, non-autoregressive framework with a conditional variational autoencoder, focusing on reducing source timbre leakage and improving linguistic-acoustic disentanglement for better style transfer. To minimize style leakage, we use multilingual discrete speech units for content representation and reinforce embeddings with augmentation-based similarity loss and mix-style layer normalization. To enhance expressivity transfer, we incorporate local F0 information via cross-attention and extract style embeddings enriched with global pitch and energy features. Experiments show our model outperforms baselines in emotion and speaker similarity, demonstrating superior style adaptation and reduced source style leakage.
Voice Conversion for Likability Control via Automated Rating of Speech Synthesis Corpora
Authors: Hitoshi Suda; Shinnosuke Takamichi; Satoru Fukayama
Venue: Interspeech 2025 (main)

Abstract: Perceived voice likability plays a crucial role in various social interactions, such as partner selection and advertising. A system that provides reference likable voice samples tailored to target audiences would enable users to adjust their speaking style and voice quality, facilitating smoother communication. To this end, we propose a voice conversion method that controls the likability of input speech while preserving both speaker identity and linguistic content. To improve training data scalability, we train a likability predictor on an existing voice likability dataset and employ it to automatically annotate a large speech synthesis corpus with likability ratings. Experimental evaluations reveal a significant correlation between the predictorâs outputs and human-provided likability ratings. Subjective and objective evaluations further demonstrate that the proposed approach effectively controls voice likability while preserving both speaker identity and linguistic content.
REWIND: Speech Time Reversal for Enhancing Speaker Representations in Diffusion-based Voice Conversion
Authors: Ishan D. Biyani; Nirmesh J. Shah; Ashishkumar P. Gudmalwar; Pankaj Wasnik; Rajiv R. Shah
Venue: Interspeech 2025 (main)

Abstract: Speech time reversal refers to the process of reversing the entire speech signal in time, causing it to play backward. Such signals are completely unintelligible since the fundamental structures of phonemes and syllables are destroyed. However, they still retain tonal patterns that enable perceptual speaker identification despite losing linguistic content. In this paper, we propose leveraging speaker representations learned from time reversed speech as an augmentation strategy to enhance speaker representation. Notably, speaker and language disentanglement in voice conversion (VC) is essential to accurately preserve a speaker's unique vocal traits while minimizing interference from linguistic content. The effectiveness of the proposed approach is evaluated in the context of state-of-the-art diffusion-based VC models. Experimental results indicate that the proposed approach significantly improves speaker similarity-related scores while maintaining high speech quality.
Training-Free Voice Conversion with Factorized Optimal Transport
Authors: Alexander Lobashev; Assel Yermekova; Maria Larchenko
Venue: Interspeech 2025 (main)

Abstract: This paper introduces Factorized MKL-VC, a training-free modification for kNN-VC pipeline. In contrast with original pipeline, our algorithm performs high quality any-to-any cross-lingual voice conversion with only 5 second of reference audio. MKL-VC replaces kNN regression with a factorized optimal transport map in WavLM embedding subspaces, derived from Monge-Kantorovich Linear solution.  Factorization addresses non-uniform variance across dimensions, ensuring effective feature transformation. Experiments on LibriSpeech and FLEURS datasets show MKL-VC significantly improves content preservation and robustness with short reference audio, outperforming kNN-VC. MKL-VC achieves performance comparable to FACodec, especially in cross-lingual voice conversion domain.
E2E-BPVC: End-to-End Background-Preserving Voice Conversion via In-Context Learning
Authors: Yihan Liu; Zhengyang Chen; Leying Zhang; Yanmin Qian
Venue: Interspeech 2025 (main)

Abstract: Voice conversion (VC) systems are commonly trained on clean speech and are unable to function properly in the presence of background sound. However, in many instances, the background sound of speech and the context in which the speech occurs are highly semantically relevant and should also be retained. Existing approaches address this issue by introducing a denoising module to separate speech from background sound before applying voice conversion, which increases complexity and may lead to extra distortion. In this paper, we propose an end-to-end background-preserving voice conversion (E2E-BPVC) framework for the first time. By leveraging in-context learning (ICL), our model simultaneously modifies speech timbre and retains background sounds without requiring a separate denoising step. Both objective and subjective evaluations demonstrate that our method achieves performance comparable to denoising-based BPVC frameworks while maintaining a more streamlined and efficient system design.
Discl-VC: Disentangled Discrete Tokens and In-Context Learning for Controllable Zero-Shot Voice Conversion
Authors: Kaidi Wang; Wenhao Guan; Ziyue Jiang; Hukai Huang; Peijie Chen; Weijie Wu; Qingyang Hong; Lin Li
Venue: Interspeech 2025 (main)

Abstract: Currently, zero-shot voice conversion systems are capable of synthesizing the voice of unseen speakers. However, most existing approaches struggle to accurately replicate the speaking style of the source speaker or mimic the distinctive speaking style of the target speaker, thereby limiting the controllability of voice conversion. In this work, we propose Discl-VC, a novel voice conversion framework that disentangles content and prosody information from self-supervised speech representations and synthesizes the target speaker's voice through in-context learning with a flow matching transformer. To enable precise control over the prosody of generated speech, we introduce a mask generative transformer that predicts discrete prosody tokens in a non-autoregressive manner based on prompts. Experimental results demonstrate the superior performance of Discl-VC in zero-shot voice conversion and its remarkable accuracy in prosody control for synthesized speech.
ReFlow-VC: Zero-shot Voice Conversion Based on Rectified Flow and Speaker Feature Optimization
Authors: Pengyu Ren; Wenhao Guan; Kaidi Wang; Peijie Chen; Qingyang Hong; Lin Li
Venue: Interspeech 2025 (main)

Abstract: In recent years, diffusion-based generative models have demonstrated remarkable performance in speech conversion, including Denoising Diffusion Probabilistic Models (DDPM) and others. However, the advantages of these models come at the cost of requiring a large number of sampling steps. This limitation hinders their practical application in real-world scenarios. In this paper, we introduce ReFlow-VC, a novel high-fidelity speech conversion method based on rectified flow. Specifically, ReFlow-VC is an Ordinary Differential Equation (ODE) model that transforms a Gaussian distribution to the true Mel-spectrogram distribution along the most direct path. Furthermore, we propose a modeling approach that optimizes speaker features by utilizing both content and pitch information, allowing speaker features to reflect the properties of the current speech more accurately. Experimental results show that ReFlow-VC performs exceptionally well in small datasets and zero-shot scenarios.