GenAI4ELab/papercli-papers huggingface dataset:
In This Environment, As That Speaker: A Text-Driven Framework for Multi-Attribute Speech Conversion
Authors: Jiawei Jin; Zhihan Yang; Yixuan Zhou; Zhiyong Wu
Venue: Interspeech 2025 (main)

Abstract: We propose TES-VC (Text-driven Environment and Speaker controllable Voice Conversion), a text-driven voice conversion framework with independent control of speaker timbre and environmental acoustics. TES-VC processes simultaneous text inputs for target voice and environment, accurately generating speech matching described timbre/environment while preserving source content. Trained on synthetic data with decoupled vocal/environment features via latent diffusion modeling, our method eliminates interference between attributes. The Retrieval-Based Timbre Control (RBTC) module enables precise manipulation using abstract descriptions without paired data. Experiments confirm TES-VC effectively generates contextually appropriate speech in both timbre and environment with high content retention and superior controllability which demonstrates its potential for widespread applications.
LinearVC: Linear Transformations of Self-Supervised Features Through the Lens of Voice Conversion
Authors: Herman Kamper; Benjamin van Niekerk; Julian ZaÃ¯di; Marc-AndrÃ© Carbonneau
Venue: Interspeech 2025 (main)

Abstract: We introduce LinearVC, a simple voice conversion method that sheds light on the structure of self-supervised representations. First, we show that simple linear transformations of self-supervised features effectively convert voices. Next, we probe the geometry of the feature space by constraining the set of allowed transformations. We find that just rotating the features is sufficient for high-quality voice conversion. This suggests that content information is embedded in a low-dimensional subspace which can be linearly transformed to produce a target voice. To validate this hypothesis, we finally propose a method that explicitly factorizes content and speaker information using singular value decomposition; the resulting linear projection with a rank of just 100 gives competitive conversion results. Our work has implications for both practical voice conversion and a broader understanding of self-supervised speech representations.
Speaker Normalization and Content Restoration for Zero-Shot Voice Conversion with Attention-Enhanced Discriminator
Authors: Desheng Hu; Yang Xiang; Jian Lu; Xinhui Hu; Xinkang Xu
Venue: Interspeech 2025 (main)

Abstract: Zero-shot voice conversion can be achieved by extracting the source linguistic content and the unseen target speaker information, then reconstructing mel-spectrograms from these representations. In this paper, we propose a novel zero-shot VC method. First, we disentangle content and speaker information by training the content encoder from scratch, integrating a supervised phoneme classification network with speaker normalization and content restoration modules. Second, we enhance the speaker encoder by applying consistency loss, ensuring the extraction of accurate and robust speaker representations. Finally, we introduce an attention-enhanced discriminator for adversarial training to generate high-fidelity mel-spectrograms. Experimental results demonstrate that our proposed method demonstrates outstanding VC performance in terms of both speech quality and speaker similarity for unseen speakers.
Optimizing Pause Context in Fine-Tuning Pre-trained Large Language Models for Dementia Detection
Authors: Xiaoquan Ke; Man-Wai Mak; Helen Meng
Venue: Interspeech 2025 (main)

Abstract: Speech pauses serve as a valuable and non-invasive biomarker for the early detection of dementia. Our study aims to examine abnormal pauses, specifically their durations, for improving the detection performance. Inspired by the proven performance of the Transformer-based models in dementia detection, we opted for integrating the abnormal pauses into these models. Specifically, we enriched the inputs for the Transformer-based models by fusing between-segment pause context into the automated transcriptions. We performed the experiments on our Cantonese elderly corpus called CU-Marvel. To improve the detection performance, we optimized the pause durations when infusing the pause context into the transcriptions. Our findings suggest that the between-segment pauses could also serve as promising biomarkers. We emphasize the importance of optimizing pause patterns across different languages or datasets. Our findings indicate that various across different languages or datasets.
WhisperD: Dementia Speech Recognition and Filler Word Detection with Whisper
Authors: Emmanuel Akinrintoyo; Nadine Abdelhalim; Nicole Salomons
Venue: Interspeech 2025 (main)

Abstract: Whisper fails to correctly transcribe dementia speech because persons with dementia (PwDs) often exhibit irregular speech patterns and disfluencies such as pauses, repetitions, and fragmented sentences. It was trained on standard speech and may have had little or no exposure to dementia-affected speech. However, correct transcription is vital for dementia speech for cost-effective diagnosis and the development of assistive technology. In this work, we fine-tune Whisper with the open-source dementia speech dataset (DementiaBank) and our in-house dataset to improve its word error rate (WER). The fine-tuning also includes filler words to ascertain the filler inclusion rate (FIR) and F1 score. The fine-tuned models significantly outperformed the off-the-shelf models. The medium-sized model achieved a WER of 0.24, outperforming previous work. Similarly, there was a notable generalisability to unseen data and speech patterns.
Acoustic and Linguistic Biomarkers for Cognitive Impairment Detection from Speech
Authors: Catarina Botelho; David Gimeno-GÃ³mez; Francisco Teixeira; John MendonÃ§a; PatrÃ­cia Pereira; Diogo A. P. Nunes; Thomas Rolland; Anna Pompili; RubÃ©n Solera-UreÃ±a; Maria Ponte; David Martins de Matos; Carlos-D. MartÃ­nez-Hinarejos; Isabel Trancoso; Alberto Abad
Venue: Interspeech 2025 (main)

Abstract: This work describes a comprehensive approach for the automatic assessment of cognitive decline from spontaneous speech in the context of the PROCESS Challenge 2025. Based on our previous experience on the use of speech and text-derived biomarkers for disease detection, we evaluate here the use of knowledge-based acoustic and text-based feature sets, as well as LLM-based macro-descriptors, and multiple neural representations (e.g., Longformer, ECAPA-TDNN, and Trillsson embeddings). The combination of these feature sets with different classifiers resulted in a large pool of systems, from which, those providing the best balance between train, development, and individual class performance were selected for model ensembling. Our final best-performing systems correspond to combinations of models that are complementary to each other, relying on acoustic and textual information from the three clinical tasks provided in the challenge dataset.
Alzheimerâs Dementia Detection Using Perplexity from Paired Large Language Models
Authors: Yao Xiao; Heidi Christensen; Stefan Goetze
Venue: Interspeech 2025 (main)

Abstract: Alzheimerâs dementia (AD) is a neurodegenerative disorder with cognitive decline that commonly impacts language ability. This work extends the paired perplexity approach to detecting AD by using a recent large language model (LLM), the instruction-following version of Mistral-7B. We improve accuracy by an average of 3.33% over the best current paired perplexity method and by 6.35% over the top-ranked method from the ADReSS 2020 challenge benchmark. Our further analysis demonstrates that the proposed approach can effectively detect AD with a clear and interpretable decision boundary in contrast to other methods that suffer from opaque decision-making processes. Finally, by prompting the fine-tuned LLMs and comparing the model-generated responses to human responses, we illustrate that the LLMs have learned the special language patterns of AD speakers, which opens up possibilities for novel methods of model interpretation and data augmentation.
Understanding Dementia Speech Alignment with Diffusion-Based Image Generation
Authors: Mansi; Anastasios Lepipas; Dominika C Woszczyk; Yiying Guan; Soteris Demetriou
Venue: Interspeech 2025 (main)

Abstract: Text-to-image models generate highly realistic images based on natural language descriptions and millions of users use them to create and share images online. While it is expected that such models can align input text and generated image in the same latent space little has been done to understand whether this alignment is possible between pathological speech and generated images. In this work, we examine the ability of such models to align dementia-related speech information with the generated images and develop methods to explain this alignment. Surprisingly, we found that dementia detection is possible from generated images alone achieving 75% accuracy on the ADReSS dataset. We then leverage explainability methods to show which parts of the language contribute to the detection.
ClaritySpeech: Dementia Obfuscation in Speech
Authors: Dominika C Woszczyk; Ranya Aloufi; Soteris Demetriou
Venue: Interspeech 2025 (main)

Abstract: Dementia, a neurodegenerative disease, alters speech patterns, creating communication barriers and raising privacy concerns. Current speech technologies, such as automatic speech transcription (ASR), struggle with dementia and atypical speech, further challenging accessibility. This paper presents a novel dementia obfuscation in speech framework, ClaritySpeech, integrating ASR, text obfuscation, and zero-shot text-to-speech (TTS) to correct dementia-affected speech while preserving speaker identity in low-data environments without fine-tuning. Results show a 16% and 10% drop in mean F1 score across various adversarial settings and modalities (audio, text, fusion) for ADReSS and ADReSSo, respectively, maintaining 50% speaker similarity. We also find that our system improves WER (from 0.73 to 0.08 for ADReSS and 0.15 for ADReSSo) and speech quality from 1.65 to ~2.15, enhancing privacy and accessibility.12
Quadruple Path Modeling with Latent Feature Transfer for Permutation-free Continuous Speech Separation
Authors: Jihyun Kim; Doyeon Kim; Hyewon Han; Jinyoung Lee; Jonguk Yoo; Chang Woo Han; Jeongook Song; Hoon-Young Cho; Hong-Goo Kang
Venue: Interspeech 2025 (main)

Abstract: This paper proposes Quadruple Path Modeling (QPM), a permutation-free and generalized continuous speech separation (CSS) model designed to handle varying speaker conditions and efficiently address the permutation problem in chunk-based streaming scenarios. QPM integrates intra-chunk feature modeling, inter-speaker and inter-chunk processing, and latent feature transfer (LFT) modules to enhance separation performance while ensuring speaker consistency across segments. By leveraging a memory-based inter-chunk mechanism and a learnable gating strategy, QPM effectively propagates relevant speaker information across segments, thus reducing speaker permutation errors in streaming CSS tasks. Designed for lightweight and low-latency applications, including live streaming, QPM demonstrates strong performance using a 2-second chunk size. Experimental results confirm the efficacy of the proposed system in resolving the permutation problem, offering a scalable and adaptable solution for CSS.
End-to-End DOA-Guided Speech Extraction in Noisy Multi-Talker Scenarios
Authors: Kangqi Jing; Wenbin Zhang; Yu Gao
Venue: Interspeech 2025 (main)

Abstract: Target Speaker Extraction (TSE) plays a critical role in enhancing speech signals in noisy and multi-speaker environments. This paper presents an end-to-end TSE model that incorporates Direction of Arrival (DOA) and beamwidth embeddings to extract speech from a specified spatial region centered around the DOA. Our approach efficiently captures spatial and temporal features, enabling robust performance in highly complex scenarios with multiple simultaneous speakers. Experimental results demonstrate that the proposed model not only significantly enhances the target speech within the defined beamwidth but also effectively suppresses interference from other directions, producing a clear and isolated target voice. Furthermore, the model achieves remarkable improvements in downstream Automatic Speech Recognition (ASR) tasks, making it particularly suitable for real-world applications.
Speaker Separation for an Unknown Number of Speakers with Encoder-Decoder-Based Contextual Information Module
Authors: Xue Yang; Guiru Shen; Yu Yang
Venue: Interspeech 2025 (main)

Abstract: Many speaker separation methods impractically assume that the number of speakers is known in advance. To tackle this issue, an encoder-decoder-based attractor module was proposed to generate multiple speaker attractors. However, the attractors are compact vectors that discard the contextual information. In this paper, an encoder-decoder-based contextual information module is proposed. During training, the LSTM decoder performs multiple iterations to sequentially derive the frame-level representations of different speakers and the mixed signal. During inference, a sufficiently large number of iterations is assumed. The LSTM decoder can autonomously determine at which iteration to derive the frame-level representation of the mixed signal, thereby determining the appropriate channel for its estimation. The number of speakers can be effectively determined through the similarity computation. Experimental results show that high performance is achieved in both speaker counting and separation.
Attractor-Based Speech Separation of Multiple Utterances by Unknown Number of Speakers
Authors: Yuzhu Wang; Archontis Politis; Konstantinos Drossos; Tuomas Virtanen
Venue: Interspeech 2025 (main)

Abstract: This paper addresses the problem of single-channel speech separation, where the number of speakers is unknown, and each speaker may speak multiple utterances. We propose a speech separation model that simultaneously performs separation, dynamically estimates the number of speakers, and detects individual speaker activities by integrating an attractor module. The proposed system outperforms existing methods by introducing an attractor-based architecture that effectively combines local and global temporal modeling for multi-utterance scenarios. To evaluate the method in reverberant and noisy conditions, a multi-speaker multi-utterance dataset was synthesized by combining Librispeech speech signals with WHAM! noise signals. The results demonstrate that the proposed system accurately estimates the number of sources. The system effectively detects source activities and separates the corresponding utterances into correct outputs in both known and unknown source count scenarios.
ReSepNet: A Unified-Light Model for Recursive Speech Separation with Unknown Speaker Count
Authors: Hadi Alizadeh; Rahil Mahdian Toroghi; Hassan Zareian
Venue: Interspeech 2025 (main)

Abstract: Single-channel speech separation remains a significant challenge, particularly when the number of concurrent speakers is unknown. Existing methods often rely on prior assumptions about speaker count, limiting their real-world applicability. This paper introduces ReSepNet (Recursive Separation Network), a novel, unified model for speaker-independent speech separation that dynamically adapts to an unknown number of speakers. ReSepNet employs a recursive separation architecture, enabling it to iteratively isolate individual voices without prior knowledge of the speaker count. To enhance efficiency and reduce model complexity, we introduce a novel objective function and workflow. We demonstrate the effectiveness of ReSepNet on the WSJ0 datasets, achieving state-of-the-art separation performance and accurate speaker count estimation. Furthermore, ReSepNet generalizes well to mixtures containing four and five speakers, showcasing its robustness and adaptability to challenging scenarios.
Deep-Simplex Multichannel Speech Separation
Authors: Tzlil Avidan; Bracha Laufer-Goldshtein
Venue: Interspeech 2025 (main)

Abstract: Numerous methods exist for sound source separation, leveraging either classical signal processing or deep learning approaches. While deep-learning-based models often outperform conventional methods, they require large training datasets and struggle to generalize to new settings. To address this, we propose Deep-Simplex, a deep prior-based method that reconstructs the probability simplex of speaker activity over time. This global activity probability guides the estimation of a local mask per frequency, identifying the dominant speaker in each time-frequency (TF) bin. We then use this mask for both spatial and spectral separation. Experimental results demonstrate that Deep-Simplex outperforms competing baselines in different reverberation conditions.
FLASepformer: Efficient Speech Separation with Gated Focused Linear Attention Transformer
Authors: Haoxu Wang; Yiheng Jiang; Gang Qiao; Pengteng Shi; Biao Tian
Venue: Interspeech 2025 (main)

Abstract: Speech separation always faces the challenge of handling prolonged time sequences. Past methods try to reduce sequence lengths and use the Transformer to capture global information. However, due to the quadratic time complexity of the attention module, memory usage and inference time still increase significantly with longer segments. To tackle this, we introduce Focused Linear Attention and build FLASepformer with linear complexity for efficient speech separation. Inspired by SepReformer and TF-Locoformer, we have two variants: FLA-SepReformer and FLA-TFLocoformer. We also add a new Gated module to improve performance further. Experimental results on various datasets show that FLASepformer matches state-of-the-art performance with less memory consumption and faster inference. FLA-SepReformer-T/B/L increases speed by 2.29x, 1.91x, and 1.49x, with 15.8%, 20.9%, and 31.9% GPU memory usage, proving our model's effectiveness.
Power Spectral Density Estimation for Acoustic Source Separation Using A Spherical Microphone Array
Authors: Liang Tao; Maoshen Jia; Yonggang Hu
Venue: Interspeech 2025 (main)

Abstract: Acoustic source separation using microphone arrays has garnered increasing attention within the audio signal processing community. Cascading a beamformer and a post-filter for source separation is widely recognized as an effective solution, where accurate estimation of the power spectral density (PSD) is crucial for post-filter. However, estimating the PSD components is a challenging task in the presence of noise. In this paper, we propose an efficient multi-source PSD estimation method using a spherical microphone array (SMA) in noisy environments for source separation. Specifically, the orthogonality of spherical harmonics (SH) inherent in an error-free sampled SMA enables effective suppression of noise components in the spatial covariance matrix (SCM) of the observation signals, facilitating accurate PSD estimation for each individual source, thus the post-filter would be effectively constructed. The source separation problem is then addressed by applying a SH-domain beamformer along with the developed post-filter. Both simulation and experimental results demonstrate the proposed algorithmâs effectiveness over the baseline methods.
Exploring Efficient Directional and Distance Cues for Regional Speech Separation
Authors: Yiheng Jiang; Haoxu Wang; Yafeng Chen; Gang Qiao; Biao Tian
Venue: Interspeech 2025 (main)

Abstract: In this paper, we introduce a neural network-based method for regional speech separation using a microphone array. This approach leverages novel spatial cues to extract the sound source not only from specified direction but also within defined distance. Specifically, our method employs an improved delay-and-sum technique to obtain directional cues, substantially enhancing the signal from the target direction. We further enhance separation by incorporating the direct-to-reverberant ratio into the input features, enabling the model to better discriminate sources within and beyond a specified distance. Experimental results demonstrate that our proposed method leads to substantial gains across multiple objective metrics. Furthermore, our method achieves state-of-the-art performance on the CHiME-8 MMCSG dataset, which was recorded in real-world conversational scenarios, underscoring its effectiveness for speech separation in practical applications.
Teacher-Free Knowledge Distillation for Improving Short-Utterance Spoken Language Identification
Authors: Spandan Dey; Hirak Mondal; Sanjay Kumar Kurmi
Venue: Interspeech 2025 (main)

Abstract: Spoken language identification (LID) systems exhibit performance degradations as the test input duration reduces. To delve deeper, we show that 36.94% of the misclassifications on 2-second (s) LID inputs occur due to out-of-scope elements like non-speech, named entities, filler words, and overlapped speech. To mitigate this, we propose a teacher-free knowledge distillation (TF-KD) using online label smoothing. This method accumulates prediction logits of correctly classified training segments from the preceding epoch and uses them as soft-labels for distillation in the next epoch. We further enhance TF-KD with dynamic weights, conditional label update, and entropy-based soft-label computation. Compared to existing KD-based solutions for 2s inputs, our approach achieves consistent Cavg improvements for both same-corpora and cross-corpora evaluations without training a separate teacher network.
LID Models are Actually Accent Classifiers: Implications and Solutions for LID on Accented Speech
Authors: Niyati Bafna; Matthew Wiesner
Venue: Interspeech 2025 (main)

Abstract: Prior research indicates that LID model performance significantly declines on accented speech; however, the specific causes, extent, and characterization of these errors remain under-explored. (i) We identify a common failure mode on accented speech whereby LID systems often misclassify L2 accented speech as the speaker's native language or a related language. (ii) We present evidence suggesting that state-of-the-art models are invariant to permutations of short spans of speech, implying they classify on the basis of short phonotactic features indicative of accent rather than language. Our analysis reveals a simple method to enhance model robustness to accents through input chunking. (iii) We present an approach that integrates sequence-level information into our model without relying on monolingual ASR systems; this reduces accent-language confusion and significantly enhances performance on accented speech while maintaining comparable results on standard LID.
Analyzing the Impact of Accent on English Speech: Acoustic and Articulatory Perspectives
Authors: Gowtham Premananth; Vinith Kugathasan; Carol Espy-Wilson
Venue: Interspeech 2025 (main)

Abstract: Advancements in AI-driven speech-based applications have transformed diverse industries ranging from healthcare to customer service. However, the increasing prevalence of non-native accented speech in global interactions poses significant challenges for speech-processing systems, which are often trained on datasets dominated by native speech. This study investigates accented English speech through articulatory and acoustic analysis, identifying simpler coordination patterns and higher average pitch than native speech. Using eigenspectra and Vocal Tract Variable-based coordination features, we establish an efficient method for quantifying accent strength without relying on resource-intensive phonetic transcriptions. Our findings provide a new avenue for research on the impacts of accents on speech intelligibility and offer insights for developing inclusive, robust speech processing systems that accommodate diverse linguistic communities.