GenAI4ELab/papercli-papers huggingface dataset:
Multistage Universal Speech Enhancement System for URGENT Challenge
Authors: Xiaohuai Le; Zhuangqi Chen; Siyu Sun; Xianjun Xia; Chuanzeng Huang
Venue: Interspeech 2025 (main)

Abstract: During audio transmission and processing, various distortions may occur. To effectively address this challenge, we developed a multistage universal speech enhancement system, consisting of four submodules, namely audio declipping, packet loss compensation, audio separation, and spectral inpainting. These modules operate across the time, sub-band, and time-frequency domains. We employed a pretrain-finetune training paradigm and introduce a self-distillation method to further improve performance. Experiments on large-scale datasets demonstrate that our system outperforms in multiple evaluation metrics, particularly in improving subjective speech quality. The proposed system ranked 1st in the URGENT 2024 challenge with a MOS of 3.52 and placed 4th in the second track of the URGENT 2025 challenge.
Scaling beyond Denoising: Submitted System and Findings in URGENT Challenge 2025
Authors: Zhihang Sun; Andong Li; Tong Lei; Rilin Chen; Meng Yu; Chengshi Zheng; Yi Zhou; Dong Yu
Venue: Interspeech 2025 (main)

Abstract: Although deep neural networks have remarkably facilitated speech enhancement (SE), current studies focus on noise suppression and narrow benchmarks, with limited exploration of realistic scenarios. This paper presents a system that jointly addresses multiple degradations through an end-to-end framework. Specifically, a variant of dual-path architecture in the time-frequency domain is proposed, involving a fast band-processing strategy to enable parallel band split and merge operations, and a channel-mixing module based on Fourier Analysis Networks to facilitate target estimation. Besides, to enable training with limited GPU resources, we propose a simple yet effective progressive block extension strategy to support model scaling training with limited GPU resources. We participate in the URGENT Challenge and our submitted system ranked first on Track 1. The related findings and analysis of scaling effects provide diverse insights for building universal SE systems.
DeepFilterGAN: A Full-band Real-time Speech Enhancement System with GAN-based Stochastic Regeneration
Authors: Sanberk Serbest; Tijana Stojkovic; Milos Cernak; Andrew Harper
Venue: Interspeech 2025 (main)

Abstract: In this work, we propose a full-band real-time speech enhancement system with GAN-based stochastic regeneration. Predictive models focus on estimating the mean of the target distribution, whereas generative models aim to learn the full distribution. This behavior of predictive models may lead to over-suppression, i.e. the removal of speech content. In the literature, it was shown that combining a predictive model with a generative one within the stochastic regeneration framework can reduce the distortion in the output. We use this framework to obtain a real-time speech enhancement system. With 3.58M parameters and a low latency, our system is designed for real-time streaming with a lightweight architecture. Experiments show that our system improves over the first stage in terms of NISQA-MOS metric. Finally, through an ablation study, we show the importance of noisy conditioning in our system. We participated in 2025 Urgent Challenge with our model and later made further improvements.
FUSE: Universal Speech Enhancement using MultiâStage Fusion of Sparse Compression and Token Generation Models for the URGENT 2025 Challenge
Authors: Nabarun Goswami; Tatsuya Harada
Venue: Interspeech 2025 (main)

Abstract: We propose a multi-stage framework for universal speech enhancement, designed for the Interspeech 2025 URGENT Challenge. Our system first employs a Sparse Compression Network to robustly separate sources and extract an initial clean speech estimate from noisy inputs. This is followed by an efficient generative model that refines speech quality by leveraging self-supervised features and optimizing a masked language modeling objective on acoustic tokens derived from a neural audio codec. In the final stage, a fusion network integrates the outputs of the first two stages with the original noisy signal, achieving a balanced improvement in both signal fidelity and perceptual quality. Additionally, a shift trick that aggregates multiple time-shifted predictions, along with output blending, further boosts performance. Experimental results on challenging multilingual datasets with variable sampling rates and diverse distortion types validate the effectiveness of our approach.
Universal Speech Enhancement with Regression and Generative Mamba
Authors: Rong Chao; Rauf Nasretdinov; Yu-Chiang Frank Wang; Ante Jukic; Szu-Wei Fu; Yu Tsao
Venue: Interspeech 2025 (main)

Abstract: The Interspeech 2025 URGENT Challenge aimed to advance universal, robust, and generalizable speech enhancement by unifying speech enhancement tasks across a wide variety of conditions, including seven different distortion types and five languages. We present Universal Speech Enhancement Mamba (USEMamba), a state-space speech enhancement model designed to handle long-range sequence modeling, time-frequency structured processing, and sampling frequency-independent feature extraction. Our approach primarily relies on regression-based modeling, which performs well across most distortions. However, for packet loss and bandwidth extension, where missing content must be inferred, a generative variant of the proposed USEMamba proves more effective. Despite being trained in only a subset of the full training data, USEMamba achieved 2nd place in Track 1 during the blind test phase, demonstrating strong generalization across diverse conditions.
Structured pruning for efficient systolic array accelerated cascade Speech-to-Text Translation
Authors: Jean-Luc Rouas; Charles Brazier; Leila Ben Letaifa; Rafael Medina; Pedro Palacios; David Atienza; Giovanni Ansaloni
Venue: Interspeech 2025 (main)

Abstract: We present in this paper a simple method for pruning tiles of weights in sparse matrices, that do not require fine-tuning or retraining. This method is applied here to the feed-forward layers of transformers. We assess in a first experiment the impact of such pruning on the performances of speech recognition, machine translation, and the cascaded speech-to-text translation, on the MuST-C database, for the English to French direction. Depending on the size of the pruned tiles (from 4x4 to 32x32), we observe that pruning rates from 15 to 40% for speech recognition and from 40 to 70% for machine translation are feasible for a performance degradation of 10%. Applying this pruning method to the systolic array accelerated version of the cascade speech-to-text translation system results in speedups up to 74x compared to the non-accelerated system. Energy consumption also benefits from structured pruning with a maximum reduction of 35%.
Scaling pseudo-labeling data for end-to-end low-resource speech translation (the case of Kurdish language)
Authors: Mohammad Mohammadamini; Aghilas Sini; Marie Tahon; Antoine Laurent
Venue: Interspeech 2025 (main)

Abstract: In this paper we propose a pseudo-labeling pipeline to generate End-to-End Speech to Text Translation (E2E S2TT) data for low-resource languages. This pipeline allows us to achieve very promising results on S2TT task without having any parallel speech corpora. The proposed pipeline is composed of a speech segmentation, followed by a speech recognition system and a machine translation system. Our study is performed on Kurdish language which doesn't have resources for E2E S2TT. In our study, we firstly fine-tune and evaluate the ASR and MT systems to achieve a degree of reliability on the these components. The pipeline is used to pseudo-label 3,200 hours of Kurdish speech aligned with English translation. The pseudo-labeled data is extensively evaluated with different E2E S2TT systems such as Seq2Seq Transfomers and Whisper model. Achieving a 20.68 BLEU score on Fleurs benchmark shows the effectiveness of the our approach and its potential for other low-resource languages. The parallel pseudo-labeled data can be retrieved from: https://lium.univ-lemans.fr/en/ckbens2tt/
Multilingual Query-by-Example KWS for Indian Languages using Transliteration
Authors: Kirandevraj R; Vinod K Kurmi; Vinay Namboodiri; CV Jawahar
Venue: Interspeech 2025 (main)

Abstract: Query-by-Example Keyword Spotting (QbE KWS) detects query audio within target audio. A common approach for multilingual QbE KWS uses phoneme posteriors as representations, with a shared phoneme dictionary across languages. We propose a novel method that replaces phoneme-based representations with transliteration, unifying transcripts from multiple Indian languages into the Devanagari script, a text script used for Hindi and Marathi. We train a Multilingual ASR model to predict transliterated Devanagari text from audio across 10 Indian languages. The character logits from this ASR serve as both query and target audio features. Using the Kathbath dataset for training and the IndicSUPERB QbE evaluation set, our approach achieves significant improvements. The average MTWV increased from 0.015 (IndicSUPERB) to 0.504, and performance rose from 0.387 to 0.504, surpassing the best-performing Marathi ASR baseline. This demonstrates the effectiveness of transliteration for multilingual KWS.
Novel Parasitic Dual-Scale Modeling for Efficient and Accurate Multilingual Speech Translation
Authors: Chenyang Le; Yinfeng Xia; Huiyan Li; Manhong Wang; Yutao Sun; Xingyang Ma; Yanmin Qian
Venue: Interspeech 2025 (main)

Abstract: Recent advancements in speech-to-text translation have led to the development of multilingual models capable of handling multiple language pairs simultaneously. However, these unified models often suffer from large parameter sizes, making it challenging to balance inference efficiency and performance, particularly in local deployment scenarios. We propose an innovative Parasitic Dual-Scale Approach, which combines an enhanced speculative sampling method with model compression and knowledge distillation techniques. Building on the Whisper Medium model, we enhance it for multilingual speech translation into whisperM2M, and integrate our novel KVSPN module, achieving state-of-the-art (SOTA) performance across six popular languages with improved inference efficiency. KVSPN enables a 40% speedup with no BLEU score degradation. Combined with distillation methods, it represents a 2.6$Ã speedup over the original Whisper Medium with superior performance.
A Multi-Dialectal Dataset for German Dialect ASR and Dialect-to-Standard Speech Translation
Authors: Verena Blaschke; Miriam Winkler; Constantin FÃ¶rster; Gabriele Wenger-Glemser; Barbara Plank
Venue: Interspeech 2025 (main)

Abstract: Although Germany has a diverse landscape of dialects, they are underrepresented in current automatic speech recognition (ASR) research. To enable studies of how robust models are towards dialectal variation, we present Betthupferl, an evaluation dataset containing four hours of read speech in three dialect groups spoken in Southeast Germany (Franconian, Bavarian, Alemannic), and half an hour of Standard German speech. We provide both dialectal and Standard German transcriptions, and analyze the linguistic differences between them. We benchmark several multilingual state-of-the-art ASR models on speech translation into Standard German, and find differences between how much the output resembles the dialectal vs. standardized transcriptions. Qualitative error analyses of the best ASR model reveal that it sometimes normalizes grammatical differences, but often stays closer to the dialectal constructions.
NIRANTAR: Continual Learning with New Languages and Domains on Real-world Speech Data
Authors: Tahir Javed; Kaushal Bhogale; Mitesh M. Khapra
Venue: Interspeech 2025 (main)

Abstract: We introduce Nirantar, a comprehensive framework for evaluating continual learning (CL) in multilingual and multi-domain ASR. Designed to reflect real-world CL challenges, Nirantar leverages data collected incrementally across 22 languages and 208 districts in India through natural episodes. This enables evaluation across Language-Incremental (LIL), Domain-Incremental (DIL), and the novel Language-Incremental Domain-Incremental Learning (LIDIL) scenarios. Unlike prior work that relies on simulated episodes, Nirantar presents dynamic, non-uniform language and domain shifts, making it an ideal testbed for CL research. With 3250 hours of human-transcribed speech, including 1720 hours newly introduced in this work, our framework enables systematic benchmarking of CL methods. We evaluate existing approaches and demonstrate that no single method performs consistently well, underscoring the need for more robust CL strategies.
Temporal Modeling of Room Impulse Response Generation via Multi-Scale Autoregressive Learning
Authors: Sheng Lyu; Yuemin Yu; Chenshu Wu
Venue: Interspeech 2025 (main)

Abstract: The rise of AIGC has revolutionized multimedia processing, including audio applications. Room Impulse Response (RIR), which models sound propagation in acoustic environments,  plays a critical role in various downstream tasks such as speech synthesis. Existing RIR generation methods, whether based on ray tracing or neural representations, fail to fully exploit the temporal dynamics inherent in RIR. In this work, we propose a novel method for temporal modeling of RIR through autoregressive learning. Our approach captures the sequential evolution of sound propagation by introducing a multi-scale generation mechanism that adaptively scales across varying temporal resolutions. Extensive evaluations demonstrate that our approach achieves respective T60 error rates of 4.1% and 5.3% on two real-world datasets, outperforming existing RIR generation methods. We believe our work opens up new directions for future research.
Effect of Noise Floor in Room Impulse Response on Speech Perception Under Spherical Harmonics-based Spatial Sound Reproduction
Authors: Yunqi C. Zhang; Dhruv Jagmohan; Hong Kit Li; C. T. Justine Hui; Yusuke Hioka
Venue: Interspeech 2025 (main)

Abstract: The current study investigates the effect of noise floor in measured room impulse responses (RIR) on the reproducibility of speech perception under spherical harmonics-based spatial sound reproduction. Subjective listening test measuring the intelligibility of speech in noise was conducted under the spatial sound reproduction implemented using practically measured RIR with varying level of noise floor. The same test was also conducted in the real rooms where the RIR were measured. The comparison of the experimental results from the spatial sound reproduction and the real room suggests using measured RIR with low noise floor contributes to reproducing speech perception in real rooms accurately when the room is highly reverberant. It also has an effect to improve the reproducibility when the sound sources are located at 5 m but not at 2 m. Truncating RIR to further remove the noise floor mostly did not help improve the reproducibility regardless of the acoustics of the room.
Direction-Aware Neural Acoustic Fields for Few-Shot Interpolation of Ambisonic Impulse Responses
Authors: Christopher Ick; Gordon Wichern; Yoshiki Masuyama; FranÃ§ois G. Germain; Jonathan Le Roux
Venue: Interspeech 2025 (main)

Abstract: The characteristics of a sound field are intrinsically linked to the geometric and spatial properties of the environment surrounding a sound source and a listener. The physics of sound propagation is captured in a time-domain signal known as a room impulse response (RIR). Prior work using neural fields (NFs) has allowed learning spatially-continuous representations of RIRs from finite RIR measurements. However, previous NF-based methods have focused on monaural omnidirectional or at most binaural listeners, which does not precisely capture the directional characteristics of a real sound field at a single point. We propose a direction-aware neural field (DANF) that more explicitly incorporates the directional information by Ambisonic-format RIRs. While DANF inherently captures spatial relations between sources and listeners, we further propose a direction-aware loss. In addition, we investigate the ability of DANF to adapt to new rooms in various ways including low-rank adaptation.
AuralNet: Hierarchical Attention-based 3D Binaural Localization of Overlapping Speakers
Authors: Linya Fu; Yu Liu; Zhijie Liu; Zedong Yang; Zhong-Qiu Wang; Youfu Li; He Kong
Venue: Interspeech 2025 (main)

Abstract: We propose AuralNet, a novel 3D multi-source binaural sound source localization approach that localizes overlapping sources in both azimuth and elevation without prior knowledge of the number of sources. AuralNet employs a gated coarse-to-fine architecture, combining a coarse classification stage with a fine-grained regression stage, allowing for flexible spatial resolution through sector partitioning. The model incorporates a multi-head self-attention mechanism to capture spatial cues in binaural signals, enhancing robustness in noisy-reverberant environments. A masked multi-task loss function is designed to jointly optimize sound detection, azimuth, and elevation estimation. Extensive experiments in noisy-reverberant conditions demonstrate the superiority of AuralNet over recent methods.
SoundSculpt: Direction and Semantics Driven Ambisonic Target Sound Extraction
Authors: Tuochao Chen; D Shin; Hakan Erdogan; Sinan Hersek
Venue: Interspeech 2025 (main)

Abstract: This paper introduces SoundSculpt, a neural network designed to extract target sound fields from ambisonic recordings. SoundSculpt employs an ambisonic-in-ambisonic-out architecture and is conditioned on both spatial information (e.g., target direction obtained by pointing at an immersive video) and semantic embeddings (e.g., derived from image segmentation and captioning). Trained and evaluated on synthetic and real ambisonic mixtures, SoundSculpt demonstrates superior performance compared to various signal processing baselines. Our results further reveal that while spatial conditioning alone can be effective, the combination of spatial and semantic information is beneficial in scenarios where there are secondary sound sources spatially close to the target. Additionally, we compare two different semantic embeddings derived from a text description of the target sound using text encoders.
TF-Mamba: A Time-Frequency Network for Sound Source Localization
Authors: Yang Xiao; Rohan Kumar Das
Venue: Interspeech 2025 (main)

Abstract: Sound source localization (SSL) determines the position of sound sources using multi-channel audio data. It is commonly used to improve speech enhancement and separation. Extracting spatial features is crucial for SSL, especially in challenging acoustic environments. Recently, a novel structure referred to as Mamba demonstrated notable performance across various sequence-based modalities. This study introduces the Mamba for SSL tasks. We consider the Mamba-based model to analyze spatial features from speech signals by fusing both time and frequency features, and we develop an SSL system called TF-Mamba. This system integrates time and frequency fusion, with Bidirectional Mamba managing both time-wise and frequency-wise processing. We conduct the experiments on the simulated and real datasets. Experiments show that TF-Mamba significantly outperforms other advanced methods. The code will be publicly released in due course.
Articulatory modeling of the S-shaped F2 trajectories observed in Ãhman's spectrographic analysis of VCV syllables
Authors: FrÃ©dÃ©ric Berthommier
Venue: Interspeech 2025 (main)

Abstract: The synthesis of Ãhman's VCV sequences with intervocalic plosive consonants was first achieved 30 years ago using the DRM model. However, this approach remains primarily acoustic and lacks articulatory constraints. In this study, the same 75 VCVs are analyzed, but generated with the Maeda model, using trajectory planning that differentiates vowel-to-vowel transitions from consonantal influences. Synthetic data exhibit similar characteristics to Ãhman's sequences, including the presence of S-shaped F2 trajectories. Furthermore, locus equations (LEs) for F2 and F3 are computed from synthetic CV data to investigate their underlying determinism, leading to a reassessment of conventional interpretations. The findings indicate that, although articulatory planning is structured separately for vowel and consonant groups, S-shaped F2 trajectories emerge from a composite mechanism governed by the coordinated synergy of all articulators.
The Role of Voiced Consonant Duration in Sung Vowel-Consonant and Consonant-Vowel Recognition
Authors: Allan Vurma; Einar Meister; Lya Meister; Jaan Ross; Marju Raju; Veeda Kala; Tuuri Dede
Venue: Interspeech 2025 (main)

Abstract: This study explores the impact of consonant duration on the intelligibility of sung consonants (/l/, /m/, /n/, and /v/) across various acoustic settings, pitch levels, and background noise conditions. Forty-two participants (13 male, 29 female; aged 16-69) completed recognition tests involving CV and VC segments containing the vowel /a/, sung and spoken by a mezzo-soprano and a baritone. Consonant durations ranged from 0 ms to 200 ms, with artificial reverberation or brown noise added to some stimuli to simulate performance environments. GLMM revealed that recognition was poorer at high pitches, in reverberant acoustics, and with accompaniment, particularly for VC segments. Extending consonant duration from 20 ms to 200 ms consistently improved recognition by up to 25 percentage points. At low pitches, recognition exceeded chance even when the stationary part of the consonant was absent; 20 ms was sufficient for 95% recognition of spoken CVs, except in the presence of noise.
How sibilant spectra shape gender perception in prepubertal children: A voice morphing study
Authors: Riccarda Funk; Melanie Weirich; Adrian Simpson
Venue: Interspeech 2025 (main)

Abstract: Spectral characteristics of sibilants have been linked to social factors such as gender or sexual orientation. This paper examines the impact of sibilant spectra on gender perception in prepubertal German-speaking children (ages 6-9) through a longitudinal study. Center of Gravity (CoG) and skewness were measured in the sibilant /z/. Two listening experiments assessed gender perception: one with natural stimuli, the other with voice morphed stimuli to isolate the effect of sibilant spectra.No overall gender differences in CoG and skewness were found. While sibilant spectra had no effect on gender perception in natural stimuli, significant effects emerged in morphed stimuli. This suggests that while broader acoustic cues may have a stronger effect on gender perception, sibilants also reflect stereotypical gender associations in isolation.
Constrained LDDMM for Dynamic Vocal Tract Morphing: Integrating Volumetric and Real-Time MRI
Authors: Tharinda Piyadasa; Joan GlaunÃ¨s; Amelia Gully; Michael Proctor; Kirrie Ballard; TÃ¼nde Szalay; Naeim Sanaei; Sheryl Foster; David Waddington; Craig Jin
Venue: Interspeech 2025 (main)

Abstract: We present a novel framework for analyzing dynamic vocal tract deformations by integrating volumetric Magnetic Resonance Imaging (MRI) data and real-time MRI (rtMRI) boundary constraints within an iterative Large Deformation Diffeomorphic Metric Mapping (LDDMM) framework. More precisely, we apply LDDMM to morph volumetric vocal tract shapes using rtMRI boundary constraints that enable a smooth and anatomically plausible articulatory transformation. We demonstrate the method and discuss the issues involved using a vowel-consonant-vowel sequence. We show the influence of varying the number of rtMRI images on the resulting articulatory transformation.