GenAI4ELab/papercli-papers huggingface dataset:
MelRe: Vision-Based Mel-Spectrogram Restoration
Authors: Kaixuan Luan; Xiaoda Yang; Shile Cai; Ruofan Hu; Minghui Fang; Wenrui Liu; Jialong Zuo; Jiaqi Duan; Yuhang Ma; Junyu Lu
Venue: Interspeech 2025 (main)

Abstract: With advancements in visual technology, an increasing number of visual techniques have recently been applied in other fields. Among them, mel spectrograms provide a bridge between audio features and visual models. Previous work has demonstrated that applying image processing methods to mel spectrograms is feasible. However, traditional image-based models operate at a relatively coarse level, focusing primarily on controlling texture and shape. In contrast, mel spectrograms are highly sensitive to detail, containing complex time-frequency information that requires more refined modeling. To address this, we propose MelRe, a visual model specifically designed for mel spectrograms, aimed at tackling complex fine-grained audio degradation issues from a visual perspective. MelRe addresses the need for fine-grained detail through pixel-level restoration methods and employs degradation alignment and noise simulation strategies to achieve high-precision restoration across varying levels of degradation, demonstrating exceptional restoration performance. Experimental results show that MelRe achieves a new state-of-the-art (SOTA) level in complex audio restoration tasks, highlighting its potential for high-quality audio repair.
SpeechRefiner: Towards Perceptual Quality Refinement for Front-End Algorithms
Authors: Sirui Li; Shuai Wang; Zhijun Liu; Zhongjie Jiang; Yannan Wang; Haizhou Li
Venue: Interspeech 2025 (main)

Abstract: Speech pre-processing techniques such as denoising, de-reverberation, and separation, are commonly employed as front-ends for various downstream speech processing tasks. However, these methods can sometimes be inadequate, resulting in residual noise or the introduction of new artifacts. Such deficiencies are typically not captured by metrics like SI-SNR but are noticeable to human listeners. To address this, we introduce SpeechRefiner, a post-processing tool that utilizes Conditional Flow Matching (CFM) to improve the perceptual quality of speech. In this study, we benchmark SpeechRefiner against recent task-specific refinement methods and evaluate its performance within our internal processing pipeline, which integrates multiple front-end algorithms. Experiments show that SpeechRefiner exhibits strong generalization across diverse impairment sources, significantly enhancing speech perceptual quality.
Modality-Specific Speech Enhancement and Noise-Adaptive Fusion for Acoustic and Body-Conduction Microphone Framework
Authors: Yunsik Kim; Yoonyoung Chung
Venue: Interspeech 2025 (main)

Abstract: Body-conduction microphone signals (BMS) bypass airborne sound, providing strong noise resistance. However, a complementary modality is required to compensate for the inherent loss of high-frequency information. In this study, we propose a novel multi-modal framework that combines BMS and acoustic microphone signals (AMS) to achieve both noise suppression and high-frequency reconstruction. Unlike conventional multi-modal approaches that simply merge features, our method employs two specialized networks: a mapping-based model to enhance BMS and a masking-based model to denoise AMS. These networks are integrated through a dynamic fusion mechanism that adapts to local noise conditions, ensuring the optimal use of each modalityâs strengths. We performed evaluations on the TAPS dataset, augmented with DNS-2023 noise clips, using objective speech quality metrics. The results clearly demonstrate that our approach outperforms single-modal solutions in a wide range of noisy environments.
Joint Rate Allocation and Sensor Selection for Speech Enhancement in Wireless Acoustic Sensor Networks
Authors: De Hu; Qilong Li
Venue: Interspeech 2025 (main)

Abstract: In wireless acoustic sensor networks (WASNs) with high spatial resolution, energy efficiency is crucial for speech enhancement (SE) due to limited node energy. To optimize energy efficiency under predefined performance constraints, sensor selection (SS), rate allocation (RA), or a joint RA and SS (RASS) approach can be employed. However, since the above criteria were developed in a frequency-dependent manner, it may lead to conflicts among SS results over different frequencies. In this work, we propose a novel RASS model by promoting bit rate sparsity in RA, eliminating NP-hard Boolean programming, and extending it to a frequency-invariant RASS (FI-RASS) for consistent SS results across frequencies. In addition, we introduce a re-weighting strategy to further reduce energy consumption in the WASN. Compared with existing RA, SS, and RASS approaches, the FI-RASS provides a remarkable superiority in terms of energy efficiency. Numerical experiments demonstrate the effectiveness of the proposed method.
Individualized speech enhancement for hearing-impaired listeners
Authors: Chuan Wen; Sarah Verhulst
Venue: Interspeech 2025 (main)

Abstract: Despite significant progress in speech enhancement (SE), improving speech intelligibility and perceptual quality in noisy environments for hearing-impaired individuals remains challenging. This paper presents an individualized speech-enhancement (ISE) framework that integrates noise reduction (NR) and sound amplification for hearing loss compensation within a unified system. Our fully differentiable, closed-loop design incorporates biophysically realistic auditory models. The framework features two pathways: one simulating the auditory response of a normal-hearing (NH) system to denoised speech and the other modeling a hearing-impaired (HI) systemâs response to noisy speech. The ISE model is trained by minimizing the difference between NH and HI auditory responses. Experimental results show that the ISE model enhances speech intelligibility and perceptual quality in noisy conditions for HI listeners, offering a promising foundation for advancing personalized noise reduction strategies.
First Analyze Then Enhance: A Task-Aware System for Speech Separation, Denoising, and Dereverberation
Authors: Shaoxiang Dang; Li Li; Shogo Seki; Hiroaki Kudo
Venue: Interspeech 2025 (main)

Abstract: This paper presents the First Analyze Then Enhance (FATE) framework for speech enhancement. In FATE, observed signals are initially classified based on their specific degradation types and then enhanced using appropriate modules tailored to each type. This design prevents the overprocessing of clean signals and reduces process complexity by eliminating unnecessary procedures. This paper focuses on commonly encountered degradations, including additive noise, reverberation, speech mixing, and their combinations. To address these degradations, FATE includes a separation submodule and a denoising/dereverberation submodule the enhancement models. Degradation types are predicted using features extracted from pretrained models based on automatic speech recognition and self-supervised learning. Experiments show that FATE can accurately identify undegraded signals and achieve comparable enhancement performance for degraded signals in each scenario while optimizing processing complexity.
A Robust Hybrid ACC-PM Approach for Personal Sound Zones
Authors: Yaqi Zhu; Lei Zhou; Hongqing Liu; Liming Shi; Lu Gan
Venue: Interspeech 2025 (main)

Abstract: The performance of personal sound systems is often degraded by inaccurate acoustic measurements. To achieve robust control while balancing acoustic contrast and signal distortion, this work proposes a robust hybrid optimization method that exploits both acoustic contrast control and pressure matching (ACC-PM). The method addresses perturbations caused by uncertainties in the acoustic transfer functions such as temperature changes, head movement, etc, modeled as norm-bounded uncertainties. Although the resulting worst-case optimization is inherently non-convex, it is reformulated as a second-order cone programming problem, which can be efficiently solved. Numerical simulations demonstrate the effectiveness of the proposed robust ACC-PM algorithm, showing an improvement over 18%  in terms of AC compared to vanilla ACC-PM.
Distilling a speech and music encoder with task arithmetic
Authors: Fabian Ritter-Gutierrez; Yi-Cheng Lin; Jui-Chiang Wei; Jeremy H.M Wong; Eng Siong Chng; Nancy F. Chen; Hung-yi Lee
Venue: Interspeech 2025 (main)

Abstract: Despite the progress in self-supervised learning (SSL) for speech and music, existing models treat these domains separately, limiting their capacity for unified audio understanding. A unified model is desirable for applications that require general representations, e.g. audio large language models. Nonetheless, directly training a general model for speech and music is computationally expensive. Knowledge Distillation of teacher ensembles may be a natural solution, but we posit that decoupling the distillation of the speech and music SSL models allows for more flexibility. Thus, we propose to learn distilled task vectors and then linearly interpolate them to form a unified speech+music model. This strategy enables flexible domain emphasis through adjustable weights and is also simpler to train. Experiments on speech and music benchmarks demonstrate that our method yields superior overall performance compared to ensemble distillation.
MSDA: Combining Pseudo-labeling and Self-Supervision for Unsupervised Domain Adaptation in ASR
Authors: Dimitrios Damianos; Georgios Paraskevopoulos; Alexandros Potamianos
Venue: Interspeech 2025 (main)

Abstract: In this work, we investigate the Meta PL unsupervised domain adaptation framework for Automatic Speech Recognition (ASR). We introduce a Multi-Stage Domain Adaptation pipeline (MSDA), a sample-efficient, two-stage adaptation approach that integrates self-supervised learning with semi-supervised techniques. MSDA is designed to enhance the robustness and generalization of ASR models, making them more adaptable to diverse conditions. It is particularly effective for low-resource languages like Greek and in weakly supervised scenarios where labeled data is scarce or noisy. Through extensive experiments, we demonstrate that Meta PL can be applied effectively to ASR tasks, achieving state-of-the-art results, significantly outperforming state-of-the-art methods, and providing more robust solutions for unsupervised domain adaptation in ASR. Our ablations highlight the necessity of utilizing a cascading approach when combining self-supervision with self-training.
REB-former: RWKV-enhanced E-branchformer for Speech Recognition
Authors: Jie Song; Wang Xiang; Jian Zhou; Cunhang Fan; Zhao Lv
Venue: Interspeech 2025 (main)

Abstract: Transformer-based architectures have achieved significant success in automatic speech recognition (ASR). However, the quadratic complexity of their self-attention mechanisms limits processing efficiency for speech sequences. To address this issue, this paper proposes the Receptance Weighted Key Value (RWKV)-enhanced E-Branchformer (REB-former). Specifically, the REB-former interleaves the E-Branchformer and RWKV layers, combining different attention mechanisms to reduce computational complexity and enhance speech modeling. To overcome RWKV's unidirectional limitation, we introduce the GroupBiRWKV module for efficient contextual feature capture. The results show that the REB-former outperforms the E-Branchformer in terms of computational efficiency and inference speed, achieving a relative reduction of up to 7.1% in the word error rate (WER). On the LibriSpeech 100h dataset, our model achieves WER of 6.0%/15.8% on test-clean/test-other, setting a new state-of-the-art performance.
PredTrAD â Prediction-based Transformer for Anomaly Detection in Multivariate Time Series Data
Authors: Jan Schuster; Alexander WÃ¶lfel; Fabian Brunner; Christian Bergler
Venue: Interspeech 2025 (main)

Abstract: Anomaly detection in multivariate time series data is an extensive field of research with significant impact on a broad spectrum of real-world applications. Building a reliable anomaly detection system is extremely challenging due to data imbalance, lack of labels, next to the actual definition of anomalies. The current study proposes a new transformer-based approach, together with a publicly available time series dataset â the TIKI data corpus â comprising various performance metrics of a Kubernetes cluster distributed over roughly 500 million timestamps. The proposed network, entitled PredTrAD, is compared with the state-of-the-art anomaly detection models â TranAD and DTAAD â in several experiments: (1) verification on three well-known benchmark corpora, (2) evaluation on the TIKI data, and (3) cross-entity anomaly detection. Across all experimental constellations, the proposed model has shown a significant performance growth, proven by F1 score enhancements of 2.41% up to 16.81%.
FairASR: Fair Audio Contrastive Learning for Automatic Speech Recognition
Authors: Jongsuk Kim; Jaemyung Yu; Minchan Kwon; Junmo Kim
Venue: Interspeech 2025 (main)

Abstract: Large-scale ASR models have achieved remarkable gains in accuracy and robustness. However, fairness issues remain largely unaddressed despite their critical importance in real-world applications.  In this work, we introduce FairASR, a system that mitigates demographic bias by learning representations that are uninformative of group membership, enabling fair generalization across demographic groups. Leveraging a multi-demographic dataset, our approach employs a gradient reversal layer to suppress demographic-discriminative features while maintaining the ability to capture generalizable speech patterns through an unsupervised contrastive loss. Experimental results show that FairASR delivers competitive overall ASR performance while significantly reducing performance disparities across different demographic groups.
Automatic Speech Recognition of African American English: Lexical and Contextual Effects
Authors: Hamid Mojarad; Kevin Tang
Venue: Interspeech 2025 (main)

Abstract: Automatic Speech Recognition (ASR) models often struggle with the phonetic, phonological, and morphosyntactic features found in African American English (AAE). This study focuses on two key AAE variables: Consonant Cluster Reduction (CCR) and ING-reduction. It examines whether the presence of CCR and ING-reduction increases ASR misrecognition. Subsequently, it investigates whether end-to-end ASR systems without an external Language Model (LM) are more influenced by lexical neighborhood effect and less by contextual predictability compared to systems with an LM. The Corpus of Regional African American Language (CORAAL) was transcribed using wav2vec 2.0 with and without an LM. CCR and ING-reduction were detected using the Montreal Forced Aligner (MFA) with pronunciation expansion. The analysis reveals a small but significant effect of CCR and ING on Word Error Rate (WER) and indicates a stronger presence of lexical neighborhood effect in ASR systems without LMs.
Improving Synthetic Data Training for Contextual Biasing Models with a Keyword-Aware Cost Function
Authors: Chin Yuen Kwok; Jia Qi Yip; Eng Siong Chng
Venue: Interspeech 2025 (main)

Abstract: Rare word recognition can be improved by adapting ASR models to synthetic data that includes these words. Further improvements can be achieved through contextual biasing, which trains and adds a biasing module into the model architecture to prioritize rare words. While training the module on synthetic rare word data is more effective than using non-rare-word data, it can lead to overfitting due to artifacts in the synthetic audio. To address this, we enhance the TCPGen-based contextual biasing approach and propose a keyword-aware loss function that additionally focuses on biased words when training biasing modules. This loss includes a masked cross-entropy term for biased word prediction and a binary classification term for detecting biased word positions. These two terms complementarily support the decoding of biased words during inference. By adapting Whisper to 10 hours of synthetic data, our method reduced the word error rate on the NSC Part 2 test set from 29.71% to 11.81%.
SOMSRED-SVC: Sequential Output Modeling with Speaker Vector Constraints for Joint Multi-Talker Overlapped ASR and Speaker Diarization
Authors: Naoki Makishima; Naotaka Kawata; Taiga Yamane; Mana Ihori; Tomohiro Tanaka; Satoshi Suzuki; Shota Orihashi; Ryo Masumura
Venue: Interspeech 2025 (main)

Abstract: We have developed a sequential output model with speaker vector constraints for the joint modeling of multi-talker automatic speech recognition (ASR) and speaker diarization. The conventional approach to joint modeling of multi-talker ASR and speaker diarization, called SOMSRED, enables the estimation of speaker embeddings from fully overlapped speech by discretizing the speaker embedding space and treating the speaker embeddings as tokens. However, the predicted speaker embeddings become less distinctive compared to the ones directly obtained from non-overlapping speech due to the discretization. To address this problem, we add a new training objective that optimizes speaker embeddings in continuous space without discretization. Experimental results show that the proposed method avoids overfitting to the discretized speaker tokens and outperforms SOMSRED in both ASR performance and speaker embedding performance.
Thinking in Directivity: Speech Large Language Model for Multi-Talker Directional Speech Recognition
Authors: Jiamin Xie; Ju Lin; Yiteng Huang; Tyler Vuong; Zhaojiang Lin; Zhaojun Yang; Peng Su; Prashant Rawat; Sangeeta Srivastava; Ming Sun; Florian Metze
Venue: Interspeech 2025 (main)

Abstract: Recent studies have demonstrated that prompting large language models (LLM) with audio encodings enables effective speech recognition capabilities. However, the ability of Speech LLMs to comprehend and process multi-channel audio with spatial cues remains a relatively uninvestigated area of research. In this work, we present directional-SpeechLlama, a novel approach that leverages the microphone array of smart glasses to achieve directional speech recognition, source localization, and bystander cross-talk suppression. To enhance the model's ability to understand directivity, we propose two key techniques: serialized directional output training (S-DOT) and contrastive direction data augmentation (CDDA). Experimental results show that our proposed directional-SpeechLlama effectively captures the relationship between textual cues and spatial audio, yielding strong performance in both speech recognition and source localization tasks.
A Study of Real-world Audio-Visual Corpus Design and Production: A Perspective from MISP Challenges
Authors: Hang Chen; Jun Du; Qing Wang; Juan Xie; Shi-Fu XIong
Venue: Interspeech 2025 (main)

Abstract: With the increasing proliferation of speech-driven applications, the challenges in their deployment environments are becoming more prominent. Accordingly, audio-visual speech processing (AVSP) proposed integrating audio and visual information to enhance performance across many speech-processing tasks. In this context, the MISP challenges were organized at ICASSP 2022, 2023, and 2024, respectively. These challenges released audio-visual corpora to support four core tasks: audio-visual wakeup, diarization, speech enhancement, and recognition. The datasets have garnered attention from the global research community, with over 110 teams downloading the corpora. This paper provides a comprehensive analysis of the MISP corpus design from various perspectives, including scenario selection, recording equipment and processes, as well as manual transcription and alignment, highlighting its strengths and limitations and offering insights and recommendations for the design of future AVSP corpora.
VCapAV: A Video-Caption Based Audio-Visual Deepfake Detection Dataset
Authors: Yuxi Wang; Yikang Wang; Qishan Zhang; Hiromitsu Nishizaki; Ming Li
Venue: Interspeech 2025 (main)

Abstract: Most existing deepfake datasets focus on speech synthesis or voice cloning, with little attention given to non-speech environmental sounds. Existing audio-focused datasets also lack video content, restricting progress in multimodal detection. To bridge gaps, we introduce VCapAV, a large-scale audio-visual dataset, designed to advance deepfake detection research involving environmental sound manipulations in multimodal scenarios. VCapAV is constructed through an innovative data generation pipeline that synthesizes realistic environmental audio using Text-to-Audio and Video-to-Audio approaches, while deepfake videos are generated through a Text-to-Video model. We establish two baseline detection tasks on this dataset: (i) audio-only deepfake detection, and (ii) visual-only deepfake detection. Experimental results show that existing detection models on the VCapAV dataset compared to standard datasets such as ASVspoof 2019 LA and AV-Deepfake1M. The dataset and baseline codes are released.
J-SPAW: Japanese speaker verification and spoofing attacks recorded in-the-wild dataset
Authors: Sayaka Shiota; Suzuka Horie; Kouta Kanno; Shinnosuke Takamichi
Venue: Interspeech 2025 (main)

Abstract: In this paper, we present J-SPAW (Japanese speaker verification and spoofing attacks recorded in-the-wild dataset), a novel speech database designed for speaker verification and spoofing detection in-the-wild environments1.J-SPAW is a unique database that simultaneously evaluates speaker verification and spoofing detection under realistic conditions, focusing on physical access scenarios, including replay attacks. The database includes diverse physical access scenarios, enhancing the variety and applicability of datasets available for anti-spoofing research. Our experimental results demonstrate that J-SPAW enables comprehensive analysis of spoofing detection from various perspectives and can be utilized for both speaker verification and spoofing detection tasks. This contribution is expected to advance state-of-the-art speaker verification and spoofing detection and provide a valuable resource for future research.
CommissionsQC: a QuÃ©bec French Speech Corpus for Automatic Speech Recognition
Authors: Coralie Serrand; Amira Morsli; Gilles Boulianne
Venue: Interspeech 2025 (main)

Abstract: We introduce CommissionsQC, a novel QuÃ©bec French speech corpus designed for training and evaluation of Automatic Speech Recognition (ASR) systems. The dataset addresses the underrepresentation of QuÃ©bec French in existing multilingual speech corpora, which mostly include European French. CommissionsQC includes over 1,000 hours of high-quality, spontaneous QuÃ©bec French speech derived from two public inquiries commissioned by QuÃ©bec's provincial government. The official pdf transcripts, produced by stenographers, were processed to standardize text and align it with audio recordings. The corpus underwent extensive normalization and filtering to ensure accurate alignment and segmentation. Quality metrics and ASR training results confirm that CommissionsQC is suitable for evaluating or enhancing of speech recognition performance in QuÃ©bec French.