GenAI4ELab/papercli-papers huggingface dataset:
A Two-Stage Hierarchical Deep Filtering Framework for Real-Time Speech Enhancement
Authors: Shenghui Lu; Hukai Huang; Jinanglong Yao; Kaidi Wang; Qingyang Hong; Lin Li
Venue: Interspeech 2025 (main)

Abstract: This paper proposes a model that integrates sub-band processing and deep filtering to fully exploit information from the target time-frequency (TF) bin and its surrounding TF bins for single-channel speech enhancement. The sub-band module captures surrounding frequency bin information at the input, while the deep filtering module applies filtering at the output to both the target TF bin and its surrounding TF bins. To further improve the model performance, we decouple deep filtering into temporal and frequency components and introduce a two-stage framework, reducing the complexity of filter coefficient prediction at each stage. Additionally, we propose the TAConv module to strengthen convolutional feature extraction. Experimental results demonstrate that the proposed hierarchical deep filtering network (HDF-Net) effectively utilizes surrounding TF bin information and outperforms other advanced systems while using fewer resources.
Real-Time Audio-Visual Speech Enhancement Using Pre-trained Visual Representations
Authors: Teng Ma; Sile Yin; Li-Chia Yang; Shuo Zhang
Venue: Interspeech 2025 (main)

Abstract: Speech enhancement in audio-only settings remains challenging, particularly in the presence of interfering speakers. This paper presents a simple yet effective real-time audio-visual speech enhancement (AVSE) system, RAVEN, which isolates and enhances the on-screen target speaker while suppressing interfering speakers and background noise. We investigate how visual embeddings learned from audio-visual speech recognition (AVSR) and active speaker detection (ASD) contribute to AVSE across different SNR conditions and numbers of interfering speakers. Our results show concatenating embeddings from AVSR and ASD models provides the greatest improvement in low-SNR, multi-speaker environments, while AVSR embeddings alone perform best in noise-only scenarios. In addition, we develop a real-time streaming system that operates on a computer CPU and we provide a video demonstration and code repository. To our knowledge, this is the first open-source implementation of a real-time AVSE system.
Lightweight Speech Enhancement Model Based on Harmonic Attention and Phase Estimation with Skin-Attachable Accelerometer
Authors: Yonghun Song; Yeeun Kim; Yoonyoung Chung
Venue: Interspeech 2025 (main)

Abstract: Skin-attachable accelerometers (ACCs) capture speech vibrations through the skin, providing a noise-robust complement to microphone (MIC) signals. However, prior multi-modal models combining these signals face trade-offs between processing overhead and performance. This study proposes a lightweight ACC-assisted U-Net (LAU-Net) for real-time speech enhancement. The LAU-Net employs a harmonic attention module to enhance spectral clarity by emphasizing speech harmonics predicted from ACC signals while only increasing the parameter count from 92.29k to 92.98k. The phase estimation block of LAU-Net adaptively integrates ACC and MIC phases based on noise levels, eliminating the need for phase data training. The LAU-Net achieves a PESQ of 2.92 with 39M MACs/s on the TAPS dataset, demonstrating a balance between speech quality and computational efficiency. These results highlight the LAU-Net as a practical solution for robust and efficient speech processing with real-time edge deployment.
TSDT-Net: Ultra-Low-Complexity Two-Stage Model Combining Dual-Path-Transformer and Transform-Average-Concatenate Network for Speech Enhancement
Authors: Yi Gao; Hangting Chen; Siyu Zhang; Qingshan Yang; Jingcong Chen
Venue: Interspeech 2025 (main)

Abstract: This paper introduces TSDT-Net, a dual-stage ultra-low-complexity architecture for speech enhancement which achieves higher denoising performance with limited parameter number and computational cost. Its first stage utilizes a simplified Dual-Path-Transformer(DPT) structure. In the second stage, the first-stage output and original noisy signal are treated as dual-channel inputs, modeled as a beamforming optimization problem. An enhanced Transform-Average-Concatenate (TAC) network processes these channels through spectral filtering and enhancement. Fast linear transformers ensure ultra-low computational overhead, while gated networks in both stages facilitate complex Ideal Ratio Mask (cIRM) construction. Residual connections between stages enable performance synergies. Evaluations on the INTERSPEECH 2020 DNS Challenge demonstrate TSDT-Net's superiority, achieving staet-of-the-art DNSMOS and PESQ scores with significant margins over single-stage models under stringent computational constraints (<700K parameters and <500M/ 200M MACs). This efficiency enables deployment across diverse embedded devices.
Structured Codebook Based Hierarchical Framework for DNN for Computationally Efficient Speech Enhancement
Authors: Chidambar B; Hanumanth Rao Naidu
Venue: Interspeech 2025 (main)

Abstract: Deep Neural Networks (DNN) based single-channel speech enhancement techniques have surpassed traditional techniques in handling non-stationary noise, however, they are computationally demanding. In this work, we introduce a novel Hierarchical Framework for DNNs (HF-DNN) for speech enhancement that replaces a single complex and computationally expensive DNN model with multiple simpler and less complex DNNs that are hierarchically connected. This is achieved by using structured codebooks of speech parameters, like log power spectra, that are generated by exploiting hierarchical relation between the speech training data. The proposed HF-DNN reduces computational complexity significantly compared to a large DNN while maintaining speech enhancement performance. Importantly, such a framework can be extended to other speech processing tasks, such as speech recognition, speaker verification, etc., where parametric models of speech data are utilized.
Evaluation of Three Automatic Alignment Tools for the Processing of Non-native French
Authors: Qian Zhou; Mathilde Hutin
Venue: Interspeech 2025 (main)

Abstract: The production of non-native speech is known to display "cross-language phonetic interference", which makes such speech uneasy to align and label automatically. Automatic phonetic alignment refers to an automated process whereby software synchronizes speech with its transcription, usually at the phone and word levels. This method has proven useful and reliable for native speech, yet this reliability usually does not extend to non-native speech. This paper proposes to test three major automatic aligners (WebMAUS, MFA and SPPAS) on non-native French uttered by two native speakers of Chinese by comparing them with two manual segmentations. This paper's goal is to offer non-computer linguists a preliminary investigation on which to rely when choosing a tool for their studies in non-native phonetics or language didactics. Results show that the best performing tool for labeling is SPPAS while the best performing tool for both word- and phone-segmentation overall is WebMAUS and MFA the worst.
CrossPhon: An Auto Phone Mapping Tool to Streamline Cross-language Modeling for Phone Alignment of Low-resource Languages
Authors: Hongchen Wu; Yixin Gu
Venue: Interspeech 2025 (main)

Abstract: Phone alignment matches spoken sounds with text, streamlining speech dataset creation and analysis. However, most trained aligners focus on Indo-European languages, leaving under-resourced languages unsupported. Developing new aligners for these languages requires expertise and large datasets, which are often scarce. Cross-language phone alignment offers a solution using aligners trained in one language to align speech in another, but it traditionally relies on expert-crafted phone mappings. Our tool, CrossPhon, automates this process, making cross-language phone alignment more efficient. In tests on 14 languages from 7 families, CrossPhon achieved agreement rates of 78.95% to 97.77% compared to human expert mappings and delivered competitive performance in cross-language phone alignment. CrossPhon provides an efficient, reliable solution for generating cross-language phone alignment in under-resourced languages, helping bridge the digital divide and efficiently study these languages.
Multi-lingual and Zero-Shot Speech Recognition by Incorporating Classification of Language-Independent Articulatory Features
Authors: Ryo Magoshi; Shinsuke Sakai; Jaeyoung Lee; Tatsuya Kawahara
Venue: Interspeech 2025 (main)

Abstract: We address multi-lingual speech recognition including unknown or zero-shot languages based on the International Phonetic Alphabet (IPA) and articulatory features. Articulatory features are language-independent representations for IPA based on phonetic knowledge. In the previous studies, however, they were mostly limited to two dimensions of place of articulation and manner of articulation. Moreover, the classification of articulatory features were not well aligned with phone recognition. In this study, we adopt a comprehensive 24-dimensional vector representation, and propose a training method in which IPA tokens and their corresponding articulatory features are simultaneously predicted based on CTC alignment. Experiments are conducted by fine-tuning the wav2vec 2.0 XLS-R model over 22 languages, and the results demonstrated significant improvements on average as well as in zero-shot language settings.
Instantaneous changes in acoustic signals reflect syllable progression and cross-linguistic syllable variation
Authors: Haley Hsu; Dani Byrd; Khalil Iskarous; Louis Goldstein
Venue: Interspeech 2025 (main)

Abstract: While abstract speech representations often exploit sequenced syllable units, how exactly syllables as abstract cognitive compositional structure relate to observable patterns in the articulatory and acoustic signals remains opaque. Previous work suggests oscillatory acoustic properties link such linguistic representations to physical events. We probe this relationship by testing temporal coordination between changes in spectral energy and amplitude with syllable boundary locations through phase-locking analyses. Results for syllabic nuclei demonstrate these phase-locking values (PLVs) track syllable progression in both English and Tashlhiyt. Further, cross-language preferences for different syllable nucleus types are found to be reflected in their respective PLVs. Overall, the findings demonstrate a tight coordination between abstract syllable units and quantifiable signal properties and additionally provide novel dynamical grounding for cross-linguistic syllable nucleus preferences.
Influence of Proficiency and L2 Experience on Dynamic Spectral Cue Utilization in L2 Vowel Perception and Production
Authors: Linda Bakkouche; Brechtje Post
Venue: Interspeech 2025 (main)

Abstract: The acquisition of English vowels as an L2 is complex, yet most studies focus on static measures, with little attention to dynamic spectral cues like Vowel-Inherent Spectral Change (VISC). It remains unclear how language experience and length of residence (LOR) in immersion-rich environments affect perception-production alignment. This study examines Polish learnersâ perception and production of /e-Ã¦/ (DRESS-TRAP) and /i-I/ (FLEECE-KIT). These contrasts are challenging due to phonetic similarity and category overlap as predicted by L2 models. Advanced learners showed greater perceptual accuracy and more consistent production, especially for /i-I/, while /e-Ã¦/ remained difficult. With higher proficiency, learners exhibited greater formant movement (20-40% of vowel duration), but LOR and language experience were not significant predictors. These findings provide insight into phonetic similarity in theoretical models of L2 vowel acquisition.
A Bayesian Approach to L2 Fluency Ratings by Native and Nonnative Listeners
Authors: Kakeru Yazawa; Takayuki Konishi
Venue: Interspeech 2025 (main)

Abstract: This study investigates how native and nonnative listeners evaluate the fluency of Japanese speakers' English using a Bayesian modeling framework. Data were obtained from 16 listeners with diverse linguistic backgrounds (Cantonese, English, French, German, Japanese, Korean, Mandarin, Polish, Punjabi, and Spanish), who rated English read speech samples from 180 Japanese speakers, in the J-AESOP corpus. Utterance fluency measures included speed (syllable- or segment-based articulation rate), breakdown (pause frequency and duration), and re pair (repetitions). Results revealed that nonnative listeners, particularly those with Asian language backgrounds, were generally more lenient and less reliant on speech rate than native listeners, highlighting inter-listener variability previously overlooked. Model comparisons also revealed that segment-based articulation rate better captures utterance speed fluency than the commonly adopted syllable-based articulation rate.
Are loan sequences different from foreign sequences? A perception study with Japanese listeners on coronal obstruent â high front vowel sequences
Authors: Silke Hamann; Andrea AliÄehajiÄ
Venue: Interspeech 2025 (main)

Abstract: Native phonotactics influences speech perception, as numerous studies have shown. The present study tackles the question whether there is a difference in perceptual performance if the involved sequence occurs only in loanwords, compared to a sequence that does not occur at all in the native language. This was tested with the native Japanese sequences of palatal affricate plus /i/, compared to /ti/ (accepted only in loanwords) versus /zi/ (not accepted in Japanese) in an online AX discrimination task with 39 Japanese speakers (21-63 years old), who also had to answer three questions on their received English input. Participants performed significantly better at discriminating the accepted loan sequence /ti/, though discrimination of the foreign sequence /zi/ was also quite high (ranging from 40-100% correct). The results indicate that discriminability is only partly guided by native phonotactics. A potential role of amount of English input measured by self-report could not be attested.
Relative cue weighting in multilingual stop voicing production
Authors: Le Xuan Chan; Annika Heuser
Venue: Interspeech 2025 (main)

Abstract: How does a multilingual speaker produce similar phonological contrasts across the different languages that they speak? Some theories predict crosslinguistic influence while others predict that multilinguals keep separate sound inventories for each language. In this paper, we present crosslinguistic data from early multilingual speakers in Malaysia. We investigate the interaction of a true voicing language (Malay), a variable voicing language (English), and an aspiration language (Mandarin). Using a random forest classification of nine acoustic correlates of stop voicing, we show that 1) all early multilinguals show language-specific productions of stop voicing, and 2) variation driven by dominance can still be observed despite this language-specificity. In addition, we present evidence that closure voicing is a salient correlate alongside aspiration in Malaysian English, and that English is more reliant on secondary correlates than Malay and Mandarin.
Variability in Intervocalic /t/ and Community Diversity in Australian English
Authors: Hannah White; Joshua Penney; Felicity Cox
Venue: Interspeech 2025 (main)

Abstract: The voiceless alveolar stop /t/ exhibits considerable variation in English. Realisations of /t/ vary depending phonetic context and social factors such as gender, age and socioeconomic status. Generally, studies on Australian English have focused on the âmainstreamâ variety, without acknowledging the wide range of linguistic diversity speakers are exposed to in contemporary multicultural Australian society. In the present paper, we explore intervocalic /t/ variation in data collected from 183 speakers as part of the Multicultural Australian English â Voices of Sydney corpus. Results show that, in certain phonetic contexts, exposure to community linguistic diversity can affect intervocalic /t/ realisation, with speakers from more diverse areas showing a preference for a single variant (the tap) compared to those from less diverse areas. We interpret this as an example of simplification that can occur in diverse communities where there is extreme variability in ambient language exposure.
Vector Quantized Cross-lingual Unsupervised Domain Adaptation for Speech Emotion Recognition
Authors: Pravin Mote; Donita Robinson; Elizabeth Richerson; Carlos Busso
Venue: Interspeech 2025 (main)

Abstract: Building speech emotion recognition (SER) models for low-resource languages is challenging due to the scarcity of labeled speech data. This limitation mandates the development of cross-lingual unsupervised domain adaptation techniques to effectively utilize labeled data from resource-rich languages. Inspired by the TransVQA framework, we propose a method that leverages a shared quantized feature space to enable knowledge transfer between labeled and unlabeled data across languages. The approach utilizes a quantized codebook to capture shared features, while reducing the domain gap, and aligning class distributions, thereby improving classification accuracy. Additionally, an information loss (InfoLoss) mechanism mitigates critical information loss during quantization. InfoLoss achieves this goal by minimizing the loss within the simplex of posterior class label distributions. The proposed method demonstrates superior performance compared to state-of-the-art baseline approaches.
HYFuse: Aligning Heterogeneous Speech Pre-Trained Representations in Hyperbolic Space for Speech Emotion Recognition
Authors: Orchid Chetia Phukan; Girish; Mohd Mujtaba Akhtar; Swarup Ranjan Behera; Pailla Balakrishna Reddy; Arun Balaji Buduru; Rajesh Sharma
Venue: Interspeech 2025 (main)

Abstract: Compression-based representations (CBRs) from neural audio codecs such as EnCodec capture intricate acoustic features like pitch and timbre, while representation-learning-based representations (RLRs) from pre-trained models trained for speech representation learning such as WavLM encode high-level semantic and prosodic information. Previous research on Speech Emotion Recognition (SER) has explored both, however, fusion of CBRs and RLRs havenât been explored yet. In this study, we solve this gap and investigate the fusion of RLRs and CBRs and hypothesize they will be more effective by providing complementary information. To this end, we propose, HYFuse, a novel framework that fuses the representations by transforming them to hyperbolic space. With HYFuse, through fusion of x-vector (RLR) and Soundstream (CBR), we achieve the top performance in comparison to individual representations as well as the homogeneous fusion of RLRs and CBRs and report SOTA.
Meta-PerSER: Few-Shot Listener Personalized Speech Emotion Recognition via Meta-learning
Authors: Shi-Xin Fang; Liang-Yeh Shen; Yi-Cheng Lin; Huang-Cheng Chou; Hung-yi Lee
Venue: Interspeech 2025 (main)

Abstract: This paper introduces Meta-PerSER, a novel meta-learning framework that personalizes Speech Emotion Recognition (SER) by adapting to each listenerâs unique way of interpreting emotion. Conventional SER systems rely on aggregated annotations, often overlooking individual subtleties and leading to inconsistent predictions. In contrast, Meta-PerSER leverages a Model-Agnostic Meta-Learning (MAML) approach enhanced with Combined-Set Meta-Training, Derivative Annealing, and per-layer per-step learning rates, enabling rapid adaptation with only a few labeled examples. By integrating robust representations from pre-trained self-supervised models, our framework first captures general emotional cues and then fine-tunes itself to personal annotation styles. Experiments on the IEMOCAP corpus demonstrate that Meta-PerSER significantly outperforms baseline methods in both seen and unseen data scenarios, highlighting its promise for personalized emotion recognition.
Breaking Resource Barriers in Speech Emotion Recognition via Data Distillation
Authors: Yi Chang; Zhao Ren; Zhonghao Zhao; Thanh Tam Nguyen; Kun Qian; Tanja Schultz; BjÃ¶rn W. Schuller
Venue: Interspeech 2025 (main)

Abstract: Speech emotion recognition (SER) plays a crucial role in human-computer interaction. The emergence of edge devices in the Internet of Things (IoT) presents challenges in constructing intricate deep learning models due to constraints in memory and computational resources. Moreover, emotional speech data often contains private information, raising concerns about privacy leakage during the deployment of SER models. To address these challenges, we propose a data distillation framework to facilitate efficient development of SER models in IoT applications using a synthesised, smaller, and distilled dataset. Our experiments demonstrate that the distilled dataset can be effectively utilised to train SER models with fixed initialisation, achieving performances comparable to those developed using the original full emotional speech dataset.
Multi-Teacher Language-Aware Knowledge Distillation for Multilingual Speech Emotion Recognition
Authors: Mehedi Hasan Bijoy; Dejan Porjazovski; TamÃ¡s GrÃ³sz; Mikko Kurimo
Venue: Interspeech 2025 (main)

Abstract: Speech Emotion Recognition (SER) is crucial for improving human-computer interaction. Despite strides in monolingual SER, extending them to build a multilingual system remains challenging. Our goal is to train a single model capable of multilingual SER by distilling knowledge from multiple teacher models. To address this, we introduce a novel language-aware multi-teacher knowledge distillation method to advance SER in English, Finnish, and French. It leverages Wav2Vec2.0 as the foundation of monolingual teacher models and then distills their knowledge into a single multilingual student model. The student model demonstrates state-of-the-art performance, with a weighted recall of 72.9 on the English dataset and an unweighted recall of 63.4 on the Finnish dataset, surpassing fine-tuning and knowledge distillation baselines. Our method excels in improving recall for sad and neutral emotions, although it still faces challenges in recognizing anger and happiness.
Learning More with Less: Self-Supervised Approaches forLow-Resource Speech Emotion Recognition
Authors: Ziwei Gong; Pengyuan Shi; Kaan Donbekci; Lin Ai; Run Chen; David Sasu; Zehui Wu; Julia Hirschberg
Venue: Interspeech 2025 (main)

Abstract: Speech Emotion Recognition (SER) has seen significant progress with deep learning, yet remains challenging for Low-Resource Languages (LRLs) due to the scarcity of annotated data. In this work, we explore unsupervised learning to improve SER in low-resource settings. Specifically, we investigate contrastive learning (CL) and Bootstrap Your Own Latent (BYOL) as self-supervised approaches to enhance cross-lingual generalization. Our methods achieve notable F1 score improvements of 10.6% in Urdu, 15.2% in German, and 13.9% in Bangla, demonstrating their effectiveness in LRLs. Additionally, we analyze model behavior to provide insights on key factors influencing performance across languages, and also highlighting challenges in low-resource SER. This work provides a foundation for developing more inclusive, explainable, and robust emotion recognition systems for underrepresented languages.