GenAI4ELab/papercli-papers huggingface dataset:
2D Immersed Boundary Method in Vocal Tract Acoustics: An EulerianâLagrangian Model for Simulation of Diphthongs
Authors: Rongshuai Wu; Debasish Ray Mohapatra; Sidney Fels
Venue: Interspeech 2025 (main)

Abstract: Modeling dynamic vocal tracts requires accurate interpolation between target vowels for diphthong synthesis. High-fidelity Finite Element models are computationally expensive, often needing complete remeshing of the computational domain to maintain interpolation stability. In contrast, Digital Waveguide models (DWM) suffer from discretization errors due to staircased approximations of tract geometries. We model dynamic tracts using the 2D Immersed Boundary Method, which represents tract contours in the Lagrangian domain and uses a 2D Finite-Difference Time-Domain scheme to solve wave equations in the Eulerian domain. This framework enables free boundary motion within a fixed Cartesian grid, avoiding remeshing and staircasing. We synthesized three diphthongs and compared their spectral features with a 3D DWM model and recorded speech. Results show that the F1-F2 trajectories closely align with those from the 3D DWM and recorded speech, achieving correlation coefficients exceeding 0.8.
Reconstruction of the Complete Vocal Tract Contour Through Acoustic to Articulatory Inversion Using Real-Time MRI Data
Authors: Sofiane Azzouz; Pierre-AndrÃ© Vuissoz; Yves Laprie
Venue: Interspeech 2025 (main)

Abstract: Articulatory to acoustic inversion has often been limited to a small part of the vocal tract because the data are generally EMA (ElectroMagnetic Articulography) data requiring sensors to be glued to easily accessible articulators. The presented acoustic to articulation model focuses on the inversion of the entire vocal tract from the glottis, the complete tongue, the velum, to the lips. It relies on a realtime dynamic MRI database of more than 3 hours of speech. The data are the denoised speech signal and the automatically segmented articulator contours. Several bidirectional LSTM-based approaches have been used, either inverting each articulator individually or inverting all articulators simultaneously. To our knowledge, this is the first complete inversion of the vocal tract. The average RMSE precision on the test set is 1.65 mm to be compared with the pixel size which is 1.62 mm.
Co-registration of real-time MRI and respiration for speech research
Authors: Yubin Zhang; Prakash Kumar; Ye Tian; Ziwei Zhao; Xuan Shi; Kevin Huang; Kevin Lee; Haley Hsu; Shrikanth Narayanan; Krishna Nayak; Louis Goldstein
Venue: Interspeech 2025 (main)

Abstract: In the past, research on speech respiration and articulation has mostly been conducted independently of each other. Due to the limited availability of methods for simultaneously measuring respiration and articulation, the coordination of these two subsystems to encode linguistic structures is yet to be fully examined. In this study, we present a protocol for performing co-registration of real-time vocal tract MRI and respiratory plethysmography for speech research. The procedures for data acquisition and analysis are first presented. Then, the preliminary results from two sample speech production experiments are reported. The first experiment is on the timing between articulation and respiration, and the second one examines articulatory and respiratory aspects of linguistic focus encoding. The implications of the method for speech research are discussed.
SPEAKtoCOPD: a flashmob study to collect COPD speech
Authors: Loes van Bemmel; Lauren G Reinders; Folkert Brijker; Bas Holverda; Frits M.E. Franssen; Hanneke van Helvoort; Visara Urovi; Marieke Spreeuwenberg; Sami O. Simons
Venue: Interspeech 2025 (main)

Abstract: Chronic Obstructive Pulmonary Disease (COPD) is the most common chronic respiratory disease in the world. Fluctuations in symptoms frequently occur in COPD and might require additional treatments. Automatic monitoring or diagnosis of these fluctuations may aid the management of people with COPD. Speech analysis is a possible remote non-invasive methods to achieve this. However, this analysis is disease and possibly language specific and the development of speech models requires high-quality and high-quantity data, which are often not available. In this study we introduce SPEAKtoCOPD: a dataset of Dutch speech collected with a flash mob study targeting people with a respiratory disease (specifically COPD). In this paper, we describe the flash mob methodology, our collected data, and perform an automatic quality check to assess whether the speech tasks have been performed correctly. We aim to publish our gathered data for researchers to improve speech analysis for respiratory health.
Developing a LeFF Transformer Model for Exacerbated Speech Detection in COPD and Asthma
Authors: Yuyang Yan; Sami O. Simons; Visara Urovi
Venue: Interspeech 2025 (main)

Abstract: The acoustic features of speech exhibit variations across different respiratory conditions, highlighting the potential of voice analysis as a valuable tool for non-invasive monitoring systems. Early detection of exacerbations plays a critical role in the effective management of chronic respiratory diseases, such as chronic obstructive pulmonary disease (COPD) and asthma. This paper presents the utilization of fused acoustic features from multiple domains, integrated with a Locally-enhanced Feed-Forward Network (LeFF) Transformer model, to classify exacerbated and stable speech in COPD and asthma patients. The proposed methodology is evaluated on the TACTICAS dataset, demonstrating superior performance compared to current state-of-the-art approaches, underscoring its potential for exacerbations monitoring in COPD and asthma patients.
Towards Pre-training an Effective Respiratory Audio Foundation Model
Authors: Daisuke Niizumi; Daiki Takeuchi; Masahiro Yasuda; Binh Thien Nguyen; Yasunori Ohishi; Noboru Harada
Venue: Interspeech 2025 (main)

Abstract: Recent advancements in foundation models have sparked interest in respiratory audio foundation models. However, the effectiveness of applying conventional pre-training schemes to datasets that are small-sized and lack diversity has not been sufficiently verified. This study aims to explore better pre-training practices for respiratory sounds by comparing numerous pre-trained audio models. Our investigation reveals that models pre-trained on AudioSet, a general audio dataset, are more effective than the models specifically pre-trained on respiratory sounds. Moreover, combining AudioSet and respiratory sound datasets for further pre-training enhances performance, and preserving the frequency-wise information when aggregating features is vital. Along with more insights found in the experiments, we establish a new state-of-the-art for the OPERA benchmark, contributing to advancing respiratory audio foundation models.
Effect of physical exercise on voice in people living with COPD
Authors: Lauren G Reinders; Loes van Bemmel; Alexander Mackay; David Nobbs; Frits M.E. Franssen; Hester Gietema; Simona SchÃ¤fer; Sami O. Simons
Venue: Interspeech 2025 (main)

Abstract: Previous research on the effect of exercise on voice has been mainly performed in a healthy population. However, the competition between breathing and speaking might be more pronounced in people with COPD. We aimed to evaluate voice features change in people living with COPD after exercise. A prolonged vowel /a/ and paced reading task were performed before and after an exercise task. In this exploratory study, differences were found in the sustained phonation for shimmer (apq5 and local), vowel duration, F1 frequency, and H1-A3 harmonic difference, when taking sex into account. Also, we saw a trend that voice features change by increasing COPD severity for HNR (mean) in the sustained phonation, and in the forced reading task for F2 frequency, H1-A3 harmonic difference (mean) and Hammarberg index. More research is needed using voice recordings from a larger sample size to confirm these findings, and possibly implement this in an exacerbation model using voice.
Adaptive Differential Denoising for Respiratory Sounds Classification
Authors: Gaoyang Dong; Zhicheng Zhang; Ping Sun; Minghui Zhang
Venue: Interspeech 2025 (main)

Abstract: Automated respiratory sound classification faces practical challenges from background noise and insufficient denoising in existing systems. We propose Adaptive Differential Denoising network, that integrates noise suppression and pathological feature preservation via three innovations: 1) Adaptive Frequency Filter with learnable spectral masks and soft shrink to eliminate noise while retaining diagnostic high-frequency components; 2) A Differential Denoise Layer using differential attention to reduce noise-induced variations through augmented sample comparisons; 3) A bias denoising loss jointly optimizing classification and robustness without clean labels.  Experiments on the ICBHI2017 dataset show that our method achieves 65.53% of the Score, which is improved by 1.99% over the previous sota method. The code is available in https://github.com/deegy666/ADD-RSC
Disentangling Dual-Encoder Masked Autoencoder for Respiratory Sound Classification
Authors: Peidong Wei; Shiyu Miao; Lin Li
Venue: Interspeech 2025 (main)

Abstract: Deep neural networks have been applied to audio spectrograms for respiratory sound classification, but it is still remains challenging to achieve satisfactory performance due to the scarcity of available data. Moreover, domain mismatch may be introduced into the trained models as a result of the respiratory sound samples being collected from various electronic stethoscopes, patient demographics, and recording environments. To tackle this issue, we proposed a modified Masked Autoencoder (MAE) model, named Disentangling Dual-Encoder MAE (DDE-MAE) for respiratory sound classification. Two independent encoders were designed to capture disease-related and disease-irrelevant information separately, achieving feature disentanglement to reduce the domain mismatch. Our method achieves a competitive performance on the ICBHI dataset.
Patient-Aware Feature Alignment for Robust Lung Sound Classification: Cohesion-Separation and Global Alignment Losses
Authors: Seung Gyu Jeong; Seong Eun Kim
Venue: Interspeech 2025 (main)

Abstract: Lung sound classification is vital for early diagnosis of respiratory diseases. However, biomedical signals often exhibit inter-patient variability even among patients with the same symptoms, requiring a learning approach that considers individual differences. We propose a Patient-Aware Feature Alignment (PAFA) framework with two novel losses, Patient Cohesion-Separation Loss (PCSL) and Global Patient Alignment Loss (GPAL). PCSL clusters features of the same patient while separating those from other patients to capture patient variability, whereas GPAL draws each patient's centroid toward a global center, preventing feature space fragmentation. Our method achieves outstanding results on the ICBHI dataset with a score of 64.84% for four-class and 72.08% for two-class classification. These findings highlight PAFA's ability to capture individualized patterns and demonstrate performance gains in distinct patient clusters, offering broader applications for patient-centered healthcare.
Improving Respiratory Sound Classification with Architecture-Agnostic Knowledge Distillation from Ensembles
Authors: Miika Toikkanen; June-Woo Kim
Venue: Interspeech 2025 (main)

Abstract: Respiratory sound datasets are limited in size and quality, making high performance difficult to achieve. Ensemble models help but inevitably increase compute cost at inference time. Soft label training distills knowledge efficiently with extra cost only at training. In this study, we explore soft labels for respiratory sound classification as an architecture-agnostic approach to distill an ensemble of teacher models into a student model. We examine different variations of our approach and find that even a single teacher, identical to the student, considerably improves performance beyond its own capability, with optimal gains achieved using only a few teachers. We achieve the new state-of-the-art Score of 64.39 on ICHBI, surpassing the previous best by 0.85 and improving average Scores across architectures by more than 1.16. Our results highlight the effectiveness of knowledge distillation with soft labels for respiratory sound classification, regardless of size or architecture.
Theoretical proposal for a unified Bayesian model of adaptation in non-interactive and interactive speech production
Authors: MÃ©len Guillaume; Anahita Basirat; Julien Diard
Venue: Interspeech 2025 (main)

Abstract: In the scientific study of speech production, as well as in speech and language therapy, a variety of experimental paradigms and tasks resulting in plasticity and adaptation of the system are used. However, existing computational models accounting for adaptation appear segregated: feedback effects during non-interactive, altered speech are mostly accounted for by speech production models, whereas accommodation effects during interactive speech are mostly accounted for by speech perception models. In this paper, we consider COSMO, a previous unified Bayesian framework of speech perception and speech production, as the basis for a theoretical proposal and Bayesian model of adaptation effects. We show how these effects in both non-interactive and interactive speech production experiments could be described in this framework, thanks for its ability to separate knowledge and representations from their involvement during speech tasks.
Self-supervised Optimality-Guided Learning of Speech Articulation
Authors: Juraj Å imko; Benjamin Elie; Alice Turk
Venue: Interspeech 2025 (main)

Abstract: This paper introduces a novel approach for modeling speech articulatory planning based on Optimal Control Theory. The presented approach uses an internal feed-forward controller model that learns to predict optimal articulatory commands minimizing a context-dependent objective function. This objective function combines conflicting tasks of minimizing articulatory effort and maximizing the recognition probability of a target vowel based on acoustic characteristics. We present a self-supervised optimality-guided architecture for training the feedforward internal model that directly uses the objective function as a training loss. Simulations involving isolated vowels of American-English show that online training of the internal model enables feedforward estimation of near-optimal articulatory parameters.
Extended High-frequency Cues to Phoneme Recognition: Insights from ASR
Authors: Zhe-chen Guo; Bharath Chandrasekaran
Venue: Interspeech 2025 (main)

Abstract: There is emerging evidence that extended high frequencies (EHFs; >8 kHz) improve speech perception in noise. Yet, the mechanisms underlying this benefit remain unclear. We investigated whether EHFs contribute to phoneme recognition using an automatic speech recognition (ASR) model. A neural network model was trained to decode phonemes from cochleagrams of broadband speech and speech low-pass filtered at 8 or 6 kHz in quiet and masked conditions with varying target-to-masker ratios (TMRs) and target-masker spatial separations. Compared with filtered speech, broadband speech improved phoneme recognition accuracy in masked conditions, particularly at lower TMRs, but showed no benefit in quiet. Removing EHFs increased the probability of the model omitting a phoneme more for consonants than vowels. The findings suggest that the EHF benefit in adverse conditions may partly arise from enhanced phoneme processing, highlighting the potential of improving audiometry and ASR by including EHFs.
Decoding Speaker-Normalized Pitch from EEG for Mandarin Perception
Authors: Jia-Xin Chen; Yi-Ming Wang; Ziyu Zhang; Jiayang Han; Yin-Long Liu; Rui Feng; Xiuyuan Liang; Zhen-Hua Ling; Jia-Hong Yuan
Venue: Interspeech 2025 (main)

Abstract: The same speech content produced by different speakers exhibits significant differences in pitch contour, yet listeners' semantic perception remains unaffected. This phenomenon may stem from the brain's perception of pitch contours being independent of individual speakers' pitch ranges. In this work, we recorded electroencephalogram (EEG) while participants listened to Mandarin monosyllables with varying tones, phonemes, and speakers. The CE-ViViT model is proposed to decode raw or speaker-normalized pitch contours directly from EEG. Experimental results demonstrate that the proposed model can decode pitch contours with modest errors, achieving performance comparable to state-of-the-art EEG regression methods. Moreover, speaker-normalized pitch contours were decoded more accurately, supporting the neural encoding of relative pitch.
SSF-DST: A Spectro-Spatial Features Enhanced Deep Spatiotemporal Network for EEG-Based Auditory Attention Detection
Authors: Tong Zhu; Xiaoke Yang; Jian Zhou; Lu Li; Zhao Lv; Cunhang Fan
Venue: Interspeech 2025 (main)

Abstract: In a cocktail party scenario, humans can selectively attend to different speakers. EEG-based auditory attention detection (AAD) aims to identify the attended speaker from brain signals. Existing AAD methods enhance features through temporal-frequency fusion, but frequency features incur heavy computational cost with limited contribution. Furthermore, many methods utilize attention mechanisms to capture temporal dynamics, which may not fully capture the complex interplay of spatial and temporal information. To address these issues, we propose a spectro-spatial features enhanced deep spatiotemporal network (SSF-DST). SSF-DST efficiently extracts spectro-spatial features from frequency domain to enhance spatial patterns of EEG, and constructs deep spatiotemporal features to capture long-term dependencies and spatial patterns. Experimental results demonstrate that SSF-DST outperforms the state-of-the-art model in classification accuracy while reducing the trainable parameters by 50%.
Overestimated performance of auditory attention decoding caused by experimental design in EEG recordings
Authors: Yujie Yan; Xiran Xu; Haolin Zhu; Songyi Li; Bo Wang; Xihong Wu; Jing Chen
Venue: Interspeech 2025 (main)

Abstract: Auditory Attention Decoding (AAD) identifies a listener's focus in complex auditory scenes based on cortical neural responses. High decoding performance using DNN-based methods has been achieved with public EEG datasets. However, performance may be overestimated as models might learn temporal-autocorrelation features rather than auditory attention-related features. While data splitting risks have been discussed, experimental design risks have not. In this work, we collected a non-block design (NBD) scalp-EEG and ear-EEG joint dataset and compared it to previous block design (BD) datasets using DNN-based models. Results show a significant accuracy drop from BD to NBD dataset, while a linear stimulus reconstruction model remains robust. Inter-trial phase coherence analysis confirms stronger neural phase-locking to attended speech in BD dataset. These findings suggest BD enhances coherence of neural response but risks overestimating AAD accuracy. Code and data are released.
A real-time MRI study on asymmetry in velum dynamics during VCV production with nasal sounds
Authors: Chetan Sharma; Vaishnavi Chandwanshi; Shreya Shrikant Karkun; Aditya Anand Gupta; Prasanta Kumar Ghosh
Venue: Interspeech 2025 (main)

Abstract: Velum movement controls airflow through nasal passage enabling production of nasal sound. The asymmetry in velum dynamics during velum lowering vs raising is not well understood, although several studies on asymmetry of other articulators exist. In this study of asymmetry in velum dynamics, we use real-time MRI videos of 68 speakers speaking symmetric vowel-consonant-vowel (VCV) sequence with V being vowels (/a/, /i/, /u/) and C being nasals (/m/, /n/). The asymmetry is analyzed in terms of the extent and speed of velum movement. The study reveals that the extent of velum displacement is significantly higher (by a factor of ~2) during V-C transition (velum lowering) compared to that during C-V transition (velum raising) for all six vowel and nasal combinations chosen. It is also found that the speed with which velum lowers is higher (~1.5 times) than that of velum raising during production of all VCVs except /ana/ and /unu/, for which no significant difference is observed.
Exploratory Analysis of Brainstem fMRI Data During Sustained Phonation
Authors: Carey Smith; Hu Cheng; Pertti Palo; Daniel Aalto; Steven M. Lulich
Venue: Interspeech 2025 (main)

Abstract: Very little work has been done on functional imaging of the cranial nerve nuclei in the brainstem in the context of motor control. This study is a preliminary step in understanding the functional organization and involvement of the brainstem during speech motor control. We aim to identify the impact of using contrastive sustained phonation tasks in isolating activation patterns. We provide preliminary observations of activation patterns in the cranial nerve nuclei. In the study 4 participants performed sustained phonation of [a] and [i] for 15 seconds, repeated over 12 trials. Phonation of [a] resulted in greater activation in the upper pons, in the vicinity of the trigeminal nucleus, compared to [i]. Phonation of [i] elicited activation more anterior and inferior, consistent with expected locations of the facial nucleus. Both phonemes resulted in activation in the medulla, consistent with the expected location of the hypoglossal nucleus.
Gaze-Enhanced Multimodal Turn-Taking Prediction in Triadic Conversations
Authors: Seongsil Heo; Christi Miller; Calvin Murdock; Michael Proulx
Venue: Interspeech 2025 (main)

Abstract: Turn-taking prediction is crucial for seamless interactions. This study introduces a novel, lightweight framework for accurate turn-taking prediction in triadic conversations without relying on computationally intensive methods. Unlike prior approaches that either disregard gaze or treat it as a passive signal, our model integrates gaze with speaker localization, structuring it within a spatial constraint to transform it into a reliable predictive cue. Leveraging egocentric behavioral cues, our experiments demonstrate that incorporating gaze data from a single-user significantly improves prediction performance, while gaze data from multiple-users further enhances it by capturing richer conversational dynamics. This study presents a lightweight and privacy-conscious approach to support adaptive, directional sound control, enhancing speech intelligibility in noisy environments, particularly for hearing assistance in smart glasses.
Visual Cues Support Robust Turn-taking Prediction in Noise
Authors: Sam O'Connor Russell; Naomi Harte
Venue: Interspeech 2025 (main)

Abstract: Accurate predictive turn-taking models (PTTMs) are essential for naturalistic human-robot interaction. However, little is known about their performance in noise. This study therefore explores PTTM performance in types of noise likely to be encountered once deployed. Our analyses reveal PTTMs are highly sensitive to noise. Hold/shift accuracy drops from 84% in clean speech to just 52% in 10 dB music noise. Training with noisy data enables a multimodal PTTM, which includes visual features to better exploit visual cues, with 72% accuracy in 10 dB music noise. The multimodal PTTM outperforms the audio-only PTTM across all noise types and SNRs, highlighting its ability to exploit visual cues; however, this does not always generalise to new types of noise. Analysis also reveals that successful training relies on accurate transcription, limiting the use of ASR-derived transcriptions to clean conditions. We make code publicly available for future research.