GenAI4ELab/papercli-papers huggingface dataset:
Voice Reconstruction through Large-Scale TTS Models: Comparing Zero-Shot and Fine-tuning Approaches to Personalise TTS in Assistive Communication
Authors: Ãva SzÃ©kely; PÃ©ter Mihajlik; MÃ¡tÃ© Soma KÃ¡dÃ¡r; LÃ¡szlÃ³ TÃ³th
Venue: Interspeech 2025 (main)

Abstract: Personalised synthetic speech can enhance communication for Augmentative and Alternative Communication (AAC) users, but achieving high-quality, speaker-specific voices depends on various factors such as the condition causing speech loss, and availability of recorded speech. Recent advancements in large-scale zero-shot TTS models may change the data requirements, as they have the potential to adapt to a wider range of inputs. This paper explores the potential of these pretrained models in various data availability scenarios, from extensive spontaneous speech to minimal or no unaffected speech. We evaluate a state-of-the-art TTS system on a case study involving a stroke survivor with dysarthria, leveraging both typical and atypical speech data. Additionally, we introduce a novel interactive approach using dysarthric speech as an audio prompt to enable user-guided prosody adaptation.
Objective and Subjective Evaluation of Diffusion-Based Speech Enhancement for Dysarthric Speech
Authors: Dimme de Groot; Tanvina Patel; Devendra Kayande; Odette Scharenborg; Zhengjun Yue
Venue: Interspeech 2025 (main)

Abstract: Dysarthric speech poses significant challenges for automatic speech recognition (ASR) systems due to its high variability and reduced intelligibility. In this work we explore the use of diffusion models for dysarthric speech enhancement, which is based on the hypothesis that using diffusion-based speech enhancement moves the distribution of dysarthric speech closer to that of typical speech, which could potentially improve dysarthric speech recognition performance. We assess the effect of two diffusion-based and one signal-processing-based speech enhancement algorithms on intelligibility and speech quality of two English dysarthric speech corpora. We applied speech enhancement to both typical and dysarthric speech and evaluate the ASR performance using Whisper-Turbo, and the subjective and objective speech quality of the original and enhanced dysarthric speech. We also fine-tuned Whisper-Turbo on the enhanced speech to assess its impact on recognition performance.
Data Augmentation using Speech Synthesis for Speaker-Independent Dysarthria Severity Classification
Authors: Minseop Kim; Minsu Han; Seokyoung Hong; Myoung-wan Koo
Venue: Interspeech 2025 (main)

Abstract: Accurate dysarthria severity classification is essential for assessing motor speech disorders, and automation can improve efficiency and accessibility in clinical settings. While deep learning has significantly advanced this field, recent studies have increasingly leveraged large foundation ASR models. However, most studies focus on speaker-dependent (SD) classification, leaving speaker-independent (SI) classification as a major challenge due to limited datasets. SI classification is crucial in real-world scenarios where patient-specific information is unavailable. To address this, we applied two types of speech synthesis models for the first time in this task. We explore various strategies for integrating zero-shot text-to-speech (ZS-TTS) and voice conversion (VC) models to enhance SI classification and propose the most effective utilization settings. Our approach significantly improves the SI severity classification performance, paving the way for further research in this area.
Fairness in Dysarthric Speech Synthesis: Understanding Intrinsic Bias in Dysarthric Speech Cloning using F5-TTS
Authors: Anuprabha M; Krishna Gurugubelli; Anil Kumar Vuppala
Venue: Interspeech 2025 (main)

Abstract: Dysarthric speech poses significant challenges in developing assistive technologies, primarily due to the limited availability of data. Recent advances in neural speech synthesis, especially zero-shot voice cloning, facilitate synthetic speech generation for data augmentation; however, they may introduce biases towards dysarthric speech. In this paper, we investigate the effectiveness of state-of-the-art F5-TTS in cloning dysarthric speech using TORGO dataset, focusing on intelligibility, speaker similarity, and prosody preservation. We also analyze potential biases using fairness metrics like Disparate Impact and Parity Difference to assess disparities across dysarthric severity levels. Results show that F5-TTS exhibits a strong bias toward speech intelligibility over speaker and prosody preservation in dysarthric speech synthesis. Insights from this study can help integrate fairness-aware dysarthric speech synthesis, fostering the advancement of more inclusive speech technologies.
Synthetic Dysarthric Speech: A Supplement, Not a Substitute for Authentic Data in Dysarthric Speech Recognition
Authors: Jingting Li; Keyi Feng; Xinran Zhao; Yan Wang; Su-Jing Wang
Venue: Interspeech 2025 (main)

Abstract: Dysarthric speech recognition (DSR) is an emerging field that can enhance social interactions and mental health for individuals with dysarthria. However, the lack of sufficient Chinese dysarthric speech data and challenges like ambiguity and individual differences hinder performance improvements. Text-to-speech (TTS) technology is well-established in normal speech recognition and can also supplement dysarthric speech data. This study explores the impact of TTS-based Chinese dysarthric speech generation on DSR performance. Speaker-dependent experiments show that synthetic dysarthric speech alone does not effectively improve DSR performance. Through statistical analysis of acoustic features, we reveal the disparities between synthetic and authentic speech in dysarthria and highlight the limitations of synthetic data for DSR. These findings provide insights for future improvements in speech generation methods.
Unsupervised Rhythm and Voice Conversion to Improve ASR on Dysarthric Speech
Authors: Karl El Hajal; Enno Hermann; Sevada Hovsepyan; Mathew Magimai Doss
Venue: Interspeech 2025 (main)

Abstract: Automatic speech recognition (ASR) systems struggle with dysarthric speech due to high inter-speaker variability and slow speaking rates. To address this, we explore dysarthric-to-healthy speech conversion for improved ASR performance. Our approach extends the Rhythm and Voice (RnV) conversion framework by introducing a syllable-based rhythm modeling method suited for dysarthric speech. We assess its impact on ASR by training LF-MMI models and fine-tuning Whisper on converted speech. Experiments on the Torgo corpus reveal that LF-MMI achieves significant word error rate reductions, especially for more severe cases of dysarthria, while fine-tuning Whisper on converted data has minimal effect on its performance. These results highlight the potential of unsupervised rhythm and voice conversion for dysarthric ASR. Code available at: https://github.com/idiap/RnV.
Audio-Based Classification and Geographic Regression of Austrian Dialects
Authors: Lorenz Gutscher; Michael Pucher
Venue: Interspeech 2025 (main)

Abstract: Dialect classification remains challenging due to regional variability and limited dialect-specific datasets. This study addresses these challenges by leveraging a novel dataset of 304 speakers from 108 locations across Austria for automatic classification of Austrian dialects. To minimize speaker-specific biases and enhance dialectal features, speaker augmentation techniques are applied. Classification is conducted at three levels: location, dialect group, and federal state. Additionally, a regression task predicts the speakers' geographic coordinates, with the wav2vec 2.0 model architecture achieving an average test-set distance error of 66.7 kilometers. This work represents a unique approach to fine-grained dialect classification and geographic location prediction for Austria. Finally, model explainability is explored using Integrated Gradients (IG), identifying the most relevant speech segments for classification within each dialect group.
Jointly Improving Dialect Identification and ASR in Indian Languages using Multimodal Feature Fusion
Authors: Saurabh Kumar; Amartyaveer; Prasanta Kumar Ghosh
Venue: Interspeech 2025 (main)

Abstract: Automatic Speech Recognition (ASR) and Dialect Identification (DID) are crucial for Indian languages, many of which are low-resource and exhibit significant dialectal differences. Existing methods often optimize ASR or DID individually, resulting in performance trade-offs. In this work, we propose a multimodal framework that jointly improves ASR and DID. Our method employs a Bottleneck Encoder to extract dialectal features from Conformer-based speech representations and a RoBERTa encoder to process ASR-generated CTC embeddings. A gating mechanism merges these features, followed by an attention encoder to refine the representations. The learned embeddings are concatenated with Conformer outputs to enhance ASR features. Evaluated on eight Indian languages with thirty-three dialects, our method achieves an average DID accuracy of 81.63% and average CER and WER of 4.65% and 17.73%, respectively. These results highlight the effectiveness of our method for joint ASR-DID modeling.
ADI-20: Arabic Dialect Identification dataset and models
Authors: Haroun Elleuch; Salima Mdhaffar; Yannick EstÃ¨ve; Fethi Bougares
Venue: Interspeech 2025 (main)

Abstract: We present ADI-20, an extension of the previously published ADI-17 Arabic Dialect Identification (ADI) dataset. ADI-20 covers all Arabic-speaking countries' dialects. It comprises 3,556 hours from 19 Arabic dialects in addition to Modern Standard Arabic (MSA). We used this dataset to train and evaluate various state-of-the-art ADI systems. We explored fine-tuning pre-trained ECAPA-TDNN-based models, as well as Whisper encoder blocks coupled with an attention pooling layer and a classification dense layer. We investigated the effect of (i) training data size and (ii) the model's number of parameters on identification performance. Our results show a small decrease in F1 score while using only 30% of the original training data. We open-source our collected data and trained models to enable the reproduction of our work, as well as support further research in ADI.
Improving Low-Resource Dialect Classification Using Retrieval-based Voice Conversion
Authors: Lea Fischbach; Akbar Karimi; Caroline Kleen; Alfred Lameli; Lucie Flek
Venue: Interspeech 2025 (main)

Abstract: Deep learning models for dialect identification are often limited by the scarcity of dialectal data. To address this challenge, we propose to use Retrieval-based Voice Conversion (RVC) as an effective data augmentation method for a low-resource German dialect classification task. By converting audio samples to a uniform target speaker, RVC minimizes speaker-related variability, enabling models to focus on dialect-specific linguistic and phonetic features. Our experiments demonstrate that RVC enhances classification performance when utilized as a standalone augmentation method. Furthermore, combining RVC with other augmentation methods such as frequency masking and segment removal leads to additional performance gains, highlighting its potential for improving dialect classification in low-resource scenarios.
Effects of Prosodic Information on Dialect Classification Using Whisper Features
Authors: Phoebe Parsons; Heming StrÃ¸mholt Bremnes; Knut Kvale; TorbjÃ¸rn Svendsen; Giampiero Salvi
Venue: Interspeech 2025 (main)

Abstract: In dialect identification (DID), a model needs to attend to subtle cues to distinguish between highly similar linguistic variants. However, the knowledge of which cues are important and why is limited. Inspired by the literature on human DID, we fine-tuned a Whisper model with modified audio to see how deprivation of various signal components would impact performance. Specifically, the audio manipulation sought to either isolate or remove (tonal) prosodic information, by either low-pass filtering or monotonizing F0, respectively. Results indicate that fine-tuning on low-pass filtered data produces a significant improvement over unmodified data. Utilizing sensitivity maps in the frequency domain, we argue that the low-pass model is able to devote more attention to lower frequency bands, thus exploiting task-relevant pitch dynamics. Though only evaluated with Norwegian, we suggest that our methodology should generalize, encouraging improvement in DID and its downstream applications.
Voice Conversion Improves Cross-Domain Robustness for Spoken Arabic Dialect Identification
Authors: Badr M. Abdullah; Matthew Baas; Bernd MÃ¶bius; Dietrich Klakow
Venue: Interspeech 2025 (main)

Abstract: Arabic dialect identification (ADI) systems are essential for large-scale data collection pipelines that enable the development of inclusive speech technologies for Arabic language varieties. However, the reliability of current ADI systems is limited by poor generalization to out-of-domain speech. In this paper, we present an effective approach based on voice conversion for training ADI models that achieves state-of-the-art performance and significantly improves robustness in cross-domain scenarios. Evaluated on a newly collected real-world test set spanning four different domains, our approach yields consistent improvements of up to +34.1% in accuracy across domains. Furthermore, we present an analysis of our approach and demonstrate that voice conversion helps mitigate the speaker bias in the ADI dataset. We release our robust ADI model and cross-domain evaluation dataset to support the development of inclusive speech technologies for Arabic.
Band-Split Self-supervised Mamba for Infant-centered Audio Analysis
Authors: Xulin Fan; Jialu Li; Mark Hasegawa-Johnson; Nancy L. McElwain
Venue: Interspeech 2025 (main)

Abstract: Infant-worn audio recorders provide a valuable means to analyze an infantâs home environment and vocal interactions with family members. Recent advances in self-supervised learning on large unlabeled datasets and supervised training on limited annotated data have improved performance in this domain. However, data scarcity remains a challenge. We introduce Band-Split SSAMBA (BS-SSAMBA), a self-supervised representation learning method that incorporates band-specific projections and a band-agnostic Mamba encoder to model temporal relationships across frequency bands. Designed for data-efficient learning, BS-SSAMBA effectively leverages both unlabeled and labeled in-domain data. Through extensive experiments on family audio recordings, we show that BS-SSAMBA outperforms vanilla SSAMBA and wav2vec2-based models, demonstrating its effectiveness for infant-centered audio tagging.
Subtyping Speech Errors in Childhood Speech Sound Disorders with Acoustic-to-Articulatory Speech Inversion
Authors: Nina R Benway; Saba Tabatabaee; Benjamin Munson; Jonathan Preston; Carol Espy-Wilson
Venue: Interspeech 2025 (main)

Abstract: Speech inversion holds much potential to describe speech errors in childhood speech sound disorders. However, the clinical interpretability of speech inversion tract variables is unknown. This study is the first to show, through linear mixed modeling, that acoustic-to-articulatory speech inversion can quantify statistically significant articulatory differences between several perceptually salient subtypes of /É¹/ and /s/ speech sound errors, and correct /É¹/ and /s/ targets, in American English.
PERCEPT-US: A Multimodal American English Child Speech Corpus Specialized for Articulatory Feedback
Authors: Amanda Eads; Heather Kabakoff; Nina Benway; Elaine Hitchcock; Jonathan Preston; Tara McAllister
Venue: Interspeech 2025 (main)

Abstract: We present PERCEPT-US, a multimodal corpus of audio and ultrasound data from 126 American English-speaking children ages 8 to 17. Collected during clinical trials investigating biofeedback in speech therapy for residual speech sound disorder (RSSD), it includes 80 children with no history of speech-language-hearing challenges and 46 children with no challenges other than RSSD. The corpus is balanced by sex (58 females, 68 males) and stratified by age and speech therapy history. Participants completed syllabic, word, and sentence level speech production tasks, totaling 24,699 utterances, with a focus on American English rhotics. The corpus demonstration uses mixed-effects linear regression on segmented acoustic and labeled ultrasound data from 69 children to show that rhotic tongue shape categories significantly predict the acoustics of higher formant frequency values - the first study to demonstrate this acoustic-articulatory relationship in a large pediatric sample.
Children's Voice Privacy: First Steps and Emerging Challenges
Authors: Ajinkya Kulkarni; Francisco Teixeira; Enno Hermann; Thomas Rolland; Isabel Trancoso; Mathew Magimai Doss
Venue: Interspeech 2025 (main)

Abstract: Children are one of the most under-represented groups in speech technologies, as well as one of the most vulnerable in terms of privacy. Despite this, anonymization techniques targeting this population have received little attention. In this study, we seek to bridge this gap, and establish a baseline for the use of voice anonymization techniques designed for adult speech when applied to children's voices. Such an evaluation is essential, as children's speech presents a distinct set of challenges when compared to that of adults. This study comprises three childrenâs datasets, six anonymization methods, and objective and subjective utility metrics for evaluation. Our results show that existing systems for adults are still able to protect children's voice privacy, but suffer from much higher utility degradation. In addition, our subjective study displays the challenges of automatic evaluation methods for speech quality in children's speech, highlighting the need for further research.
FT-Boosted SV: Towards Noise Robust Speaker Verification for English Speaking Classroom Environments
Authors: Saba Tabatabaee; Jing Liu; Carol Espy-Wilson
Venue: Interspeech 2025 (main)

Abstract: Creating Speaker Verification (SV) systems for classroom settings that are robust to classroom noises such as babble noise is crucial for the development of AI tools that assist educational environments. In this work, we study the efficacy of finetuning with augmented children datasets to adapt the x-vector and ECAPA-TDNN to classroom environments. We demonstrate that finetuning with augmented children's datasets is powerful in that regard and reduces the Equal Error Rate (EER) of x-vector and ECAPA-TDNN models for both classroom datasets and children speech datasets. Notably, this method reduces EER of the ECAPA-TDNN model on average by half (a 5% improvement) for classrooms in the MPT dataset compared to the ECAPA-TDNN baseline model. The x-vector model shows an 8% average improvement for classrooms in the NCTE dataset compared to its baseline.
Examining Test-Time Adaptation for Personalized Child Speech Recognition
Authors: Zhonghao Shi; Xuan Shi; Anfeng Xu; Tiantian Feng; Harshvardhan Srivastava; Shrikanth Narayanan; Maja Mataric
Venue: Interspeech 2025 (main)

Abstract: Automatic speech recognition (ASR) models often experience performance degradation due to data domain shifts introduced at test time, a challenge that is further amplified for child speakers. Test-time adaptation (TTA) methods have shown great potential in bridging this domain gap. However, the use of TTA to adapt ASR models to the individual differences in each childâs speech has not yet been systematically studied. In this work, we investigate the effectiveness of two widely used TTA methods-SUTA, SGEM-in adapting off-the-shelf ASR models and their fine-tuned versions for child speech recognition, with the goal of enabling continuous, unsupervised adaptation at test time. Our findings show that TTA significantly improves the performance of both off-the-shelf and fine-tuned ASR models, both on average and across individual child speakers, compared to unadapted baselines. However, while TTA helps adapt to individual variability, it may still be limited with non-linguistic child speech.
Employing self-supervised learning models for cross-linguistic child speech maturity classification
Authors: Theo Zhang; Madurya Suresh; Anne Warluamont; Kasia Hitczenko; Alejandrina Cristia; Margaret Cychosz
Venue: Interspeech 2025 (main)

Abstract: Speech technology systems struggle with many downstream tasks for child speech due to small training corpora and the difficulties that child speech pose. We apply a novel dataset, SpeechMaturity, to state-of-the-art transformer models to address a fundamental classification task: identifying child vocalizations. Unlike previous corpora, our dataset captures maximally ecologically-valid child vocalizations across an unprecedented sample, comprising children acquiring 25+ languages in the U.S., Bolivia, Vanuatu, Papua New Guinea, Solomon Islands, and France. The dataset contains 242,004 labeled vocalizations, magnitudes larger than previous work. Models were trained to distinguish between cry, laughter, mature (consonant+vowel), and immature speech (just consonant or vowel). Models trained on the dataset outperform state-of-the-art models trained on previous datasets, achieved classification accuracy comparable to humans, and were robust across rural and urban settings.
On Enhancing the Performance of Children's ASR Task in Limited Data Scenario
Authors: Ankita Ankita; Shambhavi Shambhavi; Syed Shahnawazuddin
Venue: Interspeech 2025 (main)

Abstract: In this paper, we have documented our efforts towards developing a robust children's automatic speech recognition (ASR) system in limited data scenario. At first, we have explored the effect of in-domain data augmentation so as to deal with limitations posed by data scarcity. This also helps in developing a competitive baseline ASR system. Next, we have studied the affect of modeling glottal activity parameters along with spectrum-based front-end acoustic features like the Mel-frequency cepstral coefficients (MFCC). Finally, the impact of feature normalization through feature-space maximum likelihood linear regression (fMLLR) is explored. As a consequence of applying fMLLR and then concatenating the normalized MFCC features with glottal activity parameters, a relative reduction in character error rate by 40% over the baseline is obtained.
Egocentric Speaker Classification in Child-Adult Dyadic Interactions: From Sensing to Computational Modeling
Authors: Tiantian Feng; Anfeng Xu; Xuan Shi; Somer Bishop; Shrikanth Narayanan
Venue: Interspeech 2025 (main)

Abstract: Autism spectrum disorder (ASD) is a neurodevelopmental condition characterized by challenges in social communication, repetitive behavior, and sensory processing. One important research area in ASD is evaluating children's behavioral changes over time during treatment. The standard protocol with this objective is BOSCC, which involves dyadic interactions between a child and clinicians performing a pre-defined set of activities. A fundamental aspect of understanding children's behavior in these interactions is automatic speech understanding, particularly identifying who speaks and when. Conventional methods in this area heavily rely on audio recorded from a spectator perspective, and there is limited research on egocentric speech modeling. Here, we experiment with wearable sensors to perform speech sampling in BOSCC interviews from an egocentric perspective. Our findings highlight the potential of egocentric speech collection and pre-training to improve speaker classification accuracy.