GenAI4ELab/papercli-papers huggingface dataset:
Large Language Models based ASR Error Correction for Child Conversations
Authors: Anfeng Xu; Tiantian Feng; So Hyun Kim; Somer Bishop; Catherine Lord; Shrikanth Narayanan
Venue: Interspeech 2025 (main)

Abstract: Automatic Speech Recognition (ASR) has recently shown remarkable progress, but accurately transcribing children's speech remains a significant challenge. Recent developments in Large Language Models (LLMs) have shown promise in improving ASR transcriptions. However, their applications in child speech including conversational scenarios are under-explored. In this study, we explore the use of LLMs in correcting ASR errors for conversational child speech. We demonstrate the promises and challenges of LLMs through experiments on two children's conversational speech datasets with both zero-shot and fine-tuned ASR outputs. We find that while LLMs are helpful in correcting zero-shot ASR outputs and fine-tuned CTC-based ASR outputs, it remains challenging for LLMs to improve ASR performance when incorporating contextual information or when using fine-tuned autoregressive ASR (e.g., Whisper) outputs.
Challenges in Automated Processing of Speech from Child Wearables: The Case of Voice Type Classifier
Authors: Tarek Kunze; Marianne MÃ©tais; Hadrien Titeux; Lucas Elbert; Joseph Coffey; Emmanuel Dupoux; Alejandrina Cristia; Marvin Lavechin
Venue: Interspeech 2025 (main)

Abstract: Recordings gathered with child-worn devices promised to revolutionize both fundamental and applied speech sciences by allowing the effortless capture of children's naturalistic speech environment and language production. This promise hinges on speech technologies that can transform the sheer mounds of data thus collected into usable information. This paper demonstrates several obstacles blocking progress by summarizing three years' worth of experiments aimed at improving one fundamental task: Voice Type Classification. Our experiments suggest that improvements in representation features, architecture, and parameter search contribute to only marginal gains in performance. More progress is made by focusing on data relevance and quantity, which highlights the importance of collecting data with appropriate permissions to allow sharing.
Improving Child Speech Recognition and Reading Mistake Detection by Using Prompts
Authors: Lingyun Gao; Cristian Tejedor-Garcia; Catia Cucchiarini; Helmer Strik
Venue: Interspeech 2025 (main)

Abstract: Automatic reading aloud evaluation can provide valuable support to teachers by enabling more efficient scoring of reading exercises. However, research on reading evaluation systems and applications remains limited. We present a novel multimodal approach that leverages audio and knowledge from text resources. In particular, we explored the potential of using Whisper and instruction-tuned large language models (LLMs) with prompts to improve transcriptions for child speech recognition, as well as their effectiveness in downstream reading mistake detection. Our results demonstrate the effectiveness of prompting Whisper and prompting LLM, compared to the baseline Whisper model without prompting. The best performing system achieved state-of-the-art recognition performance in Dutch child read speech, with a word error rate (WER) of 5.1%, improving the baseline WER of 9.4%. Furthermore, it significantly improved reading mistake detection, increasing the F1 score from 0.39 to 0.73.
Improving Automatic Speech Recognition for Children's Reading Assessment with Disfluency-aware Language Models
Authors: JazmÃ­n Vidal; Luciana Ferrer; Juan Esteban Kamienkowski; Pablo Riera
Venue: Interspeech 2025 (main)

Abstract: We present an approach to improve automatic speech recognition (ASR) quality for childrenâs reading assessment in Spanish by incorporating disfluencies into the decoding process. Existing ASR-based assessment methods use hybrid or end-to-end models, with some methods relying on task-specific lattices or n-gram models for modeling potentially disfluent speech. In this work, we compare both families of approaches, in combination with in-domain fine-tuning of the acoustic model, on a dataset of Spanish-speaking children reading aloud. The task-specific n-gram model is learned on a synthetic dataset of disfluent text generated automatically from the reference text with our disfluentES toolkit. Results show that modeling disfluencies improves ASR performance as well as fluency assessment performance.
Oral Reading Errors by Grade 3 Children in Indian Schools: A Hindi-English Perspective
Authors: Sneha Raman; Preeti Rao
Venue: Interspeech 2025 (main)

Abstract: We present an analysis of reading errors on a manually transcribed dataset of Grade 3 students (N=595) who read aloud level-appropriate passages in English and Hindi. Substitutions are categorised as word or non-word errors. Further, substitutions are analysed using grapheme/phoneme sequence matching and assigned typical reading error types such as initial part matches, final part matches and scaffolding errors. We compare the distribution of error categories for the two languages and discuss underlying language-dependent decoding strategies. We also apply the analysis methodology to identify the percentage scaffolding errors per utterance given its usefulness for reading assessment. Finally, we extend our work to its practical application for diagnostics by testing an automatic phone recognition system on our task.
Grammatical Error Detection on Spontaneous Children's Speech Using Iterative Pseudo Labeling
Authors: Christopher Gebauer; Lars Rumberg; Lars KÃ¶hn; Hanna Ehlert; Edith Beaulac; JÃ¶rn Ostermann
Venue: Interspeech 2025 (main)

Abstract: Language acquisition is fundamental for the development of various skills in the early stage of a childrenâs life. Unfortunately, developmental language disorder (DLD) is the most common developmental disorder during childhood. A common indicator of DLD is that children with such condition struggle to correctly use grammatical forms. Therefore, we focus in this work on automatic grammatical error detection on spontaneous childrenâs speech. We extend the state of the art by an iterative pseudo labeling scheme to account for the ambiguity of grammatical error labels. Such ambiguity becomes obvious, when it is unclear which word is incorrect, e. g., for agreement errors. In terms of the F1 gain score (FG1) we significantly improve upon the baseline on sentence- and word-level label. On automatic transcriptions of the kidsTALC corpus we increase the sentence-level FG1 from 0.38 to 0.63. Further, our best performing system achieves a recall of 0.45, while maintaining a precision of 0.36.
Why is children's ASR so difficult? Analyzing children's phonological error patterns using SSL-based phoneme recognizers
Authors: Koharu Horii; Naohiro Tawara; Atsunori Ogawa; Shoko Araki
Venue: Interspeech 2025 (main)

Abstract: Children's automatic speech recognition (child ASR) is generally more challenging than adult ASR, as children's speech differs from that of adults and continuously evolves with a child's development. To investigate the causes of this difficulty and leverage the findings to improve the child ASR performance in the future, this study examines phoneme recognition error patterns using self-supervised learning (SSL)-based phoneme recognizers (PRs). First, we apply several SSL-based child PRs to English children's speech from kindergarten to grade 10, confirming that the WavLM-based model performs best. Then, using this model, we examine whether misrecognitions are primarily due to child mispronunciations or model errors. Furthermore, analyzing the results obtained by the WavLM-based model, we investigate how error patterns change along with age. The results show that mispronunciations persist longer than previously reported, particularly in certain manners of articulation.
Automatic detection of speech sound disorders in German-speaking children: augmenting the data with typically developed speech
Authors: Darline Monika Marx; Marco Matassoni; Alessio Brutti
Venue: Interspeech 2025 (main)

Abstract: Speech Sound Disorders (SSD) are common among children, affecting their academic, social, and emotional development. Traditional diagnostic methods are based on speech-language pathologists, making them resource intensive. Due to the global shortage of experts and increasing demand, exploring deep-learning tools is crucial. Adapting a multi-task framework to fine-tune a pre-trained multilingual Wav2Vec model, this study tackles Automatic Speech Recognition and SSD classification for German children using a custom dataset. We show that incorporating public out-of-domain datasets improves robustness and generalizability. Interestingly,  combining pathological and typical speech data with mis-pronunciations benefits the performance in terms of speech recognition and SSD detection. Finally we investigate a two-step training of the model that further improves the overall performance.
Continuous Learning for Children's ASR: Overcoming Catastrophic Forgetting with Elastic Weight Consolidation and Synaptic Intelligence
Authors: Edem Ahadzi; Vishwanath Pratap Singh; Tomi Kinnunen; Ville Hautamaki
Venue: Interspeech 2025 (main)

Abstract: In this work, we present the first study addressing automatic speech recognition (ASR) for children in an online learning setting. This is particularly important for both child-centric applications and the privacy protection of minors, where training models with sequentially arriving data is critical. The conventional approach of model fine-tuning often suffers from catastrophic forgetting. To tackle this issue, we explore two established techniques: elastic weight consolidation (EWC) and synaptic intelligence (SI). Using a custom protocol on the MyST corpus, tailored to the online learning setting, we achieve relative word error rate (WER) reductions of 5.21 % with EWC and 4.36 % with SI, compared to the fine-tuning baseline.
Exploring Shared-Weight Mechanisms in Transformer and Conformer Architectures for Automatic Speech Recognition
Authors: Thomas Rolland; Alberto Abad
Venue: Interspeech 2025 (main)

Abstract: In recent years, the increasing demand for parameter-efficient automatic speech recognition (ASR) systems has driven researchers to explore innovative architectures and techniques designed for minimising model size while maintaining recognition performance. Thus, weight-sharing mechanisms stand out as a promising approach. In this study, we present a comprehensive evaluation of weight-sharing applied to the different components of both Transformer and Conformer architectures in the context of ASR. Furthermore, we investigate the behaviour of these weight-sharing configurations when fine-tuned for a low-resource task, specifically children's ASR. Additionally, we introduce Shared-Conformer, a novel architecture that achieves a 63% parameter reduction with only a minimal increase in word error rate. Our findings demonstrate that weight-sharing significantly reduces the number of parameters while preserving competitive performance in both well-resourced and low-resourced scenarios.
Advancing Pediatric ASR: The Role of Voice Generation in Disordered Speech
Authors: Karen Rosero; Ali N Salman; Shreeram Chandra; Berrak Sisman; Cortney Vanât Slot; Alex Kane; Rami R Hallac; Carlos Busso
Venue: Interspeech 2025 (main)

Abstract: Zero-shot performance of state-of-the-art automatic speech recognition (ASR) significantly declines on pediatric patients with speech sound disorders (SSDs) due to deviations in phonetic pronunciation. To address this, we train a subject-agnostic ASR system on 77 minutes of pediatric SSD transcribed data, which improved zero-shot ASR by 67.48%. Given the scarcity of data and privacy concerns with children's data, we study the suitability of voice conversion (VC) and text-to-speech (TTS) to synthesize disorder-reflective samples. Our ASR system surpassed zero-shot by 71.72% when leveraging TTS and showed potential for privacy preservation when using VC. Notably, pre-training on synthetic samples alone reduces the required real SSD data to 50 minutes (i.e., 65% of the data), while achieving metrics comparable to the model finetuned with all SSD samples. This study enables ASR technologies to assist individuals with SSDs and facilitates automatic transcription of speech therapy sessions.
CHSER: A Dataset and Case Study on Generative Speech Error Correction for Child ASR
Authors: Natarajan Balaji Shankar; Zilai Wang; Kaiyuan Zhang; Mohan Shi; Abeer Alwan
Venue: Interspeech 2025 (main)

Abstract: Automatic Speech Recognition (ASR) systems struggle with child speech due to its distinct acoustic and linguistic variability and limited availability of child speech datasets, leading to high transcription error rates. While ASR error correction (AEC) methods have improved adult speech transcription, their effectiveness on child speech remains largely unexplored. To address this, we introduce CHSER, a Generative Speech Error Correction (GenSEC) dataset for child speech, comprising 200K hypothesis-transcription pairs spanning diverse age groups and speaking styles. Results demonstrate that fine-tuning on the CHSER dataset achieves up to a 28.5% relative WER reduction in a zero-shot setting and a 13.3% reduction when applied to fine-tuned ASR systems. Additionally, our error analysis reveals that while GenSEC improves substitution and deletion errors, it struggles with insertions and child-specific disfluencies. These findings highlight the potential of GenSEC for improving child ASR.
Causal Structure Discovery for Error Diagnostics of Children's ASR
Authors: Vishwanath Pratap Singh; Md Sahidullah; Tomi Kinnunen
Venue: Interspeech 2025 (main)

Abstract: Childrenâs automatic speech recognition (ASR) often underperforms compared to that of adults due to a confluence of interdependent factors: physiological (e.g., smaller vocal tracts), cognitive (e.g., underdeveloped pronunciation), and extrinsic (e.g., vocabulary limitations, background noise). Existing analysis methods examine the impact of these factors in isolation, neglecting interdependenciesâsuch as age affecting ASR accuracy both directly and indirectly via pronunciation skills. In this paper, we introduce a causal structure discovery to unravel these interdependent relationships among physiology, cognition, extrinsic factors, and ASR errors. Then, we employ causal quantification to measure each factorâs impact on childrenâs ASR. We extend the analysis to fine-tuned models to identify which factors are mitigated by fine-tuning and which remain largely unaffected. Experiments on Whisper and Wav2Vec2.0 demonstrate the generalizability of our findings across different ASR systems.
Brain-tuned Speech Models Better Reflect Speech Processing Stages in the Brain
Authors: Omer Moussa; Mariya Toneva
Venue: Interspeech 2025 (main)

Abstract: Pretrained self-supervised speech models excel in speech tasks but do not reflect the hierarchy of human speech processing, as they encode rich semantics in middle layers and poor semantics in late layers. Recent work showed that brain-tuning (fine-tuning models using human brain recordings) improves speech models' semantic understanding. Here, we examine how well brain-tuned models further reflect the brainâs intermediate stages of speech processing. We find that late layers of brain-tuned models substantially improve over pretrained models in their alignment with semantic language regions. Further layer-wise probing reveals that early layers remain dedicated to low-level acoustic features, while late layers become the best at complex high-level tasks. These findings show that brain-tuned models not only perform better but also exhibit a well-defined hierarchical processing going from acoustic to semantic representations, making them better model organisms for human speech processing.
Enhancing Syllabic Recognition via Speech-EEG Phase Analysis and Non-Activity State Modeling
Authors: Rini Sharon; Hema A. Murthy
Venue: Interspeech 2025 (main)

Abstract: Decoding inner speech from brain activity holds transformative promise for brain-computer interfaces and speech rehabilitation. This study examines EEG-based neural patterns across speech production, imagination, and perception. Signal-level visualizations reveal distinct, phase-specific cortical signatures that emerge within precise temporal windows in spatially localized regions. These insights motivate the introduction of a non-activity(NA) state to account for resting and transitional EEG periods - akin to pauses in natural speech. Capturing and excluding these "cognitive pauses" de-noises the EEG signal, improving the focus on task-relevant neural activity. Comparative evaluation across continuous and isolated speech datasets demonstrates that NA modeling improves syllabic recognition accuracy by 1.8% and 1.15%, respectively. These results underscore the role and generalizability of cognitive pauses in imagined speech-EEG decoding, enabling robust brain computer interfaces.
Functional Connectivity and Hilbert-Based Features for Covert Speech EEG Variability Analysis and Classification
Authors: Saravanakumar Duraisamy; Maurice Rekrut; Luis A. Leiva
Venue: Interspeech 2025 (main)

Abstract: We explore inter-trial, inter-class, and inter-subject variability in covert speech imagination using Electroencephalogram (EEG) signals. Two key functional connectivity metrics (Phase Locking Value and Coherence) revealed unique and shared activation patterns across speech commands, influenced by individual word perception and affective states. We also propose a subject-independent classification model using Hilbert envelope and instantaneous phase features across EEG frequency bands with a Bidirectional Long Short-Term Memory (BiLSTM) architecture, achieving 59.14% classification accuracy across five speech categories. Our state-of-the-art results in EEG-based speech decoding contribute a new understanding of the neural dynamics underlying imagined speech and affective processing.
Neuro2Semantic: A Transfer Learning Framework for Semantic Reconstruction of Continuous Language from Human Intracranial EEG
Authors: Siavash Shams; Richard Antonello; Gavin Mischler; Stephan Bickel; Ashesh Mehta; Nima Mesgarani
Venue: Interspeech 2025 (main)

Abstract: Decoding continuous language from neural signals remains a significant challenge in the intersection of neuroscience and artificial intelligence. We introduce Neuro2Semantic, a novel framework that reconstructs the semantic content of perceived speech from intracranial EEG (iEEG) recordings. Our approach consists of two phases: first, an LSTM-based adapter aligns neural signals with pre-trained text embeddings; second, a corrector module generates continuous, natural text directly from these aligned embeddings. This flexible method overcomes the limitations of previous decoding approaches and enables unconstrained text generation. Neuro2Semantic achieves strong performance with as little as 30 minutes of neural data,  outperforming a recent state-of-the-art method in low-data settings. These results highlight the potential for practical applications in brain-computer interfaces and neural decoding technologies.
Selective Auditory Attention Decoding in Naturalistic Conversations Using EEG-Based Speech Envelope Tracking in Multi-Speaker Environments
Authors: Gabriel Ivucic; Saurav Pahuja; Dashanka Da Silva; Tanja Schultz
Venue: Interspeech 2025 (main)

Abstract: In daily life, we effortlessly switch attention between different sound sources while filtering out distractions. Prior research has shown that the target speaker can be decoded from EEG in such settings, however, how the switching of attention influences decoding has not been measured. We investigated EEG-based target speaker decoding with exogenous attention switches in a multi-talker environment, where twenty participants alternated listening to two target speakers taking turns while ignoring two background speakers. Speech envelope reconstruction from EEG showed higher accuracy for the attended speaker than distractors, with single-trial classification at 76%. Our findings showed a brief increase in target speaker reconstruction accuracy after attention switches, suggesting heightened alertness when shifting attention to a new speaker. These findings enhance the understanding of dynamic auditory attention and support the development of attention-based hearing applications.
MiSTR: Multi-Modal iEEG-to-Speech Synthesis with Transformer-Based Prosody Prediction and Neural Phase Reconstruction
Authors: Mohammed Al-Radhi; GÃ©za NÃ©meth; Branislav Gerazov
Venue: Interspeech 2025 (main)

Abstract: Speech synthesis from intracranial EEG (iEEG) signals offers a promising avenue for restoring communication in individuals with severe speech impairments. However, achieving intelligible and natural speech remains challenging due to limitations in feature representation, prosody modeling, and phase reconstruction. We introduce MiSTR, a deep-learning framework that integrates: 1) Wavelet-based feature extraction to capture fine-grained temporal, spectral, and neurophysiological representations of iEEG signals, 2) A Transformer-based decoder for prosody-aware spectrogram prediction, and 3) A neural phase vocoder enforcing harmonic consistency via adaptive spectral correction. Evaluated on a public iEEG dataset, MiSTR achieves state-of-the-art speech intelligibility, with a mean Pearson correlation of 0.91 between reconstructed and original Mel spectrograms, improving over existing neural speech synthesis baselines.
Probing Prosodic Differences Between Two Regional Varieties of Brazilian Portuguese
Authors: Gustavo Silveira; Aviad Albert; Martine Grice
Venue: Interspeech 2025 (main)

Abstract: This study compares the intonation of two Brazilian Portuguese varieties, Alagoas and SÃ£o Paulo, focusing on the last three syllables before terminal and non-terminal boundaries of declarative utterances in sociolinguistic interviews from ten male speakers. With the ProPer toolbox, two acoustic measures were extracted from syllables: (i) DeltaF0, measuring F0 change between syllables; and (ii) Synchrony, reflecting F0 trends within syllables. Analyses of 695 boundary tokens reveal continuous differences (F0 excursion size) between varieties for terminal boundaries, and both categorical (edge tone type) and continuous (primarily F0 direction) differences for non-terminal ones. Alagoans produced larger pitch falls for terminal boundaries and favored a high pitch level on post-stressed syllables for non-terminal boundaries, whereas Paulistans preferred a fall on post-stressed syllables. Both regions mark finality vs. continuation primarily with pitch accents, rather than edge tones.
Data-driven approaches to pitch modelling in two Mexican Spanish ethnolects: K-means Clustering & GAMMs
Authors: Gilly Marchini; Jeremy Steffman
Venue: Interspeech 2025 (main)

Abstract: This study models F0 data for documentation purposes. Using K-means clustering and Generative Additive Mixed Models, it documents differences in F0 contours between two Mexican Spanish ethnolects: Afro-Mexican Spanish and Altiplateau Mexican Spanish. Both dialects distinguish focus condition through contour shape and pitch range: rising contours and high frequency ranges are associated with in-focus conditions, and lower frequency plateaus with post-focus. AMS however shows delayed peak alignment in open syllables and overall higher pitch contours for those with post-vocalic /s/. We posit that these fine-grained differences in segmental alignment and pitch range would not be captured through traditional methods, e.g., Sp_ToBI. With this study's focus on phonetic detail, the ToBI model risks both the absence of phonetic variation and the assumption of underlying phonological structure. We therefore argue that this variation is best captured through clustering and GAMMs.