GenAI4ELab/papercli-papers huggingface dataset:
Iterative Refinement, Not Training Objective, Makes HuBERT Behave Differently from wav2vec 2.0
Authors: Robin Huo; Ewan Dunbar
Venue: Interspeech 2025 (main)

Abstract: Self-supervised models for speech representation learning now see widespread use for their versatility and performance on downstream tasks, but the effect of model architecture on the linguistic information learned in their representations remains under-studied. This study investigates two such models, HuBERT and wav2vec 2.0, and minimally compares two of their architectural differences: training objective and iterative pseudo-label refinement through multiple training iterations. We find that differences in canonical correlation of hidden representations to word identity, phoneme identity, and speaker identity are explained by training iteration, not training objective. We suggest that future work investigate the reason for the effectiveness of iterative refinement in encoding linguistic information in self-supervised speech representations.
On the reliability of feature attribution methods for speech classification
Authors: Gaofei Shen; Hosein Mohebbi; Arianna Bisazza; Afra Alishahi; Grzegorz Chrupala
Venue: Interspeech 2025 (main)

Abstract: As the capabilities of large-scale pre-trained models evolve, understanding the determinants of their outputs becomes more important. Feature attribution aims to reveal which parts of the input elements contribute the most to model outputs. In speech processing, the unique characteristics of the input signal make the application of feature attribution methods challenging. We study how factors such as input type and aggregation and perturbation timespan impact the reliability of standard feature attribution methods, and how these factors interact with characteristics of each classification task.  We find that standard approaches to feature attribution are generally unreliable when applied to the speech domain, with the exception of word-aligned perturbation methods when applied to word-based classification tasks.
An Exploration of Interpretable Deep Learning Models for the Assessment of Mild Cognitive Impairment
Authors: Emma Cathrine Liisborg Leschly; Oliver Roesler; Michael Neumann; Jackson Liscombe; Abhishek Hosamath; Lakshmi Arbatti; Line H. Clemmensen; Melanie Ganz; Vikram Ramanarayanan
Venue: Interspeech 2025 (main)

Abstract: Early diagnosis and intervention are crucial for mild cognitive impairment (MCI), as MCI often progresses to more severe neurodegenerative conditions. In this study, we explore utilizing deep learning for MCI detection without loosing the interpretability provided by feature-based approaches. We used a dataset consisting of 90 MCI patients and 91 controls collected via a remote assessment platform and analyzed the participantsâ spontaneous speech responses to the Patient Report of Problems (PROP) which asks patients to report their most bothersome general health problems. The proposed deep neural network, which features a bottleneck layer including 13 interpretable symptom domains, achieved an AUC of 0.62, thereby outperforming a set of feature-based classifiers while ensuring interpretability due to the bottleneck layer. We further illustrated the modelâs interpretability by examining how the predicted PROP domains influence final predictions using Shapley values.
Towards Multi-Level Transcript Segmentation: LoRA Fine-Tuning for Table-of-Contents Generation
Authors: Steffen Freisinger; Philipp Seeberger; Thomas Ranzenberger; Tobias Bocklet; Korbinian Riedhammer
Venue: Interspeech 2025 (main)

Abstract: Segmenting speech transcripts into thematic sections benefits both downstream processing and users who depend on written text for accessibility. We introduce a novel approach to hierarchical topic segmentation in transcripts, generating multi-level tables of contents that capture both topic and subtopic boundaries. We compare zero-shot prompting and LoRA fine-tuning on large language models, while also exploring the integration of high-level speech pause features. Evaluations on English meeting recordings and multilingual lecture transcripts (Portuguese, German) show significant improvements over established topic segmentation baselines. Additionally, we adapt a common evaluation measure for multi-level segmentation, taking into account all hierarchical levels within one metric.
Pick and Summarize: Integrating Extractive and Abstractive Speech Summarization
Authors: Takatomo Kano; Atsunori Ogawa; Marc Delcroix; Ryo Fukuda; William Chen; Shinji Watanabe
Venue: Interspeech 2025 (main)

Abstract: Speech summarization condenses long speech while preserving essential content. Recently, there has been growing interest in end-to-end (E2E) abstractive speech summarization, which directly generates a text summary from spoken input. However, abstractive summarization of lengthy speech sequences presents challenges, such as identifying key information within very long speech. In this paper, we hypothesize that first addressing the simpler task of extractive summarization can help with these aforementioned long-sequence challenges and improve overall summarization performance. To this end, we introduce an extractive-abstractive summarization model that exploits auxiliary information from extractive summaries generated directly from raw speech input to enhance abstractive speech summarization.  Experiments on a web presentation corpus demonstrate consistent gains with our proposed method, achieving up to 1.4-point gains in METEOR score over a strong abstractive summarization baseline.
Beyond Similarity Scoring: Detecting Entailment and Contradiction in Multilingual and Multimodal Contexts
Authors: Othman Istaiteh; Salima Mdhaffar; Yannick EstÃ¨ve
Venue: Interspeech 2025 (main)

Abstract: Natural Language Inference (NLI) determines whether a hypothesis entails, contradicts, or is neutral with respect to a premise. While text-based NLI is well-studied, its multimodal and multilingual extension remains underexplored. This paper introduces a multilingual, multimodal NLI framework classifying entailment, contradiction, and neutrality across text-text, text-speech, speech-text, and speech-speech pairs in same- and cross-lingual settings. A key motivation is improving translation assessment, where similarity-based approaches may miss contradictions. The framework complements evaluation methods and helps identify inconsistencies by detecting entailment and contradiction alongside semantic similarity. It also extends text-based datasets with speech-text and speech-speech pairs for multilingual multimodal inference. Experiments show the model outperforms BLASER in distinguishing entailment from non-entailment, achieving F1 gains of 0.19 in speech-speech and 0.13 in speech-text.
Comparison-Based Automatic Evaluation for Meeting Summarization
Authors: Ziwei Gong; Lin Ai; Harsh Deshpande; Alexander Johnson; Emmy Phung; Zehui Wu; Ahmad Emami; Julia Hirschberg
Venue: Interspeech 2025 (main)

Abstract: Large Language Models (LLMs) have spurred interest in automatic evaluation methods for summarization, offering a faster, more cost-effective alternative to human evaluation. However, existing methods often fall short when applied to complex tasks like long-context summarizations and dialogue-based meeting summarizations. In this paper, we introduce CREAM (Comparison-based Reference-free Elo-ranked Automatic evaluation for Meeting summarization), a novel framework that addresses the unique challenges of evaluating meeting summaries. CREAM leverages a combination of chain-of-thought reasoning and key facts alignment to assess conciseness and completeness of model-generated summaries without requiring reference. By employing an ELO ranking system, our approach provides a robust mechanism for comparing the quality of different models or prompt configurations.
Voxplorer: Voice data exploration and projection in an interactive dashboard
Authors: Alessandro De Luca; Srikanth Madikeri; Volker Dellwo
Venue: Interspeech 2025 (main)

Abstract: The acoustic signal of voice is inherently multi-dimensional. Early speech and voice research often focused on isolated or limited acoustic features. However, past research has demonstrated that a comprehensive understanding of voice requires analysing multiple dimensions simultaneously. The Voxplorer interactive dashboard addresses this issue by making state-of-the-art feature extraction and dimensionality reduction methods more accessible. It allows users to interactively explore, subset, and visualise pre-computed high-dimensional data, or extract features from recordings directly in the dashboard. The Voxplorer aims to provide modern technical resources for researchers, broadening the ideas and scope of future researcher in the field of voice communication sciences.
ASR-FAIRBENCH: Measuring and Benchmarking Equity Across Speech Recognition Systems
Authors: Anand Rai; Satyam Rahangdale; Utkarsh Anand; Animesh Mukherjee
Venue: Interspeech 2025 (main)

Abstract: Automatic Speech Recognition (ASR) systems have become ubiquitous in everyday applications, yet significant disparities in performance across diverse demographic groups persist. In this work, we introduce the ASR-FAIRBENCH leaderboard which is designed to assess both the accuracy and equity of ASR models in real-time. Leveraging the Meta's Fair-Speech dataset, which captures diverse demographic characteristics, we employ a mixed-effects Poisson regression model to derive an overall fairness Score. This score is integrated with traditional metrics like Word Error Rate (WER) to compute the Fairness Adjusted ASR Score (FAAS), providing a comprehensive evaluation framework. Our approach reveals significant performance disparities in SOTA ASR models across demographic groups and offers a benchmark to drive the development of more inclusive ASR technologies.
Transcribing Oral History Recordings Using the Transcription Portal
Authors: Christoph Draxler; Julian PÃ¶mp; Henk van den Heuvel; Fabio Ardolino; Arjan van Hessen
Venue: Interspeech 2025 (main)

Abstract: The Transcription Portal is a web-based service for multilingual orthographic transcription of speech, notably Oral History interviews. It is targeted at non-technical users: it provides a simple and intuitive GUI, supports several languages, and the workflow is pre-configured. Currently, the workflow consists of three steps: 1) automatic speech recognition, 2) manual correction of the transcript, 3) data export. Summarization and translation are planned. We demonstrate the portal on a set of historical Italian interviews on the RavensbrÃ¼ck concentration camps.
LiRI Corpus Platform: Demonstration of a Web-Based Infrastructure for Multimodal Corpus Analysis
Authors: Teodora VukoviÄ; Jeremy Zehr; Jonathan Schaber; Igor MustaÄ; Nikolina RajoviÄ; Daniel McDonald; Johannes GraÃ«n; Noah Bubenhofer
Venue: Interspeech 2025 (main)

Abstract: We present the LiRI Corpus Platform (LCP), a web-based infrastructure for storing, exploring, and analyzing diverse linguistic corpora with a focus on multimodal and audiovisual data. The platform supports synchronized querying of text, speech, and video through unified interfaces and a custom query language (Descriptive Query Definition - DQD). Dedicated frontends enable time-aligned exploration of gesture, audio, and spoken transcripts. LCP is designed to support researchers working with multimodal and annotated datasets, enabling cross-modal queries and layered annotation.
Speech Annotation for A: Accuracy, Access, and Application
Authors: Zirong Li; Hongchen Wu; Yixin Gu; Yao Du; Yang Yue
Venue: Interspeech 2025 (main)

Abstract: Accurate and efficient annotation of bilingual clinical recordings remains a persistent challenge, as existing solutions often require high demand for manual work by bilingual clinicians and their assistants and significant training related to annotation tools. To address this issue, we introduce Speech Annotation for A (SAFA)âan end-to-end, user-friendly âlazy modeâ annotation workflow. By pairing annotation drafts generated from large language models with chunk-based editing, real-time difference highlighting, and speaker & language tagging - even in multi-speaker code-switching scenarios - SAFA delivers high-quality audio annotations ready for research with minimal setup and minimal human check. It further provides standardized CSV/TXT exports, bridging the gap between fully automated approaches and the meticulous accuracy demanded by multilingual clinical research, while facilitating the creation and expansion of high-quality labeled datasets for downstream studies.
LATE: Open Source Toolkit for Latvian and Latgalian Speech Transcription
Authors: Arturs Znotins; Didzis Gosko; Normunds Gruzitis
Venue: Interspeech 2025 (main)

Abstract: We present a unique software toolkit LATE for automatic speech recognition (ASR) and formatted transcription. It has a lightweight front-end and a completely statically compiled and linked stateless back-end for running Whisper-based ASR models. Thus, LATE can be relatively easily deployed and scaled in a cloud infrastructure, as well as run privately on a local PC. The LATE toolkit is available as both open source and precompiled binaries. By default, it comes with fine-tuned Latvian and Latgalian ASR models, as well as the multilingual Whisper model, while compatible models for other languages can be added.
Scalable Offline ASR for Command-Style Dictation in Courtrooms
Authors: Kumarmanas Nethil; Vaibhav Mishra; Kriti Anandan; Kavya Manohar
Venue: Interspeech 2025 (main)

Abstract: We propose an open-source framework for Command-style dictation that addresses the gap between resource-intensive Online systems and high-latency Batch processing. Our approach uses Voice Activity Detection (VAD) to segment audio and transcribes these segments in parallel using Whisper models, enabling efficient multiplexing across audios. Unlike proprietary systems like SuperWhisper, this framework is also compatible with most ASR architectures, including widely used CTC-based models. Our multiplexing technique maximizes compute utilization in real-world settings, as demonstrated by its deployment in around 15% of India's courtrooms. Evaluations on live data show consistent latency reduction as user concurrency increases, compared to sequential batch processing. The live demonstration will showcase our open-sourced implementation and allow attendees to interact with it in real-time.
Towards a dynamical model of transitions between fluent and stuttered speech
Authors: Yijing Lu; Khalil Iskarous; Louis Goldstein
Venue: Interspeech 2025 (main)

Abstract: This paper introduces a dynamical systems framework for understanding stuttering, conceptualizing it as a qualitative shift in speech articulation driven by a single control parameter. Using a forced Duffing oscillator model, we demonstrate how variations in the excitation frequency can account for transitions between fluent and stuttered speech states. The model generates specific predictions about articulatory behaviors during stuttering, which we test using real-time MRI data of stuttered speech. Analysis of articulatory movements provides empirical support for the modelâs predictions, suggesting that stuttering can be understood as a dynamical diseaseâan intact system operating outside its typical parameter range. This framework offers new insights into the nature of stuttering and potential approaches to intervention.
Study of vocal fold vibration using M-mode ultrasound: a proof of concept
Authors: Juliette Dindart; AgnÃ¨s Rouxel; Crystal Lin; Trung Kien Bui; Muriel Lefort; Claire Pillot-Loiseau; Christophe TrÃ©sallet; FrÃ©dÃ©rique Frouin
Venue: Interspeech 2025 (main)

Abstract: Ultrasound has recently been suggested as an alternative to laryngoscopy for checking vocal fold movement after neck surgery. We propose to use M-mode ultrasound (MUS) to study vocal fold vibration in left and right hemilarynges. MUS is acquired along a 1D line in each hemilarynx for about 5 seconds during vowel phonation. Post-processing estimates spatio-temporal maps of fundamental and second harmonic frequencies. To validate our method, 108 recordings (MUS and voice) from 12 healthy subjects were acquired. We compared the fundamental frequency obtained by MUS with that from voice analysis. The median fundamental frequency was estimated by MUS with high accuracy (y=0.997x+0.293, r=0.999). In our current trial, estimated frequencies are limited to 250 Hz. In future work, frequency will be increased to 1 kHz to avoid aliasing at high pitch, and MUS will be tested on patients with vocal pathologies.
Articulatory Feature Prediction from Surface EMG during Speech Production
Authors: Jihwan Lee; Kevin Huang; Kleanthis Avramidis; Simon Pistrosch; Monica Gonzalez-Machorro; Yoonjeong Lee; BjÃ¶rn W. Schuller; Louis Goldstein; Shrikanth Narayanan
Venue: Interspeech 2025 (main)

Abstract: We present a model for predicting articulatory features from surface electromyography (EMG) signals during speech production. The proposed model integrates convolutional layers and a Transformer block, followed by separate predictors for articulatory features. Our approach achieves a high prediction correlation of approximately 0.9 for most articulatory features. Furthermore, we demonstrate that these predicted articulatory features can be decoded into intelligible speech waveforms. To our knowledge, this is the first method to decode speech waveforms from surface EMG via articulatory features, offering a novel approach to EMG-based speech synthesis. Additionally, we analyze the relationship between EMG electrode placement and articulatory feature predictability, providing knowledge-driven insights for optimizing EMG electrode configurations. The source code and decoded speech samples are publicly available.
Enhancing Acoustic-to-Articulatory Speech Inversion by Incorporating Nasality
Authors: Saba Tabatabaee; Suzanne Boyce; Liran Oren; Mark Tiede; Carol Espy-Wilson
Venue: Interspeech 2025 (main)

Abstract: Speech is produced through the coordination of vocal tract constricting organs: lips, tongue, velum, and glottis. Previous works developed a Speech Inversion (SI) systems to recover acoustic-to-articulatory mappings for lip and tongue constrictions, called oral tract variables (TVs), which were later enhanced by including source information (periodic and aperiodic energies, and F0 frequency) as proxies for glottal control. Comparison of the nasometric measures with high-speed nasopharyngoscopy showed that nasalance can serve as ground truth, and that an SI system trained with it reliably recovers velum movement patterns for American English speakers. Here, two SI training approaches are compared: baseline models that estimate oral TVs and nasalance independently, and a synergistic model that combines oral TVs and source features with nasalance. The synergistic model shows relative improvements of 5% in oral TVs estimation and 9% in nasalance estimation compared to the baseline models.
Modeling Probabilistic Reduction using Information Theory and Naive Discriminative Learning
Authors: Anna Stein; Kevin Tang
Venue: Interspeech 2025 (main)

Abstract: This study compares probabilistic predictors based on information theory with Naive Discriminative Learning (NDL) predictors in modeling acoustic word duration, focusing on probabilistic reduction. We examine three models using the Buckeye corpus: one with NDL-derived predictors using information-theoretic formulas, one with traditional NDL predictors, and one with N-gram probabilistic predictors. Results show that the N-gram model outperforms both NDL models, challenging the assumption that NDL is more effective due to its cognitive motivation. However, incorporating information-theoretic formulas into NDL improves model performance over the traditional model. This research highlights a) the need to incorporate not only frequency and contextual predictability but also average contextual predictability, and b) the importance of combining information-theoretic metrics of predictability and information derived from discriminative learning in modeling acoustic reduction.
Contextual predictability effects on acoustic distinctiveness in read Polish speech
Authors: Zofia Malisz; Jan Foremski; MagÅorzata Kul
Venue: Interspeech 2025 (main)

Abstract: This paper examines how contextual predictability (surprisal) influences acoustic features in Polish speech using the PRODIS dataset. The study analyzes connected read speech from Wikipedia texts on history, politics, culture, and science, extracting surprisal values from a phoneme-based language model. Results show that high surprisal increases acoustic distinctiveness, such as longer segment duration and larger vowel space, while low surprisal reduces distinctiveness. We also find effects of text topic, lexical frequency, and lexical stress on surprisal. These findings highlight the complex interplay between predictability, discourse, and prosody in speech. The work contributes to understudied analyses on predictability and acoustics in Slavic languages.
How do both phonological and syntactic complexity influence speech planning?
Authors: Ivan Yuen; Katherine Demuth; Stefanie Shattuck-Hufnagel
Venue: Interspeech 2025 (main)

Abstract: Wheeldon & Lahiri reported that speech initiation time (RT) was influenced by the phonological complexity of the initial prosodic word (PWd) in an immediate-production task, but by the number of PWds in a delayed-production task that enabled preplanning. Thus, different tasks are sensitive to different types of complexity. The current study explored RT for different complexity types: number of syllables in the PWd (monosyllabic vs disyllabic), location of a complex PWd (early vs late), and number of PWds (3 vs. 4 vs. 5) using a reading aloud task. The presence of a disyllabic noun increased RT only in 4-PWd sentences, where the syntactic complexity of the SUBJECT and OBJECT NPs differed. This suggests that RT may increase in the absence of parallel syntactic structures in SUBJECT and OBJECT position, highlighting the need for further research on the role of complexity at different levels in the speech planning process.
When focus shapes the flow: prosodic restructuring in Mandarin complex nominals
Authors: Anqi Xu; Yu-yin Hsu
Venue: Interspeech 2025 (main)

Abstract: While extensive studies have explored acoustic focus realization in Mandarin, little is known about how focus affects the prosodic phrasing of Mandarin complex nominals. This study examined how contrastive focus influences syllable duration of Mandarin numeral-classifier-noun phrases. Using a mini-dialogue paradigm, we elicited contrastive focus of different spans, alongside a baseline no-focus condition. Two production experiments revealed that when focus was placed on the numeral, the default prosodic grouping was disrupted, dissimilar to when focus encompasses the entire phrase or in neutral contexts, with tonal factors amplifying the reorganization. Our results indicate that prosodic organization in Mandarin is shaped by tones, morphosyntactic structures, focus marking and their interplay. Crucially, the results challenge rigid models of boundary phrasing and disyllabic footing, highlighting a multilevel interaction among phonetics, prosodic phrasing, and syntax in tonal languages.