GenAI4ELab/papercli-papers huggingface dataset:
âKAN you hear me?â Exploring Kolmogorov-Arnold Networks for Spoken Language Understanding
Authors: Alkis Koudounas; Moreno La Quatra; Eliana Pastor; Sabato Marco Siniscalchi; Elena Baralis
Venue: Interspeech 2025 (main)

Abstract: Kolmogorov-Arnold Networks (KANs) have recently emerged as a promising alternative to traditional neural architectures, yet their application to speech processing remains under explored. This work presents the first investigation of KANs for Spoken Language Understanding (SLU) tasks. We experiment with 2D-CNN models on two datasets, integrating KAN layers in five different configurations within the dense block. The best-performing setup, which places a KAN layer between two linear layers, is directly applied to transformer-based models and evaluated on five SLU datasets with increasing complexity. Our results show that KAN layers can effectively replace the linear layers, achieving comparable or superior performance in most cases. Finally, we provide insights into how KAN and linear layers on top of transformers differently attend to input regions of the raw waveforms.
Rasmalai : Resources for Adaptive Speech Modeling in IndiAn Languages with Accents and Intonations
Authors: Ashwin Sankar; Yoach Lacombe; Sherry Thomas; Praveen Srinivasa Varadhan; Sanchit Gandhi; Mitesh M. Khapra
Venue: Interspeech 2025 (main)

Abstract: We introduce RASMALAI, a large-scale speech dataset with rich text descriptions, designed to advance controllable and expressive text-to-speech (TTS) synthesis for 23 Indian languages and English. It comprises 13,000 hours of speech and 24 million text-description annotations with fine-grained attributes like speaker identity, accent, emotion, style, and background conditions. Using RASMALAI, we develop INDICPARLERTTS, the first open-source, text-description-guided TTS for Indian languages. Systematic evaluation demonstrates its ability to generate high-quality speech for named speakers, reliably follow text descriptions and accurately synthesize specified attributes. Additionally, it effectively transfers expressive characteristics both within and across languages. INDICPARLERTTS consistently achieves strong performance across these evaluations, setting a new standard for controllable multilingual expressive speech synthesis in Indian languages.
Kinship in Speech: Leveraging Linguistic Relatedness for Zero-Shot TTS in Indian Languages
Authors: Utkarsh Pathak; Chandra Sai Krishna Gunda; Anusha Prakash; Keshav Agarwal; Hema A. Murthy
Venue: Interspeech 2025 (main)

Abstract: Text-to-speech (TTS) systems typically require high-quality studio data and accurate transcriptions for training. India has 1369 languages, with 22 official using 13 scripts. Training a TTS system for all these languages, most of which have no digital resources, seems a Herculean task. Our work focuses on zero-shot synthesis, particularly for languages whose scripts and phonotactics come from different families. The novelty of our work is in the augmentation of a shared phone representation and modifying the text parsing rules to match the phonotactics of the target language, thus reducing the synthesiser overhead and enabling rapid adaptation. Intelligible and natural speech was generated for Sanskrit, Maharashtrian and Canara Konkani, Maithili and Kurukh by leveraging linguistic connections across languages with suitable synthesisers. Evaluations confirm the effectiveness of this approach, highlighting its potential to expand speech technology access for under-represented languages.
Can We Reconstruct a Dysarthric Voice with the Large Speech Model Parler TTS?
Authors: Ariadna Sanchez; Simon King
Venue: Interspeech 2025 (main)

Abstract: Speech disorders can make communication hard or even impossible for those who develop them. Personalised Text-to-Speech is an attractive option as a communication aid. We attempt voice reconstruction using a large speech model, with which we generate an approximation of a dysarthric speaker's voice prior to the onset of their condition. In particular, we investigate whether a state-of-the-art large speech model, Parler TTS, can generate intelligible speech while maintaining speaker identity. We curate a dataset and annotate it with relevant speaker and intelligibility information, and use this to fine-tune the model. Our results show that the model can indeed learn to generate from the distribution of this challenging data, but struggles to control intelligibility and to maintain consistent speaker identity. We propose future directions to improve controllability of this class of model, for the voice reconstruction task.
Voice Adaptation for Swiss German
Authors: Samuel Stucki; Jan Deriu; Mark Cieliebak
Venue: Interspeech 2025 (main)

Abstract: This work investigates the performance of Voice Adaptation models for Swiss German dialects, i.e., translating Standard German text to Swiss German dialect speech. For this, we preprocess a large dataset of Swiss podcasts, which we automatically transcribe and annotate with dialect classes, yielding approximately 5000 hours of weakly labeled training material. We fine-tune the XTTSv2 model on this dataset and show that it achieves good scores in human and automated evaluations and can correctly render the desired dialect. Our work shows a step towards adapting Voice Cloning technology to underrepresented languages. The resulting model achieves CMOS scores of up to -0.28 and SMOS scores of 3.8.
Gradual modeling of the Lombard effect by modifying speaker embeddings from a Text-To-Speech model
Authors: Thiago Henrique Gomes Lobato; Magnus SchÃ¤fer
Venue: Interspeech 2025 (main)

Abstract: This work proposes to modify clean speech into Lombard-like speech by modifying speaker embeddings used to condition a text-to-speech model. A feedforward network learns to map embeddings of plain speech to those of Lombard speech using paired data from the Audio-Visual Lombard Grid corpus. Signal level is then increased as per ITU-T P.1150, and a neural vocoder performs time stretching. We show that the resulting speech retains most of the speakerâs identity while incorporating relevant Lombard characteristics. Additionally, by properly interpolating embeddings, we propose an approach to gradually model Lombard speech as a function of the background noise level. Listening tests show about a 1.12-point Mean opinion score (MOS) increase in speech plausibility in a loud background context, with only a 0.5-point MOS decrease in speaker similarity compared to an ideal Lombard speech interpolation.
When Humans Growl and Birds Speak: High-Fidelity Voice Conversion from Human to Animal and Designed Sounds
Authors: Minsu Kang; Seolhee Lee; Choonghyeon Lee; Namhyun Cho
Venue: Interspeech 2025 (main)

Abstract: Human to non-human voice conversion (H2NH-VC) transforms human speech into animal or designed vocalizations. Unlike prior studies focused on dog-sounds and 16 or 22.05kHz audio transformation, this work addresses a broader range of non-speech sounds, including natural sounds (lion-roars, birdsongs) and designed voice (synthetic growls). To accomodate generation of diverse non-speech sounds and 44.1kHz high-quality audio transformation, we introduce a preprocessing pipeline and an improved CVAE-based H2NH-VC model, both optimized for human and non-human voices. Experimental results showed that the proposed method outperformed baselines in quality, naturalness, and similarity MOS, achieving effective voice conversion across diverse non-human timbres.
EEG-based Voice Conversion : Hearing the Voice of Your Brain
Authors: Yizhong Geng; Wenxin Fu; Qihang Lu; Bingsong Bai; Cong Wang; Yingming Gao; Ya Li
Venue: Interspeech 2025 (main)

Abstract: The connection between Electroencephalography (EEG) signals and human voice has gained significant attention, with studies demonstrating the feasibility of speech synthesis from EEG data. However, EEG-based voice conversion (VC) remains largely unexplored. To address this, we present the first EEG-based zero-shot VC system that converts speech into a target speakerâs voice without prior target data. Our method integrates an EEG feature extraction module with an alignment module to map EEG features to speaker-specific voice features. By leveraging an innovative three-stage training strategy and a pre-trained VC modelâtrained solely on speech dataâwe achieve zero-shot conversion. Experiments on the Single-Word-Production Dutch-iBIDS dataset confirm the systemâs ability to reliably convert speech to a target speakerâs voice. This work highlights the potential of EEG-based VC for advancing assistive communication and brain-computer interfaces. All demos are available in https://doi.org/10.5281/zenodo.15510829.
Streaming Non-Autoregressive Model for Accent Conversion and Pronunciation Improvement
Authors: Tuan-Nam Nguyen; Ngoc-Quan Pham; Åeymanur Akti; Alexander Waibel
Venue: Interspeech 2025 (main)

Abstract: We propose a first streaming accent conversion (AC) model that transforms non-native speech into a native-like accent while preserving speaker identity, prosody and improving pronunciation. Our approach enables stream processing by modifying a previous AC architecture with an Emformer encoder and an optimized inference mechanism. Additionally, we integrate a native text-to-speech (TTS) model to generate ideal ground-truth data for efficient training. Our streaming AC model achieves comparable performance to the top AC models while maintaining stable latency, making it the first AC system capable of streaming.
Zero-Shot Mono-to-Binaural Speech Synthesis
Authors: Alon Levkovitch; Julian Salazar; Soroosh Mariooryad; RJ Skerry-Ryan; Nadav Bar; Bastiaan Kleijn; Eliya Nachmani
Venue: Interspeech 2025 (main)

Abstract: We present ZeroBAS, a neural method to synthesize binaural speech from monaural speech recordings and positional information without training on any binaural data. To our knowledge, this is the first published zero-shot neural approach to mono-to-binaural speech synthesis. Specifically, we show that a parameter-free geometric time warping and amplitude scaling based on source location suffices to get an initial binaural synthesis that can be refined by iteratively applying a pretrained denoising vocoder. Furthermore, we find this leads to generalization across room conditions, which we measure by introducing a new dataset, TUT Mono-to-Binaural, to evaluate state-of-the-art monaural-to-binaural synthesis methods on unseen conditions. Our zero-shot method is perceptually on-par with the performance of supervised methods on previous standard mono-to-binaural dataset, and even surpasses them on our out-of-distribution TUT Mono-to-Binaural dataset.
Tone recognition in low-resource languages of North-East India: peeling the layers of SSL-based speech models
Authors: Parismita Gogoi; Sishir Kalita; Wendy Lalhminghlui; Viyazonuo Terhiija; Moakala Tzudir; Priyankoo Sarmah; S. R. M. Prasanna
Venue: Interspeech 2025 (main)

Abstract: This study explores the use of self-supervised learning (SSL) models for tone recognition in three low-resource languages from North Eastern India: Angami, Ao, and Mizo. We evaluate four Wav2vec2.0 base models that were pre-trained on both tonal and non-tonal languages. We analyze tone-wise performance across the layers for all three languages and compare the different models. Our results show that tone recognition works best for Mizo and worst for Angami. The middle layers of the SSL models are the most important for tone recognition, regardless of the pre-training language, i.e. tonal or non-tonal. We have also found that the tone inventory, tone types, and dialectal variations affect tone recognition. These findings provide useful insights into the strengths and weaknesses of SSL-based embeddings for tonal languages and highlight the potential for improving tone recognition in low-resource settings. The source code is available at GitHub.
Corpus-Based Insights into Mandarin Neutral Tone: Effects of Tonal Context and Structural Patterns in Spontaneous Speech
Authors: Jingyi Sun; Nicolas Audibert; Yaru Wu; Martine Adda-Decker
Venue: Interspeech 2025 (main)

Abstract: This study examines the realization of Mandarin neutral tone (NT) in spontaneous conversational speech. Drawing on 8,550 phrase-medial NT tokens surrounded by citation tones, we investigate whether robust coarticulatory influences from adjacent tones persist in natural discourse and compare two subtypes of NTâone with a nasal offset and one withoutâthereby testing the combined effects of syllable structure and grammatical function. Our analysis confirms that adjacent tones exert a dominant influence on NT realization: ultrashort NT syllables largely continue the preceding toneâs contour, while normal- to long-duration syllables exhibit clearer transitions toward the subsequent tone. NT with a nasal offset maintains a higher pitch and greater F0 dynamics while remaining stable across durations, likely link to its stronger prosodic and grammatical cues. These findings highlight the interplay of tonal coarticulation, duration, and structural patterns in shaping NT variation.
Tonal Variation and Word Meaning in Taiwanese
Authors: Yu-Ying Chuang; Sheng-Fu Wang
Venue: Interspeech 2025 (main)

Abstract: Tone sandhi is extensive in Taiwanese, where all but the final syllable in a morphosyntactically defined unit undergo sandhi. Previous studies have primarily focused on comparing sandhi tonal realizations to their corresponding citation ones to determine whether neutralization occurs. However, such a high-level comparison overlooks the variability in how tones are actually realized. This study aims to examine tonal variation in Taiwanese, focusing on whether word meaning influences tonal realization. Using spontaneous speech data, we analyzed the realizations of the high-falling tone. Our findings indicate that word meaning does contribute to tonal variability. Moreover, when meaning-induced tonal variation is accounted for, no remaining differences between sandhi and citation tones can be observed. These results suggest that tonal realizations systematically vary with word meaning and that the effect of meaning on tonal realizations should be considered when discussing neutralization.
Sounding Like a Winner? Prosodic Differences in Post-Match Interviews
Authors: Sofoklis Kakouros; Haoyu Chen
Venue: Interspeech 2025 (main)

Abstract: This study examines the prosodic characteristics associated with winning and losing in post-match tennis interviews. Additionally, this research explores the potential to classify match outcomes solely based on post-match interview recordings using prosodic features and self-supervised learning (SSL) representations. By analyzing prosodic elements such as pitch and intensity, alongside SSL models like Wav2Vec 2.0 and HuBERT, the aim is to determine whether an athlete has won or lost their match. Traditional acoustic features and deep speech representations are extracted from the data, and machine learning classifiers are employed to distinguish between winning and losing players. Results indicate that SSL representations effectively differentiate between winning and losing outcomes, capturing subtle speech patterns linked to emotional states. At the same time, prosodic cuesâsuch as pitch variabilityâremain strong indicators of victory.
Exploratory Study of Filled Pauses in Ukrainian Language: Phonetic Properties of Filled Pauses
Authors: Anna Havras; Carlos Mendes; Helena Moniz; Gueorgui Hristovsky; JoÃ£o Miranda
Venue: Interspeech 2025 (main)

Abstract: This exploratory study addresses a significant gap in research on filled pauses (FPs) in Ukrainian, an understudied language with limited phonetic studies. By analyzing acoustic features such as fundamental frequency, formants, and duration, this research offers a comprehensive typology of Ukrainian FPs. We analyzed four hours of speech across three types of contexts: broadcast news (BN), guest interviews (GI), and political interviews (PI). We identified the most frequent FPs ([a] 54%, [Éª] 36%) and documented two previously unreported variants ([na] 0.4%, [mna] 0.4%).  Acoustic analysis confirmed gender-based differences in the vocalic triangle, with female speakers exhibiting greater vowel distinction in FPs.  Our findings revealed that FPs followed regular patterns, primarily composed by generally referred as central vowels and/or nasal consonants, and are highly dependent on both the speaker and context.
Evaluating the suitability of acoustic parameters for capturing breathy voice in non-pathological female speakers
Authors: Chloe Patman; Paul Foulkes; Kirsty McDougall
Venue: Interspeech 2025 (main)

Abstract: This study evaluates the suitability of acoustic parameters for capturing breathy voice in non-pathological female speakers. Much existing literature on the acoustic analysis of voice quality (VQ) focuses on pathological speakers and/or sustained vowel productions. The limited work on more naturalistic speech (e.g. as needed for forensic casework) focuses on male speakers. Using studio-quality recordings from the PASR dataset, nine phoneticians (6 male, 3 female) read a passage in their normal (âdefaultâ) and breathy voice multiple times. Acoustic parameters (H1âH2, H1*âH2*, H1âA1, H1*âA1*, CPP, HNR05/15/25/35) were automatically extracted using VoiceSauce. A linear mixed effects regression analysis showed significant interactions between acoustic parameters and speaker sex. In support of, our study confirms CPP as a relatively robust parameter for males and extends its suitability to females, with CPP and HNR particularly suitable for female breathy voice.
Robustness of F0 Ratio as a Diagnostic: Comparing Creaky Voice in Danish and Seoul Korean
Authors: Michaela Watkins; Rasmus Puggaard-Rode; Paul Boersma; Silke Hamann
Venue: Interspeech 2025 (main)

Abstract: We present an exploratory analysis of F0 ratio, a proposed method to pick up octave jumps in the speech signal. Such jumps, often considered errors, are possibly indicative of the presence of creaky voice. This paper focuses on co-intrinsic voice quality in Seoul Korean fortis stops, building on previous data, and on the Danish contrastive voice quality stÃ¸d. The results suggest that in Seoul Korean F0 ratio captures a clear jump upwards indicating a modality switch from creaky to modal voice, with a gender difference observed. This suggests that pitch jumps are not necessarily erroneous but may reflect systematic cues to phonological contrasts cued with creak. In Danish, F0 ratio captures a rising intonational contour for non-stÃ¸d tokens and is able to categorise between stÃ¸d and non-stÃ¸d tokens with high accuracy, although this leaves open the question whether F0 ratio captures modality shifts in Danish, or rather a combination of modality shift and pitch contour differences.
Discovering Directions of Uncertainty in Speech Inpainting
Authors: Kfir Cohen; Lior Wolf; Bracha Laufer-Goldshtein
Venue: Interspeech 2025 (main)

Abstract: Speech inpainting aims to restore missing segments in audio signals. While both classical signal processing and deep learning approaches have been developed for this task, they typically generate a single output, despite the existence of multiple plausible reconstructions. In this paper, we adapt Neural Posterior Principal Component (NPPC) to recover the principal directions of variation in the posterior distribution of the original signal given its masked version. Our empirical results demonstrate that these directions capture diverse and meaningful variations in both speech content and style, while more precisely capturing the predictive error compared to a more costly Bayesian deep learning approach.
InfiniteAudio: Infinite-Length Audio Generation with Consistency
Authors: Chaeyoung Jung; Hojoon Ki; Ji-Hoon Kim; Junmo Kim; Joon Son Chung
Venue: Interspeech 2025 (main)

Abstract: This paper presents InfiniteAudio, a simple yet effective strategy for generating infinite-length audio using diffusion-based text-to-audio methods. Current approaches face memory constraints because the output size increases with input length, making long duration generation challenging. A common workaround is to concatenate short audio segments, but this often leads to inconsistencies due to the lack of shared temporal context. To address this, InfiniteAudio integrates seamlessly into existing pipelines without additional training. It introduces two key techniques: FIFO sampling, a first-in, first-out inference strategy with fixed-size inputs, and curved denoising, which selectively prioritizes key diffusion steps for efficiency. Experiments show that InfiniteAudio achieves comparable or superior performance across all metrics.
FoleyMaster: High-Quality Video-to-Audio Synthesis via MLLM-Augmented Prompt Tuning and Joint Semantic-Temporal Adaptation
Authors: Liming Liang; Luo Chen; Yuehan Jin; Xianwei Zhuang; Yuxin Xie; Yongkang Yin; Yuexian Zou
Venue: Interspeech 2025 (main)

Abstract: We study video-to-audio (V2A) generation, a critical task for automatically creating high-quality sound effects synchronized with silent video. Current V2A methods face three limitations: (1) inadequate textual annotations in existing datasets, (2) over-reliance on global video features, and (3) coarse temporal synchronization. To address these, We propose FoleyMaster with three key innovations: 1) We introduce VGGSound Plus dataset with 197,955 videos annotated by Qwen2-VL-7B for fine-grained event descriptions; 2) We develop a cross-attention semantic adapter integrating token-level text embeddings with global video features via prompt learning, enabling precise alignment between visual events and sound; 3) We develop a probabilistic temporal adapter that adjusts audio generation based on action prominence replacing binary synchronization. Extensive experiments demonstrate that FoleyMaster achieves state-of-the-art V2A performance across all metrics. Demo and dataset are available.
Video-to-Audio Generation with Fine-grained Temporal Semantics
Authors: Yuchen Hu; Yu Gu; Chenxing Li; Rilin Chen; Dong Yu
Venue: Interspeech 2025 (main)

Abstract: With recent advances of AIGC, video generation gains a surge of research interest in both academia and industry (e.g., Sora). However, it remains a challenge to produce temporally aligned audio to match the generated video, considering its complex semantic information. In this work, inspired by the recent success of text-to-audio generation, we investigate the video-to-audio (VTA) generation based on latent diffusion model (LDM). Similar to pioneering explorations, our preliminary results show great potentials of LDM in VTA task, but challenges still exist in temporal consistence. To this end, we propose to enhance the temporal alignment of VTA with frame-level semantic information. With the popular grounding segment anything model (Grounding SAM), we extract the fine-grained semantics in video frames to enable VTA to produce better-aligned audio signal. Experiments show its effectiveness in both objective and subjective evaluation, improving both audio quality and temporal alignment.
TTMBA: Towards Text To Multiple Sources Binaural Audio Generation
Authors: Yuxuan He; Xiaoran Yang; Ningning Pan; Gongping Huang
Venue: Interspeech 2025 (main)

Abstract: Most existing text-to-audio (TTA) generation methods produce mono outputs, neglecting essential spatial information for immersive auditory experiences. To address this issue, we propose a cascaded method for text-to-multisource binaural audio generation (TTMBA) with both temporal and spatial control. First, a pretrained large language model (LLM) segments the text into a structured format with time and spatial details for each sound event. Next, a pretrained mono audio generation network creates multiple mono audios with varying durations for each event. These mono audios are transformed into binaural audios using a binaural rendering neural network based on spatial data from the LLM. Finally, the binaural audios are arranged by their start times, resulting in multisource binaural audio. Experimental results demonstrate the superiority of the proposed method in terms of both audio generation quality and spatial perceptual accuracy.