GenAI4ELab/papercli-papers huggingface dataset:
Exploring auditory feedback mechanisms in speech recognition
Authors: Louise Coppieters de Gibson; Philip N. Garner
Venue: Interspeech 2025 (main)

Abstract: For many years, automatic speech recognition (ASR) has been built on compressed filter-bank features understood to be a rough model of the cochlea.  However, recent understanding, evidenced by oto-acoustic emissions, is that the cochlea is composed of driven oscillators. The Hopf mechanism arising from an oscillator model explains the well known cube-root compression. A bifurcation arises from an inner feedback loop from the outer to inner hair cells. Further, larger feedback loops exist along the efferent path of the auditory nerve.  In the present study, to the extent to which current compute power allows, we investigate how to incorporate the Hopf mechanism and the olivocochlear feedback mechanisms into ASR.  Results show that adding this larger feedback loop appears beneficial for ASR.  The results currently have modest implications for ASR, however, such technology could be used to make inference about the biological mechanism.
French schwa is not acoustically distinct from its two lexical neighbors /Ã¸/ and /Å/
Authors: Mathilde Hutin; MÃ©lanie Lancien; Noam Faust
Venue: Interspeech 2025 (main)

Abstract: French schwa, or mute-e, is one of the most studied topics in French phonology, yet its exact phonetic quality remains debated. In particular, whether schwa is acoustically similar to or distinct from its two mid front rounded neighbors has given rise to contradictory results. The main issue is that past studies generally compare schwa and its neighbors without controlling for stress, yet the latter can be stressed while the former can never be, which is bound to impact their respective realizations. In this paper, we present a pilot study to investigate the quality of schwa compared to its neighbors while controlling for stress. Twenty-two French words with schwa or with its neighbors, were embedded in carrier sentences and read aloud among filler sentences. The data from 20 native speakers of Parisian French indicate that, when controlling for stress as well as orthography, schwa and its neighbors are acoustically very similar, in terms of both formants and duration.
Apical vs. Regular Vowel Duration: A Corpus-based Analysis of Contextual Influences in Standard Mandarin
Authors: Jingyi Sun; Bowei Shao; Martine Adda-Decker
Venue: Interspeech 2025 (main)

Abstract: The syllabic fricatives, traditionally known as apical vowels in Standard Mandarin, share the place of articulation with their sibilant onsets, exhibiting a narrow tongue tip constriction at the dental/alveolar or postalveolar region. We hypothesize that, like voiced fricatives, they face aerodynamic constraints where voicing and frication cannot be fully optimized simultaneously. The narrow constriction may delay intra-oral pressure release, leading to unstable and delayed voicing onset and consequently shorter acoustic durations. To test this hypothesis while minimizing segmentation errors, we applied forced alignment to a large-scale, style-comparable corpus to analyze their durational differences. The results show that in both spontaneous and read speech, the syllabic fricatives are significantly shorter compared to regular vocalic nuclei. The findings join previous articulatory and acoustic evidence and provide further evidence on the fricative nature of these syllabic segments.
On Apical Vowels in Eastern Zhenjiang Mandarin
Authors: Xuying Wang; Fang Hu
Venue: Interspeech 2025 (main)

Abstract: There is generally no distributional constraint for the apical vowels in Eastern Zhenjiang Mandarin, and the apical vowels contrast with the high front vowels when preceded by an initial consonant [p ph m t th n l Ê¨ Ê¨h É] or occurring in an onset-less syllable. This paper analyzes acoustic characteristics and lingual articulation in the production of the apical vowels [É¿ É¥] in Eastern Zhenjiang Mandarin. The results show that although accompanied by frication noise, the apical vowels [É¿ É¥] have their own spectral properties that are distinctive to the high front vowel [i y]. The ultrasound data reveal that both the anterior and posterior parts of the tongue are involved in the production of [É¿ É¥].
Equivalence and differences: Formant patterns of labialization and pharyngealization in Tashlhiyt
Authors: Philipp Buech; Anne Hermes; Rachid Ridouane
Venue: Interspeech 2025 (main)

Abstract: Labialization and pharyngealization are secondary articulations that are produced with different vocal tract modifications. In Tashlhiyt, a language that has both secondary articulations in its consonant system, labialization is produced with lip protrusion and a backing of the tongue, while pharyngealization is articulated by a backward movement of the tongue towards the pharynx. Despite this difference, both secondary articulations are reported to have similar acoustic effects regarding F2 across languages. The current study investigates the impact of labialization and pharyngealization on the formant structure of adjacent vowels in the same language. The analysis of VCV logatomes containing /i, a, u/ produced by 35 Tashlhiyt speakers revealed that both secondary articulations show equivalent patterns related to F2, which are strongest in /i/, followed by /a/, whereas differences between labialization and pharyngealization depend mainly on F1 and vowel quality.
Temporal organization of prenuclear glides in Hefei Mandarin
Authors: Yifan Yang; Zhiheng Qian
Venue: Interspeech 2025 (main)

Abstract: This study investigates the temporal organization of prenuclear glides [j] and [w] in Hefei Mandarin using acoustic measurements. While the status of prenuclear glides in Chinese syllables remains debated, we employ a recently developed acoustic approach to analyze how CjV and CwV syllables are coordinated. Our results suggest that prenuclear glides in Hefei Mandarin are more likely to be part of the rime rather than forming complex onsets. The findings align with previous research on the status of Mandarin prenuclear [j] but reveal inconsistencies for prenuclear [w] compared to prior studies. This research contributes to our understanding of Chinese syllable structure and highlights the utility of acoustic methods in inferring abstract phonological structures.
Speaker-specific Patterns of Phonetic Covariation in Korean Word-medial Stops and the Role of Phonological and Morphological Contexts
Authors: Chloe D. Kwon
Venue: Interspeech 2025 (main)

Abstract: Individual speaker differences in speech have often been regarded as random, but growing evidence suggests that variation across speakers follows systematic patterns. Yet, it remains an open question whether such structured variation is universal across languages, spans multiple phonetic dimensions, and remains consistent across contexts. Korean word-medial stops provide a compelling test case, as these stops are subject to extensive contextual variability. This study examines patterns in individual speaker variation by analyzing the phonetic distributions and covariation patterns between word-medial stops in Seoul Korean in varying phonological and morphological contexts. The results show that while individual differences arise in phonetic realization of these stops, speakers maintain structured covariation that allows stop categories to remain distinct despite variability, providing further support for the phonetic uniformity account.
HiFiTTS-2: A Large-Scale High Bandwidth Speech Dataset
Authors: Ryan Langman; Xuesong Yang; Paarth Neekhara; Shehzeen Hussain; Edresson Casanova; Evelina Bakhturina; Jason Li
Venue: Interspeech 2025 (main)

Abstract: This paper introduces HiFiTTS-2, a large-scale speech dataset designed for high-bandwidth speech synthesis. The dataset is derived from LibriVox audiobooks, and contains approximately 36.7k hours of English speech for 22.05 kHz training, and 31.7k hours for 44.1 kHz training. We present our data processing pipeline, including bandwidth estimation, segmentation, text preprocessing, and multi-speaker detection. The dataset is accompanied by detailed utterance and audiobook metadata generated by our pipeline, enabling researchers to apply data quality filters to adapt the dataset to various use cases. Experimental results demonstrate that our data pipeline and resulting dataset can facilitate the training of high-quality, zero-shot text-to-speech (TTS) models at high bandwidths.
JIS: A Speech Corpus of Japanese Idol Speakers with Various Speaking Styles
Authors: Yuto Kondo; Hirokazu Kameoka; Kou Tanaka; Takuhiro Kaneko
Venue: Interspeech 2025 (main)

Abstract: We construct Japanese Idol Speech Corpus (JIS) to advance research in speech generation AI, including text-to-speech synthesis (TTS) and voice conversion (VC). JIS will facilitate more rigorous evaluations of speaker similarity in TTS and VC systems since all speakers in JIS belong to a highly specific category: âyoung female live idols" in Japan, and each speaker is identified by a stage name, enabling researchers to recruit listeners familiar with these idols for listening experiments. With its unique speaker attributes, JIS will foster compelling research, including generating voices tailored to listener preferencesâan area not yet widely studied. JIS will be distributed free of charge to promote research in speech generation AI, with usage restricted to non-commercial, basic research. We describe the construction of JIS, provide an overview of Japanese live idol culture to support effective and ethical use of JIS, and offer a basic analysis to guide application of JIS.
FaVC: A Validated, Transcribed, Parallel Farsi Speech Dataset for Voice Conversion
Authors: Mina Serajian; Saeed Najafzadeh Rahaghi; Hadi Veisi; Saman Haratizadeh
Venue: Interspeech 2025 (main)

Abstract: In this paper, we created the first transcribed, parallel, balanced Farsi dataset (FaVC) that can be used for all tasks of speech synthesis, including both parallel and non-parallel voice conversion. FaVC is a balanced dataset that provides all phonemes of the Farsi language in all possible phonetic combinations. A metadata file is provided, including Farsi transcriptions, normal text, and IPA phoneme transcriptions of all audio files. The first Farsi voice conversion results using FaVC are also reported in this paper. The results indicate that by using FaVC, performance is as good as the chosen baseline methods for voice conversion in English. Moreover, objective evaluations of a voice conversion system that requires a parallel speech corpus can be performed using this dataset.
SawtArabi: A Benchmark Corpus for Arabic TTS. Standard, Dialectal and Code-Switching
Authors: Vasista Sai Lodagala; Lamya Alkanhal; Daniel Izham; Shivam Mehta; Shammur Absar Chowdhury; Aqeelah Makki; Hamdy S. Hussein; Gustav Eje Henter; Ahmed Ali
Venue: Interspeech 2025 (main)

Abstract: Curating Text-to-Speech (TTS) datasets is a strenuous task given the quality considerations. While it is hard to find high-quality TTS datasets in languages other than English, it is rare to come across code-switching (CS) datasets. As a part of this work, we curate a 4-hour Arabic-English TTS corpus consisting of code-switched Egyptian-English, monolingual Modern Standard Arabic (MSA), Egyptian, and English, all recorded by the same voice talent. We demonstrate the importance of vowelization and the need for better phonemization of Arabic text. To this effect, we present the modified espeak-ng phonemizer that handles various irregularities of espeak-ng over Arabic text. Upon training baseline TTS systems over this benchmark, we demonstrate its efficacy through extensive subjective evaluations.
The Text-to-speech in the Wild (TITW) Database
Authors: Jee-weon Jung; Wangyou Zhang; Soumi Maiti; Yihan Wu; Xin Wang; Ji-Hoon Kim; Yuta Matsunaga; Seyun Um; Jinchuan Tian; Hye-jin Shim; Nicholas Evans; Joon Son Chung; Shinnosuke Takamichi; Shinji Watanabe
Venue: Interspeech 2025 (main)

Abstract: Traditional Text-to-Speech (TTS) systems rely on studio-quality speech recorded in controlled settings. Recently, an effort known as "noisy-TTS training" has emerged, aiming to utilize in-the-wild data. However, the lack of dedicated datasets has been a significant limitation. We introduce the TTS In the Wild (TITW) dataset, which is publicly available, created through a fully automated pipeline applied to the VoxCeleb1 dataset. It comprises two training sets: TITW-Hard, derived from the transcription, segmentation, and selection of raw VoxCeleb1 data, and TITW-Easy, which incorporates additional enhancement and data selection based on DNSMOS. State-of-the-art TTS models achieve over 3.0 UTMOS score with TITW-Easy, while TITW-Hard remains difficult showing UTMOS below 2.8. Beyond TTS, TITWâs unique design, leveraging a automatic speaker recognition dataset, strengthens ethical efforts to counteract malicious use of TTS models by supporting tasks such as speech deepfake detection.
Towards Emotionally Consistent Text-Based Speech Editing: Introducing EmoCorrector and The ECD-TSE Dataset
Authors: Rui Liu; Pu Gao; Jiatian Xi; Berrak Sisman; Carlos Busso; Haizhou Li
Venue: Interspeech 2025 (main)

Abstract: Text-based speech editing (TSE) modifies speech using only text, eliminating re-recording. However, existing TSE methods, mainly focus on the content accuracy and acoustic consistency of synthetic speech segments, and often overlook the emotional shifts or inconsistency issues introduced by text changes. To address this issue, we propose EmoCorrector, a novel postcorrection scheme for TSE. EmoCorrector leverages Retrieval-Augmented Generation (RAG) by extracting the edited textâs emotional features, retrieving speech samples with matching emotions, and synthesizing speech that aligns with the desired emotion while preserving the speakerâs identity and quality. To support the training and evaluation of emotional consistency modeling in TSE, we pioneer the benchmarking Emotion Correction Dataset for TSE (ECD-TSE). The prominent aspect of ECD-TSE is its inclusion of <text, speech> paired data featuring diverse text variations and a range of emotional expressions. Subjective and objective experiments and comprehensive analysis on ECD-TSE confirm that EmoCorrector significantly enhances the expression of intended emotion while addressing emotion inconsistency limitations in current TSE methods.
ArVoice: A Multi-Speaker Dataset for Arabic Speech Synthesis
Authors: Hawau Toyin; Rufael Marew; Humaid Alblooshi; Samar M. Magdy; Hanan Aldarmaki
Venue: Interspeech 2025 (main)

Abstract: We introduce ArVoice, a multi-speaker Modern Standard Arabic (MSA) speech corpus with diacritized transcriptions, intended for multi-speaker speech synthesis, and can be useful for other tasks such as speech-based diacritic restoration, voice conversion, and deepfake detection. ArVoice comprises: (1) a new professionally recorded set from six voice talents with diverse demographics, (2) a modified subset of the Arabic Speech Corpus; and (3) high-quality synthetic speech from two commercial systems. The complete corpus consists of a total of 83.52 hours of speech across 11 voices; around 10 hours consist of human voices from 7 speakers. We train three open-source TTS and two voice conversion systems to illustrate the use cases of the dataset. The corpus is available for research use.
A Dataset for Automatic Assessment of TTS Quality in Spanish
Authors: Alejandro Sosa Welford; Leonardo Pepino
Venue: Interspeech 2025 (main)

Abstract: This work addresses the development of a database for the automatic assessment of textâtoâspeech (TTS) systems in Spanish, aiming to improve the accuracy of naturalness prediction models. The dataset consists of 4,326 audio samples from 52 different TTS systems and human voices and is, up to our knowledge, the first of its kind in Spanish. To label the audios, a subjective test was designed based on the ITUâT Rec. P.807 standard and completed by 92 participants. Furthermore, the utility of the collected dataset was validated by training automatic naturalness prediction systems. We explored two approaches: fine-tuning an existing model originally trained for English, and training small downstream networks on top of frozen self-supervised speech models. Our models achieve a mean absolute error of 0.8 on a five-point MOS scale. Further analysis demonstrates the quality and diversity of the developed dataset, and its potential to advance TTS research in Spanish.
Factors affecting the in-context learning abilities of LLMs for dialogue state tracking
Authors: Pradyoth Hegde; Santosh Kesiraju; JÃ¡n Å vec; Å imon SedlÃ¡Äek; Bolaji Yusuf; OldÅich Plchot; Deepak K T; Jan ÄernockÃ½
Venue: Interspeech 2025 (main)

Abstract: This study explores the application of in-context learning (ICL) to the dialogue state tracking (DST) problem and investigates the factors that influence its effectiveness. We use a sentence embedding based k-nearest neighbour method to retrieve the suitable demonstrations for ICL. The selected demonstrations, along with the test samples, are structured within a template as input to the LLM. We then conduct a systematic study to analyse the impact of factors related to demonstration selection and prompt context on DST performance. This work is conducted using the MultiWoZ2.4 dataset and focuses primarily on the OLMo-7B-instruct, Mistral-7B-Instruct-v0.3, and Llama3.2-3B-Instruct models. Our findings provide several useful insights on in-context learning abilities of LLMs for dialogue state tracking.
Spoken Question Answering for Visual Queries
Authors: Nimrod Shabtay; Zvi Kons; Avihu Dekel; Hagai Aronowitz; Ron Hoory; Assaf Arbelle
Venue: Interspeech 2025 (main)

Abstract: Question answering (QA) systems are designed to answer natural language questions. Visual QA (VQA) and Spoken QA (SQA) systems extend the textual QA system to accept visual and spoken input respectively. This work aims to create a system that enables user interaction through both speech and images. That is achieved through the fusion of text, speech, and images modalities to tackle the task of spoken VQA (SVQA). The resulting multi-modal model has textual, visual, and spoken inputs and can answer spoken questions on images. Training and evaluating SVQA models requires a dataset for all three modalities, but no such dataset currently exists. We address this problem by synthesizing VQA datasets using two zero-shot TTS models. Our initial findings indicate that a model trained only with synthesized speech nearly reaches the performance of the upper-bounding model trained on textual QAs. In addition, we show that the choice of the TTS model has a minor impact on accuracy.
Towards Human-like Multimodal Conversational Agent by Generating Engaging Speech
Authors: Taesoo Kim; Yongsik Jo; Hyunmin Song; Taehwan Kim
Venue: Interspeech 2025 (main)

Abstract: Human conversation involves language, speech, and visual cues, with each medium providing complementary information. For instance, speech conveys a vibe or tone not fully captured by text alone. While multimodal LLMs focus on generating text responses from diverse inputs, less attention has been paid to generating natural and engaging speech. We propose a human-like agent that generates speech responses based on conversation mood and responsive style information. To achieve this, we build a novel multi-sensory conversation dataset focused on speech to enable agents to generate natural speech. We then propose a multimodal LLM-based model for generating text responses and voice descriptions, which are used to generate speech covering para-lingual information. Experimental results demonstrate the effectiveness of utilizing both visual and audio modalities in conversation to generate engaging speech.
Chain-of-Thought Training for Open E2E Spoken Dialogue Systems
Authors: Siddhant Arora; Jinchuan Tian; Hayato Futami; Jee-weon Jung; Jiatong Shi; Yosuke Kashiwagi; Emiru Tsunoo; Shinji Watanabe
Venue: Interspeech 2025 (main)

Abstract: Unlike traditional cascaded pipelines, end-to-end (E2E) spoken dialogue systems maintain full differentiability and effectively capture non-phonemic information, making them well-suited for spoken dialogue modeling. However, existing E2E approaches often require large-scale training data and struggle to produce semantically coherent responses. In this work, we propose a simple yet effective strategy leveraging a chain-of-thought (CoT) formulation, ensuring that training on conversational data remains closely aligned with the multimodal language model (LM)'s pre-training on speech recognition (ASR), text-to-speech synthesis (TTS), and text LM tasks. Our results demonstrate that our approach is highly compute-efficient, enabling the successful training of E2E spoken dialogue systems on publicly available human-human conversation datasetsâeven with as little as 300 hours of data, such as Switchboard. To support future research, we will publicly release our models and training code.
SaD: A Scenario-Aware Discriminator for Speech Enhancement
Authors: Xihao Yuan; Siqi Liu; Yan Chen; Hang Zhou; Chang Liu; Hanting Chen; Jie Hu
Venue: Interspeech 2025 (main)

Abstract: Generative adversarial network-based models have shown remarkable performance in the field of speech enhancement. However, the current optimization strategies for these models predominantly focus on refining the architecture of the generator or enhancing the quality evaluation metrics of the discriminator. This approach often overlooks the rich contextual information inherent in diverse scenarios. In this paper, we propose a scenario-aware discriminator that captures scene-specific features and performs frequency-domain division, thereby enabling a more accurate quality assessment of the enhanced speech generated by the generator. We conducted comprehensive experiments on three representative models using two publicly available datasets. The results demonstrate that our method can effectively adapt to various generator architectures without altering their structure, thereby unlocking further performance gains in speech enhancement across different scenarios.
Listen through the Sound: Generative Speech Restoration Leveraging Acoustic Context Representation
Authors: Soo-Whan Chung; Min-Seok Choi
Venue: Interspeech 2025 (main)

Abstract: This paper introduces a novel approach to speech restoration by integrating a context-related conditioning strategy. Specifically, we employ the diffusion-based generative restoration model, UNIVERSE++, as a backbone to evaluate the effectiveness of contextual representations. We incorporate acoustic context embeddings extracted from the CLAP model, which capture the environmental attributes of input audio. Additionally, we propose an Acoustic Context (ACX) representation that refines CLAP embeddings to better handle various distortion factors and their intensity in speech signals. Unlike content-based approaches that rely on linguistic and speaker attributes, ACX provides contextual information that enables the restoration model to distinguish and mitigate distortions better. Experimental results indicate that context-aware conditioning improves both restoration performance and its stability across diverse distortion conditions, reducing variability compared to content-based methods.