GenAI4ELab/papercli-papers huggingface dataset:
Tracking /r/ Deletion: Forced Alignment of Pronunciation Variants and Sociophonetic Insights into Post-Obstruent Final /r/ in French
Authors: Anisia Popescu; Lori Lamel; Marc Evrard; Ioana Vasilescu
Venue: Interspeech 2025 (main)

Abstract: This paper investigates post-obstruent final /r/ deletion in French (e.g., livre [liv(K)], votre [vot(K)]) using two large corpora and phoneme-level forced alignment to analyze both pronounced and deleted /r/ variants. The study examines factors influencing /r/ deletion, including phonetic context, speaker profession, gender, age, and speaking style. Results show that right-phone context, age, and speaking style significantly affect /r/ deletion, with specific patterns also observed for the preceding phone and speaker profession. Gender has no effect. With over 390 speakers producing 14,167 individual /r/ tokens, the study provides robust insights into phonetic and social factors shaping final post-obstruent /r/ deletion. The combination of speech technology, NLP methods, and linguistic analysis of large-scale naturalistic data improves our understanding of this phenomenon, challenging assumptions about genderâs relevance in this phenomenon.
Agent-based modelling, sound change, and metaphony in Southern Italian varieties of Italo-Romance.
Authors: Lilian von Bressensdorf; Pia Greca; Jonathan Harrington
Venue: Interspeech 2025 (main)

Abstract: The study uses an agent-based computational model to test the hypothesis that contact between two dialects that are conservative and innovative as far as a sound change is concerned produces an asymmetric shift of the conservative speakers towards the innovative ones. The computational model, which is initialized with speech data from real speakers, is tested for the first time on a morpho-phonological sound change by which cues to inflectional morphology are being transferred from a suffix to a stem vowel in two Italo-Romance dialects of Southern Italy. The results based on quantifying the extent of diphthongization and the emergence of categorical contrasts marking morphological inflection provide some support for the proposed asymmetric shift in dialect contact. The analysis is more generally consistent with feedback models of sound change in which production is stochastically updated as a consequence of memorizing speech signals in speech perception.
Modeling Vowel System Typology Using Iterated Confusion Minimization
Authors: John McGahay
Venue: Interspeech 2025 (main)

Abstract: This work presents a unified theory of vowel system typology based on a novel algorithm called Iterated Confusion Minimization, which simulates the interaction of optimal listeners and speakers over time. Compatible with existing sound change models that involve confusion-reducing adjustments to phonetic distributions, simulations over random initial vowel systems resemble observed sound change (e.g., chain shifts) and most often result in typologically dominant systems with minimal confusion, suggesting a diachronic explanation for the typological dominance of perceptually optimal vowel systems. In contrast to past vowel dispersion models, Iterated Confusion Minimization easily accounts for interior vowels like /É/, allowing for prediction of the most common systems up to 11 vowels. Finally, variation in simulation output mirrors typological variation, with a tight correlation between system emergence rate and typological frequency up to 6 vowels that outperforms past baselines.
Investigating Glottal Stop Coda Loss During Sound Change of Checked Syllables Based on Speech-EGG Voice Offset Alignment
Authors: Bingliang Zhao; Xiyu Wu
Venue: Interspeech 2025 (main)

Abstract: The glottal stop coda in Wu Chinese checked syllables diachronically weakens into non-modal phonation on the preceding vowel and finally disappears. It's crucial to capture the initial stage of this sound change when the glottal stop coda is realized as an abrupt glottal closure. Aiming to clarify the age-related differences in the phonetic realizations of the glottal stop coda, we detected the presence of the abrupt glottal closure at voice offset using speech and EGG signals from Shengzhou Wu speakers. Results show that the voice offsets in checked syllables produced by most old speakers remain aligned, indicating the sound change has not been initiated. However, the glottal stop coda is no longer realized as abrupt glottal closure in the younger generation. This study directly measures the glottal stop coda loss through the physiological basis of articulation, potentially providing insights applicable to abrupt glottal closure detection for other linguistic purposes.
FlowTSE: Target Speaker Extraction with Flow Matching
Authors: Aviv Navon; Aviv Shamsian; Yael Segal-Feldman; Neta Glazer; Gil Hetz; Joseph Keshet
Venue: Interspeech 2025 (main)

Abstract: Target speaker extraction (TSE) aims to isolate a specific speaker's speech from a mixture using speaker enrollment as a reference. While most existing approaches are discriminative, recent generative methods for TSE achieve strong results. However, generative methods for TSE remain underexplored, with most existing approaches relying on complex pipelines and pretrained components, leading to computational overhead. In this work, we present FlowTSE, a simple yet effective TSE approach based on conditional flow matching. Our model receives an enrollment audio sample and a mixed speech signal, both represented as mel-spectrograms, with the objective of extracting the target speaker's clean speech. Furthermore, for tasks where phase reconstruction is crucial, we propose a novel vocoder conditioned on the complex STFT of the mixed signal, enabling improved phase estimation. Experimental results on standard TSE benchmarks show that FlowTSE matches or outperforms strong baselines.
MTSE: Multi-Target Speaker Extraction for Conversation Scenarios
Authors: Thomas Serre; Mathieu Fontaine; Eric Benhaim; Slim Essid
Venue: Interspeech 2025 (main)

Abstract: Target Speaker Extraction (TSE) aims to capture a desired voice among other interfering ones and/or background noise using a reference excerpt acquired during the enrollment phase. While useful in many applications, existing TSE systems cannot handle the scenario where several voices need to be enrolled and/or targeted. In this work, we address this new task, called multi-target speaker extraction (MTSE), which consists of extracting multiple target speakers in a mixture, possibly involving other interfering voices, using multiple speaker embeddings. Such models can thus be used by multiple users without the re-enrollment necessity. We propose a curriculum learning scheme to adapt well-known TSE models to the MTSE task. We prove its effectiveness and obtain, for the first time, successful MTSE results on meeting-type excerpts. We also present single-speaker TSE results with multiple enrolled speakers, proving the robustness and versatility of our solution.
Location-Aware Target Speaker Extraction for Hearing Aids
Authors: Daniel-JosÃ© Alcala Padilla; Nils L. Westhausen; Swati Vivekananthan; Bernd T. Meyer
Venue: Interspeech 2025 (main)

Abstract: Target speaker extraction (TSE) using deep learning offers potential benefits for hearing-impaired listeners. However, their implementation in hearing aids requires low-latency, low-complexity algorithms capable of real-time operation. Existing models that comply with these requirements use multi-channel input from binaural hearing aids to improve speech intelligibility but are limited to extracting speech from a single fixed direction. In this work, we utilize the direction of arrival (DOA) of the target speaker to extract speech at arbitrary angles in complex acoustic environments based on a deep-learning model. We introduce a novel DOA encoding method based on complex exponentials which is compared to one-hot (oh) encoding. We explore three low-complexity methods for integrating DOA information into the model. The evaluation using objective measures demonstrates that our extended model outperforms the baseline system with our novel encoding method achieving superior performance in 16 out of 21 cases.
ClearerVoice-Studio: Bridging Advanced Speech Processing Research and Practical Deployment
Authors: Shengkui Zhao; Zexu Pan; Bin Ma
Venue: Interspeech 2025 (main)

Abstract: This paper introduces ClearerVoice-Studio, an open-source, AI-powered speech processing toolkit designed to bridge cutting-edge research and practical application. Unlike broad platforms like SpeechBrain and ESPnet, ClearerVoice-Studio focuses on interconnected speech tasks of speech enhancement, separation, super-resolution, and multimodal target speaker extraction. A key advantage is its state-of-the-art pretrained models, including FRCRN (3M+ uses) and MossFormer (2.5M+ uses), optimized for real-world scenarios. It also offers model optimization tools, multi-format audio support, the SpeechScore evaluation toolkit, and user-friendly interfaces, catering to researchers, developers, and end-users. Its rapid adoption (2.8K GitHub stars, 200+ forks) highlights its academic and industrial impact. This paper details ClearerVoice-Studioâs capabilities, architectures, training strategies, benchmarks, community impact, and future plan.
Online AV-CrossNet: a Causal and Efficient Audiovisual System for Speech Enhancement and Target Speaker Extraction
Authors: Cheng Yu; Vahid Ahmadi Kalkhorani; Buye Xu; DeLiang Wang
Venue: Interspeech 2025 (main)

Abstract: This paper presents online AV-CrossNet, a computationally efficient audiovisual speech enhancement/extraction system capable of causal and real-time processing. We aim to improve the state-of-the-art AV-CrossNet by enabling causal, frame-by-frame processing. To achieve this, we incorporate causal layers and compression techniques, reduce model size, and employ only one-frame look-ahead, thereby substantially enhancing real-world applicability. Additionally, we analyze compression ratio in both audio and visual modules, providing valuable insights into audiovisual model compression. Experimental results demonstrate an inference latency of 4.73 ms, capable of real-time processing. Moreover, the system maintains competitive performance while reducing size by a factor of 10. These findings highlight the efficiency and effectiveness of the proposed system, offering a promising solution for real-time audiovisual speech enhancement and speaker extraction in acoustically adverse environments.
Steering Deep Non-Linear Spatially Selective Filters for Weakly Guided Extraction of Moving Speakers in Dynamic Scenarios
Authors: Jakob Kienegger; Timo Gerkmann
Venue: Interspeech 2025 (main)

Abstract: Recent speaker extraction methods using deep non-linear spatial filtering perform exceptionally well when the target direction is known and stationary. However, spatially dynamic scenarios are considerably more challenging due to time-varying spatial features and arising ambiguities, e.g. when moving speakers cross. While in a static scenario it may be easy for a user to point to the target's direction, manually tracking a moving speaker is impractical. Instead of relying on accurate time-dependent directional cues, which we refer to as strong guidance, in this paper we propose a weakly guided extraction method solely depending on the targetâs initial position to cope with spatial dynamic scenarios. By incorporating our own deep tracking algorithm and developing a joint training strategy on a synthetic dataset, we demonstrate the proficiency of our approach in resolving spatial ambiguities and even outperform a mismatched, but strongly guided extraction method.
RA-CLAP: Relation-Augmented Emotional Speaking Style Contrastive Language-Audio Pretraining For Speech Retrieval
Authors: Haoqin Sun; Jingguang Tian; Jiaming Zhou; Hui Wang; Jiabei He; Shiwan Zhao; Xiangyu Kong; Desheng Hu; Xinkang Xu; Xinhui Hu; Yong Qin
Venue: Interspeech 2025 (main)

Abstract: The Contrastive Language-Audio Pretraining (CLAP) model has demonstrated excellent performance in general audio description-related tasks, such as audio retrieval. However, in the emerging field of emotional speaking style description (ESSD), cross-modal contrastive pretraining remains largely unexplored. In this paper, we propose a novel speech retrieval task called emotional speaking style retrieval (ESSR), and ESS-CLAP, an emotional speaking style CLAP model tailored for learning relationship between speech and natural language descriptions. In addition, we further propose relation-augmented CLAP (RA-CLAP) to address the limitation of traditional methods that assume a strict binary relationship between caption and audio. The model leverages self-distillation to learn the potential local matching relationships between speech and descriptions, thereby enhancing generalization ability. The experimental results validate the effectiveness of RA-CLAP, providing valuable reference in ESSD.
EmotionRankCLAP: Bridging Natural Language Speaking Styles and Ordinal Speech Emotion via Rank-N-Contrast
Authors: Shreeram Suresh Chandra; Lucas Goncalves; Junchen Lu; Carlos Busso; Berrak Sisman
Venue: Interspeech 2025 (main)

Abstract: Current emotion-based contrastive language-audio pretraining (CLAP) methods typically learn by naÃ¯vely aligning audio samples with corresponding text prompts. Consequently, this approach fails to capture the ordinal nature of emotions, hindering inter-emotion understanding and often resulting in a wide modality gap between the audio and text embeddings due to insufficient alignment. To handle these drawbacks, we introduce EmotionRankCLAP, a supervised contrastive learning approach that uses dimensional attributes of emotional speech and natural language prompts to jointly capture fine-grained emotion variations and improve cross-modal alignment. Our approach utilizes a Rank-N-Contrast objective to learn ordered relationships by contrasting samples based on their rankings in the valence-arousal space. EmotionRankCLAP outperforms existing emotion-CLAP methods in modeling emotion ordinality across modalities, measured via a cross-modal retrieval task.1
Modality-Agnostic Multimodal Emotion Recognition using a Contrastive Masked Autoencoder
Authors: Georgios Chochlakis; Turab Iqbal; Woo Hyun Kang; Zhaocheng Huang
Venue: Interspeech 2025 (main)

Abstract: Multimodal deep learning methods have greatly accelerated research in emotion recognition and have become the state of the art. However, in many scenarios, not all modalities are readily available, leading to either failure of traditional algorithms or the need for multiple models. In this work, we advance the state of the art in emotion recognition by proposing a unified, modality-agnostic transformer-based model that is inherently robust to missing modalities. To better exploit the multimodality of the data, we propose to use contrastive learning for modality alignment and masked autoencoding for multimodal reconstruction. Experimental results on the MSP-Podcast corpus show that our unified model achieves state-of-the-art performance, and improves both unimodal and multimodal baselines by 1-5% relative in respective evaluation metrics with the capability to handle missing modalities for two emotion recognition tasks in a more compact model.
Multi-Modal Multi-Task Affective States Recognition Based on Label Encoder Fusion
Authors: Maxim Markitantov; Elena Ryumina; Heysem Kaya; Alexey Karpov
Venue: Interspeech 2025 (main)

Abstract: Despite recent advances in multi-modal approaches, recognizing the full range of human affective states, including emotions and sentiments, remains challenging due to complex interactions between different modalities and the hierarchical nature of affective states. This work presents a novel approach for multi-modal multi-task emotion and sentiment recognition that integrates audio, video, and text data. We introduce a Label Encoder Fusion Strategy, which produces and processes uni-modal emotion and sentiment predictions, which are used alongside modality-specific features during the fusion process to provide additional contextual information. We conduct elaborate multi-corpus experiments on the RAMAS, MELD, and CMU-MOSEI corpora. The proposed approach achieves state-of-the-art performance in both affective tasks. On MELD, we achieve a macro F1 (MF) of 40.9% and 67.02% for emotion and sentiment recognition. On CMU-MOSEI, the mean MF is 62.30% and MF is 62.00% for the same tasks.
Triadic Multi-party Voice Activity Projection for Turn-taking in Spoken Dialogue Systems
Authors: Mikey Elmers; Koji Inoue; Divesh Lala; Tatsuya Kawahara
Venue: Interspeech 2025 (main)

Abstract: Turn-taking is a fundamental component of spoken dialogue, however conventional studies mostly involve dyadic settings. This work focuses on applying voice activity projection (VAP) to predict upcoming turn-taking in triadic multi-party scenarios. The goal of VAP models is to predict the future voice activity for each speaker utilizing only acoustic data. This is the first study to extend VAP into triadic conversation. We trained multiple models on a Japanese triadic dataset where participants discussed a variety of topics. We found that the VAP trained on triadic conversation outperformed the baseline for all models but that the type of conversation affected the accuracy. This study establishes that VAP can be used for turn-taking in triadic dialogue scenarios. Future work will incorporate this triadic VAP turn-taking model into spoken dialogue systems.
Continuous prediction of backchannel timing for human-robot interaction
Authors: Michael Paierl; Martin HagmÃ¼ller; Barbara Schuppler
Venue: Interspeech 2025 (main)

Abstract: This paper explores the continuous prediction of backchannel timing in conversational speech, with the aim to make turn-taking in human-robot interaction more natural. To assure real-time prediction, we present regression-based models based exclusively on acoustic features that can be extracted continuously from the user's speech. Comparing different machine learning models, we found lightGBM models to perform best with respect to accuracy (mean absolute error: approx. 130 ms) and efficiency, while meeting the real-time requirement. Our analysis of feature importances revealed that speaking duration, intensity and fundamental frequency are among the most important predictors of backchannel timing, when extracted in the window from 275-875 ms before a backchannel in the interlocutor's turn. Given the strong predictive performance of our models, this work provides a foundation for implementing more natural and responsive conversational agents.
Impact of Background Noise on Turn-Taking Dynamics in Triadic Conversations
Authors: Valeska Slomianka; Tobias May; Torsten Dau
Venue: Interspeech 2025 (main)

Abstract: Conversations require individuals to comprehend speech while preparing responses, with demands increasing significantly in noise or with more people involved. Here, we explore the impact of noise on turn-taking dynamics in triadic conversations. Ten groups of three young, normal-hearing participants engaged in conversations at two levels of 8-talker babble noise, with each group holding three conversations under noisy conditions and three in quiet. We extracted objective measures of turn-taking dynamics from the recorded speech and assessed subjective ratings of conversational difficulty. Conversations in noise were shorter, with reduced distance between participants, fewer overlaps-within, and fewer, more delayed, and variable floor transfers. The conversational dynamics in noise correlated with the subjective ratings. Overall, changes in turn-taking dynamics in triadic conversations in noise resemble those in dyads, reflecting alterations in conversational flow and engagement.
Multimodal Dynamics of Hand Gestures and Pauses in Multiparty Interactions
Authors: Delphine Charuau; Naomi Harte
Venue: Interspeech 2025 (main)

Abstract: This study examines multimodal patterns linking hand gestures and pauses in multiparty interactions, distinguishing between within-speaker pauses and between-speaker pauses. Using the MULTISIMO corpus, which includes annotated audio-visual recordings of collaborative dialogues, we analysed, for each category of gesture, their distribution, pause duration, and the timing of gesture onset and offset relative to the pause. Results showed distinct gestural timing patterns involving within-speaker and between-speaker pauses. Self-adaptors were associated with longer pauses, possibly reflecting increased cognitive demands or speech planning. While syntactic position influenced pause duration, with shorter pauses at utterance endings, it did not impact gesture onset and offset. Additionally, most pauses containing gestures occurred within utterances, regardless of gesture type.
TinyClick: Single-Turn Agent for Empowering GUI Automation
Authors: Pawel Pawlowski; Krystian Zawistowski; Wojciech Lapacz; Adam Wiacek; Marcin Skorupa; Sebastien Postansque; Jakub Hoscilowicz
Venue: Interspeech 2025 (main)

Abstract: We present an UI agent for user interface (UI) interaction tasks, using Vision-Language Model Florence-2-Base. The agent's primary task is identifying the screen coordinates of the UI element corresponding to the user's command. It demonstrates very strong performance on Screenspot and OmniAct annotations, while maintaining a very small size of 0.27B parameters and minimal latency. Moreover, training needs small compute budget of 56 GPU-hours (worth about $40). Relevant improvement comes from vision-specific multi-task training and MLLM-based data augmentation. We hope that decreased needs for expensive compute resources and manually annotated data will allow to facilitate more inclusive and sustainable research of UI agents.
Improving User Impression of Spoken Dialogue Systems by Controlling Para-linguistic Expression Based on Intimacy
Authors: Shoki Kawanishi; Akinori Ito; Yuya Chiba; Takashi Nose
Venue: Interspeech 2025 (main)

Abstract: The advent of large language models (LLMs) has improved the naturalness of responses in dialogue systems; however, conversations with these systems still differ significantly from those between humans. For instance, humans adjust their manner of speaking based on their relationship with the conversation partner, whereas dialogue systems respond uniformly, even after repeated interactions. By producing responses that consider the progression of relationships, it may be possible to develop dialogue systems that are more appealing to users. Previous studies attempting to achieve such systems have primarily focused on the linguistic expressions of responses,while paralinguistic expressions have been largely overlooked. In this paper, we propose a dialogue system that adapts both linguistic and para-linguistic expressions as the number of interactions increases. We also evaluate its effectiveness through dialogue experiments.
Dialogue Response Prefetching Based on Semantic Similarity and Prediction Confidence of Language Model
Authors: Kiyotada Mori; Seiya Kawano; Angel GarcÃ­a Contreras; Koichiro Yoshino
Venue: Interspeech 2025 (main)

Abstract: Prefetching of dialogue responses has been investigated to reduce user-perceived latency (UPL), which refers to the user's waiting time before receiving the system's response, in spoken dialogue systems. To reduce the UPL, it is necessary to predict complete user utterances before the end of the user's speech, typically by language models, to prepare prefetched dialogue responses. In this study, we proposed a prediction confidence model (PCM) that determines whether prefetching is possible or not by estimating the semantic similarity between the predicted complete user utterance and the complete user utterance. We evaluated our PCM based on the differences between the predicted complete user utterance and the complete user utterance.