GenAI4ELab/papercli-papers huggingface dataset:
Granary: Speech Recognition and Translation Dataset in 25 European Languages
Authors: Nithin Rao Koluguri; Monica Sekoyan; George Zelenfroynd; Sasha Meister; Shuoyang Ding; Sofia Kostandian; He Huang; Nikolay Karpov#; Jagadeesh Balam; Vitaly Lavrukhin; Yifan Peng; Sara Papi; Marco Gaido; Alessio Brutti; Boris Ginsburg
Venue: Interspeech 2025 (main)

Abstract: Multi-task and multilingual approaches benefit large models, yet speech processing for low-resource languages remains underexplored due to data scarcity. To address this, we present Granary, a large-scale collection of speech datasets for recognition and translation across 25 European languages. This is the first open-source effort at this scale for both transcription and translation. We enhance data quality using a pseudo-labeling pipeline with segmentation, two-pass inference, hallucination filtering, and punctuation restoration. We further generate translation pairs from pseudo-labeled transcriptions using EuroLLM, followed by a data filtration pipeline. Designed for efficiency, our pipeline processes vast amount of data within hours. We assess models trained on processed data by comparing their performance on previously curated datasets for both high- and low-resource languages. Our findings show that these models achieve similar performance using approx. 50% less data.
Collecting, Curating, and Annotating Good Quality Speech deepfake dataset for Famous Figures: Process and Challenges
Authors: Hashim Ali; Surya Subramani; Raksha Varahamurthy; Nithin Adupa; Lekha Bollinani; Hafiz Malik
Venue: Interspeech 2025 (main)

Abstract: Recent advances in speech synthesis have introduced unprecedented challenges in maintaining voice authenticity, particularly concerning public figures who are frequent targets of impersonation attacks. This paper presents a comprehensive methodology for collecting, curating, and generating synthetic speech data for political figures and a detailed analysis of challenges encountered. We introduce a systematic approach incorporating an automated pipeline for collecting high-quality bonafide speech samples, featuring transcription-based segmentation that significantly improves synthetic speech quality. We experimented with various synthesis approaches; from single-speaker to zero-shot synthesis, and documented the evolution of our methodology. The resulting dataset comprises bonafide and synthetic speech samples from ten public figures, demonstrating superior quality with a NISQA-TTS naturalness score of 3.69 and the highest human misclassification rate of 61.9%.
Quantifying and Reducing Speaker Heterogeneity within the Common Voice Corpus for Phonetic Analysis
Authors: Miao Zhang; Aref Farhadipour; Annie Baker; Jiachen Ma; Bogdan Pricop; Eleanor Chodroff
Venue: Interspeech 2025 (main)

Abstract: With its crosslinguistic and cross-speaker diversity, the Mozilla Common Voice Corpus (CV) has been a valuable resource for multilingual speech technology and holds tremendous potential for research in crosslinguistic phonetics and speech sciences. Properly accounting for speaker variation is, however, key to the theoretical and statistical bases of speech research. While CV provides a client ID as an approximation to a speaker ID, multiple speakers can contribute under the same ID. This study aims to quantify and reduce heterogeneity in the client ID for a better approximation of a true, though still anonymous speaker ID. Using ResNet-based voice embeddings, we obtained a similarity score among recordings with the same client ID, then implemented a speaker discrimination task to identify an optimal threshold for reducing perceived speaker heterogeneity. These results have major downstream applications for phonetic analysis and the development of speaker-based speech technology.
The Speech Accessibility Project: Best Practices for Collection and Curation of Disordered Speech
Authors: Chris Zwilling; Mark Hasegawa-Johnson; Heather Hodges; Lorraine Ramig; Adina Bradshaw; Clarion Mendes; Heejin Kim; Alexandria Barkhimer; Laura Mattie; Meg Dickinson; Shawnise Carter; Marie Moore Channell
Venue: Interspeech 2025 (main)

Abstract: Drawing lessons from the collection and curation of disordered speech samples from a large open-source database that includes dysarthria, apraxia, dysphonia, or otherwise atypical speechâthe Speech Accessibility Projectâthis paper provides best practices to guide the collection of diverse speech samples and enhance reproducibility. This paper describes data collection partnerships; the information technology architecture required to collect a massively distributed dataset spanning countries; the data collection process from the participant point of view; and the speech prompts and annotation process used for diverse speech. The open-source data set resulting from this set of best practices will provide a high-quality testbed for researchers around the world to train automatic speech recognition algorithms for disordered speech.
Challenges and practical guidelines for atypical speech data collection, annotation, usage and sharing: A multi-project perspective
Authors: Zhengjun Yue; Mara Barberis; Tanvina Patel; Judith Dineley; Willemijn Doedens; Lottie Stipdonk; YuanYuan Zhang; Elke de Witte; Erfan Loweimi; Hugo Van hamme; Djaina Satoer; Marina Ruiter; Laureano Moro Velazquez; Nicholas Cummins; Odette Scharenborg
Venue: Interspeech 2025 (main)

Abstract: Speech technologies have advanced significantly, yet they remain largely trained on typical speech, limiting their applicability to individuals with speech and language impairments. A key obstacle is the lack of well-annotated and representative atypical speech corpora. This paper conducts a multi-project survey and shares the first-hand experience on the challenges of collecting, annotating, using, and sharing atypical speech data. Experiences from seven research projects on collecting atypical speech data, involving both academic and clinical perspectives, are reported and potential issues are discussed. Furthermore, the paper provides practical guidelines that allow for standardisation and harmonisation of data collection practices, which are crucial to allow studies to be compared, replicated, and validated, which is essential for developing more inclusive and effective speech technologies.
Fifteen Years of Child-Centered Long-Form Recordings: Promises, Resources, and Remaining Challenges to Validity
Authors: Loann Peurey; Marvin Lavechin; Tarek Kunze; Manel Khentout; Lucas Gautheron; Emmanuel Dupoux; Alejandrina Cristia
Venue: Interspeech 2025 (main)

Abstract: Audio-recordings collected with a child-worn device are a fundamental tool in child language research. Long-form recordings collected over whole days promise to capture children's input and production with minimal observer bias, and therefore high validity. The sheer volume of resulting data necessitates automated analysis to extract relevant metrics for researchers and clinicians. This paper summarizes collective knowledge on this technique, providing entry points to existing resources. We also highlight various sources of error that threaten the accuracy of automated annotations and the interpretation of resulting metrics. To address this, we propose potential troubleshooting metrics to help users assess data quality. While a fully automated quality control system is not feasible, we outline practical strategies for researchers to improve data collection and contextualize their analyses.
Contextual Paralinguistic Data Creation for Multi-Modal Speech-LLM: Data Condensation and Spoken QA Generation
Authors: Qiongqiong Wang; Hardik B. Sailor; Tianchi Liu; Ai Ti Aw
Venue: Interspeech 2025 (main)

Abstract: Current speech-LLMs exhibit limited capability in contextual reasoning alongside paralinguistic understanding, primarily due to the lack of Question-Answer (QA) datasets that cover both aspects. We propose a novel framework for dataset generation from in-the-wild speech data, that integrates contextual reasoning with paralinguistic information. It consists of a pseudo paralinguistic label-based data condensation of in-the-wild speech and LLM-based Contextual Paralinguistic QA (CPQA) generation. The effectiveness is validated by a strong correlation in evaluations of the Qwen2-Audio-7B-Instruct model on a dataset created by our framework and human-generated CPQA dataset. The results also reveal the speech-LLM's limitations in handling empathetic reasoning tasks, highlighting the need for such datasets and more robust models. The proposed framework is first of its kind and has potential in training more robust speech-LLMs with paralinguistic reasoning capabilities.
Investigating Affect Mining Techniques for Annotation Sample Selection in the Creation of Finnish Affective Speech Corpus
Authors: Kalle Lahtinen; Einari Vaaras; Liisa Mustanoja; Okko RÃ¤sÃ¤nen
Venue: Interspeech 2025 (main)

Abstract: Study of affect in speech requires suitable data, as emotional expression and perception vary across languages. Until now, no corpus has existed for natural expression of affect in spontaneous Finnish, existing data being acted or from a very specific communicative setting. This paper presents the first such corpus, created by annotating 12,000 utterances for emotional arousal and valence, sampled from three large-scale Finnish speech corpora. To ensure diverse affective expression, sample selection was conducted with an affect mining approach combining acoustic, cross-linguistic speech emotion, and text sentiment features. We compare this method to random sampling in terms of annotation diversity, and conduct post-hoc analyses to identify sampling choices that would have maximized the diversity. As an outcome, the work introduces a spontaneous Finnish affective speech corpus and informs sampling strategies for affective speech corpus creation in other languages or domains.
Scalable Spontaneous Speech Dataset (SSSD): Crowdsourcing Data Collection to Promote Dialogue Research
Authors: Zaid Sheikh; Shuichiro Shimizu; Siddhant Arora; Jiatong Shi; Samuele Cornell; Xinjian Li; Shinji Watanabe
Venue: Interspeech 2025 (main)

Abstract: This paper introduces the Scalable Spontaneous Speech Dataset (SSSD) project, comprising 727 hours of spontaneous English conversations between two randomly-matched, anonymous participants on Amazon Mechanical Turk (MTurk) crowd-sourcing platform. The dataset features conversations averaging 25-30 minutes, covering a wide range of everyday topics. A key innovation of this work is our approach to maximizing the number of MTurk workers concurrently participating in our task, enabling more effective randomized matching and live two-person conversations. Data quality is ensured through a two-tiered task structure: a qualification round to select reliable workers, followed by the main recording sessions. We detail our methodology for collecting and recording spontaneous voice conversations, present analyses of the conversational content and speech quality of the dataset in comparison to other datasets, and discuss potential usage.
A Multimodal Chinese Dataset for Cross-lingual Sarcasm Detection
Authors: Xiyuan Gao; Bruce Xiao Wang; Meiling Zhang; Shuming Huang; Zhu Li; Shekhar Nayak; Matt Coler
Venue: Interspeech 2025 (main)

Abstract: Sarcasm is expressed through subtle cues like pitch, speech rate, and facial expressions, with patterns varying across languages, e.g., English speakers lower the pitch while Cantonese speakers raise it. While humans readily interpret these signals, computational models struggle, creating challenges for Human-Machine Interaction. Most multimodal sarcasm recognition research focuses on English and the lack of high-quality datasets for other languages hinders cross-lingual and cross-cultural studies. We introduce the Multimodal Chinese Sarcasm Dataset (MCSD), containing 10.57 hours of video. We propose a standardized annotation framework that captures annotator certainty to reflect the subjectivity of sarcasm, achieving a Fleissâ kappa of 0.74 (unweighted) and 0.79 (certainty-weighted). Validation of our dataset using SVM achieves a 76.64% F1-score in sarcasm detection. MCSD lays the foundation for robust cross-lingual sarcasm detection, contributing to advanced, human-centric systems.
Leveraging Large Language Models for Sarcastic Speech Annotation in Sarcasm Detection
Authors: Zhu Li; Yuqing Zhang; Xiyuan Gao; Shekhar Nayak; Matt Coler
Venue: Interspeech 2025 (main)

Abstract: Sarcasm fundamentally alters meaning through tone and context, yet detecting it in speech remains a challenge due to data scarcity. In addition, existing detection systems often rely on multimodal data, limiting their applicability in contexts where only speech is available. To address this, we propose an annotation pipeline that leverages large language models (LLMs) to generate a sarcasm dataset. Using a publicly available sarcasm-focused podcast, we employ GPT-4o and LLaMA 3 for initial sarcasm annotations, followed by human verification to resolve disagreements. We validate this approach by comparing annotation quality and detection performance on a publicly available sarcasm dataset using a collaborative gating architecture. Finally, we introduce PodSarc, a large-scale sarcastic speech dataset created through this pipeline. The detection model achieves a 73.63% F1 score, demonstrating the datasetâs potential as a benchmark for sarcasm detection research.
Analysis of the ABC Classification Backends for NIST SRE24
Authors: Sandro Cumani; Anna Silnova; Sara Barahona; Ladislav MoÅ¡ner; OldÅich Plchot; Johan Rohdin
Venue: Interspeech 2025 (main)

Abstract: We present an analysis of the classification backends of the ABC submission for the audio tracks of the NIST 2024 Speaker Recognition Evaluation (SRE24). Our analysis covers embedding pre-processing, classification and score-level normalization, calibration and fusion strategies adopted to cope with the source, language and duration mismatch challenges of SRE24. We show that Pairwise Support Vector Machines provide the best results, which can be further improved, for single frontends, through score-level fusion of additional classifiers. We also show that condition-aware score calibration can mitigate the effects of source mismatch, whereas score normalization methods proved ineffective. Finally, we show that generative calibration is able to achieve competitive results with respect to other approaches.
STCON NIST SRE24 System: Composite Speaker Recognition Solution for Challenging Scenarios
Authors: Stepan Malykh; Alexander Anikin; Nikita Khmelev; Anastasia Korenevskaya; Anastasia Zorkina; Sergey Novoselov; Vladislav Marchevskiy; Vladimir Volokhov; Andrey Shulipa; Alexander Kozlov; Alexander Melnikov; Vasiliy Galyuk; Timur Pekhovskiy
Venue: Interspeech 2025 (main)

Abstract: This paper addresses the real-world challenges of voice biometrics, specifically those highlighted by the National Institute of Standards and Technology Speaker Recognition (SR) Evaluation 2024 challenge (NIST SRE24). We present multi-module SR systems integrating speaker diarization, robust embedding extraction, and adaptive scoring to handle multi-speaker recordings, variable speech segment durations, and cross-channel/cross-lingual speaker recognition scenarios. We propose methodology for training robust speaker embedding extractors and explore a range of state-of-the-art SR neural network architectures. We focus on the combined optimization of traditionally separate components â diarization and speaker recognition â and present practical observations regarding this integrated approach. The presented findings allowed our systems to secure top leaderboard positions in the NIST SRE24.
Vo-Ve: An Explainable Voice-Vector for Speaker Identity Evaluation
Authors: Jaejun Lee; Kyogu Lee
Venue: Interspeech 2025 (main)

Abstract: In this paper, we propose Vo-Ve, a novel voice-vector embedding that captures speaker identity. Unlike conventional speaker embeddings, Vo-Ve is explainable, as it contains the probabilities of explicit voice attribute classes. Through extensive analysis, we demonstrate that Vo-Ve not only evaluates speaker similarity competitively with conventional techniques but also provides an interpretable explanation in terms of voice attributes. We strongly believe that Vo-Ve can enhance evaluation schemes across various speech tasks due to its high-level explainability.
Variability in performance across four generations of automatic speaker recognition systems
Authors: Lauren Harrington; Vincent Hughes; Philip Harrison; Paul Foulkes; Jessica Wormald; Finnian Kelly; David van der Vloed
Venue: Interspeech 2025 (main)

Abstract: The field of automatic speaker recognition (ASR) has seen a series of generational changes to speaker modelling approaches in the last 3 decades. Adoption of new approaches has mainly been driven by improvements observed in overall system-level performance metrics on common datasets. There is now considerable debate within the field around understanding why systems perform better for some speakers than others. In this study, we compare the performance of 4 generations of ASR systems with the same set of forensically-relevant test and calibration data. On a system- and individual speaker-level, we observe improvements from GMM-UBM to i-vector to x-vector but not for ECAPA-TDNN. We find that certain individuals remain difficult to recognise across all systems. Our findings show that both file- and speaker-level factors contribute to the performance of individual speakers and systems overall, which supports calls for more detailed exploration of system performance.
On the influence of language similarity in non-target speaker verification trials
Authors: Paul M. Reuter; Michael Jessen
Venue: Interspeech 2025 (main)

Abstract: In this paper, we investigate the influence of language similarity in cross-lingual non-target speaker verification trials using a state-of-the-art speaker verification system, ECAPA-TDNN, trained on multilingual and monolingual variants of the VoxCeleb dataset. Our analysis of the score distribution patterns on multilingual Globalphone and LDC CTS reveals a clustering effect in speaker comparisons involving a training language, whereby the choice of comparison language only minimally impacts scores. Conversely, we observe a language similarity effect in trials involving languages not included in the training set of the speaker verification system, with scores correlating with language similarity measured by a language classification system, especially when using multilingual training data.
The Sub-3Sec Problem: From Text-Independent to Text-Dependent Corpus
Authors: Ruichen Zuo; Kong Aik Lee; Zilong Huang; Man-Wai Mak
Venue: Interspeech 2025 (main)

Abstract: This paper introduces the sub-3sec problem in speaker verification, a short-duration task rarely explored. The issue arises from labor-intensive annotations and costly recordings for text-dependent speaker verification (TD-SV) corpora. To address this issue, we propose an automatic pipeline to extract short phrases from text-independent speaker verification (TI-SV) corpora. An ASR model identifies phrases and timestamps, with N-gram analysis ensuring phrases are common across speakers, enabling sufficient trials. Using this pipeline, we created Sub3Vox, a TD-SV corpus from VoxCeleb1, containing 3.7 million short utterances from 1,250 speakersâfar larger than existing TD-SV corpora. Results show that matching enrollment and test phrases in TD-SV reduces EER by up to 45.23%. Additionally, shortening test utterances causes significant TI-SV performance drops but only minor reduction for TD-SV, offering the first analysis of phrase length effects on sub-3-second performance.
ToxicTone: A Mandarin Audio Dataset Annotated for Toxicity and Toxic Utterance Tonality
Authors: Yu-Xiang Luo; Yi-Cheng Lin; Ming-To Chuang; Jia-Hung Chen; I-Ning Tsai; Pei Xing Kiew; Yueh-Hsuan Huang; Chien-Feng Liu; Yu-Chen Chen; Bo-Han Feng; Wenze Ren; Hung-yi Lee
Venue: Interspeech 2025 (main)

Abstract: Despite extensive research on toxic speech detection in text, a critical gap remains in handling spoken Mandarin audio. The lack of annotated datasets that capture the unique prosodic cues and culturally specific expressions in Mandarin leaves spoken toxicity underexplored. To address this, we introduce ToxicToneâthe largest public dataset of its kindâfeaturing detailed annotations that distinguish both forms of toxicity (e.g., profanity, bullying) and sources of toxicity (e.g., anger, sarcasm, dismissiveness). Our data, sourced from diverse real-world audio and organized into 13 topical categories, mirrors authentic communication scenarios. We also propose a multimodal detection framework that integrates acoustic, linguistic, and emotional features using state-of-the-art speech and emotion encoders. Extensive experiments show our approach outperforms text-only and baseline models, underscoring the essential role of speech-specific cues in revealing hidden toxic expressions.
ViToSA: Audio-Based Toxic Spans Detection on Vietnamese Speech Utterances
Authors: Huy Ba Do; Vy Le-Phuong Huynh; Luan Thanh Nguyen
Venue: Interspeech 2025 (main)

Abstract: Toxic speech on online platforms is a growing concern, impacting user experience and online safety. While text-based toxicity detection is well-studied, audio-based approaches remain underexplored, especially for low-resource languages like Vietnamese. This paper introduces ViToSA (Vietnamese Toxic Spans Audio), the first dataset for toxic spans detection in Vietnamese speech, comprising 11,000 audio samples (25 hours) with accurate human-annotated transcripts. We propose a pipeline that combines ASR and toxic spans detection for fine-grained identification of toxic content. Our experiments show that fine-tuning ASR models on ViToSA significantly reduces WER when transcribing toxic speech, while the text-based toxic spans detection (TSD) models outperform existing baselines. These findings establish a novel benchmark for Vietnamese audio-based toxic spans detection, paving the way for future research in speech content moderation.
Self-Supervised Models of Speech Processing for Haitian Creole
Authors: William N. Havard; Renauld Govain; Benjamin Lecouteux; Emmanuel Schang
Venue: Interspeech 2025 (main)

Abstract: We develop tailored speech processing models for Haitian Creole positioning it as a high-resource language in terms of SSL models of speech processing. We do so by pretraining monolingual Wav2Vec2-Base, Wav2Vec2-Large and Data2Vec-Audio-Base models from scratch, which are then finetuned on an Automatic Speech Recognition task. We compare the performance of these models with models finetuned from larger multilingual (XLSR-53, XLSR2-300m, MMS-1B) and monolingual French-based models (LeBenchmark 1 to 7K). Our results highlight the effectiveness of pretraining monolingual models from scratch, demonstrating that they can achieve performance comparable to or even surpassing larger models derived from large-scale multilingual pretraining. Our work provides essential resources for robust Haitian Creole speech recognition and offers pretrained models that can be adapted to other French-based Caribbean Creoles, opening new avenues for linguistic research and practical applications in the region.