GenAI4ELab/papercli-papers huggingface dataset:
Decoding Listener's Identity: Person Identification from EEG Signals Using a Lightweight Spiking Transformer
Authors: Zheyuan Lin; Siqi Cai; Haizhou Li
Venue: Interspeech 2025 (main)

Abstract: EEG-based person identification enables applications in security, personalized brain-computer interfaces (BCIs), and cognitive monitoring. However, existing techniques often rely on deep learning architectures at high computational cost, limiting their scope of applications. In this study, we propose a novel EEG person identification approach using spiking neural networks (SNNs) with a lightweight spiking transformer for efficiency and effectiveness. The proposed SNN model is capable of handling the temporal complexities inherent in EEG signals. On the EEG-Music Emotion Recognition Challenge dataset, the proposed model achieves 100% classification accuracy with less than 10% energy consumption of traditional deep neural networks. This study offers a promising direction for energy-efficient and high-performance BCIs.
Recreating Neural Activity During Speech Production with Language and Speech Model Embeddings
Authors: Owais Mujtaba Khanday; Pablo RodrÃ­guez San Esteban; Zubair Ahmad Lone; Marc Ouellet; Jose A. Gonzalez-Lopez
Venue: Interspeech 2025 (main)

Abstract: Understanding how neural activity encodes speech and language production is a fundamental challenge in neuroscience and artificial intelligence. This study investigates whether embeddings from large-scale, self-supervised language and speech models can effectively reconstruct high-gamma neural activity characteristics, key indicators of cortical processing, recorded during speech production. We use pre-trained embeddings from deep learning models on linguistic and acoustic data to map high-level speech features onto high-gamma signals. We analyze the extent to which these embeddings preserve the spatio-temporal dynamics of brain activity. Reconstructed neural signals are evaluated against high-gamma ground-truth activity using correlation metrics and signal reconstruction quality assessments. The results indicate High-gamma activity was effectively reconstructed using language and speech model embeddings, yielding Pearson correlation coefficients of 0.79-0.99 across all participants.
Towards Sentence Level Imagined Speech Generation from EEG signals
Authors: Sparsh Rastogi; Harsh Dadwal; Khushboo Modi; Jatin Bedi; Jasmeet Singh
Venue: Interspeech 2025 (main)

Abstract: Brain-Computer Interfaces (BCIs) have emerged as alternative means of communication for individuals with speech & motor impairments. These systems enable patients to express themselves without any articulation, by decoding speech from neural activity. However, most of the existing studies rely on invasive surgical procedures, with limited studies using non-invasive signals for phoneme or word level classification, thus covering a short vocabulary. To the best of our knowledge, this study presents the first demonstration of a framework for sentence level imagined speech synthesis from non-invasive electroencephalography (EEG) signals. Our model uses an Efficient-Net based masked auto-encoder approach for learning feature embeddings from EEG signals which are then used for fine-tuning BERT for next token generation. For this study, Large Spanish Speech EEG Dataset has been used with a mixed subject approach for both training & evaluation purposes, resulting into a 48.92% accuracy.
Word-Level Error Analysis in Decoding Systems: From Speech Recognition to Brain-Computer Interfaces
Authors: Jingya Huang; Aashish N. Patel; Sowmya Manojna Narasimha; Gal Mishne; Vikash Gilja
Venue: Interspeech 2025 (main)

Abstract: Brain-to-text (BTT) systems that decode attempted speech from neural activity have achieved 4.2% word error rate (WER). These systems demonstrate potential for daily use similar to automatic speech recognition (ASR) systems, enabling communication for individuals with profound speech loss. To examine fine-grained error patterns in both BTT and ASR, we introduce a refined alignment algorithm to detect word edits, along with four word-level metrics to assess exact correctness and semantic distance of these edits. Analyzing errors by word frequency reveals a significant performance disparity among frequent, infrequent, and rare words across all models. Although transformer-based architectures and self-supervised pre-training achieve lower error rates, the gap between frequent and infrequent words remains substantial. Our analysis indicates that misclassifying infrequent words incurs higher semantic costs, suggesting that addressing this word-level performance gap could enhance overall system usability across ASR and BTT. Our implementation is available.
NeuroSpex+: Dual-Task Training of Neuro-Guided Speaker Extraction with Speech Envelope and Waveform
Authors: Dashanka Da Silva; Siqi Cai; Saurav Pahuja; Tanja Schultz; Haizhou Li
Venue: Interspeech 2025 (main)

Abstract: Neuro-guided speaker extraction, i.e. NeuroSpex, aims to isolate the speech signal a listener is attending to in a multi-talker environment using reference cues derived from cortical activity, such as electroencephalography (EEG). Despite remarkable progress, there remains untapped potential. In this study, we propose NeuroSpex+, a novel neuro-guided speaker extraction model that integrates an additional task of reconstructing the target speech envelope. By simultaneously optimizing the model for both the target speech envelope and speech waveform, NeuroSpex+ reinforces the mask generation for speaker extraction. Experimental results demonstrate that the proposed model significantly outperforms baselines, improving overall signal quality.
DiffMV-ETS: Diffusion-based Multi-Voice Electromyography-to-Speech Conversion using Speaker-Independent Speech Training Targets
Authors: Kevin Scheck; Tom Dombeck; Zhao Ren; Peter Wu; Michael Wand; Tanja Schultz
Venue: Interspeech 2025 (main)

Abstract: Electromyography (EMG) signals have been investigated for novel voice prostheses to enable speech communication with silent articulation. In this work, we propose DiffMV-ETS, a multi-voice, diffusion-based EMG-to-speech system that converts EMG signals to speech in selectable voices. We evaluate it for scenarios where no speech of the speaker wearing EMG sensors is used for training. For this purpose, we introduce EMG-VCTK, a dataset containing EMG and audio recordings of sentences from the Voice Conversion Tool Kit corpus. We compare EMG models trained with audio of the same speaker, of auxiliary speakers, and of text-to-speech systems. Experiments indicate that models retain their intelligibility and naturalness when trained with synthetic speech. DiffMV-ETS enhances the speech naturalness and similarity to unseen voices. To the best of our knowledge, this is the first work to train multi-voice EMG-to-speech systems with speaker-independent targets.
Conformer-based Ultrasound-to-Speech Conversion
Authors: Ibrahim Ibrahimov; Csaba ZainkÃ³; GÃ¡bor Gosztolya
Venue: Interspeech 2025 (main)

Abstract: Deep neural networks have shown promising potential for ultrasound-to-speech conversion task towards Silent Speech Interfaces. In this work, we applied two Conformer-based DNN architectures (Base and one with bi-LSTM) for this task. Speaker-specific models were trained on the data of four speakers from the Ultrasuite-Tal80 dataset, while the generated mel spectrograms were synthesized to audio waveform using a HiFi-GAN vocoder. Compared to a standard 2D-CNN baseline, objective measurements (MSE and mel cepstral distortion) showed no statistically significant improvement for either model. However, a MUSHRA listening test revealed that Conformer with bi-LSTM provided better perceptual quality, while Conformer Base matched the performance of the baseline along with a 3Ã faster training time due to its simpler architecture. These findings suggest that Conformer-based models, especially the Conformer with bi-LSTM, offer a promising alternative to CNNs for ultrasound-to-speech conversion.
Training Articulatory Inversion Models for Interspeaker Consistency
Authors: Charles McGhee; Mark J.F. Gales; Kate M. Knill
Venue: Interspeech 2025 (main)

Abstract: Acoustic-to-Articulatory Inversion (AAI) attempts to model the inverse mapping from speech to articulation. Exact articulatory prediction from speech alone may be impossible, as speakers can choose different forms of articulation seemingly without reference to their vocal tract structure. However, once a speaker has selected an articulatory form, their productions vary minimally. Recent works in AAI have proposed adapting Self-Supervised Learning (SSL) models to single-speaker datasets, claiming that these single-speaker models provide a universal articulatory template. In this paper, we investigate whether SSL-adapted models trained on single and multi-speaker data produce articulatory targets which are consistent across speaker identities for English and Russian. We do this through the use of a novel evaluation method which extracts articulatory targets using minimal pair sets. We also present a training method which can improve interspeaker consistency using only speech data.
Enhancing Acoustic-to-Articulatory Inversion with Multi-Target Pretraining for Low-Resource Settings
Authors: Jesuraj Bandekar; Prasanta Kumar Ghosh
Venue: Interspeech 2025 (main)

Abstract: Acoustic-to-Articulatory Inversion (AAI) estimates vocal tract articulator movements from speech, benefiting tasks like ASR, speech synthesis, and speaker verification. While deep learning-based methods (CNNs, RNNs, Transformers) have advanced AAI, recent studies show that Self-Supervised Learning (SSL) features further enhance performance, particularly in low-resource settings. However, SSL feature extractors introduce inference latency and computational overhead. To address this, we propose a novel pretraining method leveraging three target representations - Phoneme Labels, Articulatory Feature Labels, and Critical-articulator Labels - eliminating the need for an SSL extractor during inference. We evaluate our approach against both baseline and SSL-based models across various data conditions. Results demonstrate that our method consistently improves AAI performance, particularly in low-resource scenarios, while significantly reducing inference costs without sacrificing accuracy.
Articulatory Vowel Distinctiveness in Spanish
Authors: Kristin Teplansky; Emily Rangel; Mimi LaValley; Jinuk Kwon; Beiming Cao; Jun Wang
Venue: Interspeech 2025 (main)

Abstract: This study investigated Spanish articulatory patterns in bilingual speakers through a comparative analysis with English, focusing on isolated vowels while controlling for vocal tract differences. A recently developed, wearable electromagnetic articulography device was used to track tongue and lip motion. The y and z-coordinates were used to classify 5 Spanish vowels. The vowel classification results showed medium to high accuracy. While small positional differences indicated distinct articulatory configurations, the overall size of the vowel articulatory distinctiveness space was comparable between the two languages. The similarly high classification accuracy and similar vowel distribution patterns suggest the feasibility of training Silent Speech Interfaces in both Spanish and English for bilingual speakers. To our knowledge, this is the first study on vowel kinematic distinctiveness in Spanish.
EEG-based Speech Decoding Based on Multi-mode Joint Modeling
Authors: Peiran Li; Fei Chen; Xixin Wu
Venue: Interspeech 2025 (main)

Abstract: Electroencephalography (EEG)-based speech decoding enables the development of non-invasive speech brain-computer interfaces (BCIs) for restoring communication of individuals with speech impairments. Previous work achieves much better performance in decoding spoken and intended speech from EEG signals, with imagined speech decoding lagging far behind. This paper proposes a novel framework to train a unified multi-mode decoding model for EEG signals of imagined, intended and spoken speech modes using a dynamic masking mechanism. Our multi-mode model achieves significantly better four-vowel decoding accuracies than baselines (34.95% vs. 29.18% for imagined speech). Training a single-mode model with a subset of EEG channels selected according to a multi-mode model as inputs provides superior performance than training a single-mode model from all channels. The accuracy improvements and channel selection capability demonstrate the effectiveness of the proposed joint modeling framework.
A Silent Speech Decoding System from EEG and EMG with Heterogenous Electrode Configurations
Authors: Masakazu Inoue; Motoshige Sato; Kenichi Tomeoka; Nathania Nah; Eri Hatakeyama; Kai Arulkumaran; Ilya Horiguchi; Shuntaro Sasai
Venue: Interspeech 2025 (main)

Abstract: Silent speech decoding, which performs unvocalized human speech recognition from electroencephalography/electromyography (EEG/EMG), increases accessibility for speech-impaired humans. However, data collection is difficult and performed using varying experimental setups, making it nontrivial to collect a large, homogeneous dataset. In this study we introduce neural networks that can handle EEG/EMG with heterogeneous electrode placements and show strong performance in silent speech decoding via multi-task training on large-scale EEG/EMG datasets. We achieve improved word classification accuracy in both healthy participants (95.3%), and a speech-impaired patient (54.5%), substantially outperforming models trained on single-subject data (70.1% and 13.2%). Moreover, our models also show gains in cross-language calibration performance. This increase in accuracy suggests the feasibility of developing practical silent speech decoding systems, particularly for speech-impaired patients.
NAM-to-Speech Conversion with Multitask-Enhanced Autoregressive Models
Authors: Neil Shah; Shirish Karande; Vineet Gandhi
Venue: Interspeech 2025 (main)

Abstract: We propose an alignment-free, end-to-end Non-Audible Murmur (NAM)-to-Speech conversion model. Existing methods rely on large NAM-text pairs per speaker to generate high-quality alignments for training non-autoregressive models. However, alignment quality deteriorates when trained on multi-speaker data, limiting their ability to generalize and effectively utilize the available training data. To address this, we introduce a streamlined autoregressive approach that eliminates the need for explicit alignment learning. By leveraging multi-speaker samples, synthetic training pairs, and multitask character recognition training, our method reduces the word error rate (WER) by 59.19% compared to the state-of-the-art (SOTA) on two public datasets. We demonstrate the modelâs zero-shot capability and validate the effectiveness of multitask training through ablation studies.
RESOUND: Speech Reconstruction from Silent Videos via Acoustic-Semantic Decomposed Modeling
Authors: Long-Khanh Pham; Thanh V. T. Tran; Minh-Tan Pham; Van Nguyen
Venue: Interspeech 2025 (main)

Abstract: Lip-to-speech (L2S) synthesis, which reconstructs speech from visual cues, faces challenges in accuracy and naturalness due to limited supervision in capturing linguistic content, accents, and prosody. In this paper, we propose RESOUND, a novel L2S system that generates intelligible and expressive speech from silent talking face videos. Leveraging source-filter theory, our method involves two components: an acoustic path to predict prosody and a semantic path to extract linguistic features. This separation simplifies learning, allowing independent optimization of each representation. Additionally, we enhance performance by integrating speech units, a proven unsupervised speech representation technique, into waveform generation alongside mel-spectrograms. This allows RESOUND to synthesize prosodic speech while preserving content and speaker identity. Experiments conducted on two standard L2S benchmarks confirm the effectiveness of the proposed method across various metrics.
Layer-Wise Decision Fusion for Fake Audio Detection Using XLS-R
Authors: Yixuan Xiao; Ngoc Thang Vu
Venue: Interspeech 2025 (main)

Abstract: Recent fake audio detection methods often leverage large speech models to achieve robust speech representations.  These models are typically very deep, providing multiple layer-wise representations.  However, current works often rely solely on single layer representation or feature fusion to extract one utterance-level representation for decision making.  These methods risk underutilizing rich information from multiple layers and might induce feature collapse.  We propose a novel layer-wise decision fusion method that applies fusion after per-layer decision making and achieves the best cross-dataset performance on In-the-Wild dataset (EER 6.90%) compared to other strong baselines. Our model design also makes the model more transparent, allowing us to conduct detailed analysis to reveal the underlying mechanism of decision making.
SynHate: Detecting Hate Speech in Synthetic Deepfake Audio
Authors: Rishabh Ranjan; Kishan Pipariya; Mayank Vatsa; Richa Singh
Venue: Interspeech 2025 (main)

Abstract: The rise of deepfake audio and hate speech, powered by advanced text-to-speech, threatens online safety. We present SynHate, the first multilingual dataset for detecting hate speech in synthetic audio, spanning 37 languages. SynHate uses a novel four-class scheme: Real-normal, Real-hate, Fake-normal, and Fake-hate. Built from MuTox and ADIMA datasets, it captures diverse hate speech patterns globally and in India. We evaluate five leading self-supervised models (Whisper-small/medium, XLS-R, AST, mHuBERT), finding notable performance differences by language, with Whisper-small performing best overall. Cross-dataset generalization remains a challenge. By releasing SynHate and baseline code, we aim to advance robust, culturally sensitive, and multilingual solutions against synthetic hate speech. The dataset is available.
Can Emotion Fool Anti-spoofing?
Authors: Aurosweta Mahapatra; Ismail R. Ulgen; Abinay Reddy Naini; Carlos Busso; Berrak Sisman
Venue: Interspeech 2025 (main)

Abstract: Traditional anti-spoofing focuses on models and datasets built on synthetic speech with mostly neutral state, neglecting diverse emotional variations. As a result, their robustness against high-quality, emotionally expressive synthetic speech is uncertain. We address this by introducing EmoSpoof-TTS, a corpus of emotional text-to-speech samples. Our analysis shows existing anti-spoofing models struggle with emotional synthetic speech, exposing risks of emotion-targeted attacks. Even trained on emotional data, the models underperform due to limited focus on emotional aspect and show performance disparities across emotions. This highlights the need for emotion-focused anti-spoofing paradigm in both dataset and methodology. We propose GEM, a gated ensemble of emotion-specialized models with a speech emotion recognition gating network. GEM performs effectively across all emotions and neutral state, improving defenses against spoofing attacks. We release the EmoSpoof-TTS Dataset.
Pushing the Performance of Synthetic Speech Detection with Kolmogorov-Arnold Networks and Self-Supervised Learning Models
Authors: Tuan Dat Phuong; Long-Vu Hoang; Huy Dat Tran
Venue: Interspeech 2025 (main)

Abstract: Recent advancements in speech synthesis technologies have led to increasingly advanced spoofing attacks, posing significant challenges for automatic speaker verification systems. While systems based on self-supervised learning (SSL) models, particularly the XLSR-Conformer model, have demonstrated remarkable performance in synthetic speech detection, there remains room for architectural improvements. In this paper, we propose a novel approach that replaces the traditional Multi-Layer Perceptron in the XLSR-Conformer model with a Kolmogorov-Arnold Network (KAN), a novel architecture based on the Kolmogorov-Arnold representation theorem. Our results on ASVspoof2021 demonstrate that integrating KAN into the SSL-based models can improve the performance by 60.55% relatively on LA and DF sets, further achieving 0.70% EER on the 21LA set. These findings suggest that incorporating KAN into SSL-based models is a promising direction for advances in synthetic speech detection.
Amplifying Artifacts with Speech Enhancement in Voice Anti-spoofing
Authors: Thanapat Trachu; Thanathai Lertpetchpun; Ekapol Chuangsuwanich
Venue: Interspeech 2025 (main)

Abstract: Spoofed utterances always contain artifacts introduced by generative models. While several countermeasures have been proposed to detect spoofed utterances, most primarily focus on architectural improvements. In this work, we investigate how artifacts remain hidden in spoofed speech and how to enhance their presence. We propose a model-agnostic pipeline that amplifies artifacts using speech enhancement and various types of noise. Our approach consists of three key steps: noise addition, noise extraction, and noise amplification. First, we introduce noise into the raw speech. Then, we apply speech enhancement to extract the entangled noise and artifacts. Finally, we amplify these extracted features. Moreover, our pipeline is compatible with different speech enhancement models and countermeasure architectures. Our method improves spoof detection performance by up to 44.44% on ASVspoof2019 and 26.34% on ASVspoof2021.
Thai Speech Spoofing Detection Dataset with Variations in Speaking Styles
Authors: Ticho Urai; Pachara Boonsarngsuk; Ekapol Chuangsuwanich
Venue: Interspeech 2025 (main)

Abstract: We develop the Chula Spoofed Speech (CSS) dataset, a spoofing dataset for Thai, which contains 1,332,120 utterances of both bona fide and synthetic speech. Synthetic speech samples were generated using five distinct high-quality text-to-speech (TTS) systems, all based on the same utterances as the bona fide data. The data covers various age ranges and speaking styles. Strong baselines such as AASIST and RawNet2 are trained under different conditions to uncover aspects that affect the performance of the models. Besides unseen attacks, unseen speaking styles also have a big impact on performance, indicating a need for diversity in speaking styles in anti-spoofing datasets. Furthermore, we investigate the models in telephony scenarios against additional TTS systems. The results reveal that the models still face certain challenges in this context.
CBA: Backdoor Attack on Deep Speech Classification via Audio Compression
Authors: Yuheng Huang; Ying Ren; Wenjie Zhang; Diqun Yan
Venue: Interspeech 2025 (main)

Abstract: As deep neural networks permeate various aspects of society, research has demonstrated their vulnerability to backdoor attacks, particularly with untrustworthy third-party platforms, enabling attackers to manipulate model outputs using customized triggers. However, most audio-based backdoor methods are either sample-agnostic or audible. In this paper, we propose Compression-based Backdoor Attack (CBA), a novel strategy designed specifically for audio compression scenarios, which also utilizes fusion as well as iterative optimization strategies to generate sample-specific triggers and ensure both the effectiveness and stealthiness of the triggers. Extensive experiments validate the attack efficiency and robustness.