GenAI4ELab/papercli-papers huggingface dataset:
Federated Learning for Secure Development of AI Models for Parkinsonâs Disease Detection Using Speech from Different Languages
Authors: Soroosh Tayebi Arasteh; Cristian David RÃ­os-Urrego; Elmar NÃ¶th; Andreas Maier; Seung Hee Yang; Jan Rusz; Juan Rafael Orozco-Arroyave
Venue: Interspeech 2023 (main)

Abstract: Parkinson's disease (PD) is a neurological disorder impacting a person's speech. Among automatic PD assessment methods, deep learning models have gained particular interest. Recently, the community has explored cross-pathology and cross-language models which can improve diagnostic accuracy even further. However, strict patient data privacy regulations largely prevent institutions from sharing patient speech data with each other. In this paper, we employ federated learning (FL) for PD detection using speech signals from 3 real-world language corpora of German, Spanish, and Czech, each from a separate institution. Our results indicate that the FL model outperforms all the local models in terms of diagnostic accuracy, while not performing very differently from the model based on centrally combined training sets, with the advantage of not requiring any data sharing among collaborators. This will simplify inter-institutional collaborations, resulting in enhancement of patient outcomes.
F0inTFS: A lightweight periodicity enhancement strategy for cochlear implants
Authors: Huali Zhou; Fanhui Kong; Nengheng Zheng; Qinglin Meng
Venue: Interspeech 2023 (main)

Abstract: Periodicity is one of the main cues for pitch-related speech perception but is poorly encoded in cochlear implants (CIs). Most efforts towards CI periodicity enhancement involve explicit fundamental frequency (F0) detection, which requires additional computational loads and may not be reliable in real settings. Here we propose a new strategy, namely F0inTFS, which encodes F0 information without explicit F0 detection. Our idea is inspired by the fact that temporal fine structures (TFS) at the low-frequency channels inherently contain strong F0-related periodicity cues. In F0inTFS, the TFS of the lowest-frequency channel is integrated into the temporal envelopes at all higher-frequency channels using a specifically designed algorithm. F0inTFS can be lightly implemented in the FFT-based framework of one clinically used strategy, i.e., the Advanced Combination Encoder (ACE) strategy. The benefits of F0inTFS are supported by a lexical tone perception experiment in simulated CI users.
Differentiating acoustic and physiological features in speech for hypoxia detection
Authors: Benjamin O'Brien; Adrien Gresse; Jean-Baptise Billaud; Guilhem Belda; Jean-FranÃ§ois Bonastre
Venue: Interspeech 2023 (main)

Abstract: In order to stave off the effects of hypoxia, speech may become limited at elevated altitudes. This paper evaluates the role of speech on acoustic and physiological features used to detect hypoxia. Acoustic, cerebral blood oxygenation, and cardiac signals were recorded from participants who completed control and normobaric hypoxia experimental conditions. Acoustic and physiological features were extracted from (non-)speech segments via a voice activity detection method. Support Vector Machines were used to evaluate hypoxia classification using independent and combined features produced at sea-level and simulated 5 km altitudes. Models were built upon a 4-fold cross-validation design and evaluated on an independent dataset. Our results confirmed the importance of physiological features when detecting hypoxia. When combined, acoustic features boosted performance by 10% at 5 km in comparison to sea-level. Hypoxia detection may be improved by distinguishing respiration from non-speech.
Mandarin Electrolaryngeal Speech Voice Conversion using Cross-domain Features
Authors: Hsin-Hao Chen; Yung-Lun Chien; Ming-Chi Yen; Shu-Wei Tsai; Tai-shih Chi; Hsin-Min Wang; Yu Tsao
Venue: Interspeech 2023 (main)

Abstract: Patients who have had their entire larynx removed, including the vocal folds, owing to throat cancer may experience difficulties in speaking. In such cases, electrolarynx devices are often prescribed to produce speech, which is commonly referred to as electrolaryngeal speech (EL speech). However, the quality and intelligibility of EL speech are poor. To address this problem, EL voice conversion (ELVC) is a method used to improve the intelligibility and quality of EL speech. In this paper, we propose a novel ELVC system that incorporates cross-domain features, specifically spectral features and self-supervised learning (SSL) embeddings. The experimental results show that applying cross-domain features can notably improve the conversion performance for the ELVC task compared with utilizing only traditional spectral features.
Audio-Visual Mandarin Electrolaryngeal Speech Voice Conversion
Authors: Yung-Lun Chien; Hsin-Hao Chen; Ming-Chi Yen; Shu-Wei Tsai; Hsin-Min Wang; Yu Tsao; Tai-shih Chi
Venue: Interspeech 2023 (main)

Abstract: Electrolarynx is a commonly used assistive device to help patients with removed vocal cords regain their ability to speak. Although the electrolarynx can generate excitation signals like the vocal cords, the naturalness and intelligibility of electrolaryngeal (EL) speech are very different from those of natural (NL) speech. Many deep-learning-based models have been applied to electrolaryngeal speech voice conversion (ELVC) for converting EL speech to NL speech. In this study, we propose a multimodal voice conversion (VC) model that integrates acoustic and visual information into a unified network. We compared different pre-trained models as visual feature extractors and evaluated the effectiveness of these features in the ELVC task. The experimental results demonstrate that the proposed multimodal VC model outperforms single-modal models in both objective and subjective metrics, suggesting that the integration of visual information can significantly improve the quality of ELVC.
Which aspects of motor speech disorder are captured by Mel Frequency Cepstral Coefficients? Evidence from the change in STN-DBS conditions in Parkinsonâs disease
Authors: VojtÄch Illner; Petr KrÃ½Å¾e; Jan Å vihlÃ­k; MÃ¡rio Sousa; Paul Krack; Elina Tripoliti; Robert Jech; Jan Rusz
Venue: Interspeech 2023 (main)

Abstract: One of the most popular speech parametrizations for dysarthria has been Mel Frequency Cepstral Coefficients (MFCCs). Although the MFCCs ability to capture vocal track characteristics is known, the reflected dysarthria aspects are primarily undisclosed. Thus, we investigated the relationship between key acoustic variables in Parkinson's disease (PD) and the MFCCs. 23 PD patients were recruited with ON and OFF conditions of Deep Brain Stimulation of the Subthalamic Nucleus (STN-DBS) and examined via a reading passage. The changes in dysarthria aspects were compared to changes in a global MFCC measure and individual MFCCs. A similarity was found in 2nd to 3rd MFCCs changes and voice quality. Changes in 4th to 9th MFCCs reflected articulation clarity. The global MFCC parameter outperformed individual MFCCs and acoustical measures in capturing STN-DBS conditions changes. The findings may assist in interpreting outcomes from clinical trials and improve the monitoring of disease progression.
Detecting Manifest Huntington's Disease Using Vocal Data
Authors: Vinod Subramanian; Namhee Kwon; Raymond Brueckner; Nate Blaylock; Henry O'Connell; Luis Sierra; Clementina Ullman; Karen Hildebrand; Simon Laganiere
Venue: Interspeech 2023 (main)

Abstract: Huntington's disease (HD) is an autosomal-dominant neurodegenerative disorder that leads to the devastating loss of motor control - including severe speech impairment. Current models are insufficient to predict the onset or progression of manifest symptoms and early signs of the disease remain challenging to detect and monitor. Therefore, we propose a purely speech-based, non-invasive approach to discriminate Huntington's Disease patients who are exhibiting early signs of disease from those who are not. We study various features derived from speech and machine learning models to classify HD patients. Our results show that Random Forest classifiers leveraging language features perform very well with an unweighted accuracy of 0.95. In addition, we analyze the statistical significance of features, the importance of different questions asked to the patients, and other classification problems in Huntington's disease to provide a strong foundation for this field of research.
Exploring multi-task learning and data augmentation in dementia detection with self-supervised pretrained models
Authors: Minchuan Chen; Chenfeng Miao; Jun Ma; Shaojun Wang; Jing Xiao
Venue: Interspeech 2023 (main)

Abstract: Detection of Alzheimer's Dementia (AD) is crucial for timely intervention to slow down disease progression. Using spontaneous speech to detect AD is a non-invasive, efficient and inexpensive approach. Recent innovations in self-supervised learning (SSL) have led to remarkable advances in speech processing. In this work, we investigate a set of SSL models using joint fine-tuning strategy and compare their performance with conventional classification model. Our work shows that fine-tuning the pretrained SSL models, in conjunction with multi-task learning and data augmentation, boosts the effectiveness of general-purpose speech representations in AD detection. The results surpass the baseline and are comparable to state-of-the-art performance on the popular ADReSS dataset. We also compare single- and multi-task training for AD classification, and analyze different augmentation methods to show how to achieve improved results.
GL-SSD: Global and Local Speech Style Disentanglement by vector quantization for robust sentence boundary detection in speech stream
Authors: Kuncai Zhang; Wei Zhou; Pengcheng Zhu; Haiqing Chen
Venue: Interspeech 2023 (main)

Abstract: Sentence boundary detection (SBD) in speech, aimed at segmenting the sentence units from the audio speech, plays a significant role in a broad range of tasks such as automatic speech recognition and speech translation. Previous studies have explored the solution based on basic acoustic features and high level semantic representation. Although widely studied, sentence boundary detection still remains a challenge when applied to different speech styles, including the global style and local style. To improve the robustness of SBD in the scene of different speech styles, we propose Global and Local Speech Style Disentanglement (GL-SSD) by vector quantization from the raw speech and incorporate the disentangled style representations into the semantic representation. Relevant experiments demonstrate the superior performance of the proposed method compared to other recent mainstream methods.
Semantic VAD: Low-Latency Voice Activity Detection for Speech Interaction
Authors: Mohan Shi; Yuchun Shu; Lingyun Zuo; Qian Chen; Shiliang Zhang; Jie Zhang; Li-Rong Dai
Venue: Interspeech 2023 (main)

Abstract: For speech interaction, voice activity detection (VAD) is often used as a front-end. However, traditional VAD algorithms usually need to wait for a continuous tail silence to reach a preset maximum duration before segmentation, resulting in a large latency that affects user experience. In this paper, we propose a novel semantic VAD for low-latency segmentation. Different from existing methods, a frame-level punctuation prediction task is added to the semantic VAD, and the artificial endpoint is included in the classification category in addition to the often-used speech presence and absence. To enhance the semantic information of the model, we also incorporate an automatic speech recognition (ASR) related semantic loss. Evaluations on an internal dataset show that the proposed method can reduce the average latency by 53.3% without significant deterioration of character error rate in the back-end ASR compared to the traditional VAD approach.
Dynamic Encoder RNN for Online Voice Activity Detection in Adverse Noise Conditions
Authors: Prithvi R.R. Gudepu; Jayesh M. Koroth; Kamini Sabu; Mahaboob Ali Basha Shaik
Venue: Interspeech 2023 (main)

Abstract: The majority of online Voice Activity Detection (VAD) models employ a Recurrent Neural Network (RNN) component to capture long context which helps to improve noise-robustness. These RNN components are static models which do not make efficient use of the model's predictions from previous frames. In this work, we introduce a new Dynamic Encoder RNN (DE-RNN) that encodes the target speech dynamically to facilitate distinguishing of target speech from noise. Experiments on different established baseline architectures by modifying their RNN component by the addition of DE-RNN, show improvement in both background noise and secondary competing speaker noise scenarios. We used publicly available datasets for experiments.
Point to the Hidden: Exposing Speech Audio Splicing via Signal Pointer Nets
Authors: Denise Moussa; Germans Hirsch; Sebastian Wankerl; Christian Riess
Venue: Interspeech 2023 (main)

Abstract: Verifying the integrity of voice recording evidence for criminal investigations is an integral part of an audio forensic analyst's work. Here, one focus is on detecting deletion or insertion operations, so called audio splicing. While this is a rather easy approach to alter spoken statements, careful editing can yield quite convincing results. For difficult cases or big amounts of data, automated tools can support in detecting potential editing locations. To this end, several analytical and deep learning methods have been proposed by now. Still, few address unconstrained splicing scenarios as expected in practice. With SigPointer, we propose a pointer network framework for continuous input that uncovers splice locations naturally and more efficiently than existing works. Extensive experiments on forensically challenging data like strongly compressed and noisy signals quantify the benefit of the pointer mechanism with performance increases between about 6 to 10 percentage points.
Real-Time Causal Spectro-Temporal Voice Activity Detection Based on Convolutional Encoding and Residual Decoding
Authors: Jingyuan Wang; Jie Zhang; Li-Rong Dai
Venue: Interspeech 2023 (main)

Abstract: Voice activity detection (VAD) is an essential front-end in many speech applications that aims at determining the presence or absence of speech signals in an audio frame. However, traditional VAD methods often suffer from poor performance or non-causality in low signal-to-noise ratio (SNR) environments. In this work, we therefore present a real-time causal VAD model, which mainly consists of a frequency-domain feature generation module, a convolutional-based encoding module and a residual block based decoding module. The exploitation of only current and past frames for feature extraction guarantees the causality. The effectiveness of the proposed model is verified on two datasets under various noise conditions. It is shown that the proposed method can achieve a comparable or even better performance than state-of-the-art non-causal models.
SVVAD: Personal Voice Activity Detection for Speaker Verification
Authors: Zuheng Kang; Jianzong Wang; Junqing Peng; Jing Xiao
Venue: Interspeech 2023 (main)

Abstract: Voice activity detection (VAD) improves the performance of speaker verification (SV) by preserving speech segments and attenuating the effects of non-speech. However, this scheme is not ideal: (1) it fails in noisy environments or multi-speaker conversations; (2) it is trained based on inaccurate non-SV sensitive labels. To address this, we propose a speaker verification-based voice activity detection (SVVAD) framework that can adapt the speech features according to which are most informative for SV. To achieve this, we introduce a label-free training method with triplet-like losses that completely avoids the performance degradation of SV due to incorrect labeling. Extensive experiments show that SVVAD significantly outperforms the baseline in terms of equal error rate (EER) under conditions where other speakers are mixed at different ratios. Moreover, the decision boundaries reveal the importance of the different parts of speech, which are largely consistent with human judgments.
Learning Cross-lingual Mappings for Data Augmentation to Improve Low-Resource Speech Recognition
Authors: Muhammad Umar Farooq; Thomas Hain
Venue: Interspeech 2023 (main)

Abstract: Exploiting cross-lingual resources is an effective way to compensate for data scarcity of low resource languages. Recently, a novel multilingual model fusion technique has been proposed where a model is trained to learn cross-lingual acoustic-phonetic similarities as a mapping function. However, handcrafted lexicons have been used to train hybrid DNN-HMM ASR systems. To remove this dependency, we extend the concept of learnable cross-lingual mappings for end-to-end speech recognition. Furthermore, mapping models are employed to transliterate the source languages to the target language without using parallel data. Finally, the source audio and its transliteration is used for data augmentation to retrain the target language ASR. The results show that any source language ASR model can be used for a low-resource target language recognition followed by proposed mapping model. Furthermore, data augmentation results in a relative gain up to 5% over baseline monolingual model.
AfriNames: Most ASR Models "Butcher" African Names
Authors: Tobi Olatunji; Tejumade Afonja; Bonaventure F. P. Dossou; Atnafu Lambebo Tonja; Chris Chinenye Emezue; Amina Mardiyyah Rufai; Sahib Singh
Venue: Interspeech 2023 (main)

Abstract: Useful conversational agents must accurately capture named entities to minimize error for downstream tasks, for example, asking a voice assistant to play a track from a certain artist, initiating navigation to a specific location, or documenting a laboratory result for a patient. However, where named entities such as "Ukachukwu" (Igbo), "Lakicia" (Swahili), or "Ingabire" (Rwandan) are spoken, automatic speech recognition (ASR) models' performance degrades significantly, propagating errors to downstream systems. We model this problem as a distribution shift and demonstrate that such model bias can be mitigated through multilingual pre-training, intelligent data augmentation strategies to increase the representation of African-named entities, and fine-tuning multilingual ASR models on multiple African accents. The resulting fine-tuned models show an 81.5% relative WER improvement compared with the baseline on samples with African-named entities.
Towards Dialect-inclusive Recognition in a Low-resource Language: Are Balanced Corpora the Answer?
Authors: Liam Lonergan; Mengjie Qian; Neasa NÃ­ ChiarÃ¡in; Christer Gobl; Ailbhe NÃ­ Chasaide
Venue: Interspeech 2023 (main)

Abstract: ASR systems are generally built for the spoken 'standard', and their performance declines for non-standard dialects/varieties. This is a problem for a language like Irish, where there is no single spoken standard, but rather three major dialects: Ulster (Ul), Connacht (Co) and Munster (Mu). As a diagnostic to quantify the effect of the speaker's dialect on recognition performance, 12 ASR systems were trained, firstly using baseline dialect-balanced training corpora, and then using modified versions of the baseline corpora, where dialect-specific materials were either subtracted or added. Results indicate that dialect-balanced corpora do not yield a similar performance across the dialects: the Ul dialect consistently underperforms, whereas Mu yields lowest WERs. There is a close relationship between Co and Mu dialects, but one that is not symmetrical. These results will guide future corpus collection and system building strategies to optimise for cross-dialect performance equity.
Svarah: Evaluating English ASR Systems on Indian Accents
Authors: Tahir Javed; Sakshi Joshi; Vignesh Nagarajan; Sai Sundaresan; Janki Nawale; Abhigyan Raman; Kaushal Bhogale; Pratyush Kumar; Mitesh M. Khapra
Venue: Interspeech 2023 (main)

Abstract: India is the second largest English-speaking country in the world with a speaker base of roughly 130 million. Thus, it is imperative that automatic speech recognition (ASR) systems for English should be evaluated on Indian accents. Unfortunately, Indian speakers find a very poor representation in existing English ASR benchmarks such as LibriSpeech, Switchboard, Speech Accent Archive, etc. In this work, we address this gap by creating Svarah, a benchmark that contains 9.6 hours of transcribed English audio from 117 speakers across 65 geographic locations throughout India, resulting in a diverse range of accents. Svarah comprises both read speech and spontaneous conversational data, covering various domains, such as history, culture, tourism, etc., ensuring a diverse vocabulary. We evaluate 6 open source ASR models and 2 commercial ASR systems on Svarah and show that there is clear scope for improvement on Indian accents.
N-Shot Benchmarking of Whisper on Diverse Arabic Speech Recognition
Authors: Bashar Talafha; Abdul Waheed; Muhammad Abdul-Mageed
Venue: Interspeech 2023 (main)

Abstract: Whisper, the recently developed multilingual weakly supervised model, is reported to perform well on multiple speech recognition benchmarks in both monolingual and multilingual settings. However, it is not clear how Whisper would fare under diverse conditions even on languages it was evaluated on such as Arabic. In this work, we address this gap by comprehensively evaluating Whisper on several varieties of Arabic speech for the ASR task. Our evaluation covers most publicly available Arabic speech data and is performed under n-shot (zero-, few-, and full) finetuning. We also investigate the robustness of Whisper under completely novel conditions such as in dialect-accented standard Arabic and in unseen dialects for which we develop evaluation data. Although Whisper zero-shot outperforms fully finetuned XLS-R models on all datasets, its performance deteriorates significantly in the zero-shot setting for five unseen dialects (i.e., Algeria, Jordan, Palestine, UAE, and Yemen).
The MALACH Corpus: Results with End-to-End Architectures and Pretraining
Authors: Michael Picheny; Qin Yang; Daiheng Zhang; Lining Zhang
Venue: Interspeech 2023 (main)

Abstract: The MALACH corpus contains approximately 375 hours of Holocaust survivor testimonies along with transcripts (for approximately half the data) and audio. It is an extremely difficult corpus for speech recognition, encompassing accented, emotional speech, in many cases from elderly survivors. Nevertheless, significant progress has been made on speech recognition on MALACH with WERs now typically hovering at a 20% level for hybrid speech recognition systems. The purpose of this paper is to examine if recent developments in end-to-end architectures and pretraining with self-supervision continue to drive down performance as they do on popular read corpora such as Librispeech. We also experiment with leveraging the large fraction of unlabeled corpus data by extracting pseudolabels produced from previously trained systems. It is found that the best system - a fine-tuned wav2vec2 system trained on labeled and pseudolabeled data - achieves a WER of 13.5%, a huge gain in performance.
Unsupervised speech enhancement with deep dynamical generative speech and noise models
Authors: Xiaoyu Lin; Simon Leglaive; Laurent Girin; Xavier Alameda-Pineda
Venue: Interspeech 2023 (main)

Abstract: This work builds on a previous work on unsupervised speech enhancement using a dynamical variational autoencoder (DVAE) as the clean speech model and non-negative matrix factorization (NMF) as the noise model. We propose to replace the NMF noise model with a deep dynamical generative model (DDGM) depending either on the DVAE latent variables, or on the noisy observations, or on both. This DDGM can be trained in three configurations: noise-agnostic, noise-dependent and noise adaptation after noise-dependent training. Experimental results show that the proposed method achieves competitive performance compared to state-of-the-art unsupervised speech enhancement methods, while the noise-dependent training configuration yields a much more time-efficient inference process.