GenAI4ELab/papercli-papers huggingface dataset:
Automatic Assessment of Alzheimer's across Three Languages Using Speech and Language Features
Authors: Paula A. PÃ©rez-Toro; TomÃ¡s Arias-Vergara; Franziska Braun; Florian HÃ¶nig; Carlos A. TobÃ³n-Quintero; David AguillÃ³n; Francisco Lopera; Liliana HincapiÃ©-Henao; Maria Schuster; Korbinian Riedhammer; Andreas Maier; Elmar NÃ¶th; Juan Rafael Orozco-Arroyave
Venue: Interspeech 2023 (main)

Abstract: With the increasing prevalence of Alzheimer's Disease (AD) worldwide, it is essential to develop non-invasive methods to monitor the progression of the disease. Speech and language analyses are suitable for detecting the cognitive impairment of AD patients; thus, by analyzing changes in speech patterns and language use, researchers can develop methods to monitor AD remotely. In this paper, we investigated several speech and language techniques commonly used for the automatic detection of AD. Furthermore, we considered speech recordings of 448 patients in three different languages: Spanish (57), German (205), and English (186). Cross-lingual analysis was carried out using two classification approaches: (1) training/testing in one or more languages and (2) training in one language and testing in another. We obtained unweighted average recall values of up to 83% to classify AD using the first classification approach and up to 70% with the second.
On-the-Fly Feature Based Rapid Speaker Adaptation for Dysarthric and Elderly Speech Recognition
Authors: Mengzhe Geng; Xurong Xie; Rongfeng Su; Jianwei Yu; Zengrui Jin; Tianzi Wang; Shujie Hu; Zi Ye; Helen Meng; Xunying Liu
Venue: Interspeech 2023 (main)

Abstract: Accurate recognition of dysarthric and elderly speech remain challenging tasks to date. Speaker-level heterogeneity attributed to accent or gender, when aggregated with age and speech impairment, create large diversity among these speakers. Scarcity of speaker-level data limits the practical use of data-intensive model based speaker adaptation methods. To this end, this paper proposes two novel forms of data-efficient, feature-based on-the-fly speaker adaptation methods: variance-regularized spectral basis embedding (SVR) and spectral feature driven f-LHUC transforms. Experiments conducted on UASpeech dysarthric and DementiaBank Pitt elderly speech corpora suggest the proposed on-the-fly speaker adaptation approaches consistently outperform baseline iVector adapted hybrid DNN/TDNN and E2E Conformer systems by statistically significant WER reduction of 2.48%-2.85% absolute (7.92%-8.06% relative), and offline model based LHUC adaptation by 1.82% absolute (5.63% relative) respectively.
Relationship between LTAS-based spectral moments and acoustic parameters of hypokinetic dysarthria in Parkinsonâs disease
Authors: Jan Svihlik; VojtÄch Illner; Petr Kryze; MÃ¡rio Sousa; Paul Krack; Elina Tripoliti; Robert Jech; Jan Rusz
Venue: Interspeech 2023 (main)

Abstract: Although long-term averaged spectrum (LTAS) descriptors can detect the change in dysarthria of patients with Parkinson's disease (PD) due to subthalamic nucleus deep brain stimulation (STN-DBS), the relationship between LTAS variables with measures that relate to laryngeal physiology remain unknown. We aimed to find connections between LTAS-based moments and the main acoustic characteristics of hypokinetic dysarthria in PD as the response to STN-DBS stimulation changes. We analyzed reading passages of 23 PD patients in ON and OFF STN-DBS states compared to 23 healthy controls. We found a relation between the stimulation-induced change in several spectral moments and acoustic parameters representing voice quality, articulatory decay, net speech rate, and mean fundamental frequency. While the difference between PD and controls was significant across most acoustic descriptors, only the spectral mean and fundamental frequency variability could differentiate between ON and OFF conditions.
Respiratory distress estimation in human-robot interaction scenario
Authors: Eduardo Alvarado; NicolÃ¡s GrÃ¡geda; Alejandro Luzanto; Rodrigo Mahu; Jorge Wuth; Laura Mendoza; Richard Stern; NÃ©stor Becerra Yoma
Venue: Interspeech 2023 (main)

Abstract: Social robotics and human-robot partnership are becoming very relevant topics in the next decades defining many challenges for speech technology. In addition, the COVID pandemic imposed an awareness of technology challenges to fight massive health problems. In this paper, the first system to estimate respiratory distress in a human-robot interaction (HRI) environment is presented. The training procedure of the dyspnea estimation models by simulating the HRI acoustic environment with real room impulse responses (estimated with a PR2 robot) and additive noise is described. The training and testing data were processed using two beamforming techniques: delay-and-sum and MVDR. The results suggest that it should be possible to reduce significantly the degradation in precision of estimates of respiratory distress in a real HRI scenario. The improvements in accuracy and AUC with MVDR when compared to baseline processing without beamforming are 7% and 4%, respectively.
Prediction of the Gender-based Violence Victim Condition using Speech: What do Machine Learning Models rely on?
Authors: Emma Reyner-Fuentes; Esther Rituerto-GonzÃ¡lez; Isabel Trancoso; Carmen PelÃ¡ez-Moreno
Venue: Interspeech 2023 (main)

Abstract: Women who have experienced gender-based violence (GBV) are at an increased risk of developing mental illnesses such as depression, anxiety, and post-traumatic stress disorder (PTSD). Recently, Artificial Intelligence (AI) has provided new tools to assist mental health clinical diagnosis, including speech-based detection. However, there is not much work done on the GBV victim (GBVV) condition detection. This study aims to identify specific speech features that aid this detection, analyse the relationship of such results with the user's psychological evaluation, and evaluate whether the models rely on the speaker identity or self-reported emotions to predict the GBVV condition. Our results indicate that it is possible to distinguish GBVV with controlled sequelae from non-victims, which may suggest that such differentiation for GBVV with more severe mental aftereffects-such as PTSD-may be even more meaningful. We believe that our work can help future mental health AI therapy assistants.
Whisper Encoder features for Infant Cry Classification
Authors: Monil Charola; Aastha Kachhi; Hemant A. Patil
Venue: Interspeech 2023 (main)

Abstract: Identifying the pathology in infant cry is an important and socially relevant research problem, as it can save the lives of many infants. This study proposes the use of transfer learning based approach using Whisper Encoder Module which is compared against state-of-the art MFCC feature set for classification of normal vs. pathological infant cry. Moreover, we also present multi-class pathological infant cry classification using CNN and Bi-LSTM networks. Our study finds that whisper encoder module coupled with DNN classifiers such as CNN and Bi-LSTM outperform MFCC features with absolute increment of 4% and 1% on CNN and Bi-LSTM respectively. Furthermore, whisper encoder features are analysed using statistical parameters and t-SNE plots. The experiments are performed using the 10-fold cross-validation on Baby Chillanto dataset, In-house DA-IICT dataset, and also on the datasets formed by combining these two datasets.
A neural architecture for selective attention to speech features
Authors: Nika Jurov; William Idsardi; Naomi H. Feldman
Venue: Interspeech 2023 (main)

Abstract: Speech perception is complex and demands constant adaptations to the speaker and the environment (i.e. noisy speech, accent, etc.). To adapt, the listener relies on one speech feature more than another. This cognitive mechanism is called selective attention. We present a model that captures the idea of selective attention: we show that this dynamic adaptation process can be captured in a neural architecture by using a multiple encoder beta variational auto encoder (beta-ME-VAE), which is based on rate distortion theory. This model implements the idea that optimal feature weighting looks different under different listening conditions and provides insight into how listeners can adapt their listening strategy on a moment-to-moment basis, even in listening situations they haven't experienced before.
Quantifying Informational Masking due to Masker Intelligibility in Same-talker Speech-in-speech Perception
Authors: Mingyue Huo; Yinglun Sun; Dan Fogerty; Yan Tang
Venue: Interspeech 2023 (main)

Abstract: Intelligibility of the competing speech plays a significant role in causing informational masking (IM) to the target speech during speech-in-speech perception, especially in same-talker conditions where the target and the masker share a large number of similarities in acoustics. Few studies have quantitatively measured IM as a function of intelligibility of competing speech. Evidence shows that voiced segments are robust cues for speech intelligibility. In this study, the contribution of masker intelligibility to IM was studied by adjusting the voice-to-noise ratio (VNR) on voiced segments of the competing speech, while maintaining energetic masking (EM) at different target-to-masker ratios. Although model estimations suggested that the intelligibility due to EM converged when VNR<0 dB, listener performance showed that more release from IM was received with a further decrease in VNR. It was projected that masker intelligibility could lead to target intelligibility decreased by 50%.
On the Benefits of Self-supervised Learned Speech Representations for Predicting Human Phonetic Misperceptions
Authors: Santiago Cuervo; Ricard Marxer
Venue: Interspeech 2023 (main)

Abstract: Deep neural networks (DNNs) trained by self-supervised learning (SSL) have recently been shown to produce representations similar to brain activations for the same speech input. Can SSL representations help to explain human speech perception errors? Aiming to shed light on this question, we study their use for phonetic misperception prediction. We extract representations from wav2vec 2.0, a recent SSL architecture for speech, and use them to compute features for a model predicting the presence of phonetic perception errors in speech-in-noise signals. We perform our experiments on a corpus of over 3000 consistent word-in-noise confusions in English. We consider multiple SSL-based features and compare them against conventional acoustic baselines and features obtained from DNNs fine-tuned through supervised learning for ASR. Our results show the superiority of SSL representations when extracted from the proper layer, further suggesting their potential to model human speech perception.
Predicting Perceptual Centers Located at Vowel Onset in German Speech Using Long Short-Term Memory Networks
Authors: Felicia Schulz; Mirella De Sisto; M. Paula Roncaglia-Denissen; Peter Hendrix
Venue: Interspeech 2023 (main)

Abstract: Perceptual centers (p-centers) can be defined as the perceived centers of a syllable. Previous research regarding the location of p-centers in speech relied on experimental methods, and among the suggested acoustic features contributing to the location of p-centers in Germanic languages is the transition of the consonant to the vowel onset. The current study investigates the prediction of the location of p-centers in German, by means of machine learning. Machine learning is a promising tool to capture possible non-linear relationships that may occur among the acoustic features used in the complexity that is the human perception. Therefore, an LSTM neural network approach was used for the identification of p-centers in a set of spoken German sentences, with input data features being Mel Frequency Cepstral Coefficients (MFCC), amplitude envelope and root mean squared energy. The model was able to achieve a balanced accuracy of 84% with MFCCs being the best predictor of p-center location.
Exploring the mutual intelligibility breakdown caused by sculpting speech from a competing speech signal
Authors: Martin Cooke; MarÃ­a Luisa GarcÃ­a Lecumberri
Venue: Interspeech 2023 (main)

Abstract: Passing noise through a binary mask representing speech leads to remarkably intelligible speech. However, if the mask input is a competing speech signal, both the competing speech and the target speech represented by the mask are rendered unintelligible. The current study considers potential explanations for this abrupt breakdown. Competing speech was modified to reduce the influence of properties that may have interacted adversely with those of the target, including speaker, language, F0 and spectral detail. Properties were modified by noise-vocoding, envelope substitution and preservation of temporal modulations. The outcome of a listening experiment indicated that the impact of competing speech is largely due to conflicting formant-scale spectral detail and the absence of sufficient energy in specific temporal epochs, while conflicting F0 plays no role. These findings contribute to a broader understanding of the minimal representational basis that underlies speech perception.
Perception of Incomplete Voicing Neutralization of Obstruents in Tohoku Japanese
Authors: Mafuyu Kitahara; Naoya Watabe; Hiroto Noguchi; Chuyu Huang; Ayako Hashimoto; Ai Mizoguchi
Venue: Interspeech 2023 (main)

Abstract: Intervocalic voicing neutralization has been generally accepted as a peculiar feature of Tohoku dialects. The present paper reports the results of perception experiments on this phenomenon. Natural and resynthesized stimuli spoken by Tohoku speakers were presented to both Tohoku and Tokyo listeners in a series of online experiments. A comparison between these two listener groups reveals that, for Tohoku listeners whose perception was biased by the voicing neutralization in their phonology, the boundary between voiced and voiceless tokens was more blurred compared to Tokyo listeners whose phonology had no such neutralization. These results suggest that neutralization can be bidirectional: i.e., voiced tokens become less voiced and voiceless tokens become less voiceless in contrast to the traditional view of neutralization which assumes a unidirectional process where one category remains intact and the other category merges with the former.
The emergence of obstruent-intrinsic f0 and VOT as cues to the fortis/lenis contrast in West Central Bavarian
Authors: Jasmin PÃ¶hnlein; Felicitas Kleber
Venue: Interspeech 2023 (main)

Abstract: This study examines the effects of underlying voicing in word initial and medial stops on f0 at the onset of the following stressed and unstressed vowel (CF0), respectively, in Standard German (SG) and West Central Bavarian (WB). As opposed to SG, WB hitherto did not contrast fortis and lenis stops by VOT, but the importance of this cue increases in younger WB speakers. The replacement of VOT by f0 as acoustic cue in connection with voicing mergers and tonogenesis is well-studied but not the emergence of CF0 effects together with an evolving VOT contrast. An acoustic analysis of twenty SG speakers as well as ten older and ten younger WB speakers showed higher f0 after fortis compared to lenis stops in SG but only in initial position where VOT was much longer. Younger but not older WB speakers showed signs of developing CF0 effects in initial stops as found in SG which may forecast VOT differences in this position at the population level possibly due to speaker-specific cue enhancement.
â©'âª in Tsimane': a Preliminary Investigation
Authors: William N. Havard; Yaya Sy; Camila Scaff; Loann Peurey; Alejandrina Cristia
Venue: Interspeech 2023 (main)

Abstract: Tsimane' is a language spoken in Bolivia by several thousand people. Yet, it has not been described in detail. We aim to take a step towards a better description by focusing on an aspect of language: the sound represented in spelling with â©'âª, informally described as a glottal stop. We recorded two adult speakers of Tsimane' producing (near-)minimal pairs involving this sound. Perceptual analyses suggested â©'âª is very rarely realised as a full glottal stop, and is more often cued by creaky-voiced vow- els and nasals. Despite the variability in implementation, presentation of syllabic minimal pairs to these two informants and two other adult Tsimane' listeners revealed evidence that they could easily perceive when â©'âª was intended. Together, these data suffice to rule out the hypothesis that â©'âª is systematically realised as a full stop, and suggests instead a more complex set of perceptual cues may be at speakers' and listeners' disposal.
Segmental features of Brazilian (Santa Catarina) Hunsrik
Authors: Dennis Hoffmann; Maria O'Reilly
Venue: Interspeech 2023 (main)

Abstract: This paper explores the segmentals of Brazilian Hunsrik, Santa Catarina (SC). The account proposed here is merely the starting point for a more comprehensive analysis of this unique, under-resourced dialect of German for which no written standard and only limited prior linguistic descriptions exist. The goal is to characterise the segmental features of SC Hunsrik and compare them to those reported for the Hunsrik of Rio Grande do Sul (RS). The description is based on two Praat-analysed recordings of three speakers: one saying prayers, and two chatting about daily chores. The recurrent features include vowel raising, de-rounding and schwa epenthesis; consonantal aspects involve obstruent voicing assimilation, consonant deletion and /l/-velarisation. The next steps will involve unravelling the features of German HunsrÃ¼ck and the Brazilian Portuguese influences as well as analysing the dialect's prosody.
Opening or Closing? An Electroglottographic Analysis of Voiceless Coda Consonants in Australian English
Authors: Louise Ratko; Joshua Penney; Felicity Cox
Venue: Interspeech 2023 (main)

Abstract: In voiceless sounds, the glottis may be spread or constricted. Glottal spreading is associated with breathiness, and constriction with glottalisation. In many dialects of English, glottalisation often occurs with coda /t/ and sometimes with /p, k/, suggesting coda stop voicelessness is achieved through glottal constriction. Conversely, voiceless coda fricatives are associated with breathiness of the preceding vowel, with voicelessness achieved through glottal spreading. However, analyses specifically measuring glottal activity in coda consonant contexts in English are sparse. We conducted an electroglottographic analysis of vowels preceding voiceless codas /p, t, k, s/ to examine how coda voicelessness is achieved in Australian English (AusE). We found that coda /t/ and /p/ show glottal constriction towards vowel offset. Conversely, /k/ patterns with /s/ and exhibits glottal spreading. This suggests that different glottal configurations are used to achieve coda voicelessness in AusE.
Increasing aspiration of word-medial fortis plosives in Swiss Standard German
Authors: Franka Zebe
Venue: Interspeech 2023 (main)

Abstract: There is evidence for a sound change in progress in German-speaking Switzerland: Namely, Swiss German speakers of Alemannic increasingly use aspiration in fortis plosives, particularly in word-initial position. This study aims to extend the research by investigating word-medial plosives in Swiss Standard German (SSG). Using the apparent-time paradigm, the main goal is to compare younger to older speakers. Since the increasing aspiration is probably driven by the contact to German Standard German (GSG), this study focuses on speakers from both rural and urban areas, assuming that the latter have more contact to speakers of GSG than the former. Results show that younger urban speakers produce longer VOT values in alveolar plosives than the other speakers, while all younger speakers show this pattern for bilabial plosives. Furthermore, only the younger speakers from the urban group produce shorter closure durations in fortis plosives.
Lexical Stress and Velar Palatalization in Italian: A spatio-temporal Interaction
Authors: Bowei Shao; Philipp Buech; Anne Hermes; Maria Giavazzi
Venue: Interspeech 2023 (main)

Abstract: Palatalization is the process whereby a velar stop is fronted to a palatal affricate or fricative. In Italian, it takes place at the boundary between the root and /i/ suffixes. In nouns and adjectives, palatalization occurs in words with antepenultimate stress ([Ëko.mi.tÍ¡Êi]), while it is much rarer in words with penultimate stress ([ka.Ëdu.ki]) Based on one acoustic and one articulatory study (EMA), we postulate that the resistance of post-tonic /k, g/ to palatalize is related to the stressed vowel directly preceding. In the acoustic domain, post-tonic consonants show longer closure duration. This increase in closure duration is directly related to a larger and longer tongue dorsum movement in the articulatory domain. We show an interaction between temporal (closure duration) and spatial (tongue dorsum displacement) aspects of lexical stress, which we interpret as the cause of resistance to palatalization in post-tonic velars. The findings are discussed within the Î¼-gesture framework.
Speaker Embeddings as Individuality Proxy for Voice Stress Detection
Authors: Zihan Wu; Neil Scheidwasser-Clow; Karl El Hajal; Milos Cernak
Venue: Interspeech 2023 (main)

Abstract: Since the mental states of the speaker modulate speech, stress introduced by cognitive or physical loads could be detected in the voice. The existing voice stress detection benchmark has shown that the audio embeddings extracted from the Hybrid BYOL-S self-supervised model perform well. However, the benchmark only evaluates performance separately on each dataset, but does not evaluate performance across the different types of stress and different languages. Moreover, previous studies found strong individual differences in stress susceptibility. This paper presents the design and development of voice stress detection, trained on more than 100 speakers from 9 language groups and five different types of stress. We address individual variabilities in voice stress analysis by adding speaker embeddings to the hybrid BYOL-S features. The proposed method significantly improves voice stress detection performance with an input audio length of only 3-5 seconds.
From Interval to Ordinal: A HMM based Approach for Emotion Label Conversion
Authors: Jingyao Wu; Ting Dang; Vidhyasaharan Sethu; Eliathamby Ambikairajah
Venue: Interspeech 2023 (main)

Abstract: Ordinal labels along affect dimensions are garnering increasing interest in computation paralinguistics. However, they are rarely obtained directly from raters, and instead typically obtained by conversion from interval labels. Current approaches to such conversion map interval labels to either absolute ordinal labels (AOL) (e.g., low and high), or to relative ordinal labels (ROL) (e.g., one has higher arousal than the other), but never take both into account. This paper presents a novel approach to map time-continuous interval labels to time-continuous ordinal labels. It simultaneously considers both inter-rater ambiguity about where AOLs sit on the interval label scale and the consistency amongst different raters in terms of ROLs. We validate the proposed approach by comparing the converted ordinal labels to original interval labels and the categorical labels for the same speech using the publicly available MSP-Podcast and MSP-Conversation corpora.