GenAI4ELab/papercli-papers huggingface dataset:
Challenges of German Speech Recognition: A Study on Multi-ethnolectal Speech Among Adolescents
Authors: Martha Schubert; Daniel Duran; Ingo Siegert
Venue: Interspeech 2024 (main)

Abstract: Despite significant advancements in speech recognition systems, challenges persist in accurately interpreting spontaneous speech from underrepresented groups like non-standard speakers or younger individuals. The difficulty increases when these conditions overlap. To further explore this topic, we employ a dataset featuring spontaneous as well as read speech from young speakers in Germany, including both, speakers from mono-ethnic and multi-ethnic backgrounds. Our study involves a comparative analysis of speech recognition performance, incorporating gender considerations, using three distinct Automatic Speech Recognition (ASR) engines: Whisper (OpenAI), NeMo (NVIDIA), and Wav2Vec2.0 (Meta AI). Furthermore, we conduct a comprehensive error analysis on the automatically generated transcripts, employing part-of-speech (POS) tagging. This allows us to discern the word types that pose the greatest challenge for comprehension by the ASR engines.
Just Because We Camp, Doesn't Mean We Should: The Ethics of Modelling Queer Voices.
Authors: Atli Sigurgeirsson; Eddie L. Ungless
Venue: Interspeech 2024 (main)

Abstract: Modern voice cloning models claim to be able to capture a diverse range of voices. We test the ability of a typical pipeline to capture the style known colloquially as âgay voiceâ and notice a homogenisation effect: synthesised speech is rated as sounding significantly âless gayâ (by LGBTQ+ participants) than its corresponding ground-truth for speakers with âgay voiceâ, but ratings actually increase for control speakers. Loss of âgay voiceâ has implications for accessibility. We also find that for speakers with âgay voiceâ, loss of âgay voiceâ corresponds to lower similarity ratings. However, we caution that improving the ability of such models to synthesise âgay voiceâ comes with a great number of risks. We use this pipeline as a starting point for a discussion on the ethics of modelling queer voices more broadly. Collecting âcleanâ queer data has safety and fairness ramifications, and the resulting technology may cause harms from mockery to death.
Automatic Classification of News Subjects in Broadcast News: Application to a Gender Bias Representation Analysis
Authors: Valentin Pelloin; LÃ©na Dodson; Ãmile Chapuis; Nicolas HervÃ©; David Doukhan
Venue: Interspeech 2024 (main)

Abstract: This paper introduces a computational framework designed to delineate gender distribution biases in topics covered by French TV and radio news. We transcribe a dataset of 11.7k hours, broadcasted in 2023 on 21 French channels. A Large Language Model (LLM) is used in few-shot conversation mode to obtain a topic classification on those transcriptions. Using the generated LLM annotations, we explore the finetuning of a specialized smaller classification model, to reduce the computational cost. To evaluate the performances of these models, we construct and annotate a dataset of 804 dialogues. This dataset is made available free of charge for research purposes. We show that women are notably underrepresented in subjects such as sports, politics and conflicts. Conversely, on topics such as weather, commercials and health, women have more speaking time than their overall average across all subjects. We also observe representations differences between private and public service channels.
Gender Representation in TV and Radio: Automatic Information Extraction methods versus Manual Analyses
Authors: David Doukhan; Lena Dodson; Manon Conan; Valentin Pelloin; AurÃ©lien Clamouse; MÃ©lina Lepape; GÃ©raldine Van Hille; CÃ©cile MÃ©adel; MarlÃ¨ne Coulomb-Gully
Venue: Interspeech 2024 (main)

Abstract: This study investigates the relationship between automatic information extraction descriptors and manual analyses to describe gender representation disparities in TV and Radio. Automatic descriptors, including speech time, facial categorization and speech transcriptions are compared with channel reports on a vast 32,000-hour corpus of French broadcasts from 2023. Findings reveal systemic gender imbalances, with women underrepresented compared to men across all descriptors. Notably, manual channel reports show higher womenâs presence than automatic estimates and references to women are lower than their speech time. Descriptors share common dynamics during high and low audiences, war coverage, or private versus public channels. While women are more visible than audible in French TV, this trend is inverted in news with unseen journalists depicting male protagonists. A statistical test shows 3 main effects influencing references to women: program category, channel and speaker gender.
Acoustic Effects of Facial Feminisation Surgery on Speech and Singing: A Case Study
Authors: Cliodhna Hughes; Guy Brown; Ning Ma; Nicola Dibben
Venue: Interspeech 2024 (main)

Abstract: Transfeminine people may undergo facial feminisation surgery, a term covering a range of procedures that aim to alter the appearance of facial features, thereby potentially changing characteristics of the vocal tract. Effects of facial feminisation surgery on the voice are relatively understudied, however, so, little information on the vocal effects of these surgeries is available to people considering undergoing these procedures. In this single-case study, we present an acoustic analysis of speech and singing data collected from a transgender singer before and after facial feminisation surgery, alongside an examination of longitudinal interview data from the participant. Our quantitative results suggest facial feminisation surgery can have an impact on the voice, and our qualitative analysis suggests this may not only be as a result of the altered characteristics of the vocal tract, but also as a result of the altered social context. Several issues for future research are identified.
An inclusive approach to creating a palette of synthetic voices for gender diversity
Authors: Eva Szekely; Maxwell Hope
Venue: Interspeech 2024 (main)

Abstract: Mainstream text-to-speech (TTS) technologies predominantly rely on binary, cisgender speech, failing to adequately represent the diversity of gender expansive (e.g., transgender and/or nonbinary) people. This poses challenges, particularly for users of Speech Generating Devices (SGDs) seeking TTS voices that authentically reflect their identity and desired expressive nuances. This paper introduces a novel approach for constructing a palette of controllable gender-expansive TTS voices using recordings from 14 gender-expansive speakers. We employ Constrained PCA to extract gender-independent speaker identity vectors from x-vectors, using acoustic Vocal Tract Length (aVTL) as a known component. The result is applied as a speaker embedding in neural TTS, allowing control over the aVTL and several emergent properties captured as a representation of the vocal space across speakers. In addition to quantitative metrics, we present a community evaluation conducted by nonbinary SGD users.
Speech After Gender: A Trans-Feminine Perspective on Next Steps for Speech Science and Technology
Authors: Robin Netzorg; Alyssa Cote; Sumi Koshin; Klo Vivienne Garoute; Gopala Krishna Anumanchipalli
Venue: Interspeech 2024 (main)

Abstract: As experts in voice modification, trans-feminine gender-affirming voice teachers have unique perspectives on voice that confound current understandings of speaker identity. To demonstrate this, we present the Versatile Voice Dataset (VVD), a collection of three speakers modifying their voices along gendered axes. The VVD illustrates that current approaches in speaker modeling, based on categorical notions of gender and a static understanding of vocal texture, fail to account for the flexibility of the vocal tract. Utilizing publicly-available speaker embeddings, we demonstrate that gender classification systems are highly sensitive to voice modification, and speaker verification systems fail to identify voices as coming from the same speaker as voice modification becomes more drastic. As one path towards moving beyond categorical and static notions of speaker identity, we propose modeling individual qualities of vocal texture such as pitch, resonance, and weight.
Voice Quality Variation in AAE: An Additional Challenge for Addressing Bias in ASR Models?
Authors: Li-Fang Lai; Nicole Holliday
Venue: Interspeech 2024 (main)

Abstract: Creaky voice, a non-modal phonation type often stigmatized in the U.S. media, has become increasingly prevalent in the speech of young Americans across ethnic and regional groups. This paper aims to add to our knowledge of voice quality variation and how it interacts with ASR, by conducting three analyses using a new African American English (AAE) dataset.  Acoustic analyses show robust differences between creaky voice and modal voice, suggesting cross-ethnic similarity in vocal fold articulation between AAE and Mainstream American English (MAE) speakers. In addition, we observed gender differences in creaky production both quantitatively (women > men) and qualitatively (women: medial partial creaks vs. men: final full creaks). This indicates that young AAE female speakers are participating in the phonation change taking place in MAE. We also found that the creakier the speech, the more errors in ASR output, suggesting the importance of incorporating voice quality into ASR systems.
Articulatory Configurations across Genders and Periods in French Radio and TV archives
Authors: Benjamin Elie; David Doukhan; RÃ©mi Uro; Lucas Ondel-Yang; Albert Rilliard; Simon Devauchelle
Venue: Interspeech 2024 (main)

Abstract: This paper studies changes in articulatory configurations across genders and periods using an inversion from acoustic to articulatory parameters. From a diachronic corpus based on French media archives spanning 60 years from 1955 to 2015, automatic transcription and forced alignment allowed extracting the central frame of each vowel. More than one million frames were obtained from over a thousand speakers across gender and age categories. Their formants were used from these vocalic frames to fit the parameters of Maeda's articulatory model. Evaluations of the quality of these processes are provided. We focus here on two parameters of Maedaâs model linked to total vocal tract length: the relative position of the larynx (higher for females) and the lips protrusion (more protruded for males). Implications for voice quality across genders are discussed. The effect across periods seems gender independent; thus, the assertion that females lowered their pitch with time is not supported.
On the Encoding of Gender in Transformer-based ASR Representations
Authors: Aravind Krishnan; Badr M. Abdullah; Dietrich Klakow
Venue: Interspeech 2024 (main)

Abstract: While existing literature relies on performance differences to uncover gender biases in ASR models, a deeper analysis is essential to understand how gender is encoded and utilized during transcript generation. This work investigates the encoding and utilization of gender in the latent representations of two transformer-based ASR models, Wav2Vec2 and HuBERT. Using linear erasure, we demonstrate the feasibility of removing gender information from each layer of an ASR model and show that such an intervention has minimal impacts on the ASR performance. Additionally, our analysis reveals a concentration of gender information within the first and last frames in the final layers, explaining the ease of erasing gender in these layers. Our findings suggest the prospect of creating gender-neutral embeddings that can be integrated into ASR frameworks without compromising their efficacy.
Towards a Quantitative Analysis of Coarticulation with a Phoneme-to-Articulatory Model
Authors: Chaofei Fan; Jaimie M. Henderson; Chris Manning; Francis R. Willett
Venue: Interspeech 2024 (main)

Abstract: Prior coarticulation studies focus mainly on limited phonemic sequences and specific articulators, providing only approximate descriptions of the temporal extent and magnitude of coarticulation. This paper is an initial attempt to comprehensively investigate coarticulation. We leverage existing Electromagnetic Articulography (EMA) datasets to develop and train a phoneme-to-articulatory (P2A) model that can generate realistic EMA for novel phoneme sequences and replicate known coarticulation patterns. We use model-generated EMA on 9K minimal word pairs to analyze coarticulation magnitude and extent up to eight phonemes from the coarticulation trigger, and compare coarticulation resistance across different consonants. Our findings align with earlier studies and suggest a longer-range coarticulation effect than previously found. This model-based approach can potentially compare coarticulation between adults and children and across languages, offering new insights into speech production.
A comparative study of the impact of voiceless alveolar and palato-alveolar sibilants in English on lip aperture and protrusion during VCV production
Authors: Chetan Sharma; Vaishnavi Chandwanshi; Prasanta Kumar Ghosh
Venue: Interspeech 2024 (main)

Abstract: Lip rounding and protrusion during sibilant production are known. Whether these features are discriminative among sibilants and how the discrimination changes in different vowels context are not well explored. In this work, we consider two voiceless sibilants, namely, /s/ (alveolar sibilant) and /Ê/ (palato-alveolar sibilant) in English during VCV production and show that lip aperture (LA) and lip protrusion (LP) are significantly higher in case of /Ê/ than /s/ irrespective of the vowels (/a/, /i/, /u/) context. Using the USC Speech MRI database comprising 74 subjects speaking VCV sequence, we also show that, when used for /s/ vs /Ê/ automatic classification, LA provides the highest classification accuracy of 90.57%(Â±5.91%) in case of /i/ followed by 85.09%(Â±5.19%) and 82.38%(Â±8.76%) in case of /u/ and /a/, respectively. The change in LA and LP from /s/ to /Ê/ are seen as an effect of higher displacement of lower lip than that of upper lip for /a/ and /i/ unlike that for /u/.
Measurement and simulation of pressure losses due to airflow in vocal tract models
Authors: Peter Birkholz; Patrick HÃ¤sner
Venue: Interspeech 2024 (main)

Abstract: We propose a unified model for viscous and kinetic energy losses in a discrete tube model of the vocal system including the glottis. In this model, a lossless Bernoulli flow is assumed at each transition between two tube sections if the downstream section has a smaller diameter than the upstream section, and otherwise the recovery of a fixed fraction of the dynamic pressure. For viscous losses, we propose a general equation according to which the pressure drop within a tube section is inversely proportional to a certain power of its cross-sectional area. The parameters of the model were adjusted to reproduce the results of measurements with physical replicas of the glottis and the vocal tract. The best agreement with the experimental data was achieved when 29% of the dynamic pressure were recovered at tube expansions, and when the viscous losses were proportional to the tube area to the power of -2.9. These results may improve articulatory speech synthesis.
On The Performance of EMA-synchronized Speech and Stand-alone Speech in Acoustic-to-articulatory Inversion
Authors: Qiang Fang
Venue: Interspeech 2024 (main)

Abstract: Synchronized acoustic-articulatory data is the basis of various applications, such as acoustic to articulatory inversion (AAI), articulatory to acoustic mapping (MAP), etc. Most of the studies in these fields directly trained various models with EMA synchronized speech, while the target input or output are stand-alone speech in real applications. However, the recording conditions of EMA-synchronized speech and stand-alone speech are different, which may make the EMA-synchronized speech different to the stand-alone speech and degrade the performance of downstream tasks. In this study, we attempt to present a general view of whether the differences affect the performance of AAI by training 3 latest AAI model with EMAsynchronize speech and testing them with both EMAsynchronized speech and stand-alone speech. The results indicate that the performance of all the 3 AAI models degrade dramatically in the sense of both Root-Mean-Square errors and Pearsonâs correlation coefficients.
Glottal inverse filtering and vocal tract tuning for the numerical simulation of vowel /a/ with different levels of vocal effort
Authors: Marc Freixes; Marc Arnela; Joan Claudi SocorÃ³; Luis Joglar-Ongay; Oriol Guasch; Francesc AlÃ­as-Pujol
Venue: Interspeech 2024 (main)

Abstract: Voice production models provide valuable information about the human voice generation. However, providing them with expressiveness remains a challenge. This work proposes a methodology to modify vocal effort (VE) in the numerical simulation of vowels using a glottal source Liljencrants-Fant (LF) model and a one-dimensional acoustic model based on the finite element method. Vowels recorded with high, mid, and low VE are inverse-filtered to obtain a glottal source signal, used to estimate the LF model Rd parameter. A tuning algorithm adjusts the vocal tract geometry to match the formants of the analysed vowel. Preliminary results for the vowel /a/ are presented. Objective analyses indicate the relevance of both glottal source and vocal tract changes in reproducing VE. They are also perceptually relevant for low VE, while the glottal source predominates in high VE. Perceptual assessment validates the methodology can convey different levels of VE, particularly low and medium.
Temporal Co-Registration of Simultaneous Electromagnetic Articulography and Electroencephalography for Precise Articulatory and Neural Data Alignment
Authors: Daniel Friedrichs; Monica Lancheros; Sam Kirkham; Lei He; Andrew Clark; Clemens Lutz; Volker Dellwo; Steven Moran
Venue: Interspeech 2024 (main)

Abstract: This study presents a temporal co-registration method combining electromagnetic articulography (EMA) and electroencephalography (EEG) to capture the neural planning and execution phases of speech with high precision. Traditional EEG alignment based on acoustic vocal onset is often inaccurate due to the variable lag between articulatory and acoustic onsets. Our approach synchronizes EMA-derived speech kinematics with EEG data, addressing these challenges. We also examined the interaction between EMA and EEG systems, focusing on the integrity of EMA signals in the presence of EEG equipment and the electromagnetic influence of EMA on EEG signal quality. The method achieved a mean alignment delay of 2.7 ms (SD = 0.4 ms), enabling detailed analysis of pre-articulatory brain activities. Additionally, our evaluations confirmed the robustness of EMA signals and EEG event-related potentials, supporting the method's precision, feasibility, and reliability for speech planning research.
Frication noise features of Polish voiceless dental fricative and affricate produced by children with and without speech disorder
Authors: Zuzanna Miodonska; Michal KrÄcichwost; Ewa KwaÅniok; Agata Sage; Pawel Badura
Venue: Interspeech 2024 (main)

Abstract: The study presents frication noise acoustic features of Polish dental voiceless sibilants (fricative /s/ and affricate /tÍ¡s/) in the speech samples collected from 106 children (83 with normative dental articulation and 23 with disorderedâinterdentalâarticulation) aged 4;11â8;0. We aimed to 1) verify the differences between the characteristics of the frication noise accompanying these two sounds and 2) investigate the influence of interdentality on the frication noise features. The analysis employed features of the noise band (fricative formants, formant-related measures, and noise energies) and linear-mixed effect models. The results showed significant acoustic differences between the voiceless dental fricative and affricate. Experiments suggest that the place of articulation (dental/interdental) can also be distinguished based on the spectral features of the frication band; this finding may be employed in computer-aided speech diagnosis tools.
Key Acoustic Cues for the Realization of Metrical Prominence in Tone Languages: A Cross-Dialect Study
Authors: Yiying Hu; Hui Feng
Venue: Interspeech 2024 (main)

Abstract: Few studies have quantitatively discussed the acoustic realization of metrical prominence across tone languages. This study addresses this issue in word-level prosodic units, aiming to identify effective acoustic cues for cross-language analysis based on the realization of metrical prominence. Findings include: (1) Within-dialect analyses reveal robust acoustic cues for realizing metrically strong units: duration, pitch height, and pitch slope for both left- and right-dominant prominence, and intensity, pitch range and cumulative pitch dynamics, an often-overlooked cue, for right-dominant one; (2) Cross-dialect analyses by Boruta algorithm identify that all these six acoustic cues are significant for the classification of Chinese dialects according to their realization of metrical prominence. Linear Discriminant Analysis emphasizes duration, intensity, and cumulative pitch dynamics as key discriminators across dialects.
Revisiting Pitch Jumps: F0 Ratio in Seoul Korean
Authors: Michaela Watkins; Paul Boersma; Silke Hamann
Venue: Interspeech 2024 (main)

Abstract: Pitch tracking algorithms can show upward or downward jumps in F0 by one octave. These âoctave jumpsâ are sometimes thought of as pitch-tracking âerrorsâ, in the sense that they constitute a âmistakeâ in the algorithm. Using Praat software, we discuss the point (which has been made before) that measured octave jumps often actually reflect genuine changes in periodicity and glottal-fold vibration. We illustrate this with the example of creaky voice in fortis stops in Seoul Korean. We argue (1) that when the goal is to capture periodicity or vocal-fold vibration, pitch-tracking algorithms capture F0 well, with pitch jumps possibly reflecting an important language-specific feature, and (2) that ignoring such jumps (due to assuming an error) could lead to misrepresentation of the properties of the language. To quantify these real F0 jumps, we introduce the notion of the âF0 ratioâ, which identifies potential F0 jumps and helps to chart the frequency of pitch jumps in a language.
Aerodynamics of Sakata labial-velar oral stops
Authors: Lorenzo Maselli; VÃ©ronique Delvaux
Venue: Interspeech 2024 (main)

Abstract: The present contribution represents the first in-detail exploratory account of the aerodynamics of labial-velar oral stops in Sakata, a Bantu dialect cluster of southwestern Congo. Data collection took place at the phonetics laboratory facilities of UniversitÃ© de Mons with three speakers of central Sakata. Comparative data of labial-velar and plain bilabial oral stops are presented and analysed. Descriptive statistics of the relevant variables are discussed. Given each group of variables, MANOVA results are presented for specially tailored subsets of the whole dataset to investigate variance in the corpus. Sakata labial-velar stops are shown to differ from plain bilabials for duration, airflow, and pressure patterns. Voiceless labial-velar stops exhibit pressure and airflow values consistent with a more prominent lowering of the tongue root / larynx than their voiced counter- parts. Matches and mismatches with the available typological literature are also delineated and discussed.