GenAI4ELab/papercli-papers huggingface dataset:
Comparing first spectral moment of Australian English /s/ between straight and gay voices using three analysis window sizes
Authors: TÃ¼nde Szalay; John Holik; Duy Duong Nguyen; James Morandini; Catherine J. Madill
Venue: Interspeech 2023 (main)

Abstract: Men producing /s/ with higher first spectral moment (M1) are more likely to be perceived as gay, yet it is unclear if M1 differs in production. Inconsistent results might be caused by inherent change in M1 over time. Therefore, we explored M1 change over time and tested if the length and location of the analysis window affects results on gay-straight differences. 37 gay and 29 straight male speakers of Australian English produced two /s/ tokens in continuous speech. M1 was extracted in each quarter of /s/ to explore change over time and in three windows: the entire fricative, the mid-50 ms, and the third quarter. Gay and straight men produced lower M1 in the first and last quarters relative to the midpoint. Despite the M1 change over time, we found no effect of analysis window on gay-straight differences, as gay men consistently showed higher M1. The lack of effect of analysis window on M1 is attributed to the overlap between the analysis windows caused by the duration of /s/.
Universal Automatic Phonetic Transcription into the International Phonetic Alphabet
Authors: Chihiro Taguchi; Yusuke Sakai; Parisa Haghani; David Chiang
Venue: Interspeech 2023 (main)

Abstract: This paper presents a state-of-the-art model for transcribing speech in any language into the International Phonetic Alphabet (IPA). Transcription of spoken languages into IPA is an essential yet time-consuming process in language documentation, and even partially automating this process has the potential to drastically speed up the documentation of endangered languages. Like the previous best speech-to-IPA model (Wav2Vec2Phoneme), our model is based on wav2vec 2.0 and is fine-tuned to predict IPA from audio input. We use training data from seven languages from CommonVoice 11.0, transcribed into IPA semi-automatically. Although this training dataset is much smaller than Wav2Vec2Phoneme's, its higher quality lets our model achieve comparable or better results. Furthermore, we show that the quality of our universal speech-to-IPA models is close to that of human annotators.
Voice Twins: Discovering Extremely Similar-sounding, Unrelated Speakers
Authors: Linda Gerlach; Kirsty McDougall; Finnian Kelly; Anil Alexander
Venue: Interspeech 2023 (main)

Abstract: This paper deals with extremely similar-sounding, unrelated speakers ('voice twins') and presents an automatic approach to voice twin discovery applied to different speaker databases. An automatic speaker recognition system relying on perceptually relevant phonetic features including formants and a tuned clustering algorithm DBSCAN was used to group recordings within diverse datasets. 18 voice twin pairs selected from 2-speaker clusters were evaluated by 50 listeners in a 2-alternative forced choice experiment. Same/different decisions and confidence ratings were collected for same-speaker, random different-speaker and voice twin comparisons. Listeners were unable to differentiate between the candidate voice twin pairs much better than chance level while they performed well (80% accuracy) for random same- or different-speaker comparisons indicating the voice twin speakers were perceptually very similar. The implications and forensic relevance of identifying voice twins are discussed.
Filling the population statistics gap: Swiss German reference data on F0 and speech tempo for forensic contexts
Authors: Hannah Hedegard; Andrea FrÃ¶hlich; Fabian Tomaschek; Carina Steiner; Adrian Leemann
Venue: Interspeech 2023 (main)

Abstract: The increased focus on big data in phonetics, and Bayesian statistics in the forensic sciences, prompt a fundamental issue in common applications of forensic phonetics. Relevant population distributions for most features, a key element when evaluating the similarity and distinctiveness of voices, remain lacking for a substantial number of languages and dialects. This paper provides population statistics for two phonetic features in the Swiss German context, speech tempo and F0, and outlines a potential method for big data analysis. The speech data is taken from 1000 SwG speakers and include two different style conditions: spontaneous and read speech. Results indicate significant variation for both parameters: we contradict previous findings on gender differences in speech tempo and note discrepancies for both features between the two styles. These findings constitute an important contribution to the field of forensic phonetics, as well as the field of general phonetics more broadly.
Investigating the Syntax-Discourse Interface in the Phonetic Implementation of Discourse Markers
Authors: Mathilde Hutin; Liesbeth Degand; Marc AllassonniÃ¨re-Tang
Venue: Interspeech 2023 (main)

Abstract: Discourse markers (DMs) are (chunks of) words stemming from the diachronic development of other parts-of-speech that tag the discourse's organization (ex. "well then", "innit"...). However, in synchrony, the formal accounts for the DM class vary from purely discourse-oriented definitions to models relying on a combination of lexico-grammatical and discursive information. We propose to bring new evidence into this debate by comparing the phonetic realizations of 4 DM types: stemming originally from adverbs, coordinators, subordinators and interjections. A discourse-only account would predict that the 4 types would be realized similarly, while a syntactic-discursive account predicts that subordinators would stand out, as they are less prone to syntactic independence. The analysis of various acoustic parameters (segment duration, F0, F1, F2 and HNR) in a finely-annotated 4-hour long corpus of French indicates that a hybrid approach may indeed be more accurate.
Evaluation of a Forensic Automatic Speaker Recognition System with Emotional Speech Recordings
Authors: Robert Essery; Philip Harrison; Vincent Hughes
Venue: Interspeech 2023 (main)

Abstract: In forensic contexts, speakers often feel emotional, which will likely influence their speech. Emotional mismatch between samples is therefore a source of variability which could have substantial effects on the performance of a forensic automatic speaker recognition system. This paper examines the issue of emotional speech in forensic casework, both in terms of emotional match and mismatch between test samples and in terms of the data used to calibrate the system (i.e. the reference population). Specifically, we tested system performance on samples of neutral and acted angry and fearful speech data across 37 test conditions. The best system performance was achieved when the test data and reference population conditions matched exactly. However, in 16 of the 37 tests, the system produced a Cllr greater than 0.8, 10 of which also exceeded a Cllr of 1. As a result, caution should be used to interpret the results of automatic and semi-automatic forensic analysis on emotional speech data.
An Outlier Analysis of Vowel Formants from a Corpus Phonetics Pipeline
Authors: Emily P. Ahn; Gina-Anne Levow; Richard A. Wright; Eleanor Chodroff
Venue: Interspeech 2023 (main)

Abstract: With the growing availability of large-scale spoken databases, linguists are increasingly relying on automated tools to obtain time alignments of sound units to the speech signal. A typical automated pipeline may involve grapheme-to-phoneme conversion, forced alignment, and acoustic-phonetic measurement, and each of these stages requires a strong assumption regarding the output quality. We investigate these assumptions by auditing outliers in vowel formants from two multilingual read speech corpora, CMU Wilderness and Mozilla Common Voice, across three languages: Hausa, Kazakh, and Swedish. From this audit, we develop a novel outlier taxonomy that includes the broad outlier categories of transcript errors, alignment errors, formant tracking errors, linguistic variations, and fine samples. We show the utility of this outlier analysis in identifying weaknesses in corpus-specific and corpus-general pipeline assumptions, and discovering characteristics of particular languages.
The Hidden Dance of Phonemes and Visage: Unveiling the Enigmatic Link between Phonemes and Facial Features
Authors: Liao Qu; Xianwei Zou; Xiang Li; Yandong Wen; Rita Singh; Bhiksha Raj
Venue: Interspeech 2023 (main)

Abstract: This work unveils the enigmatic link between phonemes and facial features. Traditional studies on voice-face correlations typically involve using a long period of voice input, including generating face images from voices and reconstructing 3D face meshes from voices. However, in situations like voice-based crimes, the available voice evidence may be short and limited. Additionally, from a physiological perspective, each segment of speech - phoneme corresponds to different types of airflow and movements in the face. Therefore, it is advantageous to discover the hidden link between phonemes and face attributes. In this paper, we propose an analysis pipeline to help us explore the voice-face relationship in a fine-grained manner, i.e., phonemes vs. facial anthropometric measurements (AM). We build an estimator for each phoneme-AM pair and evaluate the correlation through hypothesis testing. Our results indicate that AMs are more predictable from vowels compared to consonants, particularly with plosives. Additionally, we observe that if a specific AM exhibits more movement during phoneme pronunciation, it is more predictable. Our findings support those in physiology regarding correlation and lay the groundwork for future research on speech-face multimodal learning.
Beatboxing Kick Drum Kinematics
Authors: Reed Blaylock; Shrikanth Narayanan
Venue: Interspeech 2023 (main)

Abstract: Study of the vocal movements of beatboxing can benefit speech science in a number of ways; but while there are established models of speech motor control that make deterministic predictions about vocal kinematics, little is known about beatboxing motor control. A region of interest method was applied to real-time MRI videos of beatboxing Kick Drums to measure the time to peak velocity-a measurement that is used to assess how well models of speech action predict actual movement trajectories. The average time to peak velocity for Kick Drums is about half of the way (58%) through the total movement duration, similar to the times to peak velocity reported for speech actions. However, while the times to peak velocity for Kick Drums tend to be just above 50%, the times to peak velocity reported for speech sounds are usually a bit below 50%. Further study is needed to assess whether this difference reflects a more extreme constriction goal or a qualitatively different movement pattern.
Effects of hearing loss and amplification on Mandarin consonant perception
Authors: Huali Zhou; Xianming Bei; Nengheng Zheng; Qinglin Meng
Venue: Interspeech 2023 (main)

Abstract: This study investigates the effects of hearing loss and amplification on Mandarin consonant perception. 44 listeners with varying degrees of hearing loss were tested, both with and without the use of hearing aids. Consonant recognition was strongly correlated with the hearing threshold (r = -0.87), and was significantly improved by hearing-aid amplification (by more than 20% in group means) but was still not perfect. The underlying reasons are discussed. Furthermore, confusion patterns were analyzed and compared with those of normal-hearing listeners in the literature. The most challenging Mandarin consonants for hearing-impaired listeners include consonants with a spectral center of gravity in the high-frequency range (such as s and z), consonants with short duration (such as b, d, and g), and aspirated stops (such as p, t, and k). The findings of this study contribute to a better understanding of the difficulties experienced by Mandarin-speaking listeners with hearing loss.
An Acoustic Analysis of Fricative Variation in Three Accents of English
Authors: Roland Adams; Calbert Graham
Venue: Interspeech 2023 (main)

Abstract: This study reports on an analysis of accent and gender differences in the realisation of the fricative /s/ within three accents of English: London, Cambridge, Belfast. There were 30 speakers in the study. Using multilevel modelling, significant differences between accents in the dynamic acoustics of the alveolar fricative /s/ are evident. Significant gender differences in the fricative energy measure trajectories are also found, within each accent. We further discuss the implications of these differences to our understanding of the role of gender and accent in the realisation of spectra movements in English fricatives, highlighting the necessity of a dynamic approach to sociophonetic acoustic variation.
Acoustic cues to stress perception in Spanish â a mismatch negativity study
Authors: Karolina BroÅ
Venue: Interspeech 2023 (main)

Abstract: In this paper we investigate the cues governing stress perception in Spanish - an issue that has been subject to debate and remains largely unresolved. While there is general agreement as to the ability of Spanish listeners to detect and reliably produce stress contrasts, there is some disagreement on the roles played by individual stress cues. In this study, we focus on early stress processing in a passive oddball paradigm aimed at eliciting a mismatch negativity response to changes in stress. Individual features (spectral tilt and f0) rather than feature bundles are used to induce stress shift. A vowel change is used as a control condition. The results show that while both spectral tilt and f0 manipulations result in mismatch negativity, the latter evokes a stronger response that equals the effect of a change in the quality of the stressed vowel. The results are in line with previous studies on stress correlates in other languages, pointing to a possible cross-linguistic pattern.
Bulgarian Unstressed Vowel Reduction: Received Views vs Corpus Findings
Authors: Mitko Sabev; Bistra Andreeva; Christoph Gabriel; Jonas Gruenke
Venue: Interspeech 2023 (main)

Abstract: Bulgarian is often cited in phonological work for its vowel reduction, with the assumption that the six-vowel stressed inventory, /É a É i É¤ u/, shrinks to three unstressed contrastive vowels, /i É¤ u/, by virtue of /É a É/ raising and merging with /i É¤ u/. The literature in Bulgarian, on the other hand, maintains that /É-i/ do not merge in Standard Bulgarian; that vowels are less reduced in immediately pretonic syllables than elsewhere; that unstressed high vowels are lowered, while nonhigh vowels are raised; and that /a-É¤/ are more likely to merge than /É-u/. These claims have been challenged in recent work, and we present a new investigation based on 11,615 vowel tokens from 140 speakers in the BulPhonC speech corpus. MANOVA and GLMM results provide clear evidence that there is no unstressed high-vowel lowering, no difference between pretonic vs other unstressed vowels, and that both unstressed /a-É¤/ and /É-u/ merge completely, while /É-i/ remain spectrally distinct.
An Investigation of Indian Native Language Phonemic Influences on L2 English Pronunciations
Authors: Shelly Jain; Priyanshi Pal; Anil Kumar Vuppala; Prasanta Kumar Ghosh; Chiranjeevi Yarra
Venue: Interspeech 2023 (main)

Abstract: Speech systems are sensitive to accent variations. This is a challenge in India, which has numerous languages but few linguistic studies on pronunciation variation. The growing number of L2 English speakers reinforces the need to study accents and L1-L2 interactions. We investigate Indian English (IE) accents and report our observations on regional and shared features. Specifically, we observe phonemic variations and phonotactics in speakers' native languages and apply this to their English pronunciations. We demonstrate the influence of 18 Indian languages on IE by comparing native language features with IE pronunciations obtained from literature studies and phonetically annotated speech. Hence, we validate Indian language influences on IE by justifying pronunciation rules from the perspective of Indian language phonology. We obtain a comprehensive description of generalised and region-specific IE characteristics, which facilitates accent adaptation of existing speech systems.
Identifying Stable Sections for Formant Frequency Extraction of French Nasal Vowels Based on Difference Thresholds
Authors: Hye-Sook Park; Sunhee Kim
Venue: Interspeech 2023 (main)

Abstract: Formant frequencies of a vowel are generally extracted from midpoints or central sections on the time axis. Nasal vowels present a challenge for obtaining stable formant frequencies, as the midpoint often falls in an anti-formant section where vocal energy is lost through the nasal cavity. This study proposes a stable section for extracting nasal vowel formant frequencies using difference thresholds, which identify a vowel as being distinct when F1 is above 60 Hz and/or F2 is above 200 Hz. For the experiment, 481 disyllabic words (232 nasal vowels and 294 oral counterparts) are selected from an online French-Korean Dictionary. Each vowel is divided into 10 intervals, and the stable section is identified as one or more continuous intervals with lower frequencies than the difference thresholds. The results show that the stable section for nasal vowels is identified in 20%~50% of the vowels, while the stable section for oral vowels is identified in 20%~80% of the vowels.
Evaluation of delexicalization methods for research on emotional speech
Authors: Nicolas Audibert; Francesca Carbone; Maud Champagne-Lavau; AurÃ©lien Said Housseini; Caterina Petrone
Venue: Interspeech 2023 (main)

Abstract: Perceptual evaluation of non-controlled emotional speech requires delexicalization to neutralize semantic variation. However, most existing methods imply losing spectral cues crucial to emotional attribution, related to both laryngeal and supralaryngeal settings. We propose a method relying on voice morphing to retain part of the spectral information of the original stimuli, as an additional step to diphone synthesis delexicalization. After previous assessment of intelligibility loss, this study evaluates the naturalness of angry and neutral expressions in French films, delexicalized using low-pass filtering and the proposed method implemented with MBROLA and STRAIGHT. Results show that morphing-based delexicalization, which leads to accurate emotional attribution, is rated with a higher degree of naturalness than low-pass filtering. Implications for research in affective speech are discussed with regards to other delexicalization methods proposed in the literature.
Relationship between auditory and semantic entrainment using Deep Neural Networks (DNN)
Authors: Jay Kejriwal; Å tefan BeÅuÅ¡
Venue: Interspeech 2023 (main)

Abstract: The tendency of people to engage in similar, matching, or synchronized behaviour when interacting is known as entrainment. Many studies examined linguistic (syntactic and lexical structures) and paralinguistic (pitch, intensity) entrainment, but less attention was given to finding the relationship between them. In this study, we utilized state-of-the-art DNN embeddings such as BERT and TRIpLet Loss network (TRILL) vectors to extract features for measuring semantic and auditory similarities of turns within dialogues in two comparable spoken corpora of two different languages. We found people's tendency to entrain on semantic features more when compared to auditory features. Additionally, we found that entrainment in semantic and auditory linguistic features are positively correlated. The findings of this study might assist in implementing the mechanism of entrainment in human-machine interaction (HMI).
Unsupervised Auditory and Semantic Entrainment Models with Deep Neural Networks
Authors: Jay Kejriwal; Å tefan BeÅuÅ¡; Lina M. Rojas-Barahona
Venue: Interspeech 2023 (main)

Abstract: Speakers tend to engage in adaptive behavior, known as entrainment, when they become similar to their interlocutor in various aspects of speaking. We present an unsupervised deep learning framework that derives meaningful representation from textual features for developing semantic entrainment. We investigate the model's performance by extracting features using different variations of the BERT model (DistilBERT and XLM-RoBERTa) and Google's universal sentence encoder (USE) embeddings on two human-human (HH) corpora (The Fisher Corpus English Part 1, Columbia games corpus) and one human-machine (HM) corpus (Voice Assistant Conversation Corpus (VACC)). In addition to semantic features we also trained DNN-based models utilizing two auditory embeddings (TRIpLet Loss network (TRILL) vectors, Low-level descriptors (LLD) features) and two units of analysis (Inter pausal unit and Turn). The results show that semantic entrainment can be assessed with our model, that models can distinguish between HH and HM interactions and that the two units of analysis for extracting acoustic features provide comparable findings.
Parsing dialog turns with prosodic features in English
Authors: Elizabeth Nielsen; Mark Steedman; Sharon Goldwater
Venue: Interspeech 2023 (main)

Abstract: Parsing spoken dialogue presents challenges that parsing text does not, including a lack of clear sentence boundaries. We know from previous work that prosody helps in parsing single sentences (Tran et al. 2018), but we want to show the effect of prosody on parsing speech that isn't segmented into sentences. In experiments on the English Switchboard corpus, we find prosody helps our model both with parsing and with accurately identifying sentence boundaries. However, we find that the best-performing parser is not necessarily the parser that produces the best sentence segmentation performance. We suggest that the best parses instead come from modelling sentence boundaries jointly with other constituent boundaries.
Estimation of Listening Response Timing by Generative Model and Parameter Control of Response Substantialness Using Dynamic-Prompt-Tune
Authors: Toshiki Muromachi; Yoshinobu Kano
Venue: Interspeech 2023 (main)

Abstract: A spoken dialogue system is required to continuously listen to a human user for smooth conversation. We propose a method that simultaneously performs response generation and listener response timing estimation. Our proposed method estimates listener response timing by adding pseudo-samples where listener response should be irrelevant, which allows using text-only conversation dataset without audio information. Furthermore, our proposed method can control substantialness of responses by user-specified parameter integrated with the Dynamic-Prompt-Tune method, which uses prompt token embedding dynamically generated from the parameter. Our automatic and manual evaluation showed that the proposed method can generate responses with more natural timing and more in line with the response substantialness parameter compared to the baseline model.
Parameter Selection for Analyzing Conversations with Autism Spectrum Disorder
Authors: Tahiya Chowdhury; Veronica Romero; Amanda Stent
Venue: Interspeech 2023 (main)

Abstract: The diagnosis of autism spectrum disorder (ASD) is a complex, challenging task as it depends on the analysis of interactional behaviors by psychologists rather than the use of biochemical diagnostics. In this paper, we present a modeling approach to ASD diagnosis by analyzing acoustic/prosodic and linguistic features extracted from diagnostic conversations between a psychologist and children who either are typically developing (TD) or have ASD. We compare the contributions of different features across a range of conversation tasks. We focus on finding a minimal set of parameters that characterize conversational behaviors of children with ASD. Because ASD is diagnosed through conversational interaction, in addition to analyzing the behavior of the children, we also investigate whether the psychologist's conversational behaviors vary across diagnostic groups. Our results can facilitate fine-grained analysis of conversation data for children with ASD to support diagnosis and intervention.