pwc-archive/papers-with-abstracts huggingface dataset:
Speaker anonymization using neural audio codec language models
Authors: Michele Panariello, Francesco Nespoli, Massimiliano Todisco, Nicholas Evans
Date: 2023-09-25 00:00:00
Tasks: Language Modeling, Language Modelling, Speaker anonymization

The vast majority of approaches to speaker anonymization involve the extraction of fundamental frequency estimates, linguistic features and a speaker embedding which is perturbed to obfuscate the speaker identity before an anonymized speech waveform is resynthesized using a vocoder. Recent work has shown that x-vector transformations are difficult to control consistently: other sources of speaker information contained within fundamental frequency and linguistic features are re-entangled upon vocoding, meaning that anonymized speech signals still contain speaker information. We propose an approach based upon neural audio codecs (NACs), which are known to generate high-quality synthetic speech when combined with language models. NACs use quantized codes, which are known to effectively bottleneck speaker-related information: we demonstrate the potential of speaker anonymization systems based on NAC language modeling by applying the evaluation framework of the Voice Privacy Challenge 2022.
Haha-Pod: An Attempt for Laughter-based Non-Verbal Speaker Verification
Authors: Yuke Lin, Xiaoyi Qin, Ning Jiang, Guoqing Zhao, Ming Li
Date: 2023-09-25 00:00:00
Tasks: Speaker Verification

It is widely acknowledged that discriminative representation for speaker verification can be extracted from verbal speech. However, how much speaker information that non-verbal vocalization carries is still a puzzle. This paper explores speaker verification based on the most ubiquitous form of non-verbal voice, laughter. First, we use a semi-automatic pipeline to collect a new Haha-Pod dataset from open-source podcast media. The dataset contains over 240 speakers' laughter clips with corresponding high-quality verbal speech. Second, we propose a Two-Stage Teacher-Student (2S-TS) framework to minimize the within-speaker embedding distance between verbal and non-verbal (laughter) signals. Considering Haha-Pod as a test set, two trials (S2L-Eval) are designed to verify the speaker's identity through laugh sounds. Experimental results demonstrate that our method can significantly improve the performance of the S2L-Eval test set with only a minor degradation on the VoxCeleb1 test set. The resources for the Haha-Pod dataset can be found at https://github.com/nevermoreLin/HahaPod.
Unsupervised Accent Adaptation Through Masked Language Model Correction Of Discrete Self-Supervised Speech Units
Authors: Jakob Poncelet, Hugo Van hamme
Date: 2023-09-25 00:00:00
Tasks: Accented Speech Recognition, Language Modeling, Language Modelling, speech-recognition, Speech Recognition

Self-supervised pre-trained speech models have strongly improved speech recognition, yet they are still sensitive to domain shifts and accented or atypical speech. Many of these models rely on quantisation or clustering to learn discrete acoustic units. We propose to correct the discovered discrete units for accented speech back to a standard pronunciation in an unsupervised manner. A masked language model is trained on discrete units from a standard accent and iteratively corrects an accented token sequence by masking unexpected cluster sequences and predicting their common variant. Small accent adapter blocks are inserted in the pre-trained model and fine-tuned by predicting the corrected clusters, which leads to an increased robustness of the pre-trained model towards a target accent, and this without supervision. We are able to improve a state-of-the-art HuBERT Large model on a downstream accented speech recognition task by altering the training regime with the proposed method.
Frame-wise streaming end-to-end speaker diarization with non-autoregressive self-attention-based attractors
Authors: Di Liang, Nian Shao, Xiaofei Li
Date: 2023-09-25 00:00:00
Tasks: Decoder, speaker-diarization, Speaker Diarization

This work proposes a frame-wise online/streaming end-to-end neural diarization (FS-EEND) method in a frame-in-frame-out fashion. To frame-wisely detect a flexible number of speakers and extract/update their corresponding attractors, we propose to leverage a causal speaker embedding encoder and an online non-autoregressive self-attention-based attractor decoder. A look-ahead mechanism is adopted to allow leveraging some future frames for effectively detecting new speakers in real time and adaptively updating speaker attractors. The proposed method processes the audio stream frame by frame, and has a low inference latency caused by the look-ahead frames. Experiments show that, compared with the recently proposed block-wise online methods, our method FS-EEND achieves state-of-the-art diarization results, with a low inference latency and computational cost.
AutoPrep: An Automatic Preprocessing Framework for In-the-Wild Speech Data
Authors: Jianwei Yu, Hangting Chen, Yanyao Bian, Xiang Li, Yi Luo, Jinchuan Tian, Mengyang Liu, Jiayi Jiang, Shuai Wang
Date: 2023-09-25 00:00:00
Tasks: Automatic Speech Recognition, Speech Enhancement, Speech Extraction, speech-recognition, Speech Recognition

Recently, the utilization of extensive open-sourced text data has significantly advanced the performance of text-based large language models (LLMs). However, the use of in-the-wild large-scale speech data in the speech technology community remains constrained. One reason for this limitation is that a considerable amount of the publicly available speech data is compromised by background noise, speech overlapping, lack of speech segmentation information, missing speaker labels, and incomplete transcriptions, which can largely hinder their usefulness. On the other hand, human annotation of speech data is both time-consuming and costly. To address this issue, we introduce an automatic in-the-wild speech data preprocessing framework (AutoPrep) in this paper, which is designed to enhance speech quality, generate speaker labels, and produce transcriptions automatically. The proposed AutoPrep framework comprises six components: speech enhancement, speech segmentation, speaker clustering, target speech extraction, quality filtering and automatic speech recognition. Experiments conducted on the open-sourced WenetSpeech and our self-collected AutoPrepWild corpora demonstrate that the proposed AutoPrep framework can generate preprocessed data with similar DNSMOS and PDNSMOS scores compared to several open-sourced TTS datasets. The corresponding TTS system can achieve up to 0.68 in-domain speaker similarity.
A Two-Step Approach for Narrowband Source Localization in Reverberant Rooms
Authors: Wei-Ting Lai, Lachlan Birnie, Thushara Abhayapala, Amy Bastine, Shaoheng Xu, Prasanga Samarasinghe
Date: 2023-09-25 00:00:00

This paper presents a two-step approach for narrowband source localization within reverberant rooms. The first step involves dereverberation by modeling the homogeneous component of the sound field by an equivalent decomposition of planewaves using Iteratively Reweighted Least Squares (IRLS), while the second step focuses on source localization by modeling the dereverberated component as a sparse representation of point-source distribution using Orthogonal Matching Pursuit (OMP). The proposed method enhances localization accuracy with fewer measurements, particularly in environments with strong reverberation. A numerical simulation in a conference room scenario, using a uniform microphone array affixed to the wall, demonstrates real-world feasibility. Notably, the proposed method and microphone placement effectively localize sound sources within the 2D-horizontal plane without requiring prior knowledge of boundary conditions and room geometry, making it versatile for application in different room types.
Cross-modal Alignment with Optimal Transport for CTC-based ASR
Authors: Xugang Lu, Peng Shen, Yu Tsao, Hisashi Kawai
Date: 2023-09-24 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), cross-modal alignment, Language Modelling, speech-recognition, Speech Recognition, Transfer Learning

Temporal connectionist temporal classification (CTC)-based automatic speech recognition (ASR) is one of the most successful end to end (E2E) ASR frameworks. However, due to the token independence assumption in decoding, an external language model (LM) is required which destroys its fast parallel decoding property. Several studies have been proposed to transfer linguistic knowledge from a pretrained LM (PLM) to the CTC based ASR. Since the PLM is built from text while the acoustic model is trained with speech, a cross-modal alignment is required in order to transfer the context dependent linguistic knowledge from the PLM to acoustic encoding. In this study, we propose a novel cross-modal alignment algorithm based on optimal transport (OT). In the alignment process, a transport coupling matrix is obtained using OT, which is then utilized to transform a latent acoustic representation for matching the context-dependent linguistic features encoded by the PLM. Based on the alignment, the latent acoustic feature is forced to encode context dependent linguistic information. We integrate this latent acoustic feature to build conformer encoder-based CTC ASR system. On the AISHELL-1 data corpus, our system achieved 3.96% and 4.27% character error rate (CER) for dev and test sets, respectively, which corresponds to relative improvements of 28.39% and 29.42% compared to the baseline conformer CTC ASR system without cross-modal knowledge transfer.
Efficient Black-Box Speaker Verification Model Adaptation with Reprogramming and Backend Learning
Authors: Jingyu Li, Tan Lee
Date: 2023-09-24 00:00:00
Tasks: Domain Adaptation, Speaker Verification

The development of deep neural networks (DNN) has significantly enhanced the performance of speaker verification (SV) systems in recent years. However, a critical issue that persists when applying DNN-based SV systems in practical applications is domain mismatch. To mitigate the performance degradation caused by the mismatch, domain adaptation becomes necessary. This paper introduces an approach to adapt DNN-based SV models by manipulating the learnable model inputs, inspired by the concept of adversarial reprogramming. The pre-trained SV model remains fixed and functions solely in the forward process, resembling a black-box model. A lightweight network is utilized to estimate the gradients for the learnable parameters at the input, which bypasses the gradient backpropagation through the black-box model. The reprogrammed output is processed by a two-layer backend learning module as the final adapted speaker embedding. The number of parameters involved in the gradient calculation is small in our design. With few additional parameters, the proposed method achieves both memory and parameter efficiency. The experiments are conducted in language mismatch scenarios. Using much less computation cost, the proposed method obtains close or superior performance to the fully finetuned models in our experiments, which demonstrates its effectiveness.
Attention Is All You Need For Blind Room Volume Estimation
Authors: Chunxi Wang, Maoshen Jia, Meiran Li, Changchun Bao, Wenyu Jin
Date: 2023-09-23 00:00:00
Tasks: All, Data Augmentation, parameter estimation, Transfer Learning

In recent years, dynamic parameterization of acoustic environments has raised increasing attention in the field of audio processing. One of the key parameters that characterize the local room acoustics in isolation from orientation and directivity of sources and receivers is the geometric room volume. Convolutional neural networks (CNNs) have been widely selected as the main models for conducting blind room acoustic parameter estimation, which aims to learn a direct mapping from audio spectrograms to corresponding labels. With the recent trend of self-attention mechanisms, this paper introduces a purely attention-based model to blindly estimate room volumes based on single-channel noisy speech signals. We demonstrate the feasibility of eliminating the reliance on CNN for this task and the proposed Transformer architecture takes Gammatone magnitude spectral coefficients and phase spectrograms as inputs. To enhance the model performance given the task-specific dataset, cross-modality transfer learning is also applied. Experimental results demonstrate that the proposed model outperforms traditional CNN models across a wide range of real-world acoustics spaces, especially with the help of the dedicated pretraining and data augmentation schemes.
Contrastive Speaker Embedding With Sequential Disentanglement
Authors: Youzhi Tu, Man-Wai Mak, Jen-Tzung Chien
Date: 2023-09-23 00:00:00
Tasks: Contrastive Learning, Disentanglement

Contrastive speaker embedding assumes that the contrast between the positive and negative pairs of speech segments is attributed to speaker identity only. However, this assumption is incorrect because speech signals contain not only speaker identity but also linguistic content. In this paper, we propose a contrastive learning framework with sequential disentanglement to remove linguistic content by incorporating a disentangled sequential variational autoencoder (DSVAE) into the conventional SimCLR framework. The DSVAE aims to disentangle speaker factors from content factors in an embedding space so that only the speaker factors are used for constructing a contrastive loss objective. Because content factors have been removed from the contrastive learning, the resulting speaker embeddings will be content-invariant. Experimental results on VoxCeleb1-test show that the proposed method consistently outperforms SimCLR. This suggests that applying sequential disentanglement is beneficial to learning speaker-discriminative embeddings.
Personality Profiling: How informative are social media profiles in predicting personal information?
Authors: Joshua Watt, Lewis Mitchell, Jonathan Tuke
Date: 2023-09-15 00:00:00
Tasks: regression

Personality profiling has been utilised by companies for targeted advertising, political campaigns and public health campaigns. However, the accuracy and versatility of such models remains relatively unknown. Here we explore the extent to which peoples' online digital footprints can be used to profile their Myers-Briggs personality type. We analyse and compare four models: logistic regression, naive Bayes, support vector machines (SVMs) and random forests. We discover that a SVM model achieves the best accuracy of 20.95% for predicting a complete personality type. However, logistic regression models perform only marginally worse and are significantly faster to train and perform predictions. Moreover, we develop a statistical framework for assessing the importance of different sets of features in our models. We discover some features to be more informative than others in the Intuitive/Sensory (p = 0.032) and Thinking/Feeling (p = 0.019) models. Many labelled datasets present substantial class imbalances of personal characteristics on social media, including our own. We therefore highlight the need for attentive consideration when reporting model performance on such datasets and compare a number of methods to fix class-imbalance problems.
Analysis and Experimental Validation of the WPT Efficiency of the Both-Sides Retrodirective System
Authors: Charleston Dale M. Ambatali, Shinichi Nakasuka, Bo Yang, Naoki Shinohara
Date: 2023-09-25 00:00:00

The retrodirective antenna array is considered as a mechanism to enable target tracking of a power receiver for long range wireless power transfer (WPT) due to its simplicity in implementation using only analog circuits. By installing the retrodirective capability on both the generator and rectenna arrays, a feedback loop that produces a high efficiency WPT channel is created. In this paper, we characterize the dynamics of this phenomenon using a discrete-time state-space model based on S-parameters and show that the system can naturally achieve maximum theoretical WPT efficiency. We further confirmed the theoretical analysis through a hardware experiment using a 12-port circuit board with measurable S-parameters mimicking a static wireless channel. The results collected from the hardware experiment show agreement with the proposed theoretical framework by comparing the theoretical efficiency with the measured efficiency and by showing that the collected data points follow the predicted condition to achieve maximum efficiency.
Target Controllability and Target Observability of Structured Network Systems
Authors: Arthur N. Montanari, Chao Duan, Adilson E. Motter
Date: 2023-09-25 00:00:00
Tasks: State Estimation

The duality between controllability and observability enables methods developed for full-state control to be applied to full-state estimation, and vice versa. In applications in which control or estimation of all state variables is unfeasible, the generalized notions of output controllability and functional observability establish the minimal conditions for the control and estimation of a target subset of state variables, respectively. Given the seemly unrelated nature of these properties, thus far methods for target control and target estimation have been developed independently in the literature. Here, we characterize the graph-theoretic conditions for target controllability and target observability (which are, respectively, special cases of output controllability and functional observability for structured systems). This allow us to rigorously establish a weak and strong duality between these generalized properties. When both properties are equivalent (strongly dual), we show that efficient algorithms developed for target controllability can be used for target observability, and vice versa, for the optimal placement of sensors and drivers. These results are applicable to large-scale networks, in which control and monitoring are often sought for small subsets of nodes.
Transcending the Acceleration-Bandwidth Trade-off: Lightweight Precision Stages with Active Control of Flexible Dynamics
Authors: Jingjie Wu, Lei Zhou
Date: 2023-09-25 00:00:00

Micro/Nano-positioning stages are of great importance in a wide range of manufacturing machines and instruments. In recent years, the drastically growing demand for higher throughput and reduced power consumption in various IC manufacturing equipment calls for the development of next-generation precision positioning systems with unprecedented acceleration capability while maintaining exceptional positioning accuracy and high control bandwidth. Reducing the stage's weight is an effective approach to achieving this goal. However, the reduction of stages' weight tends to decrease its structural resonance frequency, which limits the closed-loop control bandwidth and can even cause stability issues. Aiming to overcome the aforementioned challenge and thus create new lightweight precision stages with substantially improved acceleration capability without sacrificing stage control performance, this research presents a novel sequential structure and control design framework for lightweight stages with low-frequency flexible modes of the stage being actively controlled. Additional actuators and sensors are placed to actively control the flexible structural dynamics of the lightweight stage to attain high control bandwidth. A case study is simulated to evaluate the effectiveness of the proposed approach, where a stage weight reduction of 24% is demonstrated compared to a baseline case, which demonstrates the potential of the proposed design framework. Experimental evaluation of the designed stage's motion performance will be performed on a magnetically levitated linear motor platform for performance demonstration.
Competitive Networked Bivirus SIS spread over Hypergraphs
Authors: Sebin Gracy, Brian D. O. Anderson, Mengbin Ye, Cesar A. Uribe
Date: 2023-09-25 00:00:00

The paper deals with the spread of two competing viruses over a network of population nodes, accounting for pairwise interactions and higher-order interactions (HOI) within and between the population nodes. We study the competitive networked bivirus susceptible-infected-susceptible (SIS) model on a hypergraph introduced in Cui et al. [1]. We show that the system has, in a generic sense, a finite number of equilibria, and the Jacobian associated with each equilibrium point is nonsingular; the key tool is the Parametric Transversality Theorem of differential topology. Since the system is also monotone, it turns out that the typical behavior of the system is convergence to some equilibrium point. Thereafter, we exhibit a tri-stable domain with three locally exponentially stable equilibria. For different parameter regimes, we establish conditions for the existence of a coexistence equilibrium (both viruses infect separate fractions of each population node).
Harnessing Supervised Learning for Adaptive Beamforming in Multibeam Satellite Systems
Authors: Flor Ortiz, Juan A. Vasquez-Peralvo, Jorge Querol, Eva Lagunas, Jorge L. Gonzalez Rios, Luis Garces, Victor Monzon-Baeza, Symeon Chatzinotas
Date: 2023-09-25 00:00:00

In today's ever-connected world, the demand for fast and widespread connectivity is insatiable, making multibeam satellite systems an indispensable pillar of modern telecommunications infrastructure. However, the evolving communication landscape necessitates a high degree of adaptability. This adaptability is particularly crucial for beamforming, as it enables the adjustment of peak throughput and beamwidth to meet fluctuating traffic demands by varying the beamwidth, side lobe level (SLL), and effective isotropic radiated power (EIRP). This paper introduces an innovative approach rooted in supervised learning to efficiently derive the requisite beamforming matrix, aligning it with system requirements. Significantly reducing computation time, this method is uniquely tailored for real-time adaptation, enhancing the agility and responsiveness of satellite multibeam systems. Exploiting the power of supervised learning, this research enables multibeam satellites to respond quickly and intelligently to changing communication needs, ultimately ensuring uninterrupted and optimized connectivity in a dynamic world.
A selection of PID type controller settings via LQR approach for two-wheeled balancing robot
Authors: Krzysztof Laddach, Mateusz Czyżniewski, Rafał Łangowski
Date: 2023-09-25 00:00:00

The problem of PID type controller tuning has been addressed in this paper. In particular, a method of selection of PD settings based on the solution of linear-quadratic optimisation problem using the energy criterion has been investigated. Thus, the possibility of transforming optimal settings of the linear-quadratic regulator into the settings of the controller in the classical control system has been given. The presented methodology has been used during synthesis of control system for a two-wheeled balancing robot. Finally, the performance of the proposed control system has been validated by simulation in Matlab-Simulink environment with the use of a two-wheeled balancing robot model.