pwc-archive/papers-with-abstracts huggingface dataset:
Self-Supervised Training of Speaker Encoder with Multi-Modal Diverse Positive Pairs
Authors: Ruijie Tao, Kong Aik Lee, Rohan Kumar Das, Ville Hautamäki, Haizhou Li
Date: 2022-10-27 00:00:00
Tasks: Contrastive Learning, Self-Supervised Learning, Speaker Recognition

We study a novel neural architecture and its training strategies of speaker encoder for speaker recognition without using any identity labels. The speaker encoder is trained to extract a fixed-size speaker embedding from a spoken utterance of various length. Contrastive learning is a typical self-supervised learning technique. However, the quality of the speaker encoder depends very much on the sampling strategy of positive and negative pairs. It is common that we sample a positive pair of segments from the same utterance. Unfortunately, such poor-man's positive pairs (PPP) lack necessary diversity for the training of a robust encoder. In this work, we propose a multi-modal contrastive learning technique with novel sampling strategies. By cross-referencing between speech and face data, we study a method that finds diverse positive pairs (DPP) for contrastive learning, thus improving the robustness of the speaker encoder. We train the speaker encoder on the VoxCeleb2 dataset without any speaker labels, and achieve an equal error rate (EER) of 2.89\%, 3.17\% and 6.27\% under the proposed progressive clustering strategy, and an EER of 1.44\%, 1.77\% and 3.27\% under the two-stage learning strategy with pseudo labels, on the three test sets of VoxCeleb1. This novel solution outperforms the state-of-the-art self-supervised learning methods by a large margin, at the same time, achieves comparable results with the supervised learning counterpart. We also evaluate our self-supervised learning technique on LRS2 and LRW datasets, where the speaker information is unknown. All experiments suggest that the proposed neural architecture and sampling strategies are robust across datasets.
Multi-dimensional Edge-based Audio Event Relational Graph Representation Learning for Acoustic Scene Classification
Authors: Yuanbo Hou, Siyang Song, Chuang Yu, Yuxin Song, Wenwu Wang, Dick Botteldooren
Date: 2022-10-27 00:00:00
Tasks: Acoustic Scene Classification, Graph Representation Learning, Representation Learning, Scene Classification

Most existing deep learning-based acoustic scene classification (ASC) approaches directly utilize representations extracted from spectrograms to identify target scenes. However, these approaches pay little attention to the audio events occurring in the scene despite they provide crucial semantic information. This paper conducts the first study that investigates whether real-life acoustic scenes can be reliably recognized based only on the features that describe a limited number of audio events. To model the task-specific relationships between coarse-grained acoustic scenes and fine-grained audio events, we propose an event relational graph representation learning (ERGL) framework for ASC. Specifically, ERGL learns a graph representation of an acoustic scene from the input audio, where the embedding of each event is treated as a node, while the relationship cues derived from each pair of event embeddings are described by a learned multidimensional edge feature. Experiments on a polyphonic acoustic scene dataset show that the proposed ERGL achieves competitive performance on ASC by using only a limited number of embeddings of audio events without any data augmentations. The validity of the proposed ERGL framework proves the feasibility of recognizing diverse acoustic scenes based on the event relational graph. Our code is available on our homepage (https://github.com/Yuanbo2020/ERGL).
Multi-class Detection of Pathological Speech with Latent Features: How does it perform on unseen data?
Authors: Dominik Wagner, Ilja Baumann, Franziska Braun, Sebastian P. Bayerl, Elmar Nöth, Korbinian Riedhammer, Tobias Bocklet
Date: 2022-10-27 00:00:00
Tasks: Binary Classification

The detection of pathologies from speech features is usually defined as a binary classification task with one class representing a specific pathology and the other class representing healthy speech. In this work, we train neural networks, large margin classifiers, and tree boosting machines to distinguish between four pathologies: Parkinson's disease, laryngeal cancer, cleft lip and palate, and oral squamous cell carcinoma. We show that latent representations extracted at different layers of a pre-trained wav2vec 2.0 system can be effectively used to classify these types of pathological voices. We evaluate the robustness of our classifiers by adding room impulse responses to the test data and by applying them to unseen speech corpora. Our approach achieves unweighted average F1-Scores between 74.1% and 97.0%, depending on the model and the noise conditions used. The systems generalize and perform well on unseen data of healthy speakers sampled from a variety of different sources.
Robust Data2vec: Noise-robust Speech Representation Learning for ASR by Combining Regression and Improved Contrastive Learning
Authors: Qiu-Shi Zhu, Long Zhou, Jie Zhang, Shu-Jie Liu, Yu-Chen Hu, Li-Rong Dai
Date: 2022-10-27 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Contrastive Learning, regression, Representation Learning, speech-recognition, Speech Recognition, Speech Representation Learning

Self-supervised pre-training methods based on contrastive learning or regression tasks can utilize more unlabeled data to improve the performance of automatic speech recognition (ASR). However, the robustness impact of combining the two pre-training tasks and constructing different negative samples for contrastive learning still remains unclear. In this paper, we propose a noise-robust data2vec for self-supervised speech representation learning by jointly optimizing the contrastive learning and regression tasks in the pre-training stage. Furthermore, we present two improved methods to facilitate contrastive learning. More specifically, we first propose to construct patch-based non-semantic negative samples to boost the noise robustness of the pre-training model, which is achieved by dividing the features into patches at different sizes (i.e., so-called negative samples). Second, by analyzing the distribution of positive and negative samples, we propose to remove the easily distinguishable negative samples to improve the discriminative capacity for pre-training models. Experimental results on the CHiME-4 dataset show that our method is able to improve the performance of the pre-trained model in noisy scenarios. We find that joint training of the contrastive learning and regression tasks can avoid the model collapse to some extent compared to only training the regression task.
Learning Music Representations with wav2vec 2.0
Authors: Alessandro Ragano, Emmanouil Benetos, Andrew Hines
Date: 2022-10-27 00:00:00
Tasks: Music Classification

Learning music representations that are general-purpose offers the flexibility to finetune several downstream tasks using smaller datasets. The wav2vec 2.0 speech representation model showed promising results in many downstream speech tasks, but has been less effective when adapted to music. In this paper, we evaluate whether pre-training wav2vec 2.0 directly on music data can be a better solution instead of finetuning the speech model. We illustrate that when pre-training on music data, the discrete latent representations are able to encode the semantic meaning of musical concepts such as pitch and instrument. Our results show that finetuning wav2vec 2.0 pre-trained on music data allows us to achieve promising results on music classification tasks that are competitive with prior work on audio representations. In addition, the results are superior to the pre-trained model on speech embeddings, demonstrating that wav2vec 2.0 pre-trained on music data can be a promising music representation model.
A Fast and Accurate Pitch Estimation Algorithm Based on the Pseudo Wigner-Ville Distribution
Authors: Yisi Liu, Peter Wu, Alan W Black, Gopala K. Anumanchipalli
Date: 2022-10-27 00:00:00
Tasks: Speech Synthesis

Estimation of fundamental frequency (F0) in voiced segments of speech signals, also known as pitch tracking, plays a crucial role in pitch synchronous speech analysis, speech synthesis, and speech manipulation. In this paper, we capitalize on the high time and frequency resolution of the pseudo Wigner-Ville distribution (PWVD) and propose a new PWVD-based pitch estimation method. We devise an efficient algorithm to compute PWVD faster and use cepstrum-based pre-filtering to avoid cross-term interference. Evaluating our approach on a database with speech and electroglottograph (EGG) recordings yields a state-of-the-art mean absolute error (MAE) of around 4Hz. Our approach is also effective at voiced/unvoiced classification and handling sudden frequency changes.
Solving Audio Inverse Problems with a Diffusion Model
Authors: Eloi Moliner, Jaakko Lehtinen, Vesa Välimäki
Date: 2022-10-27 00:00:00
Tasks: Audio inpainting, Bandwidth Extension

This paper presents CQT-Diff, a data-driven generative audio model that can, once trained, be used for solving various different audio inverse problems in a problem-agnostic setting. CQT-Diff is a neural diffusion model with an architecture that is carefully constructed to exploit pitch-equivariant symmetries in music. This is achieved by preconditioning the model with an invertible Constant-Q Transform (CQT), whose logarithmically-spaced frequency axis represents pitch equivariance as translation equivariance. The proposed method is evaluated with objective and subjective metrics in three different and varied tasks: audio bandwidth extension, inpainting, and declipping. The results show that CQT-Diff outperforms the compared baselines and ablations in audio bandwidth extension and, without retraining, delivers competitive performance against modern baselines in audio inpainting and declipping. This work represents the first diffusion-based general framework for solving inverse problems in audio processing.
Streaming Voice Conversion Via Intermediate Bottleneck Features And Non-streaming Teacher Guidance
Authors: Yuanzhe Chen, Ming Tu, Tang Li, Xin Li, Qiuqiang Kong, Jiaxin Li, Zhichao Wang, Qiao Tian, Yuping Wang, Yuxuan Wang
Date: 2022-10-27 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition, Voice Conversion

Streaming voice conversion (VC) is the task of converting the voice of one person to another in real-time. Previous streaming VC methods use phonetic posteriorgrams (PPGs) extracted from automatic speech recognition (ASR) systems to represent speaker-independent information. However, PPGs lack the prosody and vocalization information of the source speaker, and streaming PPGs contain undesired leaked timbre of the source speaker. In this paper, we propose to use intermediate bottleneck features (IBFs) to replace PPGs. VC systems trained with IBFs retain more prosody and vocalization information of the source speaker. Furthermore, we propose a non-streaming teacher guidance (TG) framework that addresses the timbre leakage problem. Experiments show that our proposed IBFs and the TG framework achieve a state-of-the-art streaming VC naturalness of 3.85, a content consistency of 3.77, and a timbre similarity of 3.77 under a future receptive field of 160 ms which significantly outperform previous streaming VC systems.
Acoustically-Driven Phoneme Removal That Preserves Vocal Affect Cues
Authors: Camille Noufi, Jonathan Berger, Karen J. Parker, Daniel L. Bowling
Date: 2022-10-26 00:00:00

In this paper, we propose a method for removing linguistic information from speech for the purpose of isolating paralinguistic indicators of affect. The immediate utility of this method lies in clinical tests of sensitivity to vocal affect that are not confounded by language, which is impaired in a variety of clinical populations. The method is based on simultaneous recordings of speech audio and electroglottographic (EGG) signals. The speech audio signal is used to estimate the average vocal tract filter response and amplitude envelop. The EGG signal supplies a direct correlate of voice source activity that is mostly independent of phonetic articulation. The dynamic energy of the speech audio and the average vocal tract filter are applied to the EGG signal create a third signal designed to capture as much paralinguistic information from the vocal production system as possible -- maximizing the retention of bioacoustic cues to affect -- while eliminating phonetic cues to verbal meaning. To evaluate the success of this method, we studied the perception of corresponding speech audio and transformed EGG signals in an affect rating experiment with online listeners. The results show a high degree of similarity in the perceived affect of matched signals, indicating that our method is effective.
Risk-Averse Model Predictive Control for Priced Timed Automata
Authors: Mostafa Tavakkoli Anbarani, Efe C. Balta, Rômulo Meira-Góes, Ilya Kovalenko
Date: 2022-10-27 00:00:00
Tasks: Decision Making, Model Predictive Control

In this paper, we propose a Risk-Averse Priced Timed Automata (PTA) Model Predictive Control (MPC) framework to increase flexibility of cyber-physical systems. To improve flexibility in these systems, our risk-averse framework solves a multi-objective optimization problem to minimize the cost and risk, simultaneously. While minimizing cost ensures the least effort to achieve a task, minimizing risk provides guarantees on the feasibility of the task even during uncertainty. Our framework explores the trade-off between these two qualities to obtain risk-averse control actions. The solution of risk-averse PTA MPC dynamic decision-making algorithm reacts relatively better to PTA changes compared to PTA MPC without risk-averse feature. An example from manufacturing systems is presented to show the application of the proposed control strategy.
Data-Driven Analytic Differentiation via High Gain Observers and Gaussian Process Priors
Authors: Biagio Trimarchi, Lorenzo Gentilini, Fabrizio Schiano, Lorenzo Marconi
Date: 2022-10-27 00:00:00

The presented paper tackles the problem of modeling an unknown function, and its first $r-1$ derivatives, out of scattered and poor-quality data. The considered setting embraces a large number of use cases addressed in the literature and fits especially well in the context of control barrier functions, where high-order derivatives of the safe set are required to preserve the safety of the controlled system. The approach builds on a cascade of high-gain observers and a set of Gaussian process regressors trained on the observers' data. The proposed structure allows for high robustness against measurement noise and flexibility with respect to the employed sampling law. Unlike previous approaches in the field, where a large number of samples are required to fit correctly the unknown function derivatives, here we suppose to have access only to a small window of samples, sliding in time. The paper presents performance bounds on the attained regression error and numerical simulations showing how the proposed method outperforms previous approaches.
Appendix for Nonparametric Multivariate Probability Density Forecast in Smart Grids With Deep Learning
Authors: Zichao Meng, Ye Guo, Wenjun Tang, Hongbin Sun
Date: 2022-10-27 00:00:00

This paper proposes a nonparametric multivariate density forecast model based on deep learning. It not only offers the whole marginal distribution of each random variable in forecasting targets, but also reveals the future correlation between them. Differing from existing multivariate density forecast models, the proposed method requires no a priori hypotheses on the forecasted joint probability distribution of forecasting targets. In addition, based on the universal approximation capability of neural networks, the real joint cumulative distribution functions of forecasting targets are well-approximated by a special positive-weighted deep neural network in the proposed method. Numerical tests from different scenarios were implemented under a comprehensive verification framework for evaluation, including the very short-term forecast of the wind speed, wind power, and the day-ahead forecast of the aggregated electricity load. Testing results corroborate the superiority of the proposed method over current multivariate density forecast models considering the accordance with reality, prediction interval width, and correlations between different random variables.
A Classification Scheme for Local Energy Trading
Authors: Jens Hönen, Johann L. Hurink, Bert Zwart
Date: 2022-10-27 00:00:00
Tasks: Classification, energy management, energy trading, Management

The current trend towards more renewable and sustainable energy generation leads to an increased interest in new energy management systems and the concept of a smart grid. One important aspect of this is local energy trading, which is an extension of existing electricity markets by including prosumers, who are consumers also producing electricity. Prosumers having a surplus of energy may directly trade this surplus with other prosumers, which are currently in demand. In this paper, we present an overview of the literature in the area of local energy trading. In order to provide structure to the broad range of publications, we identify key characteristics, define the various settings, and cluster the considered literature along these characteristics. We identify three main research lines, each with a distinct setting and research question. We analyze and compare the settings, the used techniques, and the results and findings within each cluster and derive connections between the clusters. In addition, we identify important aspects, which up to now have to a large extent been neglected in the considered literature and highlight interesting research directions, and open problems for future work.
Data-Driven Process Optimization of Fused Filament Fabrication based on In Situ Measurements
Authors: Xavier Guidetti, Marino Kühne, Yannick Nagel, Efe C. Balta, Alisa Rupenyan, John Lygeros
Date: 2022-10-27 00:00:00
Tasks: Bayesian Optimization

The tuning of fused filament fabrication parameters is notoriously challenging. We propose an autonomous data-driven method to select parameters based on in situ measurements. We use a laser sensor to evaluate the surface roughness of a printed part. We then correlate the roughness to the mechanical properties of the part, and show how print quality affects mechanical performance. Finally, we use Bayesian optimization to search for optimal print parameters. We demonstrate our method by printing liquid crystal polymer samples, and successfully find parameters that produce high-performance prints and maximize the manufacturing process efficiency.
Torque Controlled Locomotion of a Biped Robot with Link Flexibility
Authors: Nahuel A Villa, Pierre Fernbach, Maximilien Naveau, Guilhem Saurel, Ewen Dantec, Nicolas Mansard, Olivier Stasse
Date: 2022-10-27 00:00:00

When a big and heavy robot moves, it exerts large forces on the environment and on its own structure, its angular momentum can varysubstantially, and even the robot's structure can deform if there is a mechanical weakness. Under these conditions, standard locomotion controllers can fail easily. In this article, we propose a complete control scheme to work with heavy robots in torque control. The full centroidal dynamics is used to generate walking gaits online, link deflections are taken into account to estimate the robot posture and all postural instructions are designed to avoid conflicting with each other, improving balance. These choices reduce model and control errors, allowing our centroidal stabilizer to compensate for the remaining residual errors. The stabilizer and motion generator are designed together to ensure feasibility under the assumption of bounded errors. We deploy this scheme to control the locomotion of the humanoid robot Talos, whose hip links flex when walking. It allows us to reach steps of 35~cm, for an average speed of 25~cm/sec, which is among the best performances so far for torque-controlled electric robots.
Robust output regulation of linear system subject to modeled and unmodeled uncertainty
Authors: Zhicheng Zhang, Zhiqiang Zuo, Xiang Chen, Ying Tan, Yijing Wang
Date: 2022-10-27 00:00:00

In this paper, a novel robust output regulation control framework is proposed for the system subject to noise, modeled disturbance and unmodeled disturbance to seek tracking performance and robustness simultaneously. The output regulation scheme is utilized in the framework to track the reference in the presence of modeled disturbance, and the effect of unmodeled disturbance is reduced by an $\mathcal{H}_\infty$ compensator. The Kalman filter can be also introduced in the stabilization loop to deal with the white noise. Furthermore, the tracking error in the presence/absence of noise and disturbance is estimated. The effectiveness and performance of our proposed control framework is verified in the numerical example by applying in the Furuta Inverted Pendulum system.
Generalizability of Functional Forms for Interatomic Potential Models Discovered by Symbolic Regression
Authors: Alberto Hernandez, Tim Mueller
Date: 2022-10-27 00:00:00
Tasks: Attribute, regression, Symbolic Regression

In recent years there has been great progress in the use of machine learning algorithms to develop interatomic potential models. Machine-learned potential models are typically orders of magnitude faster than density functional theory but also orders of magnitude slower than physics-derived models such as the embedded atom method. In our previous work, we used symbolic regression to develop fast, accurate and transferrable interatomic potential models for copper with novel functional forms that resemble those of the embedded atom method. To determine the extent to which the success of these forms was specific to copper, here we explore the generalizability of these models to other face-centered cubic transition metals and analyze their out-of-sample performance on several material properties. We found that these forms work particularly well on elements that are chemically similar to copper. When compared to optimized Sutton-Chen models, which have similar complexity, the functional forms discovered using symbolic regression perform better across all elements considered except gold where they have a similar performance. They perform similarly to a moderately more complex embedded atom form on properties on which they were trained, and they are more accurate on average on other properties. We attribute this improved generalized accuracy to the relative simplicity of the models discovered using symbolic regression. The genetic programming models are found to outperform other models from the literature about 50% of the time in a variety of property predictions, with about 1/10th the model complexity on average. We discuss the implications of these results to the broader application of symbolic regression to the development of new potentials and highlight how models discovered for one element can be used to seed new searches for different elements.
A Numerical Method to Compute Brain Injury Associated to Concussion
Authors: C. Bastien, A. Scattina, C. Neal-Sturgess, R. Panno, V. Shrinivas
Date: 2022-10-25 00:00:00

This research proposes a new a numerical method to compute brain injury associated with concussion using the Peak Virtual Power method, using the THUMS 4.02 head model. The results indicate that mild and severe concussions could be prevented for lateral collisions and frontal impacts with PVP values lower than 0.928mW and 9.405mW, respectively, and no concussion would happen in the head vertical direction for a PVP value less than 1.184mW. This innovative method proposes a new paradigm to improve helmet designs, assess sports injuries and improve people's wellbeing.