pwc-archive/papers-with-abstracts huggingface dataset:
Speech Enhancement with Score-Based Generative Models in the Complex STFT Domain
Authors: Simon Welker, Julius Richter, Timo Gerkmann
Date: 2022-03-31 00:00:00
Tasks: Speech Enhancement

Score-based generative models (SGMs) have recently shown impressive results for difficult generative tasks such as the unconditional and conditional generation of natural images and audio signals. In this work, we extend these models to the complex short-time Fourier transform (STFT) domain, proposing a novel training task for speech enhancement using a complex-valued deep neural network. We derive this training task within the formalism of stochastic differential equations (SDEs), thereby enabling the use of predictor-corrector samplers. We provide alternative formulations inspired by previous publications on using generative diffusion models for speech enhancement, avoiding the need for any prior assumptions on the noise distribution and making the training task purely generative which, as we show, results in improved enhancement performance.
Equivariant Diffusion for Molecule Generation in 3D
Authors: Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, Max Welling
Date: 2022-03-31 00:00:00
Tasks: Unconditional Molecule Generation

This work introduces a diffusion model for molecule generation in 3D that is equivariant to Euclidean transformations. Our E(3) Equivariant Diffusion Model (EDM) learns to denoise a diffusion process with an equivariant network that jointly operates on both continuous (atom coordinates) and categorical features (atom types). In addition, we provide a probabilistic analysis which admits likelihood computation of molecules using our model. Experimentally, the proposed method significantly outperforms previous 3D molecular generative methods regarding the quality of generated samples and efficiency at training time.
Conditional Autoregressors are Interpretable Classifiers
Authors: Nathan Elazar
Date: 2022-03-31 00:00:00
Tasks: Classification, image-classification, Image Classification, Knowledge Distillation

We explore the use of class-conditional autoregressive (CA) models to perform image classification on MNIST-10. Autoregressive models assign probability to an entire input by combining probabilities from each individual feature; hence classification decisions made by a CA can be readily decomposed into contributions from each each input feature. That is to say, CA are inherently locally interpretable. Our experiments show that naively training a CA achieves much worse accuracy compared to a standard classifier, however this is due to over-fitting and not a lack of expressive power. Using knowledge distillation from a standard classifier, a student CA can be trained to match the performance of the teacher while still being interpretable.
SingAug: Data Augmentation for Singing Voice Synthesis with Cycle-consistent Training Strategy
Authors: Shuai Guo, Jiatong Shi, Tao Qian, Shinji Watanabe, Qin Jin
Date: 2022-03-31 00:00:00
Tasks: Data Augmentation, Singing Voice Synthesis

Deep learning based singing voice synthesis (SVS) systems have been demonstrated to flexibly generate singing with better qualities, compared to conventional statistical parametric based methods. However, neural systems are generally data-hungry and have difficulty to reach reasonable singing quality with limited public available training data. In this work, we explore different data augmentation methods to boost the training of SVS systems, including several strategies customized to SVS based on pitch augmentation and mix-up augmentation. To further stabilize the training, we introduce the cycle-consistent training strategy. Extensive experiments on two public singing databases demonstrate that our proposed augmentation methods and the stabilizing training strategy can significantly improve the performance on both objective and subjective evaluations.
Message Passing Neural Networks for Hypergraphs
Authors: Sajjad Heydari, Lorenzo Livi
Date: 2022-03-31 00:00:00
Tasks: Graph Neural Network, Hypergraph representations, Node Classification

Hypergraph representations are both more efficient and better suited to describe data characterized by relations between two or more objects. In this work, we present a new graph neural network based on message passing capable of processing hypergraph-structured data. We show that the proposed model defines a design space for neural network models for hypergraphs, thus generalizing existing models for hypergraphs. We report experiments on a benchmark dataset for node classification, highlighting the effectiveness of the proposed model with respect to other state-of-the-art methods for graphs and hypergraphs. We also discuss the benefits of using hypergraph representations and, at the same time, highlight the limitation of using equivalent graph representations when the underlying problem has relations among more than two objects.
Acoustic-Net: A Novel Neural Network for Sound Localization and Quantification
Authors: Guanxing Zhou, Hao Liang, Xinghao Ding, Yue Huang, Xiaotong Tu, Saqlain Abbas
Date: 2022-03-31 00:00:00

Acoustic source localization has been applied in different fields, such as aeronautics and ocean science, generally using multiple microphones array data to reconstruct the source location. However, the model-based beamforming methods fail to achieve the high-resolution of conventional beamforming maps. Deep neural networks are also appropriate to locate the sound source, but in general, these methods with complex network structures are hard to be recognized by hardware. In this paper, a novel neural network, termed the Acoustic-Net, is proposed to locate and quantify the sound source simply using the original signals. The experiments demonstrate that the proposed method significantly improves the accuracy of sound source prediction and the computing speed, which may generalize well to real data. The code and trained models are available at https://github.com/JoaquinChou/Acoustic-Net.
PADA: Pruning Assisted Domain Adaptation for Self-Supervised Speech Representations
Authors: Lodagala V S V Durga Prasad, Sreyan Ghosh, S. Umesh
Date: 2022-03-31 00:00:00
Tasks: Domain Adaptation, Language Modelling, Representation Learning, Speech Representation Learning

While self-supervised speech representation learning (SSL) models serve a variety of downstream tasks, these models have been observed to overfit to the domain from which the unlabelled data originates. To alleviate this issue, we propose PADA (Pruning Assisted Domain Adaptation) and zero out redundant weights from models pre-trained on large amounts of out-of-domain (OOD) data. Intuitively, this helps to make space for the target-domain ASR finetuning. The redundant weights can be identified through various pruning strategies which have been discussed in detail as a part of this work. Specifically, we investigate the effect of the recently discovered Task-Agnostic and Task-Aware pruning on PADA and propose a new pruning paradigm based on the latter, which we call Cross-Domain Task-Aware Pruning (CD-TAW). CD-TAW obtains the initial pruning mask from a well fine-tuned OOD model, which makes it starkly different from the rest of the pruning strategies discussed in the paper. Our proposed CD-TAW methodology achieves up to 20.6% relative WER improvement over our baseline when fine-tuned on a 2-hour subset of Switchboard data without language model (LM) decoding. Furthermore, we conduct a detailed analysis to highlight the key design choices of our proposed method.
A data-driven approach for the closure of RANS models by the divergence of the Reynolds Stress Tensor
Authors: Stefano Berrone, Davide Oberto
Date: 2022-03-31 00:00:00

In the present paper a new data-driven model is proposed to close and increase accuracy of RANS equations. The divergence of the Reynolds Stress Tensor (RST) is obtained through a Neural Network (NN) whose architecture and input choice guarantee both Galilean and coordinates-frame rotation. The former derives from the input choice of the NN while the latter from the expansion of the divergence of the RST into a vector basis. This approach has been widely used for data-driven models for the anisotropic RST or the RST discrepancies and it is here proposed for the divergence of the RST. Hence, a constitutive relation of the divergence of the RST from mean quantities is proposed to obtain such expansion. Moreover, once the proposed data-driven approach is trained, there is no need to run any classic turbulence model to close the equations. The well-known tests of flow in a square duct and over periodic hills are used to show advantages of the present method compared to standard turbulence models.
A unified theory of learning
Authors: Taisuke Katayose
Date: 2022-03-31 00:00:00
Tasks: Data Compression

Recently machine learning using neural networks (NN) has been developed, and many new methods have been suggested. These methods are optimized for the type of input data and work very effectively, but they cannot be used with any kind of input data universally. On the other hand, the human brain is universal for any kind of problem, and we will be able to construct artificial general intelligence if we can mimic the system of how the human brain works. We consider how the human brain learns things uniformly, and find that the essence of learning is the compression of information. We suggest a toy NN model which mimics the system of the human brain, and we show that the NN can compress the input information without ad hoc treatment, only by setting the loss function properly. The loss function is expressed as the sum of the self-information to remember and the loss of the information along with the compression, and its minimum corresponds to the self-information of the original data. To evaluate the self-information to remember, we provided the concept of memory. The memory expresses the compressed information, and the learning proceeds by referring to previous memories. There are many similarities between this NN and the human brain, and this NN is a realization of the free-energy principle which is considered to be a unified theory of the human brain. This work can be applied to any kind of data analysis and cognitive science.
Direction of Arrival Estimation of Sound Sources Using Icosahedral CNNs
Authors: David Diaz-Guerra, Antonio Miguel, Jose R. Beltran
Date: 2022-03-31 00:00:00
Tasks: Direction of Arrival Estimation

In this paper, we present a new model for Direction of Arrival (DOA) estimation of sound sources based on an Icosahedral Convolutional Neural Network (CNN) applied over SRP-PHAT power maps computed from the signals received by a microphone array. This icosahedral CNN is equivariant to the 60 rotational symmetries of the icosahedron, which represent a good approximation of the continuous space of spherical rotations, and can be implemented using standard 2D convolutional layers, having a lower computational cost than most of the spherical CNNs. In addition, instead of using fully connected layers after the icosahedral convolutions, we propose a new soft-argmax function that can be seen as a differentiable version of the argmax function and allows us to solve the DOA estimation as a regression problem interpreting the output of the convolutional layers as a probability distribution. We prove that using models that fit the equivariances of the problem allows us to outperform other state-of-the-art models with a lower computational cost and more robustness, obtaining root mean square localization errors lower than 10{\deg} even in scenarios with a reverberation time $T_{60}$ of 1.5 s.
Hypergraph Convolutional Networks via Equivalency between Hypergraphs and Undirected Graphs
Authors: Jiying Zhang, Fuyang Li, Xi Xiao, Tingyang Xu, Yu Rong, Junzhou Huang, Yatao Bian
Date: 2022-03-31 00:00:00
Tasks: Graph Learning

As a powerful tool for modeling complex relationships, hypergraphs are gaining popularity from the graph learning community. However, commonly used frameworks in deep hypergraph learning focus on hypergraphs with edge-independent vertex weights (EIVWs), without considering hypergraphs with edge-dependent vertex weights (EDVWs) that have more modeling power. To compensate for this, we present General Hypergraph Spectral Convolution (GHSC), a general learning framework that not only handles EDVW and EIVW hypergraphs, but more importantly, enables theoretically explicitly utilizing the existing powerful Graph Convolutional Neural Networks (GCNNs) such that largely ease the design of Hypergraph Neural Networks. In this framework, the graph Laplacian of the given undirected GCNNs is replaced with a unified hypergraph Laplacian that incorporates vertex weight information from a random walk perspective by equating our defined generalized hypergraphs with simple undirected graphs. Extensive experiments from various domains including social network analysis, visual objective classification, and protein learning demonstrate the state-of-the-art performance of the proposed framework.
HiFi-VC: High Quality ASR-Based Voice Conversion
Authors: A. Kashkin, I. Karpukhin, S. Shishkin
Date: 2022-03-31 00:00:00
Tasks: speech-recognition, Speech Recognition, Vocal Bursts Intensity Prediction, Voice Conversion

The goal of voice conversion (VC) is to convert input voice to match the target speaker's voice while keeping text and prosody intact. VC is usually used in entertainment and speaking-aid systems, as well as applied for speech data generation and augmentation. The development of any-to-any VC systems, which are capable of generating voices unseen during model training, is of particular interest to both researchers and the industry. Despite recent progress, any-to-any conversion quality is still inferior to natural speech. In this work, we propose a new any-to-any voice conversion pipeline. Our approach uses automated speech recognition (ASR) features, pitch tracking, and a state-of-the-art waveform prediction model. According to multiple subjective and objective evaluations, our method outperforms modern baselines in terms of voice quality, similarity and consistency.
Learning from few examples with nonlinear feature maps
Authors: Ivan Y. Tyukin, Oliver Sutton, Alexander N. Gorban
Date: 2022-03-31 00:00:00

In this work we consider the problem of data classification in post-classical settings were the number of training examples consists of mere few data points. We explore the phenomenon and reveal key relationships between dimensionality of AI model's feature space, non-degeneracy of data distributions, and the model's generalisation capabilities. The main thrust of our present analysis is on the influence of nonlinear feature transformations mapping original data into higher- and possibly infinite-dimensional spaces on the resulting model's generalisation capabilities. Subject to appropriate assumptions, we establish new relationships between intrinsic dimensions of the transformed data and the probabilities to learn successfully from few presentations.
WavThruVec: Latent speech representation as intermediate features for neural speech synthesis
Authors: Hubert Siuzdak, Piotr Dura, Pol van Rijn, Nori Jacoby
Date: 2022-03-31 00:00:00
Tasks: Speech Synthesis, text-to-speech, Text to Speech, Voice Conversion

Recent advances in neural text-to-speech research have been dominated by two-stage pipelines utilizing low-level intermediate speech representation such as mel-spectrograms. However, such predetermined features are fundamentally limited, because they do not allow to exploit the full potential of a data-driven approach through learning hidden representations. For this reason, several end-to-end methods have been proposed. However, such models are harder to train and require a large number of high-quality recordings with transcriptions. Here, we propose WavThruVec - a two-stage architecture that resolves the bottleneck by using high-dimensional Wav2Vec 2.0 embeddings as intermediate speech representation. Since these hidden activations provide high-level linguistic features, they are more robust to noise. That allows us to utilize annotated speech datasets of a lower quality to train the first-stage module. At the same time, the second-stage component can be trained on large-scale untranscribed audio corpora, as Wav2Vec 2.0 embeddings are already time-aligned. This results in an increased generalization capability to out-of-vocabulary words, as well as to a better generalization to unseen speakers. We show that the proposed model not only matches the quality of state-of-the-art neural models, but also presents useful properties enabling tasks like voice conversion or zero-shot synthesis.
Neural Architecture Search for Speech Emotion Recognition
Authors: Xixin Wu, Shoukang Hu, Zhiyong Wu, Xunying Liu, Helen Meng
Date: 2022-03-31 00:00:00
Tasks: Emotion Recognition, Neural Architecture Search, Speech Emotion Recognition

Deep neural networks have brought significant advancements to speech emotion recognition (SER). However, the architecture design in SER is mainly based on expert knowledge and empirical (trial-and-error) evaluations, which is time-consuming and resource intensive. In this paper, we propose to apply neural architecture search (NAS) techniques to automatically configure the SER models. To accelerate the candidate architecture optimization, we propose a uniform path dropout strategy to encourage all candidate architecture operations to be equally optimized. Experimental results of two different neural structures on IEMOCAP show that NAS can improve SER performance (54.89\% to 56.28\%) while maintaining model parameter sizes. The proposed dropout strategy also shows superiority over the previous approaches.
Assessing the risk of re-identification arising from an attack on anonymised data
Authors: Anna Antoniou, Giacomo Dossena, Julia MacMillan, Steven Hamblin, David Clifton, Paula Petrone
Date: 2022-03-31 00:00:00
Tasks: Benchmarking

Objective: The use of routinely-acquired medical data for research purposes requires the protection of patient confidentiality via data anonymisation. The objective of this work is to calculate the risk of re-identification arising from a malicious attack to an anonymised dataset, as described below. Methods: We first present an analytical means of estimating the probability of re-identification of a single patient in a k-anonymised dataset of Electronic Health Record (EHR) data. Second, we generalize this solution to obtain the probability of multiple patients being re-identified. We provide synthetic validation via Monte Carlo simulations to illustrate the accuracy of the estimates obtained. Results: The proposed analytical framework for risk estimation provides re-identification probabilities that are in agreement with those provided by simulation in a number of scenarios. Our work is limited by conservative assumptions which inflate the re-identification probability. Discussion: Our estimates show that the re-identification probability increases with the proportion of the dataset maliciously obtained and that it has an inverse relationship with the equivalence class size. Our recursive approach extends the applicability domain to the general case of a multi-patient re-identification attack in an arbitrary k-anonymisation scheme. Conclusion: We prescribe a systematic way to parametrize the k-anonymisation process based on a pre-determined re-identification probability. We observed that the benefits of a reduced re-identification risk that come with increasing k-size may not be worth the reduction in data granularity when one is considering benchmarking the re-identification probability on the size of the portion of the dataset maliciously obtained by the adversary.
Mutual information estimation for graph convolutional neural networks
Authors: Marius C. Landverk, Signe Riemer-Sørensen
Date: 2022-03-31 00:00:00
Tasks: Inductive Bias, Information Plane, Mutual Information Estimation

Measuring model performance is a key issue for deep learning practitioners. However, we often lack the ability to explain why a specific architecture attains superior predictive accuracy for a given data set. Often, validation accuracy is used as a performance heuristic quantifying how well a network generalizes to unseen data, but it does not capture anything about the information flow in the model. Mutual information can be used as a measure of the quality of internal representations in deep learning models, and the information plane may provide insights into whether the model exploits the available information in the data. The information plane has previously been explored for fully connected neural networks and convolutional architectures. We present an architecture-agnostic method for tracking a network's internal representations during training, which are then used to create the mutual information plane. The method is exemplified for graph-based neural networks fitted on citation data. We compare how the inductive bias introduced in graph-based architectures changes the mutual information plane relative to a fully connected neural network.
An Optimal Control Method to Compute the Most Likely Transition Path for Stochastic Dynamical Systems with Jumps
Authors: Wei Wei, Ting Gao, Jinqiao Duan, Xiaoli Chen
Date: 2022-03-31 00:00:00

Many complex real world phenomena exhibit abrupt, intermittent or jumping behaviors, which are more suitable to be described by stochastic differential equations under non-Gaussian L\'evy noise. Among these complex phenomena, the most likely transition paths between metastable states are important since these rare events may have a high impact in certain scenarios. Based on the large deviation principle, the most likely transition path could be treated as the minimizer of the rate function upon paths that connect two points. One of the challenges to calculate the most likely transition path for stochastic dynamical systems under non-Gaussian L\'evy noise is that the associated rate function can not be explicitly expressed by paths. For this reason, we formulate an optimal control problem to obtain the optimal state as the most likely transition path. We then develop a neural network method to solve this issue. Several experiments are investigated for both Gaussian and non-Gaussian cases.
JETS: Jointly Training FastSpeech2 and HiFi-GAN for End to End Text to Speech
Authors: Dan Lim, Sunghee Jung, Eesung Kim
Date: 2022-03-31 00:00:00
Tasks: text-to-speech, Text to Speech

In neural text-to-speech (TTS), two-stage system or a cascade of separately learned models have shown synthesis quality close to human speech. For example, FastSpeech2 transforms an input text to a mel-spectrogram and then HiFi-GAN generates a raw waveform from a mel-spectogram where they are called an acoustic feature generator and a neural vocoder respectively. However, their training pipeline is somewhat cumbersome in that it requires a fine-tuning and an accurate speech-text alignment for optimal performance. In this work, we present end-to-end text-to-speech (E2E-TTS) model which has a simplified training pipeline and outperforms a cascade of separately learned models. Specifically, our proposed model is jointly trained FastSpeech2 and HiFi-GAN with an alignment module. Since there is no acoustic feature mismatch between training and inference, it does not requires fine-tuning. Furthermore, we remove dependency on an external speech-text alignment tool by adopting an alignment learning objective in our joint training framework. Experiments on LJSpeech corpus shows that the proposed model outperforms publicly available, state-of-the-art implementations of ESPNet2-TTS on subjective evaluation (MOS) and some objective evaluations.