pwc-archive/papers-with-abstracts huggingface dataset:
Audio Similarity is Unreliable as a Proxy for Audio Quality
Authors: Pranay Manocha, Zeyu Jin, Adam Finkelstein
Date: 2022-06-27 00:00:00

Many audio processing tasks require perceptual assessment. However, the time and expense of obtaining ``gold standard'' human judgments limit the availability of such data. Most applications incorporate full reference or other similarity-based metrics (e.g. PESQ) that depend on a clean reference. Researchers have relied on such metrics to evaluate and compare various proposed methods, often concluding that small, measured differences imply one is more effective than another. This paper demonstrates several practical scenarios where similarity metrics fail to agree with human perception, because they: (1) vary with clean references; (2) rely on attributes that humans factor out when considering quality, and (3) are sensitive to imperceptible signal level differences. In those scenarios, we show that no-reference metrics do not suffer from such shortcomings and correlate better with human perception. We conclude therefore that similarity serves as an unreliable proxy for audio quality.
Detection of Doctored Speech: Towards an End-to-End Parametric Learn-able Filter Approach
Authors: Rohit Arora
Date: 2022-06-27 00:00:00
Tasks: Speaker Verification

The Automatic Speaker Verification systems have potential in biometrics applications for logical control access and authentication. A lot of things happen to be at stake if the ASV system is compromised. The preliminary work presents a comparative analysis of the wavelet and MFCC-based state-of-the-art spoof detection techniques developed in these papers, respectively (Novoselov et al., 2016) (Alam et al., 2016a). The results on ASVspoof 2015 justify our inclination towards wavelet-based features instead of MFCC features. The experiments on the ASVspoof 2019 database show the lack of credibility of the traditional handcrafted features and give us more reason to progress towards using end-to-end deep neural networks and more recent techniques. We use Sincnet architecture as our baseline. We get E2E deep learning models, which we call WSTnet and CWTnet, respectively, by replacing the Sinc layer with the Wavelet Scattering and Continuous wavelet transform layers. The fusion model achieved 62% and 17% relative improvement over traditional handcrafted models and our Sincnet baseline when evaluated on the modern spoofing attacks in ASVspoof 2019. The final scale distribution and the number of scales used in CWTnet are far from optimal for the task at hand. So to solve this problem, we replaced the CWT layer with a Wavelet Deconvolution(WD) (Khan and Yener, 2018) layer in our CWTnet architecture. This layer calculates the Discrete-Continuous Wavelet Transform similar to the CWTnet but also optimizes the scale parameter using back-propagation. The WDnet model achieved 26% and 7% relative improvement over CWTnet and Sincnet models respectively when evaluated over ASVspoof 2019 dataset. This shows that more generalized features are extracted as compared to the features extracted by CWTnet as only the most important and relevant frequency regions are focused upon.
Extended U-Net for Speaker Verification in Noisy Environments
Authors: Ju-ho Kim, Jungwoo Heo, Hye-jin Shim, Ha-Jin Yu
Date: 2022-06-27 00:00:00
Tasks: Denoising, Speaker Identification, Speaker Verification

Background noise is a well-known factor that deteriorates the accuracy and reliability of speaker verification (SV) systems by blurring speech intelligibility. Various studies have used separate pretrained enhancement models as the front-end module of the SV system in noisy environments, and these methods effectively remove noises. However, the denoising process of independent enhancement models not tailored to the SV task can also distort the speaker information included in utterances. We argue that the enhancement network and speaker embedding extractor should be fully jointly trained for SV tasks under noisy conditions to alleviate this issue. Therefore, we proposed a U-Net-based integrated framework that simultaneously optimizes speaker identification and feature enhancement losses. Moreover, we analyzed the structural limitations of using U-Net directly for noise SV tasks and further proposed Extended U-Net to reduce these drawbacks. We evaluated the models on the noise-synthesized VoxCeleb1 test set and VOiCES development set recorded in various noisy scenarios. The experimental results demonstrate that the U-Net-based fully joint training framework is more effective than the baseline, and the extended U-Net exhibited state-of-the-art performance versus the recently proposed compensation systems.
Joint Optimization of Sampling Rate Offsets Based on Entire Signal Relationship Among Distributed Microphones
Authors: Yoshiki Masuyama, Kouei Yamaoka, Nobutaka Ono
Date: 2022-06-27 00:00:00

In this paper, we propose to simultaneously estimate all the sampling rate offsets (SROs) of multiple devices. In a distributed microphone array, the SRO is inevitable, which deteriorates the performance of array signal processing. Most of the existing SRO estimation methods focused on synchronizing two microphones. When synchronizing more than two microphones, we select one reference microphone and estimate the SRO of each non-reference microphone independently. Hence, the relationship among signals observed by non-reference microphones is not considered. To address this problem, the proposed method jointly optimizes all SROs based on a probabilistic model of a multichannel signal. The SROs and model parameters are alternately updated to increase the log-likelihood based on an auxiliary function. The effectiveness of the proposed method is validated on mixtures of various numbers of speakers.
Transport-Oriented Feature Aggregation for Speaker Embedding Learning
Authors: Yusheng Tian, Jingyu Li, Tan Lee
Date: 2022-06-26 00:00:00
Tasks: Speaker Verification

Pooling is needed to aggregate frame-level features into utterance-level representations for speaker modeling. Given the success of statistics-based pooling methods, we hypothesize that speaker characteristics are well represented in the statistical distribution over the pre-aggregation layer's output, and propose to use transport-oriented feature aggregation for deriving speaker embeddings. The aggregated representation encodes the geometric structure of the underlying feature distribution, which is expected to contain valuable speaker-specific information that may not be represented by the commonly used statistical measures like mean and variance. The original transport-oriented feature aggregation is also extended to a weighted-frame version to incorporate the attention mechanism. Experiments on speaker verification with the Voxceleb dataset show improvement over statistics pooling and its attentive variant.
Predicting within and across language phoneme recognition performance of self-supervised learning speech pre-trained models
Authors: Hang Ji, Tanvina Patel, Odette Scharenborg
Date: 2022-06-24 00:00:00
Tasks: Phoneme Recognition, Self-Supervised Learning

In this work, we analyzed and compared speech representations extracted from different frozen self-supervised learning (SSL) speech pre-trained models on their ability to capture articulatory features (AF) information and their subsequent prediction of phone recognition performance for within and across language scenarios. Specifically, we compared CPC, wav2vec 2.0, and HuBert. First, frame-level AF probing tasks were implemented. Subsequently, phone-level end-to-end ASR systems for phoneme recognition tasks were implemented, and the performance on the frame-level AF probing task and the phone accuracy were correlated. Compared to the conventional speech representation MFCC, all SSL pre-trained speech representations captured more AF information, and achieved better phoneme recognition performance within and across languages, with HuBert performing best. The frame-level AF probing task is a good predictor of phoneme recognition performance, showing the importance of capturing AF information in the speech representations. Compared with MFCC, in the within-language scenario, the performance of these SSL speech pre-trained models on AF probing tasks achieved a maximum relative increase of 34.4%, and it resulted in the lowest PER of 10.2%. In the cross-language scenario, the maximum relative increase of 26.7% also resulted in the lowest PER of 23.0%.
A Simple Baseline for Domain Adaptation in End to End ASR Systems Using Synthetic Data
Authors: Raviraj Joshi, Anupam Singh
Date: 2022-06-22 00:00:00
Proceeding: ECNLP (ACL) 2022 5
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Domain Adaptation, speech-recognition, Speech Recognition, text-to-speech, Text to Speech

Automatic Speech Recognition(ASR) has been dominated by deep learning-based end-to-end speech recognition models. These approaches require large amounts of labeled data in the form of audio-text pairs. Moreover, these models are more susceptible to domain shift as compared to traditional models. It is common practice to train generic ASR models and then adapt them to target domains using comparatively smaller data sets. We consider a more extreme case of domain adaptation where text-only corpus is available. In this work, we propose a simple baseline technique for domain adaptation in end-to-end speech recognition models. We convert the text-only corpus to audio data using single speaker Text to Speech (TTS) engine. The parallel data in the target domain is then used to fine-tune the final dense layer of generic ASR models. We show that single speaker synthetic TTS data coupled with final dense layer only fine-tuning provides reasonable improvements in word error rates. We use text data from address and e-commerce search domains to show the effectiveness of our low-cost baseline approach on CTC and attention-based models.
A Comprehensive Survey on Video Saliency Detection with Auditory Information: the Audio-visual Consistency Perceptual is the Key!
Authors: Chenglizhao Chen, Mengke Song, Wenfeng Song, Li Guo, Muwei Jian
Date: 2022-06-20 00:00:00
Tasks: Saliency Detection, Video Saliency Detection

Video saliency detection (VSD) aims at fast locating the most attractive objects/things/patterns in a given video clip. Existing VSD-related works have mainly relied on the visual system but paid less attention to the audio aspect, while, actually, our audio system is the most vital complementary part to our visual system. Also, audio-visual saliency detection (AVSD), one of the most representative research topics for mimicking human perceptual mechanisms, is currently in its infancy, and none of the existing survey papers have touched on it, especially from the perspective of saliency detection. Thus, the ultimate goal of this paper is to provide an extensive review to bridge the gap between audio-visual fusion and saliency detection. In addition, as another highlight of this review, we have provided a deep insight into key factors which could directly determine the performances of AVSD deep models, and we claim that the audio-visual consistency degree (AVC) -- a long-overlooked issue, can directly influence the effectiveness of using audio to benefit its visual counterpart when performing saliency detection. Moreover, in order to make the AVC issue more practical and valuable for future followers, we have newly equipped almost all existing publicly available AVSD datasets with additional frame-wise AVC labels. Based on these upgraded datasets, we have conducted extensive quantitative evaluations to ground our claim on the importance of AVC in the AVSD task. In a word, both our ideas and new sets serve as a convenient platform with preliminaries and guidelines, all of which are very potential to facilitate future works in promoting state-of-the-art (SOTA) performance further.
A Generalized Probabilistic Monitoring Model with Both Random and Sequential Data
Authors: Wanke Yu, Min Wu, Biao Huang, Chengda Lu
Date: 2022-06-27 00:00:00

Many multivariate statistical analysis methods and their corresponding probabilistic counterparts have been adopted to develop process monitoring models in recent decades. However, the insightful connections between them have rarely been studied. In this study, a generalized probabilistic monitoring model (GPMM) is developed with both random and sequential data. Since GPMM can be reduced to various probabilistic linear models under specific restrictions, it is adopted to analyze the connections between different monitoring methods. Using expectation maximization (EM) algorithm, the parameters of GPMM are estimated for both random and sequential cases. Based on the obtained model parameters, statistics are designed for monitoring different aspects of the process system. Besides, the distributions of these statistics are rigorously derived and proved, so that the control limits can be calculated accordingly. After that, contribution analysis methods are presented for identifying faulty variables once the process anomalies are detected. Finally, the equivalence between monitoring models based on classical multivariate methods and their corresponding probabilistic graphic models is further investigated. The conclusions of this study are verified using a numerical example and the Tennessee Eastman (TE) process. Experimental results illustrate that the proposed monitoring statistics are subject to their corresponding distributions, and they are equivalent to statistics in classical deterministic models under specific restrictions.
Safe, Learning-Based MPC for Highway Driving under Lane-Change Uncertainty: A Distributionally Robust Approach
Authors: Mathijs Schuurmans, Alexander Katriniok, Christopher Meissen, H. Eric Tseng, Panagiotis Patrinos
Date: 2022-06-27 00:00:00
Tasks: Model Predictive Control, Motion Planning

We present a case study applying learning-based distributionally robust model predictive control to highway motion planning under stochastic uncertainty of the lane change behavior of surrounding road users. The dynamics of road users are modelled using Markov jump systems, in which the switching variable describes the desired lane of the vehicle under consideration and the continuous state describes the pose and velocity of the vehicles. We assume the switching probabilities of the underlying Markov chain to be unknown. As the vehicle is observed and thus, samples from the Markov chain are drawn, the transition probabilities are estimated along with an ambiguity set which accounts for misestimations of these probabilities. Correspondingly, a distributionally robust optimal control problem is formulated over a scenario tree, and solved in receding horizon. As a result, a motion planning procedure is obtained which through observation of the target vehicle gradually becomes less conservative while avoiding overconfidence in estimates obtained from small sample sizes. We present an extensive numerical case study, comparing the effects of several different design aspects on the controller performance and safety.
State and Input Constrained Model Reference Adaptive Control
Authors: Poulomee Ghosh, Shubhendu Bhasin
Date: 2022-06-27 00:00:00
Tasks: model

Satisfaction of state and input constraints is one of the most critical requirements in control engineering applications. In classical model reference adaptive control (MRAC) formulation, although the states and the input remain bounded, the bound is neither user-defined nor known a-priori. In this paper, an MRAC is developed for multivariable linear time-invariant (LTI) plant with user-defined state and input constraints using a simple saturated control design coupled with a barrier Lyapunov function (BLF). Without any restrictive assumptions that may limit practical implementation, the proposed controller guarantees that both the plant state and the control input remain within a user-defined safe set for all time while simultaneously ensuring that the plant state trajectory tracks the reference model trajectory. The controller ensures that all the closed-loop signals remain bounded and the trajectory tracking error converges to zero asymptotically. Simulation results validate the efficacy of the proposed constrained MRAC in terms of better tracking performance and limited control effort compared to the standard MRAC algorithm.
Attitude estimation from vector measurements: Necessary and sufficient conditions and convergent observer design
Authors: Bowen Yi, Lei Wang, Ian R. Manchester
Date: 2022-06-27 00:00:00

The paper addresses the problem of attitude estimation for rigid bodies using (possibly time-varying) vector measurements, for which we provide a necessary and sufficient condition of distinguishability. Such a condition is shown to be strictly weaker than those previously used for attitude observer design. Thereafter, we show that even for the single vector case the resulting condition is sufficient to design almost globally convergent attitude observers, and two explicit designs are obtained. To overcome the weak excitation issue, the first design employs to make full use of historical information, whereas the second scheme dynamically generates a virtual reference vector, which remains non-collinear to the given vector measurement. Simulation results illustrate the accurate estimation despite noisy measurements.
Safe Schedule Verification for Urban Air Mobility Networks with Node Closures
Authors: Qinshuang Wei, Gustav Nilsson, Samuel Coogan
Date: 2022-06-27 00:00:00

In Urban Air Mobility (UAM) networks, takeoff and landing sites, called vertiports, are likely to experience intermittent closures due to, e.g., adverse weather. To ensure safety, all in-flight Urban Air Vehicles (UAVs) in a UAM network must therefore have alternative landing sites with sufficient landing capacity in the event of a vertiport closure. In this paper, we study the problem of safety verification of UAM schedules in the face of vertiport closures. We first provide necessary and sufficient conditions for a given UAM schedule to be safe in the sense that, if a vertiport closure occurs, then all UAVs will be able to safely land at a backup landing site. Next, we convert these conditions to an efficient algorithm for verifying safety of a UAM schedule via a linear program by using properties of totally unimodular matrices. Our algorithm allows for uncertain travel time between UAM vertiports and scales quadratically with the number of scheduled UAVs. We demonstrate our algorithm on a UAM network with up to 1,000 UAVs.
Energy Circuit-based Integrated Energy Management System: Theory, Implementation, and Application
Authors: Binbin Chen, Qinglai Guo, Guanxiong Yin, Bin Wang, Zhaoguang Pan, Yuwei Chen, Wenchuan Wu, Hongbin Sun
Date: 2022-06-26 00:00:00
Tasks: energy management, Management, State Estimation

Integrated energy systems (IESs), in which various energy flows are interconnected and coordinated to release potential flexibility for more efficient and secure operation, have drawn increasing attention in recent years. In this article, an integrated energy management system (IEMS) that performs online analysis and optimization on coupling energy flows in an IES is comprehensively introduced. From the theory perspective, an energy circuit method (ECM) that models natural gas networks and heating networks in the frequency domain is discussed. This method extends the electric circuit modeling of power systems to IESs and enables the IEMS to manage large-scale IESs. From the implementation perspective, the architecture design and function development of the IEMS are presented. Tutorial examples with illustrative case studies are provided to demonstrate its functions of dynamic state estimation, energy flow analysis, security assessment and control, and optimal energy flow. From the application perspective, real-world engineering demonstrations that apply IEMSs in managing building-scale, park-scale, and city-scale IESs are reported. The economic and environmental benefits obtained in these demonstration projects indicate that the IEMS has broad application prospects for a low/zero-carbon future energy system.
An Efficient Optimal Energy Flow Model for Integrated Energy Systems Based on Energy Circuit Modeling in the Frequency Domain
Authors: Binbin Chen, Wenchuan Wu, Qinglai Guo, Hongbin Sun
Date: 2022-06-26 00:00:00

With more energy networks being interconnected to form integrated energy systems (IESs), the optimal energy flow (OEF) problem has drawn increasing attention. Extant studies on OEF models mostly utilize the finite difference method (FDM) to address partial-differential-equation (PDE) constraints related to the dynamics in natural gas networks (NGNs) and district heating networks (DHNs). However, this time-domain approach suffers from a heavy computational burden with regard to achieving high finite-difference accuracy. In this paper, a novel OEF model that formulates NGN and DHN constraints in the frequency domain and corresponding model compaction techniques for efficient solving are contributed. First, an energy circuit method (ECM) that algebraizes the PDEs of NGNs and DHNs in the frequency domain is introduced. Then, an ECM-based OEF model is formulated, which contains fewer variables and constraints than an FDM-based OEF model and thereby yields better solving efficiency. Finally, variable space projection is employed to remove implicit variables, by which another constraint generation algorithm is enabled to remove redundant constraints. These two techniques further compact the OEF model and bring about a second improvement in solving efficiency. Numerical tests on actual systems indicate the final OEF model reduces variables and constraints by more than 95% and improves the solving efficiency by more than 10 times. In conclusion, the proposed OEF model and solving techniques well meet the optimization needs of large-scale IESs.
A Planning-free Longitudinal Controller Design for Vehicles in Dynamic Traffic Environments
Authors: Wubing B. Qin
Date: 2022-06-25 00:00:00

This paper investigates the longitudinal control problem in a dynamic traffic environment where driving scenarios change between free-driving scenarios and car-following scenarios. A comprehensive longitudinal controller is proposed to ensure reasonable transient response and steady-state response in scenarios changes, which is independent of planning algorithms. This design takes into account passenger comfort, safety concerns and disturbance rejections, and attempts to meet the requirement of lower cost, faster response, increased comfort, enhanced safety and elevated extendability from the automated vehicle industry. Design insights and intuitions are provided in detail. Comprehensive simulations are conducted to demonstrate the efficacy of the proposed controller in different driving scenarios.