pwc-archive/papers-with-abstracts huggingface dataset:
Multimodal Estimation of Change Points of Physiological Arousal in Drivers
Authors: Kleanthis Avramidis, Tiantian Feng, Digbalay Bose, Shrikanth Narayanan
Date: 2022-10-28 00:00:00
Tasks: Time Series, Time Series Analysis

Detecting unsafe driving states, such as stress, drowsiness, and fatigue, is an important component of ensuring driving safety and an essential prerequisite for automatic intervention systems in vehicles. These concerning conditions are primarily connected to the driver's low or high arousal levels. In this study, we describe a framework for processing multimodal physiological time-series from wearable sensors during driving and locating points of prominent change in drivers' physiological arousal state. These points of change could potentially indicate events that require just-in-time intervention. We apply time-series segmentation on heart rate and breathing rate measurements and quantify their robustness in capturing change points in electrodermal activity, treated as a reference index for arousal, as well as on self-reported stress ratings, using three public datasets. Our experiments demonstrate that physiological measures are veritable indicators of change points of arousal and perform robustly across an extensive ablation study.
A Rosenbrock framework for tangential interpolation of port-Hamiltonian descriptor systems
Authors: Tim Moser, Boris Lohmann
Date: 2022-10-28 00:00:00
Tasks: Form

We present a new structure-preserving model order reduction (MOR) framework for large-scale port-Hamiltonian descriptor systems (pH-DAEs). Our method exploits the structural properties of the Rosenbrock system matrix for this system class and utilizes condensed forms which often arise in applications and reveal the solution behaviour of a system. Provided that the original system has such a form, our method produces reduced-order models (ROMs) of minimal dimension, which tangentially interpolate the original model's transfer function and are guaranteed to be again in pH-DAE form. This allows the ROM to be safely coupled with other dynamical systems when modelling large system networks, which is useful, for instance, in electric circuit simulation.
Target-Speaker Voice Activity Detection via Sequence-to-Sequence Prediction
Authors: Ming Cheng, Weiqing Wang, Yucong Zhang, Xiaoyi Qin, Ming Li
Date: 2022-10-28 00:00:00
Tasks: Action Detection, Activity Detection, speaker-diarization, Speaker Diarization

Target-speaker voice activity detection is currently a promising approach for speaker diarization in complex acoustic environments. This paper presents a novel Sequence-to-Sequence Target-Speaker Voice Activity Detection (Seq2Seq-TSVAD) method that can efficiently address the joint modeling of large-scale speakers and predict high-resolution voice activities. Experimental results show that larger speaker capacity and higher output resolution can significantly reduce the diarization error rate (DER), which achieves the new state-of-the-art performance of 4.55% on the VoxConverse test set and 10.77% on Track 1 of the DIHARD-III evaluation set under the widely-used evaluation metrics.
Parameter-efficient transfer learning of pre-trained Transformer models for speaker verification using adapters
Authors: Junyi Peng, Themos Stafylakis, Rongzhi Gu, Oldřich Plchot, Ladislav Mošner, Lukáš Burget, Jan Černocký
Date: 2022-10-28 00:00:00
Tasks: Speaker Verification, Transfer Learning

Recently, the pre-trained Transformer models have received a rising interest in the field of speech processing thanks to their great success in various downstream tasks. However, most fine-tuning approaches update all the parameters of the pre-trained model, which becomes prohibitive as the model size grows and sometimes results in overfitting on small datasets. In this paper, we conduct a comprehensive analysis of applying parameter-efficient transfer learning (PETL) methods to reduce the required learnable parameters for adapting to speaker verification tasks. Specifically, during the fine-tuning process, the pre-trained models are frozen, and only lightweight modules inserted in each Transformer block are trainable (a method known as adapters). Moreover, to boost the performance in a cross-language low-resource scenario, the Transformer model is further tuned on a large intermediate dataset before directly fine-tuning it on a small dataset. With updating fewer than 4% of parameters, (our proposed) PETL-based methods achieve comparable performances with full fine-tuning methods (Vox1-O: 0.55%, Vox1-E: 0.82%, Vox1-H:1.73%).
Laugh Betrays You? Learning Robust Speaker Representation From Speech Containing Non-Verbal Fragments
Authors: Yuke Lin, Xiaoyi Qin, Huahua Cui, Zhenyi Zhu, Ming Li
Date: 2022-10-28 00:00:00
Tasks: Speaker Verification

The success of automatic speaker verification shows that discriminative speaker representations can be extracted from neutral speech. However, as a kind of non-verbal voice, laughter should also carry speaker information intuitively. Thus, this paper focuses on exploring speaker verification about utterances containing non-verbal laughter segments. We collect a set of clips with laughter components by conducting a laughter detection script on VoxCeleb and part of the CN-Celeb dataset. To further filter untrusted clips, probability scores are calculated by our binary laughter detection classifier, which is pre-trained by pure laughter and neutral speech. After that, based on the clips whose scores are over the threshold, we construct trials under two different evaluation scenarios: Laughter-Laughter (LL) and Speech-Laughter (SL). Then a novel method called Laughter-Splicing based Network (LSN) is proposed, which can significantly boost performance in both scenarios and maintain the performance on the neutral speech, such as the VoxCeleb1 test set. Specifically, our system achieves relative 20% and 22% improvement on Laughter-Laughter and Speech-Laughter trials, respectively. The meta-data and sample clips have been released at https://github.com/nevermoreLin/Laugh_LSN.
SG-VAD: Stochastic Gates Based Speech Activity Detection
Authors: Jonathan Svirsky, Ofir Lindenbaum
Date: 2022-10-28 00:00:00
Tasks: Action Detection, Activity Detection, Denoising

We propose a novel voice activity detection (VAD) model in a low-resource environment. Our key idea is to model VAD as a denoising task, and construct a network that is designed to identify nuisance features for a speech classification task. We train the model to simultaneously identify irrelevant features while predicting the type of speech event. Our model contains only 7.8K parameters, outperforms the previously proposed methods on the AVA-Speech evaluation set, and provides comparative results on the HAVIC dataset. We present its architecture, experimental results, and ablation study on the model's components. We publish the code and the models here https://www.github.com/jsvir/vad.
Dysfluencies Seldom Come Alone -- Detection as a Multi-Label Problem
Authors: Sebastian P. Bayerl, Dominik Wagner, Florian Hönig, Tobias Bocklet, Elmar Nöth, Korbinian Riedhammer
Date: 2022-10-28 00:00:00
Tasks: Multi-class Classification, speech-recognition, Speech Recognition

Specially adapted speech recognition models are necessary to handle stuttered speech. For these to be used in a targeted manner, stuttered speech must be reliably detected. Recent works have treated stuttering as a multi-class classification problem or viewed detecting each dysfluency type as an isolated task; that does not capture the nature of stuttering, where one dysfluency seldom comes alone, i.e., co-occurs with others. This work explores an approach based on a modified wav2vec 2.0 system for end-to-end stuttering detection and classification as a multi-label problem. The method is evaluated on combinations of three datasets containing English and German stuttered speech, yielding state-of-the-art results for stuttering detection on the SEP-28k-Extended dataset. Experimental results provide evidence for the transferability of features and the generalizability of the method across datasets and languages.
End-to-end Ensemble-based Feature Selection for Paralinguistics Tasks
Authors: Tamás Grósz, Mittul Singh, Sudarsana Reddy Kadiri, Hemant Kathania, Mikko Kurimo
Date: 2022-10-28 00:00:00
Tasks: Feature Engineering, feature selection

The events of recent years have highlighted the importance of telemedicine solutions which could potentially allow remote treatment and diagnosis. Relatedly, Computational Paralinguistics, a unique subfield of Speech Processing, aims to extract information about the speaker and form an important part of telemedicine applications. In this work, we focus on two paralinguistic problems: mask detection and breathing state prediction. Solutions developed for these tasks could be invaluable and have the potential to help monitor and limit the spread of a virus like COVID-19. The current state-of-the-art methods proposed for these tasks are ensembles based on deep neural networks like ResNets in conjunction with feature engineering. Although these ensembles can achieve high accuracy, they also have a large footprint and require substantial computational power reducing portability to devices with limited resources. These drawbacks also mean that the previously proposed solutions are infeasible to be used in a telemedicine system due to their size and speed. On the other hand, employing lighter feature-engineered systems can be laborious and add further complexity making them difficult to create a deployable system quickly. This work proposes an ensemble-based automatic feature selection method to enable the development of fast and memory-efficient systems. In particular, we propose an output-gradient-based method to discover essential features using large, well-performing ensembles before training a smaller one. In our experiments, we observed considerable (25-32%) reductions in inference times using neural network ensembles based on output-gradient-based features. Our method offers a simple way to increase the speed of the system and enable real-time usage while maintaining competitive results with larger-footprint ensemble using all spectral features.
Influence of Utterance and Speaker Characteristics on the Classification of Children with Cleft Lip and Palate
Authors: Ilja Baumann, Dominik Wagner, Franziska Braun, Sebastian P. Bayerl, Elmar Nöth, Korbinian Riedhammer, Tobias Bocklet
Date: 2022-10-28 00:00:00
Tasks: Binary Classification, Classification

Recent findings show that pre-trained wav2vec 2.0 models are reliable feature extractors for various speaker characteristics classification tasks. We show that latent representations extracted at different layers of a pre-trained wav2vec 2.0 system can be used as features for binary classification to distinguish between children with Cleft Lip and Palate (CLP) and a healthy control group. The results indicate that the distinction between CLP and healthy voices, especially with latent representations from the lower and middle encoder layers, reaches an accuracy of 100%. We test the classifier to find influencing factors for classification using unseen out-of-domain healthy and pathologic corpora with varying characteristics: age, spoken content, and acoustic conditions. Cross-pathology and cross-healthy tests reveal that the trained classifiers are unreliable if there is a mismatch between training and out-of-domain test data in, e.g., age, spoken content, or acoustic conditions.
Speaker recognition with two-step multi-modal deep cleansing
Authors: Ruijie Tao, Kong Aik Lee, Zhan Shi, Haizhou Li
Date: 2022-10-28 00:00:00
Tasks: Representation Learning, Speaker Recognition, Vocal Bursts Valence Prediction

Neural network-based speaker recognition has achieved significant improvement in recent years. A robust speaker representation learns meaningful knowledge from both hard and easy samples in the training set to achieve good performance. However, noisy samples (i.e., with wrong labels) in the training set induce confusion and cause the network to learn the incorrect representation. In this paper, we propose a two-step audio-visual deep cleansing framework to eliminate the effect of noisy labels in speaker representation learning. This framework contains a coarse-grained cleansing step to search for the peculiar samples, followed by a fine-grained cleansing step to filter out the noisy labels. Our study starts from an efficient audio-visual speaker recognition system, which achieves a close to perfect equal-error-rate (EER) of 0.01\%, 0.07\% and 0.13\% on the Vox-O, E and H test sets. With the proposed multi-modal cleansing mechanism, four different speaker recognition networks achieve an average improvement of 5.9\%. Code has been made available at: \textcolor{magenta}{\url{https://github.com/TaoRuijie/AVCleanse}}.
UX-NET: Filter-and-Process-based Improved U-Net for Real-time Time-domain Audio Separation
Authors: Kashyap Patel, Anton Kovalyov, Issa Panahi
Date: 2022-10-28 00:00:00

This study presents UX-Net, a time-domain audio separation network (TasNet) based on a modified U-Net architecture. The proposed UX-Net works in real-time and handles either single or multi-microphone input. Inspired by the filter-and-process-based human auditory behavior, the proposed system introduces novel mixer and separation modules, which result in cost and memory efficient modeling of speech sources. The mixer module combines encoded input in a latent feature space and outputs a desired number of output streams. Then, in the separation module, a modified U-Net (UX) block is applied. The UX block first filters the encoded input at various resolutions followed by aggregating the filtered information and applying recurrent processing to estimate masks of separated sources. The letter 'X' in UX-Net is a name placeholder for the type of recurrent layer employed in the UX block. Empirical findings on the WSJ0-2mix benchmark dataset show that one of the UX-Net configurations outperforms the state-of-the-art Conv-TasNet system by 0.85 dB SI-SNR while using only 16% of the model parameters, 58% fewer computations, and maintaining low latency.
Conditioning and Sampling in Variational Diffusion Models for Speech Super-Resolution
Authors: Chin-Yun Yu, Sung-Lin Yeh, György Fazekas, Hao Tang
Date: 2022-10-27 00:00:00
Tasks: Super-Resolution

Recently, diffusion models (DMs) have been increasingly used in audio processing tasks, including speech super-resolution (SR), which aims to restore high-frequency content given low-resolution speech utterances. This is commonly achieved by conditioning the network of noise predictor with low-resolution audio. In this paper, we propose a novel sampling algorithm that communicates the information of the low-resolution audio via the reverse sampling process of DMs. The proposed method can be a drop-in replacement for the vanilla sampling process and can significantly improve the performance of the existing works. Moreover, by coupling the proposed sampling method with an unconditional DM, i.e., a DM with no auxiliary inputs to its noise predictor, we can generalize it to a wide range of SR setups. We also attain state-of-the-art results on the VCTK Multi-Speaker benchmark with this novel formulation.
Proceedings of the ACII Affective Vocal Bursts Workshop and Competition 2022 (A-VB): Understanding a critically understudied modality of emotional expression
Authors: Alice Baird, Panagiotis Tzirakis, Jeffrey A. Brooks, Christopher B. Gregory, Björn Schuller, Anton Batliner, Dacher Keltner, Alan Cowen
Date: 2022-10-27 00:00:00

This is the Proceedings of the ACII Affective Vocal Bursts Workshop and Competition (A-VB). A-VB was a workshop-based challenge that introduces the problem of understanding emotional expression in vocal bursts -- a wide range of non-verbal vocalizations that includes laughs, grunts, gasps, and much more. With affective states informing both mental and physical wellbeing, the core focus of the A-VB workshop was the broader discussion of current strategies in affective computing for modeling vocal emotional expression. Within this first iteration of the A-VB challenge, the participants were presented with four emotion-focused sub-challenges that utilize the large-scale and `in-the-wild' Hume-VB dataset. The dataset and the four sub-challenges draw attention to new innovations in emotion science as it pertains to vocal expression, addressing low- and high-dimensional theories of emotional expression, cultural variation, and `call types' (laugh, cry, sigh, etc.).
Nonlinear Stability of Complex Droop Control in Converter-Based Power Systems
Authors: Xiuqiang He, Verena Häberle, Irina Subotić, Florian Dörfler
Date: 2022-10-28 00:00:00

In this letter, we study the nonlinear stability problem of converter-based power systems, where the converter dynamics are governed by a complex droop control. This complex droop control augments the well-known power-frequency (p-f) droop control, and it proves to be equivalent to the state-of-the-art dispatchable virtual oscillator control (dVOC). In this regard, it is recognized as a promising grid-forming solution to address the high penetration of converters in future power systems. In previous work, the global stability of dVOC (i.e., complex droop control) has been proven by prespecifying a nominal synchronous steady state. For a general case of non-nominal (i.e., drooped) synchronous steady states, however, the stability problem requires further investigation. In this letter, we provide parametric conditions under which a non-nominal synchronous steady state exists and the system is almost globally asymptotically stable with respect to this non-nominal synchronous steady state.
A Framework for Output-Feedback Symbolic Control
Authors: Mahmoud Khaled, Kuize Zhang, Majid Zamani
Date: 2020-11-30 00:00:00

Symbolic control is an abstraction-based controller synthesis approach that provides, algorithmically, certifiable-by-construction controllers for cyber-physical systems. Symbolic control approaches usually assume that full-state information is available which is not suitable for many real-world applications with partially-observable states or output information. This article introduces a framework for output-feedback symbolic control. We propose relations between original systems and their symbolic models based on outputs. They enable designing symbolic controllers and refining them to enforce complex requirements on original systems. We provide example methodologies to synthesize and refine output-feedback symbolic controllers.
Magnetorheological Axisymmetric Actuator with Permanent Magnet
Authors: Jakub Bernat, Paulina Superczynska, Piotr Gajewski, Agnieszka Marcinkowska
Date: 2022-10-28 00:00:00

This study examines the concept of axisymmetric actuator based on the magnetorheological membrane, electromagnet and permanent magnet. The construction of the actuator enables its application in wide range of practical devices like pumping, loudspeaker or varying-stiffness button. This work will highlight its working principle especially the influence of permanent magnet. Furthermore, the model of devices will be defined relaying on the Hammerstein model. To show the properties of the actuator and to perform the model identification, the set of experiments was run taking into account static and dynamic working conditions.
Brain Modeling for Control: A Review
Authors: Gagan Acharya, Sebastian F. Ruf, Erfan Nozari
Date: 2022-10-28 00:00:00

Neurostimulation technologies have seen a recent surge in interest from the neuroscience and controls communities alike due to their proven potential to treat conditions such as Parkinson's Disease, and depression. The provided stimulation can be of different types, such as electric, and optogenetic, and is generally applied to a specific region of the brain in order to drive the local and/or global dynamics to a desired state of (in)activity. However, an underlying theoretical understanding of the efficacy of neurostimulation is still lacking. From a control-theoretic perspective, it is important to understand how each stimulus modality interacts with the complex brain network in order to assess the controllability of the system and develop neurophysiologically relevant computational models that can be used to design the stimulation profile in a closed-loop manner. In this paper, we review the computational modeling studies of (i) deep brain stimulation, (ii) transcranial magnetic stimulation, (iii) direct current stimulation, (iv) transcranial electrical stimulation, and (v) optogenetics as five of the most popular neurostimulation technologies in research and clinical settings. For each technology, we split the reviewed studies into (a)theory-driven biophysical models capturing the low-level physics of the interactions between the stimulation source and neuronal tissue, (b) data-driven stimulus-response models which capture the end-to-end effects of stimulation on various biomarkers of interest and (c) data-driven dynamical system models that extract the precise dynamics of the brain's response to neurostimulation from neural data. While our focus is particularly on the latter category due to their greater utility in control design, we review key works in the former two categories as the basis and context in which dynamical system models have been and will be developed.
Data-driven Output Regulation via Gaussian Processes and Luenberger Internal Models
Authors: Lorenzo Gentilini, Michelangelo Bin, Lorenzo Marconi
Date: 2022-10-28 00:00:00
Tasks: Gaussian Processes

This paper deals with the problem of adaptive output regulation for multivariable nonlinear systems by presenting a learning-based adaptive internal model-based design strategy. The approach builds on the recently proposed adaptive internal model design techniques based on the theory of nonlinear Luenberger observers, and the adaptation side is approached as a probabilistic regression problem. In particular, Gaussian process priors are employed to cope with the learning problem. Unlike the previous approaches in the field, here only coarse assumptions about the friend structure are required, making the proposed approach suitable for applications where the exosystem is highly uncertain. The paper presents performance bounds on the attained regulation error and numerical simulations showing how the proposed method outperforms previous approaches.
Dynamic Event-Triggered Discrete-Time Linear Time-Varying System with Privacy-Preservation
Authors: Xuefeng Yang, Li Liu, Wenju Zhou, Jing Shi, Yinggang Zhang, Xin Hu, Huiyu Zhou
Date: 2022-10-28 00:00:00
Tasks: Privacy Preserving

This paper focuses on discrete-time wireless sensor networks with privacy-preservation. In practical applications, information exchange between sensors is subject to attacks. For the information leakage caused by the attack during the information transmission process, privacy-preservation is introduced for system states. To make communication resources more effectively utilized, a dynamic event-triggered set-membership estimator is designed. Moreover, the privacy of the system is analyzed to ensure the security of the real data. As a result, the set-membership estimator with differential privacy is analyzed using recursive convex optimization. Then the steady-state performance of the system is studied. Finally, one example is presented to demonstrate the feasibility of the proposed distributed filter containing privacy-preserving analysis.