pwc-archive/papers-with-abstracts huggingface dataset:
Localizing Spatial Information in Neural Spatiospectral Filters
Authors: Annika Briegleb, Thomas Haubner, Vasileios Belagiannis, Walter Kellermann
Date: 2023-03-14 00:00:00
Tasks: Speech Enhancement

Beamforming for multichannel speech enhancement relies on the estimation of spatial characteristics of the acoustic scene. In its simplest form, the delay-and-sum beamformer (DSB) introduces a time delay to all channels to align the desired signal components for constructive superposition. Recent investigations of neural spatiospectral filtering revealed that these filters can be characterized by a beampattern similar to one of traditional beamformers, which shows that artificial neural networks can learn and explicitly represent spatial structure. Using the Complex-valued Spatial Autoencoder (COSPA) as an exemplary neural spatiospectral filter for multichannel speech enhancement, we investigate where and how such networks represent spatial information. We show via clustering that for COSPA the spatial information is represented by the features generated by a gated recurrent unit (GRU) layer that has access to all channels simultaneously and that these features are not source -- but only direction of arrival-dependent.
Native Multi-Band Audio Coding within Hyper-Autoencoded Reconstruction Propagation Networks
Authors: Darius Petermann, Inseon Jang, Minje Kim
Date: 2023-03-14 00:00:00
Tasks: Bandwidth Extension

Spectral sub-bands do not portray the same perceptual relevance. In audio coding, it is therefore desirable to have independent control over each of the constituent bands so that bitrate assignment and signal reconstruction can be achieved efficiently. In this work, we present a novel neural audio coding network that natively supports a multi-band coding paradigm. Our model extends the idea of compressed skip connections in the U-Net-based codec, allowing for independent control over both core and high band-specific reconstructions and bit allocation. Our system reconstructs the full-band signal mainly from the condensed core-band code, therefore exploiting and showcasing its bandwidth extension capabilities to its fullest. Meanwhile, the low-bitrate high-band code helps the high-band reconstruction similarly to MPEG audio codecs' spectral bandwidth replication. MUSHRA tests show that the proposed model not only improves the quality of the core band by explicitly assigning more bits to it but retains a good quality in the high-band as well.
Multi-Channel Masking with Learnable Filterbank for Sound Source Separation
Authors: Wang Dai, Archontis Politis, Tuomas Virtanen
Date: 2023-03-14 00:00:00

This work proposes a learnable filterbank based on a multi-channel masking framework for multi-channel source separation. The learnable filterbank is a 1D Conv layer, which transforms the raw waveform into a 2D representation. In contrast to the conventional single-channel masking method, we estimate a mask for each individual microphone channel. The estimated masks are then applied to the transformed waveform representation like in the traditional filter-and-sum beamforming operation. Specifically, each mask is used to multiply the corresponding channel's 2D representation, and the masked output of all channels are then summed. At last, a 1D transposed Conv layer is used to convert the summed masked signal into the waveform domain. The experimental results show our method outperforms single-channel masking with a learnable filterbank and can outperform multi-channel complex masking with STFT complex spectrum in the STGCSEN model if a learnable filterbank is transformed to a higher feature dimension. The spatial response analysis also verifies that multi-channel masking in the learnable filterbank domain has spatial selectivity.
Detecting post-stroke aphasia using EEG-based neural envelope tracking of natural speech
Authors: Pieter De Clercq, Jill Kries, Ramtin Mehraram, Jonas Vanthornhout, Tom Francart, Maaike Vandermosten
Date: 2023-03-14 00:00:00
Tasks: Diagnostic, EEG, Electroencephalogram (EEG)

[Objective]. After a stroke, one-third of patients suffer from aphasia, a language disorder that impairs communication ability. The standard behavioral tests used to diagnose aphasia are time-consuming and have low ecological validity. Neural tracking of the speech envelope is a promising tool for investigating brain responses to natural speech. The speech envelope is crucial for speech understanding, encompassing cues for processing linguistic units. In this study, we aimed to test the potential of the neural envelope tracking technique for detecting language impairments in individuals with aphasia (IWA). [Approach]. We recorded EEG from 27 IWA in the chronic phase after stroke and 22 controls while they listened to a story. We quantified neural envelope tracking in a broadband frequency range as well as in the delta, theta, alpha, beta, and gamma frequency bands using mutual information analysis. Besides group differences in neural tracking measures, we also tested its suitability for detecting aphasia using a Support Vector Machine (SVM) classifier. We further investigated the required recording length for the SVM to detect aphasia and to obtain reliable outcomes. [Results]. IWA displayed decreased neural envelope tracking compared to controls in the broad, delta, theta, and gamma band. Neural tracking in these frequency bands effectively captured aphasia at the individual level (SVM accuracy 84%, AUC 88%). High-accuracy and reliable detection could be obtained with 5-7 minutes of recording time. [Significance]. Our study shows that neural tracking of speech is an effective biomarker for aphasia. We demonstrated its potential as a diagnostic tool with high reliability, individual-level detection of aphasia, and time-efficient assessment. This work represents a significant step towards more automatic, objective, and ecologically valid assessments of language impairments in aphasia.
TEA-PSE 3.0: Tencent-Ethereal-Audio-Lab Personalized Speech Enhancement System For ICASSP 2023 DNS Challenge
Authors: Yukai Ju, Jun Chen, Shimin Zhang, Shulin He, Wei Rao, Weixin Zhu, Yannan Wang, Tao Yu, Shidong Shang
Date: 2023-03-14 00:00:00
Tasks: Speech Enhancement

This paper introduces the Unbeatable Team's submission to the ICASSP 2023 Deep Noise Suppression (DNS) Challenge. We expand our previous work, TEA-PSE, to its upgraded version -- TEA-PSE 3.0. Specifically, TEA-PSE 3.0 incorporates a residual LSTM after squeezed temporal convolution network (S-TCN) to enhance sequence modeling capabilities. Additionally, the local-global representation (LGR) structure is introduced to boost speaker information extraction, and multi-STFT resolution loss is used to effectively capture the time-frequency characteristics of the speech signals. Moreover, retraining methods are employed based on the freeze training strategy to fine-tune the system. According to the official results, TEA-PSE 3.0 ranks 1st in both ICASSP 2023 DNS-Challenge track 1 and track 2.
Lightweight feature encoder for wake-up word detection based on self-supervised speech representation
Authors: Hyungjun Lim, Younggwan Kim, Kiho Yeom, Eunjoo Seo, Hoodong Lee, Stanley Jungkyu Choi, Honglak Lee
Date: 2023-03-14 00:00:00
Tasks: Dimensionality Reduction, Self-Supervised Learning

Self-supervised learning method that provides generalized speech representations has recently received increasing attention. Wav2vec 2.0 is the most famous example, showing remarkable performance in numerous downstream speech processing tasks. Despite its success, it is challenging to use it directly for wake-up word detection on mobile devices due to its expensive computational cost. In this work, we propose LiteFEW, a lightweight feature encoder for wake-up word detection that preserves the inherent ability of wav2vec 2.0 with a minimum scale. In the method, the knowledge of the pre-trained wav2vec 2.0 is compressed by introducing an auto-encoder-based dimensionality reduction technique and distilled to LiteFEW. Experimental results on the open-source "Hey Snips" dataset show that the proposed method applied to various model structures significantly improves the performance, achieving over 20% of relative improvements with only 64k parameters.
Towards Real-Time Single-Channel Speech Separation in Noisy and Reverberant Environments
Authors: Julian Neri, Sebastian Braun
Date: 2023-03-14 00:00:00
Tasks: Decoder, Speech Separation

Real-time single-channel speech separation aims to unmix an audio stream captured from a single microphone that contains multiple people talking at once, environmental noise, and reverberation into multiple de-reverberated and noise-free speech tracks, each track containing only one talker. While large state-of-the-art DNNs can achieve excellent separation from anechoic mixtures of speech, the main challenge is to create compact and causal models that can separate reverberant mixtures at inference time. In this paper, we explore low-complexity, resource-efficient, causal DNN architectures for real-time separation of two or more simultaneous speakers. A cascade of three neural network modules are trained to sequentially perform noise-suppression, separation, and de-reverberation. For comparison, a larger end-to-end model is trained to output two anechoic speech signals directly from noisy reverberant speech mixtures. We propose an efficient single-decoder architecture with subtractive separation for real-time recursive speech separation for two or more speakers. Evaluation on real monophonic recordings of speech mixtures, according to speech separation measures like SI-SDR, perceptual measures like DNS-MOS, and a novel proposed channel separation metric, show that these compact causal models can separate speech mixtures with low latency, and perform on par with large offline state-of-the-art models like SepFormer.
Speech Intelligibility Classifiers from 550k Disordered Speech Samples
Authors: Subhashini Venugopalan, Jimmy Tobin, Samuel J. Yang, Katie Seaver, Richard J. N. Cave, Pan-Pan Jiang, Neil Zeghidour, Rus Heywood, Jordan Green, Michael P. Brenner
Date: 2023-03-13 00:00:00

We developed dysarthric speech intelligibility classifiers on 551,176 disordered speech samples contributed by a diverse set of 468 speakers, with a range of self-reported speaking disorders and rated for their overall intelligibility on a five-point scale. We trained three models following different deep learning approaches and evaluated them on ~94K utterances from 100 speakers. We further found the models to generalize well (without further training) on the TORGO database (100% accuracy), UASpeech (0.93 correlation), ALS-TDI PMP (0.81 AUC) datasets as well as on a dataset of realistic unprompted speech we gathered (106 dysarthric and 76 control speakers,~2300 samples).
Online Binaural Speech Separation of Moving Speakers With a Wavesplit Network
Authors: Cong Han, Nima Mesgarani
Date: 2023-03-13 00:00:00
Tasks: Online Clustering, Speaker Separation, Speech Separation

Binaural speech separation in real-world scenarios often involves moving speakers. Most current speech separation methods use utterance-level permutation invariant training (u-PIT) for training. In inference time, however, the order of outputs can be inconsistent over time particularly in long-form speech separation. This situation which is referred to as the speaker swap problem is even more problematic when speakers constantly move in space and therefore poses a challenge for consistent placement of speakers in output channels. Here, we describe a real-time binaural speech separation model based on a Wavesplit network to mitigate the speaker swap problem for moving speaker separation. Our model computes a speaker embedding for each speaker at each time frame from the mixed audio, aggregates embeddings using online clustering, and uses cluster centroids as speaker profiles to track each speaker throughout the long duration. Experimental results on reverberant, long-form moving multitalker speech separation show that the proposed method is less prone to speaker swap and achieves comparable performance with u-PIT based models with ground truth tracking in both separation accuracy and preserving the interaural cues.
A processing framework to access large quantities of whispered speech found in ASMR
Authors: Pablo Perez Zarazaga, Gustav Eje Henter, Zofia Malisz
Date: 2023-03-13 00:00:00
Tasks: Action Detection, Activity Detection

Whispering is a ubiquitous mode of communication that humans use daily. Despite this, whispered speech has been poorly served by existing speech technology due to a shortage of resources and processing methodology. To remedy this, this paper provides a processing framework that enables access to large and unique data of high-quality whispered speech. We obtain the data from recordings submitted to online platforms as part of the ASMR media-cultural phenomenon. We describe our processing pipeline and a method for improved whispered activity detection (WAD) in the ASMR data. To efficiently obtain labelled, clean whispered speech, we complement the automatic WAD by using Edyson, a bulk audio-annotation tool with human-in-the-loop. We also tackle a problem particular to ASMR: separation of whisper from other acoustic triggers present in the genre. We show that the proposed WAD and the efficient labelling allows to build extensively augmented data and train a classifier that extracts clean whisper segments from ASMR audio. Our large and growing dataset enables whisper-capable, data-driven speech technology and linguistic analysis. It also opens opportunities in e.g. HCI as a resource that may elicit emotional, psychological and neuro-physiological responses in the listener.
Learning for Precision Motion of an Interventional X-ray System: Add-on Physics-Guided Neural Network Feedforward Control
Authors: Johan Kon, Naomi de Vos, Dennis Bruijnen, Jeroen van de Wijdeven, Marcel Heertjes, Tom Oomen
Date: 2023-03-14 00:00:00
Tasks: Friction

Tracking performance of physical-model-based feedforward control for interventional X-ray systems is limited by hard-to-model parasitic nonlinear dynamics, such as cable forces and nonlinear friction. In this paper, these nonlinear dynamics are compensated using a physics-guided neural network (PGNN), consisting of a physical model, embedding prior knowledge of the dynamics, in parallel with a neural network to learn hard-to-model dynamics. To ensure that the neural network learns only unmodelled effects, the neural network output in the subspace spanned by the physical model is regularized via an orthogonal projection-based approach, resulting in complementary physical model and neural network contributions. The PGNN feedforward controller reduces the tracking error of an interventional X-ray system by a factor of 5 compared to an optimally tuned physical model, successfully compensating the unmodeled parasitic dynamics.
On the Impact of Smartification Strategies for the State Estimation of Low Voltage Grids
Authors: Armin Fatemi, Franziska Tischbein, Frank Wirtz, Christin Schmoger, Stefan Dorendorf, Annika Schurtz, David Echternacht, Andreas Ulbig
Date: 2023-03-14 00:00:00
Tasks: State Estimation

The decarbonization of for example the energy or heat sector leads to the transformation of distribution grids. The expansion of decentralized energy resources and the integration of new consumers due to sector coupling (e.g. heat pumps or electric vehicles) into low voltage grids increases the need for grid expansion and usage of flexibilities in the grid. A high observability of the current grid status is needed to perform these tasks efficiently and effectively. Therefore, there is a need to increase the observability of low voltage grids by installing measurement technologies (e.g. smart meters). Multiple different measurement technologies are available for low voltage grids which can vary in their benefit to observation quality and their installation costs. Therefore, Bayernwerk Netz GmbH and E.DIS AG in cooperation with E-Bridge Consulting GmbH and the Institute for High Voltage Equipment and Grids, Digitalization and Energy Economics (IAEW) investigated the effectiveness of different strategies for the smartification of low voltage grids. This paper presents the methodology used for the investigation and exemplary results focusing on the impact of intelligent cable distribution cabinets and smart meters on the quality of the state estimation.
A Kernel-Based Identification Approach to LPV Feedforward: With Application to Motion Systems
Authors: Max van Haren, Lennart Blanken, Tom Oomen
Date: 2023-03-14 00:00:00
Tasks: Scheduling

The increasing demands for motion control result in a situation where Linear Parameter-Varying (LPV) dynamics have to be taken into account. Inverse-model feedforward control for LPV motion systems is challenging, since the inverse of an LPV system is often dynamically dependent on the scheduling sequence. The aim of this paper is to develop an identification approach that directly identifies dynamically scheduled feedforward controllers for LPV motion systems from data. In this paper, the feedforward controller is parameterized in basis functions, similar to, e.g., mass-acceleration feedforward, and is identified by a kernel-based approach such that the parameter dependency for LPV motion systems is addressed. The resulting feedforward includes dynamic dependence and is learned accurately. The developed framework is validated on an example.
Models, metrics, and their formulas for typical electric power system resilience events
Authors: Ian Dobson
Date: 2023-03-14 00:00:00

Poisson process models are defined in terms of their rates for outage and restore processes in power system resilience events. These outage and restore processes easily yield the performance curves that track the evolution of resilience events, and the area, nadir, and duration of the performance curves are standard resilience metrics. This letter analyzes typical resilience events by analyzing the area, nadir, and duration of mean performance curves. Explicit and intuitive formulas for these metrics are derived in terms of the Poisson process model parameters, and these parameters can be estimated from utility data. This clarifies the calculation of metrics of typical resilience events, and shows what they depend on. The metric formulas are derived with lognormal, exponential, or constant rates of restoration. The method is illustrated with a typical North American transmission event. Similarly nice formulas are obtained for the area metric for empirical power system data.
A Concentration-Based Approach for Optimizing the Estimation Performance in Stochastic Sensor Selection
Authors: Christopher I. Calle, Shaunak D. Bopardikar
Date: 2023-03-14 00:00:00

In this work, we consider a sensor selection drawn at random by a sampling with replacement policy for a linear time-invariant dynamical system subject to process and measurement noise. We employ the Kalman filter to estimate the state of the system. However, the statistical properties of the filter are not deterministic due to the stochastic selection of sensors. As a consequence, we derive concentration inequalities to bound the estimation error covariance of the Kalman filter in the semi-definite sense. Concentration inequalities provide a framework for deriving semi-definite bounds that hold in a probabilistic sense. Our main contributions are three-fold. First, we develop algorithmic tools to aid in the implementation of a matrix concentration inequality. Second, we derive concentration-based bounds for three types of stochastic selections. Third, we propose a polynomial-time procedure for finding a sampling distribution that indirectly minimizes the maximum eigenvalue of the estimation error covariance. Our proposed sampling policy is also shown to empirically outperform three other sampling policies: uniform, deterministic greedy, and randomized greedy.
Distributed least square solution method to linear algebraic equations over multiagent networks
Authors: Viet Hoang Pham, Hyo-Sung Ahn
Date: 2023-03-14 00:00:00

This paper designs a distributed least square solution method for a linear algebraic equation over a multiagent network. The coefficient matrix is divided into multiple blocks, and each agent only knows a subset of these blocks. The designed method is discrete-time and based on a proximal ADMM algorithm. By applying the designed method, each agent can find its corresponding part in one least square solution of the considered linear algebraic equation while using only its information and communicating with its neighbors. Numerical simulations verify the effectiveness of the designed method in MATLAB.
Fair Energy Allocation in Risk-aware Energy Communities
Authors: Eleni Stai, Lesia Mitridati, Ioannis Stavrakakis, Evangelia Kokolaki, Petros Tatoulis, Gabriela Hug
Date: 2023-03-14 00:00:00

This work introduces a decentralized mechanism for the fair and efficient allocation of limited renewable energy sources among consumers in an energy community. In the proposed non-cooperative game, the self-interested community members independently decide whether to compete or not for access to RESs during peak hours and shift their loads analogously. In the peak hours, a proportional allocation (PA) policy is used to allocate the limited RESs among the competitors. The existence of a Nash equilibrium (NE) or dominant strategies in this non-cooperative game is shown, and closed-form expressions of the renewable energy demand and social cost are derived. Moreover, a decentralized algorithm for choosing consumers' strategies that lie on NE states is designed. The work shows that the risk attitude of the consumers can have a significant impact on the deviation of the induced social cost from the optimal. Besides, the proposed decentralized mechanism with the PA policy is shown to attain a much lower social cost than one using the naive equal sharing policy.
Mitigating the Impact of Uncertain Wildfire Risk on Power Grids through Topology Control
Authors: Yuqi Zhou, Kaarthik Sundar, Deepjyoti Deka, Hao Zhu
Date: 2023-03-14 00:00:00
Tasks: Decision Making

Wildfires pose a significant threat to the safe and reliable operations of the electric grid. To mitigate wildfire risk, system operators resort to public safety power shutoffs, or PSPS, that shed load for a subset of customers. As wildfire risk forecasts are stochastic, such decision-making may often be sub-optimal. This paper proposes a two-stage topology control problem that jointly minimizes generation and load-shedding costs in the face of uncertain fire risk. Compared to existing work, we include preand post-event topology control actions and consider scenarios where the wildfire risk is known with low and high confidence. The effectiveness of the proposed approach is demonstrated using a benchmark test system, artificially geo-located in Southern California, and using stochastic wildfire risk data that exists in the literature. Our work provides a crucial study of the comparative benefits of pre-event versus post-event control and the effects of wildfire risk accuracy on each control strategy.
An Improved Approach for Output Feedback Model Predictive Control of Hybrid Systems
Authors: Soroush Sadeghnejad, Farshad Khadivar, Mojtaba Esfandiari, Golchehr Amirkhani, Hamed Moradi, Farzam Farahmand, Gholamreza Vossoughi
Date: 2023-03-13 00:00:00
Tasks: Model Predictive Control

In this paper, a modified robust model predictive control scheme is proposed for linear parametric variable (LPV) and hybrid systems based on a quasi-min-max algorithm. Using a new cost function resulted in reduced unwanted disturbances during switching. In addition, the effects of uncertainties are reduced in the prediction dynamics, and robust stability of piecewise switching LPV systems subjected to linear matrix inequality (LMI) input constraints are guaranteed. Simulation results demonstrate the performance of the proposed controller compared to prior research.