pwc-archive/papers-with-abstracts huggingface dataset:
Listening to Multi-talker Conversations: Modular and End-to-end Perspectives
Authors: Desh Raj
Date: 2024-02-14 00:00:00
Tasks: GPU, speaker-diarization, Speaker Diarization, speech-recognition, Speech Recognition, Target Speaker Extraction

Since the first speech recognition systems were built more than 30 years ago, improvement in voice technology has enabled applications such as smart assistants and automated customer support. However, conversation intelligence of the future requires recognizing free-flowing multi-party conversations, which is a crucial and challenging component that still remains unsolved. In this dissertation, we focus on this problem of speaker-attributed multi-talker speech recognition, and propose two perspectives which result from its probabilistic formulation. In the modular perspective, we build a pipeline of sub-tasks involving speaker diarization, target speaker extraction, and speech recognition. Our first contribution is a method to perform overlap-aware diarization by reformulating spectral clustering as a constrained optimization problem. We also describe an algorithm to ensemble diarization outputs, either to combine overlap-aware systems or to perform multi-channel diarization by late fusion. Once speaker segments are identified, we robustly extract single-speaker utterances from the mixture using a GPU-accelerated implementation of guided source separation, which allows us to use an off-the-shelf ASR system to obtain speaker-attributed transcripts. Since the modular approach suffers from error propagation, we propose an alternate "end-to-end" perspective on the problem. For this, we describe the Streaming Unmixing and Recognition Transducer (SURT). We show how to train SURT models efficiently by carefully designing the network architecture, objective functions, and mixture simulation techniques. Finally, we add an auxiliary speaker branch to enable joint prediction of speaker labels synchronized with the speech tokens. We demonstrate that training on synthetic mixtures and adapting with real data helps these models transfer well for streaming transcription of real meeting sessions.
Sound Field Reconstruction Using a Compact Acoustics-informed Neural Network
Authors: Fei Ma, Sipei Zhao, Ian S. Burnett
Date: 2024-02-14 00:00:00
Tasks: valid

Sound field reconstruction (SFR) augments the information of a sound field captured by a microphone array. Conventional SFR methods using basis function decomposition are straightforward and computationally efficient, but may require more microphones than needed to measure the sound field. Recent studies show that pure data-driven and learning-based methods are promising in some SFR tasks, but they are usually computationally heavy and may fail to reconstruct a physically valid sound field. This paper proposes a compact acoustics-informed neural network (AINN) method for SFR, whereby the Helmholtz equation is exploited to regularize the neural network. As opposed to pure data-driven approaches that solely rely on measured sound pressures, the integration of the Helmholtz equation improves robustness of the neural network against variations during the measurement processes and prompts the generation of physically valid reconstructions. The AINN is designed to be compact, and is able to predict not only the sound pressures but also sound pressure gradients within a spatial region of interest based on measured sound pressures along the boundary. Numerical experiments with acoustic transfer functions measured in different environments demonstrate the superiority of the AINN method over the traditional cylinder harmonic decomposition and the singular value decomposition methods.
Benchmarking multi-component signal processing methods in the time-frequency plane
Authors: Juan M. Miramont, Rémi Bardenet, Pierre Chainais, Francois Auger
Date: 2024-02-13 00:00:00
Tasks: Benchmarking, Denoising

Signal processing in the time-frequency plane has a long history and remains a field of methodological innovation. For instance, detection and denoising based on the zeros of the spectrogram have been proposed since 2015, contrasting with a long history of focusing on larger values of the spectrogram. Yet, unlike neighboring fields like optimization and machine learning, time-frequency signal processing lacks widely-adopted benchmarking tools. In this work, we contribute an open-source, Python-based toolbox termed MCSM-Benchs for benchmarking multi-component signal analysis methods, and we demonstrate our toolbox on three time-frequency benchmarks. First, we compare different methods for signal detection based on the zeros of the spectrogram, including unexplored variations of previously proposed detection tests. Second, we compare zero-based denoising methods to both classical and novel methods based on large values and ridges of the spectrogram. Finally, we compare the denoising performance of these methods against typical spectrogram thresholding strategies, in terms of post-processing artifacts commonly referred to as musical noise. At a low level, the obtained results provide new insight on the assessed approaches, and in particular research directions to further develop zero-based methods. At a higher level, our benchmarks exemplify the benefits of using a public, collaborative, common framework for benchmarking.
Channel-Combination Algorithms for Robust Distant Voice Activity and Overlapped Speech Detection
Authors: Théo Mariotte, Anthony Larcher, Silvio Montrésor, Jean-Hugh Thomas
Date: 2024-02-13 00:00:00
Tasks: Action Detection, Activity Detection, speaker-diarization, Speaker Diarization

Voice Activity Detection (VAD) and Overlapped Speech Detection (OSD) are key pre-processing tasks for speaker diarization. In the meeting context, it is often easier to capture speech with a distant device. This consideration however leads to severe performance degradation. We study a unified supervised learning framework to solve distant multi-microphone joint VAD and OSD (VAD+OSD). This paper investigates various multi-channel VAD+OSD front-ends that weight and combine incoming channels. We propose three algorithms based on the Self-Attention Channel Combinator (SACC), previously proposed in the literature. Experiments conducted on the AMI meeting corpus exhibit that channel combination approaches bring significant VAD+OSD improvements in the distant speech scenario. Specifically, we explore the use of learned complex combination weights and demonstrate the benefits of such an approach in terms of explainability. Channel combination-based VAD+OSD systems are evaluated on the final back-end task, i.e. speaker diarization, and show significant improvements. Finally, since multi-channel systems are trained given a fixed array configuration, they may fail in generalizing to other array set-ups, e.g. mismatched number of microphones. A channel-number invariant loss is proposed to learn a unique feature representation regardless of the number of available microphones. The evaluation conducted on mismatched array configurations highlights the robustness of this training strategy.
Unrestricted Global Phase Bias-Aware Single-channel Speech Enhancement with Conformer-based Metric GAN
Authors: Shiqi Zhang, Zheng Qiu, Daiki Takeuchi, Noboru Harada, Shoji Makino
Date: 2024-02-13 00:00:00
Tasks: Speech Enhancement

With the rapid development of neural networks in recent years, the ability of various networks to enhance the magnitude spectrum of noisy speech in the single-channel speech enhancement domain has become exceptionally outstanding. However, enhancing the phase spectrum using neural networks is often ineffective, which remains a challenging problem. In this paper, we found that the human ear cannot sensitively perceive the difference between a precise phase spectrum and a biased phase (BP) spectrum. Therefore, we propose an optimization method of phase reconstruction, allowing freedom on the global-phase bias instead of reconstructing the precise phase spectrum. We applied it to a Conformer-based Metric Generative Adversarial Networks (CMGAN) baseline model, which relaxes the existing constraints of precise phase and gives the neural network a broader learning space. Results show that this method achieves a new state-of-the-art performance without incurring additional computational overhead.
If Turing played piano with an artificial partner
Authors: Dobromir Dotov, Dante Camarena, Zack Harris, Joanna Spyra, Pietro Gagliano, Laurel Trainor
Date: 2024-02-09 00:00:00

Music is an inherently social activity that allows people to share experiences and feel connected with one another. There has been little progress in designing artificial partners exhibiting a similar social experience as playing with another person. Neural network architectures that implement generative models, such as large language models, are suited for producing musical scores. Playing music socially, however, involves more than playing a score; it must complement the other musicians' ideas and keep time correctly. We addressed the question of whether a convincing social experience is made possible by a generative model trained to produce musical scores, not necessarily optimized for synchronization and continuation. The network, a variational autoencoder trained on a large corpus of digital scores, was adapted for a timed call-and-response task with a human partner. Participants played piano with a human or artificial partner-in various configurations-and rated the performance quality and first-person experience of self-other integration. Overall, the artificial partners held promise but were rated lower than human partners. The artificial partner with simplest design and highest similarity parameter was not rated differently from the human partners on some measures, suggesting that interactive rather than generative sophistication is important in enabling social AI.
The impact of load placement on grid resonances during grid restoration
Authors: Adolfo Anta, Diego Cifelli
Date: 2024-02-14 00:00:00

As inverter-based generation is being massively deployed in the grid, these type of units have to take over the current roles of conventional generation, including the capability of restoring the grid. In this context, the resonances of the grid during the first steps of a black start can be concerning, given that the grid is lightly loaded. Especially relevant are the low frequency resonances, that may be excited by the harmonic components of the inverter. A typical strategy to avoid or minimize the effect of such resonances relies on connecting load banks. This was fairly feasible with conventional generation, but given the limited ratings of inverters, the amount of load that can be connected at the beginning is very limited. In this paper we consider the energization of a transmission line, and investigate the optimal location of a load along a line in order to maximize the damping in the system. By analysing the spectral properties as a function of the load location, we formally prove that placing the load in the middle of the transmission line maximizes the damping ratio of the first resonance of the system.
Uncertainty-Aware Transient Stability-Constrained Preventive Redispatch: A Distributional Reinforcement Learning Approach
Authors: Zhengcheng Wang, Fei Teng, Yanzhen Zhou, Qinglai Guo, Hongbin Sun
Date: 2024-02-14 00:00:00
Tasks: Deep Reinforcement Learning, Distributional Reinforcement Learning, Graph Neural Network, reinforcement-learning, Reinforcement Learning

Transient stability-constrained preventive redispatch plays a crucial role in ensuring power system security and stability. Since redispatch strategies need to simultaneously satisfy complex transient constraints and the economic need, model-based formulation and optimization become extremely challenging. In addition, the increasing uncertainty and variability introduced by renewable sources start to drive the system stability consideration from deterministic to probabilistic, which further exaggerates the complexity. In this paper, a Graph neural network guided Distributional Deep Reinforcement Learning (GD2RL) method is proposed, for the first time, to solve the uncertainty-aware transient stability-constrained preventive redispatch problem. First, a graph neural network-based transient simulator is trained by supervised learning to efficiently generate post-contingency rotor angle curves with the steady-state and contingency as inputs, which serves as a feature extractor for operating states and a surrogate time-domain simulator during the environment interaction for reinforcement learning. Distributional deep reinforcement learning with explicit uncertainty distribution of system operational conditions is then applied to generate the redispatch strategy to balance the user-specified probabilistic stability performance and economy preferences. The full distribution of the post-redispatch transient stability index is directly provided as the output. Case studies on the modified New England 39-bus system validate the proposed method.
Balancing the Norwegian regulated power market anno 2016 to 2022
Authors: Pål Forr Austnes, Signe Riemer-Sørensen, David Andreas Bordvik, Christian Andre Andresen
Date: 2024-02-14 00:00:00

The balancing market for power is designed to account for the difference between predicted supply/demand of electricity and the realised supply/demand. However, increased electrification of society changes the consumption patterns, and increased production from renewable sources leads to larger un-predicted fluctuations in production, both effects potentially leading to increased balancing. We analyse public market data for the balancing market (manual Frequency Restoration Reserve) for Norway from 2016 to 2022 to investigate and document these effects. The data is newer than for similar analyses and the eight years of data is more than double the time span previously covered. The main findings are: a) The balancing volumes are dominated by hours of zero regulation but for non-zero hours, the balancing volumes are increasing during the eight-year period. b) The balancing prices are primarily correlated with day-ahead prices and secondary with balancing volumes. The latter correlation is found to be increasingly non-linear with time. c) The balancing volumes and the price difference between balancing price and day-ahead price are strongly correlated with the previous hour. d) The increasing share of wind power has not impacted the frequency of balancing, which has remained stable during the 8 years studied. However, the volumes and share of balancing power compared to overall production have increased, suggesting that the hours which are inherently difficult to predict remain the same. e) Market data alone cannot predict balancing volumes. If attempting, the auto-correlation becomes the main source of information.
A Practical and Online Trajectory Planner for Autonomous Ships' Berthing, Incorporating Speed Control
Authors: Agnes Ngina Mwange, Dimas Maulana Rachman, Rin Suyama, Atsuo Maki
Date: 2024-02-14 00:00:00
Tasks: Collision Avoidance, Trajectory Planning

Autonomous ships are essentially designed and equipped to perceive their internal and external environment and subsequently perform appropriate actions depending on the predetermined objective(s) without human intervention. Consequently, trajectory planning algorithms for autonomous berthing must consider factors such as system dynamics, ship actuators, environmental disturbances, and the safety of the ship, other ships, and port structures, among others. In this study, basing the ship dynamics on the low-speed MMG model, trajectory planning for an autonomous ship is modeled as an optimal control problem (OCP) that is transcribed into a nonlinear programming problem (NLP) using the direct multiple shooting technique. To enhance berthing safety, besides considering wind disturbances, speed control, actuators' limitations, and collision avoidance features are incorporated as constraints in the NLP, which is then solved using the Sequential Quadratic Programming (SQP) algorithm in MATLAB. Finally, the performance of the proposed planner is evaluated through (i) comparison with solutions obtained using CMA-ES for two different model ships, (ii) trajectory planning for different harbor entry and berth approach scenarios, and (iii) feasibility study using stochastically generated initial conditions and positions within the port boundaries. Simulation results indicate enhanced berthing safety as well as practical and computational feasibility making the planner suitable for real-time applications.
Distributed Secret Securing in Discrete-Event Systems
Authors: Shoma Matsui, Kai Cai, Karen Rudie
Date: 2024-02-14 00:00:00

In this paper, we study a security problem of protecting secrets in distributed systems. Specifically, we employ discrete-event systems to describe the structure and behaviour of distributed systems, in which global secret information is separated into pieces and stored in local component agents. The goal is to prevent such secrets from being exposed to intruders by imposing appropriate protection measures. This problem is formulated as to ensure that at least one piece of every distributed global secret is secured by a required number of protections, while the overall cost to apply protections is minimum. We first characterize the solvability of this security problem by providing a necessary and sufficient condition, and then develop an algorithm to compute a solution based on the supervisory control theory of discrete-event systems. Finally, we illustrate the effectiveness of our solution with an example system comprising distributed databases.
Infinite-horizon optimal scheduling for feedback control
Authors: Siyi Wang, Sandra Hirche
Date: 2024-02-13 00:00:00
Tasks: Scheduling

Emerging cyber-physical systems impel the development of communication protocols that optimize resource utilization. This article investigates infinite-horizon optimal scheduling for resource-aware networked control systems by addressing the rate-regulation tradeoff. Consider a scenario where the sensor and the controller communicate via a networked channel, the transmission scheduling problem is formulated as a Markov decision process on unbounded general state space controlled by scheduling decisions. The value of information (VoI) serves as a metric to assess the importance of sensory data for transmission. We derive the optimal scheduling law for feedback control based on VoI and show that it is deterministic and stationary, with an explicit expression obtained via value iteration. The closed-loop system under the designed scheduling law is shown to be stochastically stable. By analyzing the dynamic behavior of the iteration process, we show that the VoI function and the optimal scheduling law exhibit symmetry. Furthermore, when the system matrix is diagonalizable, the VoI function is monotone and quasi-convex. Consequently, the optimal scheduling law is shown to exhibit a threshold structure and takes a quadratic form, with the threshold region explicitly characterized. Finally, the numerical simulation illustrates the theoretical result of the VoI-based scheduling.
An Adaptive System Architecture for Multimodal Intelligent Transportation Systems
Authors: Muhammad Farooq, Nima Afraz, Fatemeh Golpayegani
Date: 2024-02-13 00:00:00
Tasks: Data Integration, Decision Making

Multimodal intelligent transportation systems (M-ITS) encompass a range of transportation services that utilise various modes of transport and incorporate intelligent technologies for enhanced efficiency and user experience. There are several challenges in M-ITS including data integration, Interoperability, scalability, user experience, etc. To address these challenges, such a system requires an adaptive system architecture that enables M-ITS to operate as an integrated ecosystem. In this paper, we provide an adaptive, user-centric, and layered architecture for multimodal transportation systems. The proposed architecture ensures scalability for seamless interactions of various subcomponents, that are often managed by different stakeholders. Concurrently, the data architecture is detailed, covering diverse data sources, advanced analytics, and stringent governance, providing a robust basis for intelligent decision-making. We provide two example use cases of the proposed architecture, showing how the data architecture and the system architecture can be fused and serve multimodal intelligent transport services.
SEASONS: Signal and Energy Aware Sensing on iNtermittent Systems
Authors: Pouya Mahdi Gholami, Henry Hoffmann
Date: 2024-02-13 00:00:00

Both energy-aware, batteryless intermittent systems and signal-aware adaptive sampling algorithms (ASA) aim to maximize sensor data accuracy under energy constraints in edge devices. Intuitively, combining both into a signal- & energy-aware solution would yield even better accuracy. Unfortunately, ASAs and intermittent systems rely on conflicting energy availability assumptions. So, a straightforward combination cannot achieve their combined benefits. Therefore, we propose SEASONS, the first framework for signal- and energy-aware intermittent systems. SEASONS buffers signal data in time, monitoring queue dynamics to ensure the data is reported within a user-specified latency constraint. SEASONS improves sensor data accuracy by 31% without increasing energy.
A New Hybrid Approach for Identifying Obsolescence Features: Applied to Railway Signaling Infrastructure
Authors: Elie Saad, Mariem Besbes, Marc Zolghadri, Victor Czmil, Vincent Bourgeois
Date: 2024-02-13 00:00:00
Tasks: Management

Electrical component obsolescence poses a major issue especially within systems with large life cycles. Thus, finding the optimal management solution for each obsolescence case is as crucial as knowing what to consider when faced with an obsolescence case. In this paper, a novel hybrid approach for identifying features affecting electrical component obsolescence management is introduced, which combines features engineering techniques and expert knowledge. The method then uses machine learning to predict obsolescence resolution techniques in order to find the optimal resolution. The motivation behind this research is driven by the imperative need for SNCF RESEAU to optimally address and mitigate the challenges posed by electrical component obsolescence in railway infrastructure.
Why Studying Cut-ins? Comparing Cut-ins and Other Lane Changes Based on Naturalistic Driving Data
Authors: Yun Lu, Dejiang Zheng, Rong Su, Avalpreet Singh Brar, Niels de Boer, Yong Liang Guan
Date: 2024-02-13 00:00:00

Extensive research has been conducted to explore vehicle lane changes, while the study on cut-ins has not received sufficient attention. The existing studies have not addressed the fundamental question of why studying cut-ins is crucial, despite the extensive investigation into lane changes. To tackle this issue, it is important to demonstrate how cut-ins, as a special type of lane change, differ from other lane changes. In this paper, we explore to compare driving characteristics of cut-ins and other lane changes based on naturalistic driving data. The highD dataset is employed to conduct the comparison. We extract all lane-change events from the dataset and exclude events that are not suitable for our comparison. Lane-change events are then categorized into the cut-in events and other lane-change events based on various gap-based rules. Several performance metrics are designed to measure the driving characteristics of the two types of events. We prove the significant differences between the cut-in behavior and other lane-change behavior by using the Wilcoxon rank-sum test. The results suggest the necessity of conducting specialized studies on cut-ins, offering valuable insights for future research in this field.