pwc-archive/papers-with-abstracts huggingface dataset:
Parameter-Varying Koopman Operator for Nonlinear System Modeling and Control
Authors: Changyu Lee, Kiyong Park, Jinwhan Kim
Date: 2023-09-19 00:00:00
Tasks: Model Predictive Control

This paper proposes a novel approach for modeling and controlling nonlinear systems with varying parameters. The approach introduces the use of a parameter-varying Koopman operator (PVKO) in a lifted space, which provides an efficient way to understand system behavior and design control algorithms that account for underlying dynamics and changing parameters. The PVKO builds on a conventional Koopman model by incorporating local time-invariant linear systems through interpolation within the lifted space. This paper outlines a procedure for identifying the PVKO and designing a model predictive control using the identified PVKO model. Simulation results demonstrate that the proposed approach improves model accuracy and enables predictions based on future parameter information. The feasibility and stability of the proposed control approach are analyzed, and their effectiveness is demonstrated through simulation.
Joint Minimum Processing Beamforming and Near-end Listening Enhancement
Authors: Andreas J. Fuglsig, Jesper Jensen, Zheng-Hua Tan, Lars S. Bertelsen, Jens Christian Lindof, Jan Østergaard
Date: 2023-09-20 00:00:00
Tasks: Speech Enhancement

We consider speech enhancement for signals picked up in one noisy environment that must be rendered to a listener in another noisy environment. For both far-end noise reduction and near-end listening enhancement, it has been shown that excessive focus on noise suppression or intelligibility maximization may lead to excessive speech distortions and quality degradations in favorable noise conditions, where intelligibility is already at ceiling level. Recently [1,2] propose to remedy this with a minimum processing framework that either reduces noise or enhances listening a minimum amount given that a certain intelligibility criterion is still satisfied Additionally, it has been shown that joint consideration of both environments improves speech enhancement performance. In this paper, we formulate a joint far- and near-end minimum processing framework, that improves intelligibility while limiting speech distortions in favorable noise conditions. We provide closed-form solutions to specific boundary scenarios and investigate performance for the general case using numerical optimization. We also show concatenating existing minimum processing far- and near-end enhancement methods preserves the effects of the initial methods. Results show that the joint optimization can further improve performance compared to the concatenated approach.
Ensembling Multilingual Pre-Trained Models for Predicting Multi-Label Regression Emotion Share from Speech
Authors: Bagus Tris Atmaja, Akira Sasou
Date: 2023-09-20 00:00:00
Tasks: Emotion Recognition, Ensemble Learning, Speech Emotion Recognition

Speech emotion recognition has evolved from research to practical applications. Previous studies of emotion recognition from speech have focused on developing models on certain datasets like IEMOCAP. The lack of data in the domain of emotion modeling emerges as a challenge to evaluate models in the other dataset, as well as to evaluate speech emotion recognition models that work in a multilingual setting. This paper proposes an ensemble learning to fuse results of pre-trained models for emotion share recognition from speech. The models were chosen to accommodate multilingual data from English and Spanish. The results show that ensemble learning can improve the performance of the baseline model with a single model and the previous best model from the late fusion. The performance is measured using the Spearman rank correlation coefficient since the task is a regression problem with ranking values. A Spearman rank correlation coefficient of 0.537 is reported for the test set, while for the development set, the score is 0.524. These scores are higher than the previous study of a fusion method from monolingual data, which achieved scores of 0.476 for the test and 0.470 for the development.
Discrete Audio Representation as an Alternative to Mel-Spectrograms for Speaker and Speech Recognition
Authors: Krishna C. Puvvada, Nithin Rao Koluguri, Kunal Dhawan, Jagadeesh Balam, Boris Ginsburg
Date: 2023-09-19 00:00:00
Tasks: Language Modeling, Language Modelling, Quantization, Representation Learning, Speaker Verification, speech-recognition, Speech Recognition

Discrete audio representation, aka audio tokenization, has seen renewed interest driven by its potential to facilitate the application of text language modeling approaches in audio domain. To this end, various compression and representation-learning based tokenization schemes have been proposed. However, there is limited investigation into the performance of compression-based audio tokens compared to well-established mel-spectrogram features across various speaker and speech related tasks. In this paper, we evaluate compression based audio tokens on three tasks: Speaker Verification, Diarization and (Multi-lingual) Speech Recognition. Our findings indicate that (i) the models trained on audio tokens perform competitively, on average within $1\%$ of mel-spectrogram features for all the tasks considered, and do not surpass them yet. (ii) these models exhibit robustness for out-of-domain narrowband data, particularly in speaker tasks. (iii) audio tokens allow for compression to 20x compared to mel-spectrogram features with minimal loss of performance in speech and speaker related tasks, which is crucial for low bit-rate applications, and (iv) the examined Residual Vector Quantization (RVQ) based audio tokenizer exhibits a low-pass frequency response characteristic, offering a plausible explanation for the observed results, and providing insight for future tokenizer designs.
An Active Noise Control System Based on Soundfield Interpolation Using a Physics-informed Neural Network
Authors: Yile, Zhang, Fei Ma, Thushara Abhayapala, Prasanga Samarasinghe, Amy Bastine
Date: 2023-09-19 00:00:00

Conventional multiple-point active noise control (ANC) systems require placing error microphones within the region of interest (ROI), inconveniencing users. This paper designs a feasible monitoring microphone arrangement placed outside the ROI, providing a user with more freedom of movement. The soundfield within the ROI is interpolated from the microphone signals using a physics-informed neural network (PINN). PINN exploits the acoustic wave equation to assist soundfield interpolation under a limited number of monitoring microphones, and demonstrates better interpolation performance than the spherical harmonic method in simulations. An ANC system is designed to take advantage of the interpolated signal to reduce noise signal within the ROI. The PINN-assisted ANC system reduces noise more than that of the multiple-point ANC system in simulations.
Incorporating Ultrasound Tongue Images for Audio-Visual Speech Enhancement
Authors: Rui-Chen Zheng, Yang Ai, Zhen-Hua Ling
Date: 2023-09-19 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Knowledge Distillation, Speech Enhancement, speech-recognition, Speech Recognition

Audio-visual speech enhancement (AV-SE) aims to enhance degraded speech along with extra visual information such as lip videos, and has been shown to be more effective than audio-only speech enhancement. This paper proposes the incorporation of ultrasound tongue images to improve the performance of lip-based AV-SE systems further. To address the challenge of acquiring ultrasound tongue images during inference, we first propose to employ knowledge distillation during training to investigate the feasibility of leveraging tongue-related information without directly inputting ultrasound tongue images. Specifically, we guide an audio-lip speech enhancement student model to learn from a pre-trained audio-lip-tongue speech enhancement teacher model, thus transferring tongue-related knowledge. To better model the alignment between the lip and tongue modalities, we further propose the introduction of a lip-tongue key-value memory network into the AV-SE model. This network enables the retrieval of tongue features based on readily available lip features, thereby assisting the subsequent speech enhancement task. Experimental results demonstrate that both methods significantly improve the quality and intelligibility of the enhanced speech compared to traditional lip-based AV-SE baselines. Moreover, both proposed methods exhibit strong generalization performance on unseen speakers and in the presence of unseen noises. Furthermore, phone error rate (PER) analysis of automatic speech recognition (ASR) reveals that while all phonemes benefit from introducing ultrasound tongue images, palatal and velar consonants benefit most.
Code Representation Pre-training with Complements from Program Executions
Authors: Jiabo Huang, Jianyu Zhao, Yuyang Rong, Yiwen Guo, Yifeng He, Hao Chen
Date: 2023-09-04 00:00:00
Tasks: Code Search, Language Modeling, Language Modelling

Large language models (LLMs) for natural language processing have been grafted onto programming language modeling for advancing code intelligence. Although it can be represented in the text format, code is syntactically more rigorous in order to be properly compiled or interpreted to perform a desired set of behaviors given any inputs. In this case, existing works benefit from syntactic representations to learn from code less ambiguously in the forms of abstract syntax tree, control-flow graph, etc. However, programs with the same purpose can be implemented in various ways showing different syntactic representations while the ones with similar implementations can have distinct behaviors. Though trivially demonstrated during executions, such semantics about functionality are challenging to be learned directly from code, especially in an unsupervised manner. Hence, in this paper, we propose FuzzPretrain to explore the dynamic information of programs revealed by their test cases and embed it into the feature representations of code as complements. The test cases are obtained with the assistance of a customized fuzzer and are only required during pre-training. FuzzPretrain yielded more than 6%/9% mAP improvements on code search over its counterparts trained with only source code or AST, respectively. Our extensive experimental results show the benefits of learning discriminative code representations with program executions.
Multi-Agent Control Synthesis from Global Temporal Logic Tasks with Synchronous Satisfaction Requirements
Authors: Tiange Yang, Yuanyuan Zou, Jinfeng Liu, ShaoYuan Li, Xiaohu Zhao
Date: 2023-09-20 00:00:00
Tasks: Motion Planning

This paper addresses the multi-agent control problem under global temporal logic tasks, considering agents with heterogeneous capabilities. These global tasks involve not only absolute and relative temporal and spatial constraints, but also group behaviors, including task completion times, agent capabilities, and task interdependencies such as the need for synchronous execution. The global tasks are formally formulated into global signal temporal logic (STL) formulae, and a synchronous robustness metric is designed to evaluate the synchronization quality with real values. A mixed-integer linear programming (MILP) encoding method is further proposed to realize task-satisfied motion planning with high synchronicity and minimum control efforts. The encoding method uses a logarithmic number of binary variables to fully capture synchronous robustness, leading to only linear computational complexity. Simulations are conducted to demonstrate the efficiency of the proposed control strategy.
Distributed Finite-Time Cooperative Localization for Three-Dimensional Sensor Networks
Authors: Jinze Wu, Lorenzo Zino, Zhiyun Lin, Alessandro Rizzo
Date: 2023-09-20 00:00:00
Tasks: Position

This paper addresses the distributed localization problem for a network of sensors placed in a three-dimensional space, in which sensors are able to perform range measurements, i.e., measure the relative distance between them, and exchange information on a network structure. First, we derive a necessary and sufficient condition for node localizability using barycentric coordinates. Then, building on this theoretical result, we design a distributed localizability verification algorithm, in which we propose and employ a novel distributed finite-time algorithm for sum consensus. Finally, we develop a distributed localization algorithm based on conjugate gradient method, and we derive a theoretical guarantee on its performance, which ensures finite-time convergence to the exact position for all localizable nodes. The efficiency of our algorithm compared to the existing ones from the state-of-the-art literature is further demonstrated through numerical simulations.
Social interactions for a sustainable lifestyle: The design of an experimental case study
Authors: Angela Fontan, Mahsa Farjadnia, Joe Llewellyn, Cecilia Katzeff, Marco Molinari, Vladimir Cvetkovic, Karl Henrik Johansson
Date: 2023-09-20 00:00:00

Every day we face numerous lifestyle decisions, some dictated by habits and some more conscious, which may or may not promote sustainable living. Aided by digital technology, sustainable behaviors can diffuse within social groups and inclusive communities. This paper outlines a longitudinal experimental study of social influence in behavioral changes toward sustainability, in the context of smart residential homes. Participants are residing in the housing on campus referred to as KTH Live-In Lab, whose behaviors are observed w.r.t. key lifestyle choices, such as food, resources, mobility, consumption, and environmental citizenship. The focus is on the preparatory phase of the case study and the challenges and limitations encountered during its setup. In particular, this work proposes a definition of sustainability indicators for environmentally significant behaviors, and hypothesizes that, through digitalization of a household into a social network of interacting tenants, sustainable living can be promoted. Preliminary results confirm the feasibility of the proposed experimental methodology.
Sample- and computationally efficient data-driven predictive control
Authors: Mohammad Alsalti, Manuel Barkey, Victor G. Lopez, Matthias A. Müller
Date: 2023-09-20 00:00:00

Recently proposed data-driven predictive control schemes for LTI systems use non-parametric representations based on the image of a Hankel matrix of previously collected, persistently exciting, input-output data. Persistence of excitation necessitates that the data is sufficiently long and, hence, the computational complexity of the corresponding finite-horizon optimal control problem increases. In this paper, we propose an efficient data-driven predictive control (eDDPC) scheme which is both more sample efficient (requires less offline data) and computationally efficient (uses less decision variables) compared to existing schemes. This is done by leveraging an alternative data-based representation of the trajectories of LTI systems. We analytically and numerically compare the performance of this scheme to existing ones from the literature.
Pointing-and-Acquisition for Optical Wireless in 6G: From Algorithms to Performance Evaluation
Authors: Hyung-Joo Moon, Chan-Byoung Chae, Kai-Kit Wong, Mohamed-Slim Alouini
Date: 2023-09-20 00:00:00

The increasing demand for wireless communication services has led to the development of non-terrestrial networks, which enables various air and space applications. Free-space optical (FSO) communication is considered one of the essential technologies capable of connecting terrestrial and non-terrestrial layers. In this article, we analyze considerations and challenges for FSO communications between gateways and aircraft from a pointing-and-acquisition perspective. Based on the analysis, we first develop a baseline method that utilizes conventional devices and mechanisms. Furthermore, we propose an algorithm that combines angle of arrival (AoA) estimation through supplementary radio frequency (RF) links and beam tracking using retroreflectors. Through extensive simulations, we demonstrate that the proposed method offers superior performance in terms of link acquisition and maintenance.
Data-driven design of complex network structures to promote synchronization
Authors: Marco Coraggio, Mario di Bernardo
Date: 2023-09-19 00:00:00

We consider the problem of optimizing the interconnection graphs of complex networks to promote synchronization. When traditional optimization methods are inapplicable, due to uncertain or unknown node dynamics, we propose a data-driven approach leveraging datasets of relevant examples. We analyze two case studies, with linear and nonlinear node dynamics. First, we show how including node dynamics in the objective function makes the optimal graphs heterogeneous. Then, we compare various design strategies, finding that the best either utilize data samples close to a specific Pareto front or a combination of a neural network and a genetic algorithm, with statistically better performance than the best examples in the datasets.
Physics-Informed Machine Learning for Data Anomaly Detection, Classification, Localization, and Mitigation: A Review, Challenges, and Path Forward
Authors: Mehdi Jabbari Zideh, Paroma Chatterjee, Anurag K. Srivastava
Date: 2023-09-19 00:00:00
Tasks: Anomaly Detection, Physics-informed machine learning, Reinforcement Learning (RL)

Advancements in digital automation for smart grids have led to the installation of measurement devices like phasor measurement units (PMUs), micro-PMUs ($\mu$-PMUs), and smart meters. However, a large amount of data collected by these devices brings several challenges as control room operators need to use this data with models to make confident decisions for reliable and resilient operation of the cyber-power systems. Machine-learning (ML) based tools can provide a reliable interpretation of the deluge of data obtained from the field. For the decision-makers to ensure reliable network operation under all operating conditions, these tools need to identify solutions that are feasible and satisfy the system constraints, while being efficient, trustworthy, and interpretable. This resulted in the increasing popularity of physics-informed machine learning (PIML) approaches, as these methods overcome challenges that model-based or data-driven ML methods face in silos. This work aims at the following: a) review existing strategies and techniques for incorporating underlying physical principles of the power grid into different types of ML approaches (supervised/semi-supervised learning, unsupervised learning, and reinforcement learning (RL)); b) explore the existing works on PIML methods for anomaly detection, classification, localization, and mitigation in power transmission and distribution systems, c) discuss improvements in existing methods through consideration of potential challenges while also addressing the limitations to make them suitable for real-world applications.
Inverse Optimal Control Design of a VTOL Aircraft
Authors: Kinza Rehman, Hafiz Zeeshan Iqbal Khan, Muhammad Farooq Haydar
Date: 2023-09-19 00:00:00

A Vertical Takeoff and Landing (VTOL) aircraft is capable of short/vertical takeoffs and landings, thus eliminating the requirement of long runways. This feature makes them suitable for a broader variety of missions, generally not achievable by a traditional aircraft. The dynamics of the VTOL aircraft in the vertical plane has been used as a benchmark problem to demonstrate the effectiveness of different nonlinear control techniques, mainly due to being highly coupled and non-minimum phase. This paper presents the design of an inverse optimal control for a simplified VTOL aircraft model. The proposed control law is based on a Control Lyapunov Function (CLF) which exploits the normal form typically used for control design through feedback linearization. The suggested control law is compared with dynamic feedback linearization based control law and a comparison is drawn based on their efficacy and robustness.
Generalized Cactus and Structural Controllability of Switched Linear Continuous-Time Systems
Authors: Yuan Zhang, Yuanqing Xia, Aming Li
Date: 2023-09-19 00:00:00

This paper explores the structural controllability of switched linear continuous-time systems. It first identifies a gap in the proof for a pivotal criterion for the structural controllability of switched linear systems in the literature. To address this void, we develop novel graph-theoretic concepts, such as multi-layer dynamic graphs, generalized stems/buds, and generalized cacti, and based on them, provide a comprehensive proof for this criterion. Our approach also induces a new, generalized cactus based graph-theoretic criterion for structural controllability. This not only extends Lin's cactus-based graph-theoretic condition to switched systems for the first time, but also provides a lower bound for the generic dimension of controllable subspaces.
Formal Abstraction of General Stochastic Systems via Noise Partitioning
Authors: John Skovbekk, Luca Laurenti, Eric Frew, Morteza Lahijanian
Date: 2023-09-19 00:00:00

Verifying the performance of safety-critical, stochastic systems with complex noise distributions is difficult. We introduce a general procedure for the finite abstraction of nonlinear stochastic systems with non-standard (e.g., non-affine, non-symmetric, non-unimodal) noise distributions for verification purposes. The method uses a finite partitioning of the noise domain to construct an interval Markov chain (IMC) abstraction of the system via transition probability intervals. Noise partitioning allows for a general class of distributions and structures, including multiplicative and mixture models, and admits both known and data-driven systems. The partitions required for optimal transition bounds are specified for systems that are monotonic with respect to the noise, and explicit partitions are provided for affine and multiplicative structures. By the soundness of the abstraction procedure, verification on the IMC provides guarantees on the stochastic system against a temporal logic specification. In addition, we present a novel refinement-free algorithm that improves the verification results. Case studies on linear and nonlinear systems with non-Gaussian noise, including a data-driven example, demonstrate the generality and effectiveness of the method without introducing excessive conservatism.
Interval Signal Temporal Logic from Natural Inclusion Functions
Authors: Luke Baird, Akash Harapanahalli, Samuel Coogan
Date: 2023-09-19 00:00:00

We propose an interval extension of Signal Temporal Logic (STL) called Interval Signal Temporal Logic (\ISTL). Given an STL formula, we consider an interval inclusion function for each of its predicates. Then, we use minimal inclusion functions for the $\min$ and $\max$ functions to recursively build an interval robustness that is a natural inclusion function for the robustness of the original STL formula. The resulting interval semantics accommodate, for example, uncertain signals modeled as a signal of intervals and uncertain predicates modeled with appropriate inclusion functions. In many cases, verification or synthesis algorithms developed for STL apply to \ISTL with minimal theoretic and algorithmic changes, and existing code can be readily extended using interval arithmetic packages at negligible computational expense. To demonstrate \ISTL, we present an example of offline monitoring from an uncertain signal trace obtained from a hardware experiment and an example of robust online control synthesis enforcing an STL formula with uncertain predicates.