pwc-archive/papers-with-abstracts huggingface dataset:
Transformers are Efficient Compilers, Provably
Authors: Xiyu Zhai, Runlong Zhou, Liao Zhang, Simon Shaolei Du
Date: 2024-10-07 00:00:00

Transformer-based large language models (LLMs) have demonstrated surprisingly robust performance across a wide range of language-related tasks, including programming language understanding and generation. In this paper, we take the first steps towards a formal investigation of using transformers as compilers from an expressive power perspective. To this end, we introduce a representative programming language, Mini-Husky, which encapsulates key features of modern C-like languages. We show that if the input code sequence has a bounded depth in both the Abstract Syntax Tree (AST) and type inference (reasonable assumptions based on the clean code principle), then the number of parameters required by transformers depends only on the logarithm of the input sequence length to handle compilation tasks, such as AST construction, symbol resolution, and type analysis. A significant technical challenge stems from the fact that transformers operate at a low level, where each layer processes the input sequence as raw vectors without explicitly associating them with predefined structure or meaning. In contrast, high-level compiler tasks necessitate managing intricate relationships and structured program information. Our primary technical contribution is the development of a domain-specific language, Cybertron, which generates formal proofs of the transformer's expressive power, scaling to address compiler tasks. We further establish that recurrent neural networks (RNNs) require at least a linear number of parameters relative to the input sequence, leading to an exponential separation between transformers and RNNs. Finally, we empirically validate our theoretical results by comparing transformers and RNNs on compiler tasks within Mini-Husky.
Multi-Level Speaker Representation for Target Speaker Extraction
Authors: Ke Zhang, Junjie Li, Shuai Wang, Yangjie Wei, Yi Wang, Yannan Wang, Haizhou Li
Date: 2024-10-21 00:00:00
Tasks: Target Speaker Extraction

Target speaker extraction (TSE) relies on a reference cue of the target to extract the target speech from a speech mixture. While a speaker embedding is commonly used as the reference cue, such embedding pre-trained with a large number of speakers may suffer from confusion of speaker identity. In this work, we propose a multi-level speaker representation approach, from raw features to neural embeddings, to serve as the speaker reference cue. We generate a spectral-level representation from the enrollment magnitude spectrogram as a raw, low-level feature, which significantly improves the model's generalization capability. Additionally, we propose a contextual embedding feature based on cross-attention mechanisms that integrate frame-level embeddings from a pre-trained speaker encoder. By incorporating speaker features across multiple levels, we significantly enhance the performance of the TSE model. Our approach achieves a 2.74 dB improvement and a 4.94% increase in extraction accuracy on Libri2mix test set over the baseline.
AC-Mix: Self-Supervised Adaptation for Low-Resource Automatic Speech Recognition using Agnostic Contrastive Mixup
Authors: Carlos Carvalho, Alberto Abad
Date: 2024-10-18 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Domain Adaptation, GPU, Self-Supervised Learning, speech-recognition, Speech Recognition

Self-supervised learning (SSL) leverages large amounts of unlabelled data to learn rich speech representations, fostering improvements in automatic speech recognition (ASR), even when only a small amount of labelled data is available for fine-tuning. Despite the advances in SSL, a significant challenge remains when the data used for pre-training (source domain) mismatches the fine-tuning data (target domain). To tackle this domain mismatch challenge, we propose a new domain adaptation method for low-resource ASR focused on contrastive mixup for joint-embedding architectures named AC-Mix (agnostic contrastive mixup). In this approach, the SSL model is adapted through additional pre-training using mixed data views created by interpolating samples from the source and the target domains. Our proposed adaptation method consistently outperforms the baseline system, using approximately 11 hours of adaptation data and requiring only 1 hour of adaptation time on a single GPU with WavLM-Large.
Observing the Southern US Culture of Honor Using Large-Scale Social Media Analysis
Authors: Juho Kim, Michael Guerzhoy
Date: 2024-10-11 00:00:00

A \textit{culture of honor} refers to a social system where individuals' status, reputation, and esteem play a central role in governing interpersonal relations. Past works have associated this concept with the United States (US) South and related with it various traits such as higher sensitivity to insult, a higher value on reputation, and a tendency to react violently to insults. In this paper, we hypothesize and confirm that internet users from the US South, where a \textit{culture of honor} is more prevalent, are more likely to display a trait predicted by their belonging to a \textit{culture of honor}. Specifically, we test the hypothesis that US Southerners are more likely to retaliate to personal attacks by personally attacking back. We leverage OpenAI's GPT-3.5 API to both geolocate internet users and to automatically detect whether users are insulting each other. We validate the use of GPT-3.5 by measuring its performance on manually-labeled subsets of the data. Our work demonstrates the potential of formulating a hypothesis based on a conceptual framework, operationalizing it in a way that is amenable to large-scale LLM-aided analysis, manually validating the use of the LLM, and drawing a conclusion.
Nonlinear Magnetics Model for Permanent Magnet Synchronous Machines Capturing Saturation and Temperature Effects
Authors: Kishan Srinivasan, Heath Hofmann, Jing Sun
Date: 2024-10-21 00:00:00

This paper proposes a nonlinear magnetics model for Permanent Magnet Synchronous Machines (PMSMs) that accurately captures the effects of magnetic saturation in the machine iron and variations in rotor temperature on the permanent magnet excitation. The proposed model considers the permanent magnet as a current source rather than the more commonly used flux-linkage source. A comparison of the two modelling approaches is conducted using Finite Element Analysis (FEA) for different machine designs as well as experimental validation, where it is shown that the proposed model has substantially better accuracy. The proposed model decouples magnetic saturation and rotor temperature effects in the current/flux-linkage relationship, allowing for adaptive estimation of the PM excitation.
Fast Physics-Informed Model Predictive Control Approximation for Lyapunov Stability
Authors: Josue N. Rivera, Jianqi Ruan, Xiaolin Xu, Shuting Yang, Dengfeng Sun, Neera Jain
Date: 2024-10-21 00:00:00
Tasks: Model Predictive Control

At the forefront of control techniques is Model Predictive Control (MPC). While MPCs are effective, their requisite to recompute an optimal control given a new state leads to sparse response to the system and may make their implementation infeasible in small systems with low computational resources. To address these limitations in stability control, this research presents a small deterministic Physics-Informed MPC Surrogate model (PI-MPCS). PI-MPCS was developed to approximate the control by an MPC while encouraging stability and robustness through the integration of the system dynamics and the formation of a Lyapunov stability profile. Empirical results are presented on the task of 2D quadcopter landing. They demonstrate a rapid and precise MPC approximation on a non-linear system along with an estimated two times speed up on the computational requirements when compared against an MPC. PI-MPCS, in addition, displays a level of stable control for in- and out-of-distribution states as encouraged by the discrete dynamics residual and Lyapunov stability loss functions. PI-MPCS is meant to serve as a surrogate to MPC on situations in which the computational resources are limited.
Lossless optimal transient control for rigid bodies in 3D space
Authors: Riccardo Zanella, Federico Califano, Antonio Franchi, Stefano Stramigioli
Date: 2024-10-21 00:00:00
Tasks: Model Predictive Control

In this letter, we propose a control scheme for rigid bodies designed to optimise transient behaviors. The search space for the optimal control input is parameterized to yield a passive, specifically lossless, nonlinear feedback controller. As a result, it can be combined with other stabilizing controllers without compromising the stability of the closed-loop system. The controller commands torques generating fictitious gyroscopic effects characteristics of 3D rotational rigid body motions, and as such does not inject nor extract kinetic energy from the system. We validate the controller in simulation using a model predictive control (MPC) scheme, successfully combining stability and performance in a stabilization task with obstacle avoidance constraints.
A New Method For Flushing of Subsea Production Systems Prior to Decommissioning or Component Disconnection
Authors: Lucas Cantinelli Sevillano, Milan Stanko, Sigbjoern Sangesland
Date: 2024-10-21 00:00:00

This paper outlines a novel subsea flushing system which uses a subsea tool to improve the performance of the flushing operation. The new method outlined in this paper uses a small-diameter, high-pressure supply line and a subsea deployed tool containing a pump which recirculates the cleaning fluid through the component or system to be retrieved. The main benefit of this method when compared against conventional practices is that it allows achieving higher fluid speeds inside the subsea equipment being flushed, while injecting smaller flow rates from the surface vessel. The high fluid speeds are achieved with the recirculation pump. The higher fluid speeds ensure efficient sweeping of hydrocarbons from complex paths. A reduced flow rate from the surface vessel also allows a small diameter high pressure supply line to be used, which allows for reduced weight and storage. The study is a numerical simulation of the method applied to a subsea jumper geometry. The injection flow rates required to achieve an efficient flushing were determined from previous experimental work. Calculations were made to estimate the pressure and power requirements for performing the flushing operation as well as the design requirements for the supply line concerning dimensions, material properties and the storage space needed on the support vessel. The performance of the proposed novel system was compared to that of conventional flushing systems. As environmental concerns increase, the presented method has the potential to make the flushing process more efficient while reducing costs associated with support vessels and the materials needed. The novel system may also be deployed using a low-cost Inspection Maintenance and Repair (IMR) vessel. The subsea tool is connected to the subsea production system, either through dedicated connection ports or using pipe clamp connectors with pipe wall penetrators.
Nonlinear Bayesian Filtering with Natural Gradient Gaussian Approximation
Authors: Wenhan Cao, Tianyi Zhang, Zeju Sun, Chang Liu, Stephen S. -T. Yau, Shengbo Eben Li
Date: 2024-10-21 00:00:00
Tasks: LEMMA

Practical Bayes filters often assume the state distribution of each time step to be Gaussian for computational tractability, resulting in the so-called Gaussian filters. When facing nonlinear systems, Gaussian filters such as extended Kalman filter (EKF) or unscented Kalman filter (UKF) typically rely on certain linearization techniques, which can introduce large estimation errors. To address this issue, this paper reconstructs the prediction and update steps of Gaussian filtering as solutions to two distinct optimization problems, whose optimal conditions are found to have analytical forms from Stein's lemma. It is observed that the stationary point for the prediction step requires calculating the first two moments of the prior distribution, which is equivalent to that step in existing moment-matching filters. In the update step, instead of linearizing the model to approximate the stationary points, we propose an iterative approach to directly minimize the update step's objective to avoid linearization errors. For the purpose of performing the steepest descent on the Gaussian manifold, we derive its natural gradient that leverages Fisher information matrix to adjust the gradient direction, accounting for the curvature of the parameter space. Combining this update step with moment matching in the prediction step, we introduce a new iterative filter for nonlinear systems called \textit{N}atural Gr\textit{a}dient Gaussia\textit{n} Appr\textit{o}ximation filter, or NANO filter for short. We prove that NANO filter locally converges to the optimal Gaussian approximation at each time step. Furthermore, the estimation error is proven exponentially bounded for nearly linear measurement equation and low noise levels through constructing a supermartingale-like property across consecutive time steps.
SPARC: Prediction-Based Safe Control for Coupled Controllable and Uncontrollable Agents with Conformal Predictions
Authors: Shuqi Wang, Siqi Wang, ShaoYuan Li, Xiang Yin
Date: 2024-10-21 00:00:00
Tasks: Autonomous Driving, Conformal Prediction, Prediction

We investigate the problem of safe control synthesis for systems operating in environments with uncontrollable agents whose dynamics are unknown but coupled with those of the controlled system. This scenario naturally arises in various applications, such as autonomous driving and human-robot collaboration, where the behavior of uncontrollable agents, like pedestrians, cannot be directly controlled but is influenced by the actions of the autonomous vehicle or robot. In this paper, we present SPARC (Safe Prediction-Based Robust Controller for Coupled Agents), a novel framework designed to ensure safe control in the presence of coupled uncontrollable agents. SPARC leverages conformal prediction to quantify uncertainty in data-driven prediction of agent behavior. Particularly, we introduce a joint distribution-based approach to account for the coupled dynamics of the controlled system and uncontrollable agents. By integrating the control barrier function (CBF) technique, SPARC provides provable safety guarantees at a high confidence level. We illustrate our framework with a case study involving an autonomous driving scenario with walking pedestrians.
A Global Coordinate-Free Approach to Invariant Contraction on Homogeneous Manifolds
Authors: Akash Harapanahalli, Samuel Coogan
Date: 2024-10-20 00:00:00

In this work, we provide a global condition for contraction with respect to an invariant Riemannian metric on reductive homogeneous spaces. Using left-invariant frames, vector fields on the manifold are horizontally lifted to the ambient Lie group, where the Levi-Civita connection is globally characterized as a real matrix multiplication. By linearizing in these left-invariant frames, we characterize contraction using matrix measures on real square matrices, avoiding the use of local charts. Applying this global condition, we provide a necessary condition for a prescribed subset of the manifold to possibly admit a contracting system with respect to an invariant metric. Applied to the sphere, this condition implies that no closed hemisphere can be contained in a contraction region. Finally, we apply our results to compute reachable sets for an attitude control problem.
A Distributed Primal-Dual Method for Constrained Multi-agent Reinforcement Learning with General Parameterization
Authors: Ali Kahe, Hamed Kebriaei
Date: 2024-10-20 00:00:00
Tasks: Multi-agent Reinforcement Learning

This paper proposes a novel distributed approach for solving a cooperative Constrained Multi-agent Reinforcement Learning (CMARL) problem, where agents seek to minimize a global objective function subject to shared constraints. Unlike existing methods that rely on centralized training or coordination, our approach enables fully decentralized online learning, with each agent maintaining local estimates of both primal and dual variables. Specifically, we develop a distributed primal-dual algorithm based on actor-critic methods, leveraging local information to estimate Lagrangian multipliers. We establish consensus among the Lagrangian multipliers across agents and prove the convergence of our algorithm to an equilibrium point, analyzing the sub-optimality of this equilibrium compared to the exact solution of the unparameterized problem. Furthermore, we introduce a constrained cooperative Cournot game with stochastic dynamics as a test environment to evaluate the algorithm's performance in complex, real-world scenarios.
Multi-class within-day dynamic traffic equilibrium with strategic travel time information
Authors: Xiaoyu Ma, Xiaozheng He
Date: 2024-10-20 00:00:00

Most research on within-day dynamic traffic equilibrium with information provision implicitly considers travel time information, often assuming information to be perfect or imperfect based on travelers' perception error. However, lacking explicit formulation of information limits insightful analysis of information impact on dynamic traffic equilibrium and the potential benefits of leveraging information provision to improve system-level performance. To address this gap, this paper proposes a within-day dynamic traffic equilibrium model that explicitly formulates strategic information provision as an endogenous element. In the proposed framework, two classes of travelers receive different types of travel time information: one class receives instantaneous travel time reflecting the prevailing traffic conditions, while the other class receives strategic forecasts of travel times, generated by accounting for travelers' reactions to instantaneous information based on strategic thinking from behavioral game theory. The resulting multi-class within-day dynamic equilibrium differs from existing models by explicitly modeling information provision and consideration of information consistency. The inherent dynamics of real-time updated traffic information, traffic conditions, and travelers' choice behaviors are analytically modeled, with the resulting dynamic equilibrium formulated as a fixed-point problem. The theoretical propositions and numerical findings offer rich insights into the impact of information on the traffic network, strategic forecast information penetration, the relationship between the proposed equilibrium and traditional dynamic traffic equilibria, and information accuracy. This research contributes to a deeper understanding of the interplay between information and traffic dynamics, paving the way for more effective traffic management strategies.
A Comparative Analysis of Nigeria's Power Sector with and without Grid-Scale Storage: Future Implications for Emission and Renewable Energy Integration
Authors: Stanley Eshiemogie, Peace Aielumoh, Tobechukwu Okamkpa, Miracle Jude, Lois Efe, Andrew Amenaghawon
Date: 2024-10-19 00:00:00
Tasks: Bayesian Optimization

This research proposes a framework for modeling and comparing two electricity scenarios for Nigeria by 2050, focusing on the inclusion and exclusion of electricity storage technologies. A Central Composite Design (CCD) was used to generate a design matrix for data collection, with EnergyPLAN software used to create energy system simulations on the CCD data for four outputs: total annual cost, CO2 emissions, critical excess electricity production (CEEP), and electricity import. Three machine learning algorithms, support vector regression (SVR), extreme gradient boosting (XGBoost), and multi-layer perceptron (MLP), were tuned using Bayesian optimization to develop models mapping the inputs to outputs. A genetic algorithm was employed for multi-objective optimization to determine the optimal input capacities that minimize the outputs. Results indicated that incorporating electricity storage technologies (EST) leads to a 37% increase in renewable electricity sources (RES) share, resulting in a 19.14% reduction in CO2 emissions. EST such as battery energy storage systems (BESS), pumped hydro storage (PHS), and vehicle-to-grid (V2G) storage allow for the storage of the critical excess electricity that comes with increasing RES share. Integrating EST in Nigeria's 2050 energy landscape is crucial for incorporating more renewable electricity sources into the energy system, thereby reducing CO2 emissions and managing excess electricity production. This study outlines a plan for optimal electricity production to meet Nigeria's 2050 demand, highlighting the need for a balanced approach that combines fossil fuels, renewable energy, nuclear power, and advanced storage solutions to achieve a sustainable and efficient electricity system.
Design and Implementation of Hedge Algebra Controller using Recursive Semantic Values for Cart-pole System
Authors: Nhat-Minh Dzoan, Thi-Thoa Mac, Hoang-Hiep Ly, Xuan-Thuan Nguyen
Date: 2024-10-19 00:00:00

This paper presents a novel approach to designing a Hedge Algebra Controller named Hedge Algebra Controller with Recursive Semantic Values (RS-HAC). This approach incorporates several newly introduced concepts, including Semantically Quantifying Simplified Mapping (SQSM) featuring a recursive algorithm, Infinite General Semantization (IGS), and Infinite General De-semantization (IGDS). These innovations aim to enhance the optimizability, scalability, and flexibility of hedge algebra theory, allowing the design of a hedge algebra-based controller to be carried out more efficiently and straightforward. An application of stabilizing an inverted pendulum on a cart is conducted to illustrate the superiority of the proposed approach. Comparisons are made between RS-HAC and a fuzzy controller of Takagi-Sugeno type (FC), as well as a linear quadratic regulator (LQR). The results indicate that the RS-HAC surpasses the FC by up to 400\% in control efficiency and is marginally better than the LQR regarding transient time in balancing an inverted pendulum on a cart.
EDRF: Enhanced Driving Risk Field Based on Multimodal Trajectory Prediction and Its Applications
Authors: Junkai Jiang, Zeyu Han, Yuning Wang, Mengchi Cai, Qingwen Meng, Qing Xu, Jianqiang Wang
Date: 2024-10-19 00:00:00
Tasks: Autonomous Vehicles, Trajectory Planning, Trajectory Prediction

Driving risk assessment is crucial for both autonomous vehicles and human-driven vehicles. The driving risk can be quantified as the product of the probability that an event (such as collision) will occur and the consequence of that event. However, the probability of events occurring is often difficult to predict due to the uncertainty of drivers' or vehicles' behavior. Traditional methods generally employ kinematic-based approaches to predict the future trajectories of entities, which often yield unrealistic prediction results. In this paper, the Enhanced Driving Risk Field (EDRF) model is proposed, integrating deep learning-based multimodal trajectory prediction results with Gaussian distribution models to quantitatively capture the uncertainty of traffic entities' behavior. The applications of the EDRF are also proposed. It is applied across various tasks (traffic risk monitoring, ego-vehicle risk analysis, and motion and trajectory planning) through the defined concept Interaction Risk (IR). Adequate example scenarios are provided for each application to illustrate the effectiveness of the model.