pwc-archive/papers-with-abstracts huggingface dataset:
Reinforcement Learning for Freeway Lane-Change Regulation via Connected Vehicles
Authors: Ke Sun, Huan Yu
Date: 2024-12-05 00:00:00
Tasks: Autonomous Vehicles, Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning

Lane change decision-making is a complex task due to intricate vehicle-vehicle and vehicle-infrastructure interactions. Existing algorithms for lane-change control often depend on vehicles with a certain level of autonomy (e.g., autonomous or connected autonomous vehicles). To address the challenges posed by low penetration rates of autonomous vehicles and the high costs of precise data collection, this study proposes a dynamic lane change regulation design based on multi-agent reinforcement learning (MARL) to enhance freeway traffic efficiency. The proposed framework leverages multi-lane macroscopic traffic models that describe spatial-temporal dynamics of the density and speed for each lane. Lateral traffic flow between adjacent lanes, resulting from aggregated lane-changing behaviors, is modeled as source terms exchanged between the partial differential equations (PDEs). We propose a lane change regulation strategy using MARL, where one agent is placed at each discretized lane grid. The state of each agent is represented by aggregated vehicle attributes within its grid, generated from the SUMO microscopic simulation environment. The agent's actions are lane-change regulations for vehicles in its grid. Specifically, lane-change regulation signals are computed at a centralized traffic management center and then broadcast to connected vehicles in the corresponding lane grids. Compared to vehicle-level maneuver control, this approach achieves a higher regulation rate by leveraging vehicle connectivity while introducing no critical safety concerns, and accommodating varying levels of connectivity and autonomy within the traffic system. The proposed model is simulated and evaluated in varied traffic scenarios and demand conditions. Experimental results demonstrate that the method improves overall traffic efficiency with minimal additional energy consumption while maintaining driving safety.
Action Mapping for Reinforcement Learning in Continuous Environments with Constraints
Authors: Mirco Theile, Lukas Dirnberger, Raphael Trumpp, Marco Caccamo, Alberto L. Sangiovanni-Vincentelli
Date: 2024-12-05 00:00:00
Tasks: Deep Reinforcement Learning

Deep reinforcement learning (DRL) has had success across various domains, but applying it to environments with constraints remains challenging due to poor sample efficiency and slow convergence. Recent literature explored incorporating model knowledge to mitigate these problems, particularly through the use of models that assess the feasibility of proposed actions. However, integrating feasibility models efficiently into DRL pipelines in environments with continuous action spaces is non-trivial. We propose a novel DRL training strategy utilizing action mapping that leverages feasibility models to streamline the learning process. By decoupling the learning of feasible actions from policy optimization, action mapping allows DRL agents to focus on selecting the optimal action from a reduced feasible action set. We demonstrate through experiments that action mapping significantly improves training performance in constrained environments with continuous action spaces, especially with imperfect feasibility models.
Stealthy Optimal Range-Sensor Placement for Target Localization
Authors: Mohammad Hussein Yoosefian Nooshabadi, Rifat Sipahi, Laurent Lessard
Date: 2024-12-05 00:00:00

We study a stealthy range-sensor placement problem where a set of range sensors are to be placed with respect to targets to effectively localize them while maintaining a degree of stealthiness from the targets. This is an open and challenging problem since two competing objectives must be balanced: (a) optimally placing the sensors to maximize their ability to localize the targets and (b) minimizing the information the targets gather regarding the sensors. We provide analytical solutions in 2D for the case of any number of sensors that localize two targets.
A Spatial-Domain Coordinated Control Method for CAVs at Unsignalized Intersections Considering Motion Uncertainty
Authors: Tong Zhao, Nikolce Murgovski, Baigen Cai, Wei Shangguan
Date: 2024-12-05 00:00:00
Tasks: Collision Avoidance, Computational Efficiency, Trajectory Planning

Coordinated control of connected and automated vehicles (CAVs) emerges as a promising technology to improve traffic safety, efficiency, and sustainability. Meanwhile, mixed traffic, where CAVs coexist with conventional human-driven vehicles (HDVs), represents an upcoming and necessary stage in the development of intelligent transportation systems. Considering the motion uncertainty of HDVs, this paper proposes a coordinated control method for trajectory planning of CAVs at an unsignalized intersection in mixed traffic. By sampling in distance and using an exact change of variables, the coordinated control problem is formulated in the spatial domain as a nonlinear program, thereby allowing for unified linear collision avoidance constraints to handle vehicle crossing, following, merging, and diverging conflicts. The motion uncertainty of HDVs is decoupled and modeled as path uncertainty and speed uncertainty, whereby the robustness of collision avoidance is ensured in both spatial and temporal dimensions. The prediction deviation for HDVs is compensated by receding horizon optimization, and a real-time iteration (RTI) scheme is developed to improve computational efficiency. Simulation case studies are conducted to validate the efficacy, robustness, and potential for real-time application of the proposed methods. The results show that the proposed control scheme provides collision-free and smooth trajectories with state and control constraints satisfied. Compared with the converged baseline, the RTI scheme reduces the computation time by orders of magnitude, and the solution deviation is less than 2.3%, demonstrating a favorable trade-off between computational effort and optimality.
Model-Agnostic Meta-Learning for Fault Diagnosis of Induction Motors in Data-Scarce Environments with Varying Operating Conditions and Electric Drive Noise
Authors: Ali Pourghoraba, MohammadSadegh KhajueeZadeh, Ali Amini, Abolfazl Vahedi, Gholam Reza Agah, Akbar Rahideh
Date: 2024-12-05 00:00:00
Tasks: Fault Detection, Fault Diagnosis, Meta-Learning

Reliable mechanical fault detection with limited data is crucial for the effective operation of induction machines, particularly given the real-world challenges present in industrial datasets, such as significant imbalances between healthy and faulty samples and the scarcity of data representing faulty conditions. This research introduces an innovative meta-learning approach to address these issues, focusing on mechanical fault detection in induction motors across diverse operating conditions while mitigating the adverse effects of drive noise in scenarios with limited data. The process of identifying faults under varying operating conditions is framed as a few-shot classification challenge and approached through a model-agnostic meta-learning strategy. Specifically, this approach begins with training a meta-learner across multiple interconnected fault-diagnosis tasks conducted under different operating conditions. In this stage, cross-entropy is utilized to optimize parameters and develop a robust representation of the tasks. Subsequently, the parameters of the meta-learner are fine-tuned for new tasks, enabling rapid adaptation using only a small number of samples. This method achieves excellent accuracy in fault detection across various conditions, even when data availability is restricted. The findings indicate that the proposed model outperforms other sophisticated techniques, providing enhanced generalization and quicker adaptation. The accuracy of fault diagnosis reaches a minimum of 99%, underscoring the model's effectiveness for reliable fault identification.
Exploring Behaviors of Hybrid Systems via the Voronoi Bias over Output Signals
Authors: Gidon Ernst, Jiří Fejlek
Date: 2024-12-05 00:00:00
Tasks: Motion Planning

In this paper, we consider an analysis of temporal properties of hybrid systems based on simulations, so-called falsification of requirements. We present a novel exploration-based algorithm for falsification of black-box models of hybrid systems based on the Voronoi bias in the output space. This approach is inspired by techniques used originally in motion planning: rapidly exploring random trees. Instead of commonly employed exploration that is based on coverage of inputs, the proposed algorithm aims to cover all possible outputs directly. Compared to other state-of-the-art falsification tools, it also does not require robustness or other guidance metrics tied to a specific behavior that is being falsified. This allows our algorithm to falsify specifications for which robustness is not conclusive enough to guide the falsification procedure.
Utilizing phase-type distributions for queueing-based railway junction performance determination
Authors: Tamme Emunds, Nils Nießen
Date: 2024-12-05 00:00:00

To ensure the effective and objective development of transportation networks, it is crucial to identify performance limitations across various subsystems. A timetable-independent assessment of infrastructure capacity at railway junctions is a fundamental aspect of long-term rail network planning. While recent research introduced queueing-based methods to quantify route-based railway junction performance, modelling arrival and service processes has been limited to exponential distributions. This work utilizes Phase-Type Distributions to propose an extension to a previously described Continuous-Time Markov Chain model. In a comparison between assumed distribution combinations, the effect of a more detailed stochastic process modelling is described. Furthermore, an analysis of the differences to a simulation method is conducted for an exemplary railway junction. The introduced method enables infrastructure managers to accurately model stochastic processes for performance determination in the early stages of the strategic planning phase.
Non-Asymptotic Bounds for Closed-Loop Identification of Unstable Nonlinear Stochastic Systems
Authors: Seth Siriya, Jingge Zhu, Dragan Nešić, Ye Pu
Date: 2024-12-05 00:00:00
Tasks: parameter estimation

We consider the problem of least squares parameter estimation from single-trajectory data for discrete-time, unstable, closed-loop nonlinear stochastic systems, with linearly parameterised uncertainty. Assuming a region of the state space produces informative data, and the system is sub-exponentially unstable, we establish non-asymptotic guarantees on the estimation error at times where the state trajectory evolves in this region. If the whole state space is informative, high probability guarantees on the error hold for all times. Examples are provided where our results are useful for analysis, but existing results are not.
Decentralized Dynamic Event-triggered Output-feedback Control of Stochastic Non-triangular Interconnected Systems with Unknown Time-varying Sensor Sensitivity
Authors: Libei Sun, Yongduan Song, Maolong Lv
Date: 2024-12-05 00:00:00
Tasks: Sensitivity

This study addresses the intricate challenge of decentralized output-feedback control for stochastic non-triangular nonlinear interconnected systems with unknown time-varying sensor sensitivity in a dynamic event-triggered context. The presence of stochastic disturbances, non-triangular structural uncertainties, and evolving sensor sensitivity distinguishes this problem of global asymptotic stability from conventional event-triggered control scenarios. Existing event-triggered control approaches with static event conditions encounter difficulties in simultaneously ensuring zero tracking/stabilization error and preventing the occurrence of Zeno behavior. In this work, we develop a novel solution to address this complex issue. Firstly, we establish a linear relationship between the state vector of each interconnected subsystem and two error vectors through a unique coordinate transformation. This transformation effectively handles the complexities introduced by non-triangular structural uncertainties. Secondly, we introduce a decentralized dynamic event-triggered output-feedback control strategy, which involves a state observer and a decentralized output-feedback controller. Unlike conventional event-triggered control methods with static event conditions, this strategy formulates a modified clock-based dynamic triggering mechanism by introducing an auxiliary variable that evolves based on predicted plant state values, while utilizing a clock variable to guarantee the existence of a positive lower bound on inter-execution times. Rigorous Lyapunov analysis confirms the global asymptotic stability in probability of the closed-loop system, with the states and the output of each local subsystem converging to the equilibrium at the origin in probability. Additionally, the existence of a minimal dwell-time between triggering instants is guaranteed.
Optimal demand response policies for inertial thermal loads under stochastic renewable sources
Authors: Gaurav Sharma, P R Kumar
Date: 2024-12-05 00:00:00

In this paper, we consider the problem of preferentially utilizing intermittent renewable power, such as wind, optimally to support thermal inertial loads in a microgrid environment. Thermal inertial loads can be programmed to preferentially consume from renewable sources. The flexibility in power consumption of inertial loads therefore can be used to absorb the fluctuations in intermittently available renewable power sources, and promote reduction of fossil fuel based costly non-renewable generation. Under a model which promotes renewable consumption by penalizing the non-renewable, but does not account for variations in the end-user requirements, the optimal solution leads to all the users' temperatures behave in a lockstep fashion, that is the power is allocated in such a fashion that all the temperatures are brought to a common value and they are kept the same after that point, resulting in synchronization among all the loads. In the first part, we showed that under a model which additionally penalizes the comfort range violation, the optimal solution is in-fact of desynchronization nature, where the temperatures are intentionally kept apart to avoid power surges resulting from simultaneous comfort violation from many loads.
Semi-automated transmission control for motorcycle gearshift: design, data-driven tuning and experimental validation
Authors: Edoardo Catenaro, Giulio Panzani, Davide Sette, Sergio M. Savaresi
Date: 2024-12-05 00:00:00
Tasks: Bayesian Optimization

This brief addresses the gearshifting problem for Semi-Automated Manual Transmissions (S-AMT) in powered two-wheelers, a powertrain setup that allows fast and smooth gear shifts with minimal modifications to the traditional manual powertrain layout. We show that with a proper synchronization between the electronic clutch and engine torque, excellent gearshift performance can be obtained, but requires precise parameters calibration. We thus propose the use of a data-driven approach with Constrained Bayesian Optimization to optimize control parameters. The procedure's effectiveness is demonstrated on a real vehicle, assessing performance in terms of optimality, convergence rate, and repeatability.
A Physics-Informed Scenario Approach with Data Mitigation for Safety Verification of Nonlinear Systems
Authors: Ali Aminzadeh, MohammadHossein Ashoori, Amy Nejati, Abolfazl Lavaei
Date: 2024-12-05 00:00:00

This paper develops a physics-informed scenario approach for safety verification of nonlinear systems using barrier certificates (BCs) to ensure that system trajectories remain within safe regions over an infinite time horizon. Designing BCs often relies on an accurate dynamics model; however, such models are often imprecise due to the model complexity involved, particularly when dealing with highly nonlinear systems. In such cases, while scenario approaches effectively address the safety problem using collected data to construct a guaranteed BC for the dynamical system, they often require substantial amounts of data-sometimes millions of samples-due to exponential sample complexity. To address this, we propose a physics-informed scenario approach that selects data samples such that the outputs of the physics-based model and the observed data are sufficiently close (within a specified threshold). This approach guides the scenario optimization process to eliminate redundant samples and significantly reduce the required dataset size. We demonstrate the capability of our approach in mitigating the amount of data required for scenario optimizations with both deterministic (i.e., confidence 1) and probabilistic (i.e., confidence between 0 and 1) guarantees. We validate our physics-informed scenario approach through two physical case studies, showcasing its practical application in reducing the required data.
Traffic Co-Simulation Framework Empowered by Infrastructure Camera Sensing and Reinforcement Learning
Authors: Talha Azfar, Ruimin Ke
Date: 2024-12-05 00:00:00
Tasks: Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Traffic Signal Control, vehicle detection

Traffic simulations are commonly used to optimize traffic flow, with reinforcement learning (RL) showing promising potential for automated traffic signal control. Multi-agent reinforcement learning (MARL) is particularly effective for learning control strategies for traffic lights in a network using iterative simulations. However, existing methods often assume perfect vehicle detection, which overlooks real-world limitations related to infrastructure availability and sensor reliability. This study proposes a co-simulation framework integrating CARLA and SUMO, which combines high-fidelity 3D modeling with large-scale traffic flow simulation. Cameras mounted on traffic light poles within the CARLA environment use a YOLO-based computer vision system to detect and count vehicles, providing real-time traffic data as input for adaptive signal control in SUMO. MARL agents, trained with four different reward structures, leverage this visual feedback to optimize signal timings and improve network-wide traffic flow. Experiments in the test-bed demonstrate the effectiveness of the proposed MARL approach in enhancing traffic conditions using real-time camera-based detection. The framework also evaluates the robustness of MARL under faulty or sparse sensing and compares the performance of YOLOv5 and YOLOv8 for vehicle detection. Results show that while better accuracy improves performance, MARL agents can still achieve significant improvements with imperfect detection, demonstrating adaptability for real-world scenarios.
Learning Robust Safety Controllers for Uncertain Input-Affine Polynomial Systems
Authors: Omid Akbarzadeh, Abolfazl Lavaei
Date: 2024-12-05 00:00:00

This paper offers a direct data-driven approach for learning robust control barrier certificates (R-CBCs) and robust safety controllers (R-SCs) for discrete-time input-affine polynomial systems with unknown dynamics under unknown-but-bounded disturbances. The proposed method relies on data from input-state observations collected over a finite-time horizon while satisfying a specific rank condition to ensure the system is persistently excited. Our data-driven scheme enables the synthesis of R-CBCs and R-SCs directly from observed data, bypassing the need for explicit modeling of the system's dynamics and thus ensuring robust system safety against disturbances within a finite time horizon. Our proposed approach is formulated as a sum-of-squares (SOS) optimization problem, providing a structured design framework. Two case studies showcase our method's capability to provide robust safety guarantees for unknown input-affine polynomial systems under bounded disturbances, demonstrating its practical effectiveness.
Certified Learning of Incremental ISS Controllers for Unknown Nonlinear Polynomial Dynamics
Authors: Mahdieh Zaker, David Angeli, Abolfazl Lavaei
Date: 2024-12-05 00:00:00
Tasks: LEMMA

Incremental input-to-state stability (delta-ISS) offers a robust framework to ensure that small input variations result in proportionally minor deviations in the state of a nonlinear system. This property is essential in practical applications where input precision cannot be guaranteed. However, analyzing delta-ISS demands detailed knowledge of system dynamics to assess the state's incremental response to input changes, posing a challenge in real-world scenarios where mathematical models are unknown. In this work, we develop a data-driven approach to design delta-ISS Lyapunov functions together with their corresponding delta-ISS controllers for continuous-time input-affine nonlinear systems with polynomial dynamics, ensuring the delta-ISS property is achieved without requiring knowledge of the system dynamics. In our data-driven scheme, we collect only two sets of input-state trajectories from sufficiently excited dynamics, as introduced by Willems et al.'s fundamental lemma. By fulfilling a specific rank condition, we design delta-ISS controllers using the collected samples through formulating a sum-of-squares optimization program. The effectiveness of our data-driven approach is evidenced by its application on a physical case study.
Abstraction-based Control of Unknown Continuous-Space Models with Just Two Trajectories
Authors: Behrad Samari, Mahdieh Zaker, Abolfazl Lavaei
Date: 2024-12-05 00:00:00

Finite abstractions (a.k.a. symbolic models) offer an effective scheme for approximating the complex continuous-space systems with simpler models in the discrete-space domain. A crucial aspect, however, is to establish a formal relation between the original system and its symbolic model, ensuring that a discrete controller designed for the symbolic model can be effectively implemented as a hybrid controller (using an interface map) for the original system. This task becomes even more challenging when the exact mathematical model of the continuous-space system is unknown. To address this, the existing literature mainly employs scenario-based data-driven methods, which require collecting a large amount of data from the original system. In this work, we propose a data-driven framework that utilizes only two input-state trajectories collected from unknown nonlinear polynomial systems to synthesize a hybrid controller, enabling the desired behavior on the unknown system through the controller derived from its symbolic model. To accomplish this, we employ the concept of alternating simulation functions (ASFs) to quantify the closeness between the state trajectories of the unknown system and its data-driven symbolic model. By satisfying a specific rank condition on the collected data, which intuitively ensures that the unknown system is persistently excited, we directly design an ASF and its corresponding hybrid controller using finite-length data without explicitly identifying the unknown system, while providing correctness guarantees. This is achieved through proposing a data-based sum-of-squares (SOS) optimization program, enabling a systematic approach to the design process. We illustrate the effectiveness of our data-driven approach through a case study.