pwc-archive/papers-with-abstracts huggingface dataset:
Policy Verification in Stochastic Dynamical Systems Using Logarithmic Neural Certificates
Authors: Thom Badings, Wietze Koops, Sebastian Junges, Nils Jansen
Date: 2024-06-02 00:00:00

We consider the verification of neural network policies for discrete-time stochastic systems with respect to reach-avoid specifications. We use a learner-verifier procedure that learns a certificate for the specification, represented as a neural network. Verifying that this neural network certificate is a so-called reach-avoid supermartingale (RASM) proves the satisfaction of a reach-avoid specification. Existing approaches for such a verification task rely on computed Lipschitz constants of neural networks. These approaches struggle with large Lipschitz constants, especially for reach-avoid specifications with high threshold probabilities. We present two key contributions to obtain smaller Lipschitz constants than existing approaches. First, we introduce logarithmic RASMs (logRASMs), which take exponentially smaller values than RASMs and hence have lower theoretical Lipschitz constants. Second, we present a fast method to compute tighter upper bounds on Lipschitz constants based on weighted norms. Our empirical evaluation shows we can consistently verify the satisfaction of reach-avoid specifications with probabilities as high as 99.9999%.
A Digital Twin Framework for Reinforcement Learning with Real-Time Self-Improvement via Human Assistive Teleoperation
Authors: Kabirat Olayemi, Mien Van, Luke Maguire, Sean McLoone
Date: 2024-06-02 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning (RL)

Reinforcement Learning (RL) or Deep Reinforcement Learning (DRL) is a powerful approach to solving Markov Decision Processes (MDPs) when the model of the environment is not known a priori. However, RL models are still faced with challenges such as handling covariate shifts and ensuring the quality of human demonstration. To address these challenges and further advance DRL models, our work develops a human-in-the-loop DRL framework via digital twin that leverages human intelligence after deployment to retrain the DRL model in real time. First, we develop a pre-trained model fully based on learning through trial and error in the simulated environment allowing scalability and automation while eliminating variability and biases that can come from subjective human guidance. Second, instead of deploying the trained model directly on the UGV, we create a digital twin which controls the physical UGV from the virtual environment. Third, to allow continuous learning without catastrophic forgetting, we introduce the ability of the model to self-improve with the help of small human guidance at the start of the retraining. We test the performance of our proposed model in both simulation and real-world environments with both static and dynamic obstacles. The results indicate that our proposed approach not only outperforms the baseline models in terms of reward accumulation but also demonstrates superior training efficiency.
Multi-technology co-optimization approach for sustainable hydrogen and electricity supply chains considering variability and demand scale
Authors: Sunwoo Kim, Joungho Park, Jay H. Lee
Date: 2024-06-02 00:00:00

In the pursuit of a carbon-neutral future, hydrogen emerges as a pivotal element, serving as a carbon-free energy carrier and feedstock. As efforts to decarbonize sectors such as heating and transportation intensify, understanding and navigating through the dynamics of hydrogen demand expansion becomes critical. Transitioning to hydrogen economy is complicated by varying regional scales and types of hydrogen demand, with forecasts indicating a rise in variable demand that calls for diverse production technologies. Currently, steam methane reforming is prevalent, but its significant carbon emissions make a shift to cleaner alternatives like blue and green hydrogen imperative. Each production method possesses distinct characteristics, necessitating a thorough exploration and co-optimization with electricity supply chains as well as carbon capture, utilization, and storage systems. Our study fills existing research gaps by introducing a superstructure optimization framework that accommodates various demand scenarios and technologies. Through case studies in California, we underscore the critical role of demand profiles in shaping the optimal configurations and economics of supply chains and emphasize the need for diversified portfolios and co-optimization to facilitate sustainable energy transitions.
Integrating solid direct air capture systems with green hydrogen production: Economic synergy of sector coupling
Authors: Sunwoo Kim, Joungho Park, Jay H. Lee
Date: 2024-06-02 00:00:00

In the global pursuit of sustainable energy solutions, mitigating carbon dioxide (CO2) emissions stands as a pivotal challenge. With escalating atmospheric CO2 levels, the imperative of direct air capture (DAC) systems becomes evident. Simultaneously, green hydrogen (GH) emerges as a pivotal medium for renewable energy. Nevertheless, the substantial expenses associated with these technologies impede widespread adoption, primarily due to significant installation costs and underutilized operational advantages when deployed independently. Integration through sector coupling enhances system efficiency and sustainability, while shared power sources and energy storage devices offer additional economic benefits. In this study, we assess the economic viability of polymer electrolyte membrane electrolyzers versus alkaline electrolyzers within the context of sector coupling. Our findings indicate that combining GH production with solid DAC systems yields significant economic advantages, with approximately a 10% improvement for PEM electrolyzers and a 20% enhancement for alkaline electrolyzers. These results highlight a substantial opportunity to improve the efficiency and economic viability of renewable energy and green hydrogen initiatives, thereby facilitating the broader adoption of cleaner technologies.
Log-Scale Quantization in Distributed First-Order Methods: Gradient-based Learning from Distributed Data
Authors: Mohammadreza Doostmohammadian, Muhammad I. Qureshi, Mohammad Hossein Khalesi, Hamid R. Rabiee, Usman A. Khan
Date: 2024-06-02 00:00:00
Tasks: Distributed Optimization, Quantization

Decentralized strategies are of interest for learning from large-scale data over networks. This paper studies learning over a network of geographically distributed nodes/agents subject to quantization. Each node possesses a private local cost function, collectively contributing to a global cost function, which the considered methodology aims to minimize. In contrast to many existing papers, the information exchange among nodes is log-quantized to address limited network-bandwidth in practical situations. We consider a first-order computationally efficient distributed optimization algorithm (with no extra inner consensus loop) that leverages node-level gradient correction based on local data and network-level gradient aggregation only over nearby nodes. This method only requires balanced networks with no need for stochastic weight design. It can handle log-scale quantized data exchange over possibly time-varying and switching network setups. We study convergence over both structured networks (for example, training over data-centers) and ad-hoc multi-agent networks (for example, training over dynamic robotic networks). Through experimental validation, we show that (i) structured networks generally result in a smaller optimality gap, and (ii) log-scale quantization leads to a smaller optimality gap compared to uniform quantization.
Hydrogen and Battery Based Energy Storage System (ESS) for Future DC Microgrids
Authors: Massiagbe Diabate, Timothy Vriend, Harish S Krishnamoorthy, Jian Shi
Date: 2024-06-02 00:00:00

In this paper, a hydrogen-based energy storage system (ESS) is proposed for DC microgrids, which can potentially be integrated with battery ESS to meet the needs of future grids with high renewable penetration. Hydrogen-based ESS can provide a stable energy supply for a long time but has a slower response than battery ESSs. However, a combination of battery and hydrogen storage provides stable energy for an extended period of time and can easily handle the sudden demands and surpluses of the microgrid. One of the main challenges in this system is the integration of power electronics with fuel cell technology to convert renewable energy into electricity seamlessly. This paper proposes a system that uses an isolated DC-DC converter to activate clean hydrogen production using an electrolyzer and then pressurize the hydrogen to store in a tank. The pressured hydrogen becomes an essential input to the fuel cell, which regulates and transforms it into electricity. The electricity produced is then transferred to the grid using a DC-DC boost converter. A Simulink model of the hybrid system with a 1 kV DC bus voltage is used to demonstrate the hydrogen production and fuel cell behavior based on the demand and surplus power of the loads. The proposed system simulates aspects of the power conversion, electrolyzer, storage tank, and fuel cell needed for the proposed hybrid ESS. Due to its economic feasibility, the polymer electrolyte membrane (PEM) is the primary technology considered for the electrolyzer and fuel cell.
Model Predictive Control and Reinforcement Learning: A Unified Framework Based on Dynamic Programming
Authors: Dimitri P. Bertsekas
Date: 2024-06-02 00:00:00
Tasks: Model Predictive Control, Reinforcement Learning (RL)

In this paper we describe a new conceptual framework that connects approximate Dynamic Programming (DP), Model Predictive Control (MPC), and Reinforcement Learning (RL). This framework centers around two algorithms, which are designed largely independently of each other and operate in synergy through the powerful mechanism of Newton's method. We call them the off-line training and the on-line play algorithms. The names are borrowed from some of the major successes of RL involving games; primary examples are the recent (2017) AlphaZero program (which plays chess, [SHS17], [SSS17]), and the similarly structured and earlier (1990s) TD-Gammon program (which plays backgammon, [Tes94], [Tes95], [TeG96]). In these game contexts, the off-line training algorithm is the method used to teach the program how to evaluate positions and to generate good moves at any given position, while the on-line play algorithm is the method used to play in real time against human or computer opponents. Significantly, the synergy between off-line training and on-line play also underlies MPC (as well as other major classes of sequential decision problems), and indeed the MPC design architecture is very similar to the one of AlphaZero and TD-Gammon. This conceptual insight provides a vehicle for bridging the cultural gap between RL and MPC, and sheds new light on some fundamental issues in MPC. These include the enhancement of stability properties through rollout, the treatment of uncertainty through the use of certainty equivalence, the resilience of MPC in adaptive control settings that involve changing system parameters, and the insights provided by the superlinear performance bounds implied by Newton's method.
Optimal Transmission Power Scheduling for Networked Control System under DoS Attack
Authors: Siyi Wang, Yulong Gao, Sandra Hirche
Date: 2024-06-01 00:00:00
Tasks: Scheduling

Designing networked control systems that are reliable and resilient against adversarial threats, is essential for ensuring the security of cyber-physical systems. This paper addresses the communication-control co-design problem for networked control systems under denial-of-service (DoS) attacks. In the wireless channel, a transmission power scheduler periodically determines the power level for sensory data transmission. Yet DoS attacks render data packets unavailable by disrupting the communication channel. This paper co-designs the control and power scheduling laws in the presence of DoS attacks and aims to minimize the sum of regulation control performance and transmission power consumption. Both finite- and infinite-horizon discounted cost criteria are addressed, respectively. By delving into the information structure between the controller and the power scheduler under attack, the original co-design problem is divided into two subproblems that can be solved individually without compromising optimality. The optimal control is shown to be certainty equivalent, and the optimal transmission power scheduling is solved using a dynamic programming approach. Moreover, in the infinite-horizon scenario, we analyze the performance of the designed scheduling policy and develop an upper bound of the total costs. Finally, a numerical example is provided to demonstrate the theoretical results.
Deep Learning based Performance Testing for Analog Integrated Circuits
Authors: Jiawei Cao, Chongtao Guo, Hao Li, Zhigang Wang, Houjun Wang, Geoffrey Ye Li
Date: 2024-06-01 00:00:00
Tasks: Deep Learning

In this paper, we propose a deep learning based performance testing framework to minimize the number of required test modules while guaranteeing the accuracy requirement, where a test module corresponds to a combination of one circuit and one stimulus. First, we apply a deep neural network (DNN) to establish the mapping from the response of the circuit under test (CUT) in each module to all specifications to be tested. Then, the required test modules are selected by solving a 0-1 integer programming problem. Finally, the predictions from the selected test modules are combined by a DNN to form the specification estimations. The simulation results validate the proposed approach in terms of testing accuracy and cost.
Modal Analysis of Power System with High CIG Penetration Based on Impedance Models
Authors: Le Zheng, Jiajie Zheng, Jiajian Lin, Chongru Liu
Date: 2024-06-01 00:00:00
Tasks: Sensitivity

This paper explores the modal analysis of power systems with high Converter-Interfaced Generation (CIG) penetration utilizing an impedance-based modeling approach. Traditional modal analysis based on the state-space model (MASS) requires comprehensive control structures and parameters of each system element, a challenging prerequisite as converters increasingly integrate into power systems and their internal specifics remain largely inaccessible. Conversely, the proposed modal analysis based on the impedance model (MAI) leverages only the impedance port characteristics to pinpoint system elements significantly influencing unstable modes. This study is the first to confirm the theoretical equivalency between MASS and MAI in terms of transfer functions, eigenvalues, and sensitivities, thus bridging the gap between detailed theoretical modeling and practical, accessible analyses. We further provide enhancements to the MAI method, including a revised element participation index, a transformer ratio-based admittance sensitivity adjustment, and an impedance splitting-based sensitivity analysis considering parameter variations. Validation through numerical simulations on a modified IEEE 14-bus system underscores the efficacy of our approach. By examining the interplay between different elements and system modes in high CIG environments, this study offers insights and a foundational framework for delineating the oscillatory modes' participation and stability characteristics of power systems with substantial CIG integration.
Reconfigurable FPGA-Based Solvers For Sparse Satellite Control
Authors: Anis Hamadouche, Yun Wu, Mathini Sellathurai, Andrew M. Wallace, Joao F. C. Mota
Date: 2024-06-01 00:00:00

This paper introduces a novel reconfigurable and power-efficient FPGA (Field-Programmable Gate Array) implementation of an operator splitting algorithm for Non-Terrestial Network's (NTN) relay satellites model predictive orientation control (MPC). Our approach ensures system stability and introduces an innovative reconfigurable bit-width FPGA-based optimization solver. To demonstrate its efficacy, we employ a real FPGA-In-the-Loop hardware setup to control simulated satellite dynamics. Furthermore, we conduct an in-depth comparative analysis, examining various fixed-point configurations to evaluate the combined system's closed-loop performance and power efficiency, providing a holistic understanding of the proposed implementation's advantages.
Performance Evaluation of Damping Systems in Civil Engineering Structures Via Minimal Sensor
Authors: Xinhao He, Dan Li
Date: 2024-06-01 00:00:00

To control structural responses under various actions, the growing use of supplementary damping systems in modern civil engineering structures necessitates inspecting and evaluating their operational performance postinstallation. However, due to the dispersed placement and complex nonlinearities of these devices, difficulties arise in determining minimal sensor configuration. This is inherently connected to a pivotal challenge: establishing a reliable input-output mapping, which comprises both the mathematical model and sensor arrangements. Prior work indicates this can be achieved through theoretical observability analysis or Lie symmetries analysis, both of which provide different perspectives on the existence of a way to access the solutions of a system identification problem uniquely (at least locally). The present study introduces a unified framework, enhanced by algorithm realization as an application guide, for analyzing the observability and Lie symmetries of a given input-output mapping. We demonstrate its implementation via examples of a building structure with various damping systems under different conditions such as seismic loads, wind loads, and operational vibrations. Finally, we present a case study for an isolation building with an inerter damper and minimal sensor arrangement under seismic action. The results demonstrate that the unscented Kalman filter, a system identification method, can precisely estimate structural responses and assess damping device performance once a reliable input-output mapping is established.
Quasiperiodic Disturbance Observer for Wideband Harmonic Suppression
Authors: Hisayoshi Muramatsu
Date: 2024-06-01 00:00:00

Periodic disturbances composed of harmonics typically occur during periodic operations, impairing performance of mechanical and electrical systems. To improve the performance, control of periodic-disturbance suppression has been studied, such as repetitive control and periodic-disturbance observers. However, actual periodic disturbances are typically quasiperiodic owing to perturbations in each cycle, identification errors of the period, variations in the period, and/or aperiodic disturbances. For robustness against quasiperiodicity, although wideband harmonic suppression is expected, conventional methods have trade-offs among harmonic suppression bandwidth, amplification of aperiodic disturbances, and deviation of harmonic suppression frequencies. This paper proposes a quasiperiodic disturbance observer to compensate for quasiperiodic disturbances while simultaneously achieving the wideband harmonic suppression, non-amplification of aperiodic disturbances, and proper harmonic suppression frequencies. A quasiperiodic disturbance is defined as comprising harmonics and surrounding signals. On the basis of this definition, the quasiperiodic disturbance observer is designed using a periodic-pass filter of a first-order periodic/aperiodic separation filter for its Q-filter, time delay integrated with a zero-phase low-pass filter, and an inverse plant model with a first-order low-pass filter. The periodic-pass filter achieves the wideband harmonic suppression while the zero-phase and first-order low-pass filters prevent the amplification of aperiodic disturbances and deviation of harmonic suppression frequencies. For the implementation, the Q-filter is discretized by an exact mapping of the s-plane to the z-plane, and the inverse plant model is discretized by the backward Euler method.
Learning to Stabilize Unknown LTI Systems on a Single Trajectory under Stochastic Noise
Authors: Ziyi Zhang, Yorie Nakahira, Guannan Qu
Date: 2024-05-31 00:00:00

We study the problem of learning to stabilize unknown noisy Linear Time-Invariant (LTI) systems on a single trajectory. It is well known in the literature that the learn-to-stabilize problem suffers from exponential blow-up in which the state norm blows up in the order of $\Theta(2^n)$ where $n$ is the state space dimension. This blow-up is due to the open-loop instability when exploring the $n$-dimensional state space. To address this issue, we develop a novel algorithm that decouples the unstable subspace of the LTI system from the stable subspace, based on which the algorithm only explores and stabilizes the unstable subspace, the dimension of which can be much smaller than $n$. With a new singular-value-decomposition(SVD)-based analytical framework, we prove that the system is stabilized before the state norm reaches $2^{O(k \log n)}$, where $k$ is the dimension of the unstable subspace. Critically, this bound avoids exponential blow-up in state dimension in the order of $\Theta(2^n)$ as in the previous works, and to the best of our knowledge, this is the first paper to avoid exponential blow-up in dimension for stabilizing LTI systems with noise.
Estimates on the domain of validity for Lyapunov-Schmidt reduction
Authors: Pranav Gupta, Anastasia Bizyaeva, Ravi Banavar
Date: 2024-05-30 00:00:00
Tasks: Dimensionality Reduction

Lyapunov-Schmidt reduction is a dimensionality reduction technique in nonlinear systems analysis that is commonly utilised in the study of bifurcation problems in high-dimensional systems. The method is a systematic procedure for reducing the dimensionality of systems of algebraic equations that have singular points, preserving essential features of their solution sets. In this article, we establish estimates for the region of validity of the reduction by leveraging recently derived bounds on the Implicit Function Theorem. We then apply these bounds to an illustrative example of a two-dimensional system with a pitchfork bifurcation.
Optimizing Bio-energy Supply Chain to Achieve Alternative Energy Targets
Authors: Jubin Thomas
Date: 2024-05-29 00:00:00

In response to global warming and the dwindling reservoirs of fossil fuels, Thailand has increasingly embraced alternative energy sources. Central to its energy development strategy is the Alternative Energy Development Plan (AEDP), which aims to reduce energy intensity, capitalize on residual resources, and mitigate greenhouse gas emissions. While significant strides have been made in meeting various consumption targets set forth by the AEDP, notable challenges persist, particularly in the realms of bio-mass-derived electricity generation, bio-gas utilization, and bio-ethanol production from bio-mass. Therefore, this study delves into the factors contributing to the shortfall in achieving AEDP targets and proposes strategies to enhance the efficiency of the bioenergy supply chain. Leveraging mathematical and linear programming techniques, our research optimizes the supply chain dynamics, accounting for monthly supplier profiles spanning 77 provinces, 17 distinct biomasses, and 427 bioenergy plants equipped with diverse energy conversion technologies. Our findings indicate that Thailand currently boasts adequate bio-mass resources to fulfill the electricity and bio-ethanol targets outlined in the AEDP provided enhancements are made to supply chain efficiency. To fully realize the objectives of the AEDP, we recommend augmenting bio-mass cultivation efforts and implementing new power plant installations. Additionally, we advocate for the consideration of high-methane content fuels, such as solid waste, as a means to alleviate bio-mass demand. This study underscores the paramount importance of strategic planning and optimization in propelling Thailand towards its alternative energy ambitions while surmounting supply chain impediments.