pwc-archive/papers-with-abstracts huggingface dataset:
Black-Box Safety Validation of Autonomous Systems: A Multi-Fidelity Reinforcement Learning Approach
Authors: Jared J. Beard, Ali Baheri
Date: 2022-03-07 00:00:00
Tasks: Decision Making, Reinforcement Learning (RL)

The increasing use of autonomous and semi-autonomous agents in society has made it crucial to validate their safety. However, the complex scenarios in which they are used may make formal verification impossible. To address this challenge, simulation-based safety validation is employed to test the complex system. Recent approaches using reinforcement learning are prone to excessive exploitation of known failures and a lack of coverage in the space of failures. To address this limitation, a type of Markov decision process called the "knowledge MDP" has been defined. This approach takes into account both the learned model and its metadata, such as sample counts, in estimating the system's knowledge through the "knows what it knows" framework. A novel algorithm that extends bidirectional learning to multiple fidelities of simulators has been developed to solve the safety validation problem. The effectiveness of this approach is demonstrated through a case study in which an adversary is trained to intercept a test model in a grid-world environment. Monte Carlo trials compare the sample efficiency of the proposed algorithm to learning with a single-fidelity simulator and show the importance of incorporating knowledge about learned models into the decision-making process.
Scalable multi-agent reinforcement learning for distributed control of residential energy flexibility
Authors: Flora Charbonnier, Thomas Morstyn, Malcolm D. McCulloch
Date: 2022-03-07 00:00:00
Tasks: Multi-agent Reinforcement Learning, Q-Learning, reinforcement-learning, Reinforcement Learning (RL)

This paper proposes a novel scalable type of multi-agent reinforcement learning-based coordination for distributed residential energy. Cooperating agents learn to control the flexibility offered by electric vehicles, space heating and flexible loads in a partially observable stochastic environment. In the standard independent Q-learning approach, the coordination performance of agents under partial observability drops at scale in stochastic environments. Here, the novel combination of learning from off-line convex optimisations on historical data and isolating marginal contributions to total rewards in reward signals increases stability and performance at scale. Using fixed-size Q-tables, prosumers are able to assess their marginal impact on total system objectives without sharing personal data either with each other or with a central coordinator. Case studies are used to assess the fitness of different combinations of exploration sources, reward definitions, and multi-agent learning frameworks. It is demonstrated that the proposed strategies create value at individual and system levels thanks to reductions in the costs of energy imports, losses, distribution network congestion, battery depreciation and greenhouse gas emissions.
Stochastic Aperiodic Control of Networked Systems with i.i.d. Time-Varying Communication Delays
Authors: Yohei Hosoe
Date: 2022-03-07 00:00:00

This paper studies stochastic aperiodic stabilization of a networked control system (NCS) consisting of a continuous-time plant and a discrete-time controller. The plant and the controller are assumed to be connected by communication channels with i.i.d. time-varying delays. The delays are theoretically not required to be bounded even when the plant is unstable in the deterministic sense. In our NCS, the sampling interval is supposed to be determined directly by such communication delays. A necessary and sufficient inequality condition is presented for designing a state-feedback controller stabilizing the NCS at sampling points in a stochastic sense. The results are also illustrated numerically.
Co-Optimization of On-Ramp Merging and Plug-In Hybrid Electric Vehicle Power Split Using Deep Reinforcement Learning
Authors: Yuan Lin, John McPhee, Nasser L. Azad
Date: 2022-03-07 00:00:00
Tasks: Deep Reinforcement Learning, energy management, Management

Current research on Deep Reinforcement Learning (DRL) for automated on-ramp merging neglects vehicle powertrain and dynamics. This work considers automated on-ramp merging for a power-split Plug-In Hybrid Electric Vehicle (PHEV), the 2015 Toyota Prius Plug-In, using DRL. The on-ramp merging control and the PHEV energy management are co-optimized such that the DRL policy directly outputs the power split between the engine and the electric motor. The testing results show that DRL can be successfully used for co-optimization, leading to collision-free on-ramp merging. When compared with sequential approaches wherein the upper-level on-ramp merging control and the lower-level PHEV energy management are performed independently and in sequence, we found that co-optimization results in economic but jerky on-ramp merging while sequential approaches may result in collisions due to neglecting powertrain power limit constraints in designing the upper-level on-ramp merging controller.
Load-Flow Solvability under Security Constraints in DC Distribution Networks
Authors: Zhe Chen, Cong Wang
Date: 2022-03-07 00:00:00

We present sufficient conditions for the load-flow solvability under security constraints in DC distribution networks. In addition, we show that a load-flow solution that fulfills security constraints can be obtained via a convex optimization.
On First Integrals of Hamiltonian System with Holonomic Hamiltonian
Authors: Tomoyuki Iori
Date: 2022-03-07 00:00:00

In this study, the solution of the Hamilton-Jacobi equation (HJE) with holonomic Hamiltonian is investigated in terms of the first integrals of the corresponding Hamiltonian system. Holonomic functions are related to a specific type of partial differential equations called Pfaffian systems, whose solution space can be regarded as a finite-dimensional real vector space. In the finite-dimensional solution space, the existence of first integrals that define a solution of the HJE is characterized by a finite number of algebraic equations for finite-dimensional vectors, which can be easily solved and verified. The derived characterization was illustrated through a numerical example.
Deep Reinforcement Learning based Model-free On-line Dynamic Multi-Microgrid Formation to Enhance Resilience
Authors: Jin Zhao, Member, Fangxing Li, Fellow, Srijib Mukherjee, Senior Member, Christopher Sticht
Date: 2022-03-06 00:00:00
Tasks: Deep Reinforcement Learning, Reinforcement Learning (RL)

Multi-microgrid formation (MMGF) is a promising solution to enhance power system resilience. This paper proposes a new deep reinforcement learning (RL) based model-free on-line dynamic multi-MG formation (MMGF) scheme. The dynamic MMGF problem is formulated as a Markov decision process, and a complete deep RL framework is specially designed for the topology-transformable micro-grids. In order to reduce the large action space caused by flexible switch operations, a topology transformation method is proposed and an action-decoupling Q-value is applied. Then, a CNN based multi-buffer double deep Q-network (CM-DDQN) is developed to further improve the learning ability of original DQN method. The proposed deep RL method provides real-time computing to support on-line dynamic MMGF scheme, and the scheme handles a long-term resilience enhancement problem using adaptive on-line MMGF to defend changeable conditions. The effectiveness of the proposed method is validated using a 7-bus system and the IEEE 123-bus system. The results show strong learning ability, timely response for varying system conditions and convincing resilience enhancement.
Extended Load Flexibility of Industrial P2H Plants: A Process Constraint-Aware Scheduling Approach
Authors: Yiwei Qiu, Buxiang Zhou, Tianlei Zang, Yi Zhou, Ruomei Qi, Jin Lin
Date: 2022-03-06 00:00:00
Tasks: Scheduling

The operational flexibility of industrial power-to-hydrogen (P2H) plants enables admittance of volatile renewable power and provides auxiliary regulatory services for the power grid. Aiming to extend the flexibility of the P2H plant further, this work presents a scheduling method by considering detailed process constraints of the alkaline electrolyzers. Unlike existing works that assume constant load range, the presented scheduling framework fully exploits the dynamic processes of the electrolyzer, including temperature and hydrogen-to-oxygen (HTO) crossover, to improve operational flexibility. Varying energy conversion efficiency under different load levels and temperature is also considered. The scheduling model is solved by proper mathematical transformation as a mixed-integer linear programming (MILP), which determines the on-off-standby states and power levels of different electrolyzers in the P2H plant for daily operation. With experiment-verified constraints, a case study show that compared to the existing scheduling approach, the improved flexibility leads to a 1.627% profit increase when the P2H plant is directly coupled to the photovoltaic power.
Systematic, Lyapunov-Based, Safe and Stabilizing Controller Synthesis for Constrained Nonlinear Systems
Authors: Reza Lavaei, Leila Bridgeman
Date: 2022-03-05 00:00:00

A controller synthesis method for state- and input-constrained nonlinear systems is presented that seeks continuous piecewise affine (CPA) Lyapunov-like functions and controllers simultaneously. Non-convex optimization problems are formulated on triangulated subsets of the admissible states that can be refined to meet primary control objectives, such as stability and safety, alongside secondary performance objectives. A multi-stage design is also given that enlarges the region of attraction (ROA) sequentially while allowing exclusive performance for each stage. A clear boundary for an invariant subset of closed-loop system's ROA is obtained from the resulting Lipschitz Lyapunov function. For control-affine nonlinear systems, the non-convex problem is formulated as a series of conservative, but well-posed, semi-definite programs. These decrease the cost function iteratively until the design objectives are met. Since the resulting CPA Lyapunov-like functions are also Lipschitz control (or barrier) Lyapunov functions, they can be used in online quadratic programming to find minimum-norm control inputs. Numerical examples are provided to demonstrate the effectiveness of the method.
Data-driven input reconstruction and experimental validation
Authors: Jicheng Shi, Yingzhao Lian, Colin N. Jones
Date: 2022-03-05 00:00:00
Tasks: LEMMA, UIE

This paper addresses a data-driven input reconstruction problem based on Willems' Fundamental Lemma in which unknown input estimators (UIEs) are constructed directly from historical I/O data. Given only output measurements, the inputs are estimated by the UIE, which is shown to asymptotically converge to the true input without knowing the initial conditions. Both open-loop and closed-loop UIEs are developed based on Lyapunov conditions and the Luenberger-observer-type feedback, whose convergence properties are studied. An experimental study is presented demonstrating the efficacy of the closed-loop UIE for estimating the occupancy of a building on the EPFL campus via measured carbon dioxide levels.
Convergence of the Distributed SG Algorithm Under Cooperative Excitation Condition
Authors: Die Gan, Zhixin Liu
Date: 2022-03-05 00:00:00
Tasks: regression

In this paper, a distributed stochastic gradient (SG) algorithm is proposed where the estimators are aimed to collectively estimate an unknown time-invariant parameter from a set of noisy measurements obtained by distributed sensors. The proposed distributed SG algorithm combines the consensus strategy of the estimation of neighbors with the diffusion of regression vectors. A cooperative excitation condition is introduced, under which the convergence of the distributed SG algorithm can be obtained without relying on the independency and stationarity assumptions of regression vectors which are commonly used in existing literature. Furthermore, the convergence rate of the algorithm can be established. Finally, we show that all sensors can cooperate to fulfill the estimation task even though any individual sensor can not by a simulation example.
Distributed Sparse Identification for Stochastic Dynamic Systems under Cooperative Non-Persistent Excitation Condition
Authors: Die Gan, Zhixin Liu
Date: 2022-03-05 00:00:00

This paper considers the distributed sparse identification problem over wireless sensor networks such that all sensors cooperatively estimate the unknown sparse parameter vector of stochastic dynamic systems by using the local information from neighbors. A distributed sparse least squares algorithm is proposed by minimizing a local information criterion formulated as a linear combination of accumulative local estimation error and L_1-regularization term. The upper bounds of the estimation error and the regret of the adaptive predictor of the proposed algorithm are presented. Furthermore, by designing a suitable adaptive weighting coefficient based on the local observation data, the set convergence of zero elements with a finite number of observations is obtained under a cooperative non-persistent excitation condition. It is shown that the proposed distributed algorithm can work well in a cooperative way even though none of the individual sensors can fulfill the estimation task. Our theoretical results are obtained without relying on the independency assumptions of regression signals that have been commonly used in the existing literature. Thus, our results are expected to be applied to stochastic feedback systems. Finally, the numerical simulations are provided to demonstrate the effectiveness of our theoretical results.
Online Adversarial Stabilization of Unknown Networked Systems
Authors: Jing Yu, Dimitar Ho, Adam Wierman
Date: 2022-03-05 00:00:00

We investigate the problem of stabilizing an unknown networked linear system under communication constraints and adversarial disturbances. We propose the first provably stabilizing algorithm for the problem. The algorithm uses a distributed version of nested convex body chasing to maintain a consistent estimate of the network dynamics and applies system level synthesis to determine a distributed controller based on this estimated model. Our approach avoids the need for system identification and accommodates a broad class of communication delay while being fully distributed and scaling favorably with the number of subsystems.
A Scenario Approach to Risk-Aware Safety-Critical System Verification
Authors: Prithvi Akella, Mohamadreza Ahmadi, Aaron D. Ames
Date: 2022-03-04 00:00:00

With the growing interest in deploying robots in unstructured and uncertain environments, there has been increasing interest in factoring risk into safety-critical control development. Similarly, the authors believe risk should also be accounted in the verification of these controllers. In pursuit of sample-efficient methods for uncertain black-box verification then, we first detail a method to estimate the Value-at-Risk of arbitrary scalar random variables without requiring \textit{apriori} knowledge of its distribution. Then, we reformulate the uncertain verification problem as a Value-at-Risk estimation problem making use of our prior results. In doing so, we provide fundamental sampling requirements to bound with high confidence the volume of states and parameters for a black-box system that could potentially yield unsafe phenomena. We also show that this procedure works independent of system complexity through simulated examples of the Robotarium.
Optimization of Traffic Control in MMAP[c]/PH[c]/S Catastrophic Queueing Model with PH Retrial Times and Controllable Preemptive Repeat Priority Policy
Authors: Raina Raj, Vidyottama Jain
Date: 2022-03-04 00:00:00

The presented study elaborates a multi-server catastrophic retrial queueing model considering preemptive repeat priority policy with phase-type (PH) distributed retrial times. For the sake of comprehension, the scenario of model operation prior and later to the occurrence of the disaster is referred to as the normal scenario and as the catastrophic scenario, respectively. In the normal scenario, the incoming heterogeneous calls are categorized as handoff calls and new calls. Handoff calls are provided controllable preemptive priority over new calls. In the catastrophic scenario, when a disaster causes the shut down of the entire system and failure of all functioning channels, a set of backup channels is quickly deployed to restore services. Due to the emergency situation in the concerned area, the incoming heterogeneous calls are divided into three categories: handoff, new call, and emergency calls. Emergency calls are provided controllable preemptive priority over new/handoff calls due to the pressing need to save lives in such situations. The Markov chain's ergodicity criteria are established by demonstrating that it belongs to the class of asymptotically quasi-Toeplitz Markov chains (AQTMC). Further, a multi-objective optimization problem to obtain optimal number of backup channels has been formulated and dealt by employing non-dominated sorting genetic algorithm-II (NSGA-II) approach.
Virtual Histological Staining of Label-Free Total Absorption Photoacoustic Remote Sensing (TA-PARS)
Authors: Marian Boktor, Benjamin Ecclestone, Vlad Pekar, Deepak Dinakaran, John R. Mackey, Paul Fieguth, Parsin Haji Reza
Date: 2022-03-04 00:00:00
Tasks: Colorization, Diagnostic, Generative Adversarial Network

Histopathological visualizations are a pillar of modern medicine and biological research. Surgical oncology relies exclusively on post-operative histology to determine definitive surgical success and guide adjuvant treatments. The current histology workflow is based on bright-field microscopic assessment of histochemical stained tissues and has some major limitations. For example, the preparation of stained specimens for brightfield assessment requires lengthy sample processing, delaying interventions for days or even weeks. Hence, there is a pressing need for improved histopathology methods. In this paper, we present a deep-learning-based approach for virtual label-free histochemical staining of total-absorption photoacoustic remote sensing (TA-PARS) images of unstained tissue. TA-PARS provides an array of directly measured label-free contrasts such as scattering and total absorption (radiative and non-radiative), ideal for developing H&E colorizations without the need to infer arbitrary tissue structures. We use a Pix2Pix generative adversarial network (GAN) to develop visualizations analogous to H&E staining from label-free TA-PARS images. Thin sections of human skin tissue were first virtually stained with the TA-PARS, then were chemically stained with H&E producing a one-to-one comparison between the virtual and chemical staining. The one-to-one matched virtually- and chemically- stained images exhibit high concordance validating the digital colorization of the TA-PARS images against the gold standard H&E. TA-PARS images were reviewed by four dermatologic pathologists who confirmed they are of diagnostic quality, and that resolution, contrast, and color permitted interpretation as if they were H&E. The presented approach paves the way for the development of TA-PARS slide-free histology, which promises to dramatically reduce the time from specimen resection to histological imaging.
Parallel Fourier Ptychography reconstruction
Authors: Guocheng Zhou, Shaohui Zhang, Yao Hu, Lei Cao, Yong Huang, Qun Hao
Date: 2022-03-04 00:00:00
Tasks: CPU

Fourier ptychography has attracted a wide range of focus for its ability of large space-bandwidth-produce, and quantative phase measurement. It is a typical computational imaging technique which refers to optimizing both the imaging hardware and reconstruction algorithms simultaneously. The data redundancy and inverse problem algorithms are the sources of FPM's excellent performance. But at the same time, this large amount of data processing and complex algorithms also greatly reduce the imaging speed. In this article, we propose a parallel Fourier ptychography reconstruction framework consisting of three levels of parallel computing parts and implemented it with both central processing unit (CPU) and compute unified device architecture (CUDA) platform. In the conventional FPM reconstruction framework, the sample image is divided into multiple sub-regions for separately processing because the illumination angles for different subregions are varied for the same LED and different subregions contain different defocus distances due to the non-planar distribution or non-ideal posture of biological sample. We first build a parallel computing sub-framework in spatial domain based on the above-mentioned characteristics. And then, by utilizing the sequential characteristics of different spectrum regions to update, a parallel computing sub-framework in the spectrum domain is carried out in our scheme. The feasibility of the proposed parallel FPM reconstruction framework is verified with different experimental results acquired with the system we built.
Competitive Exclusion in an Artificial Foraging Ecosystem
Authors: John C. Stevenson
Date: 2022-03-05 00:00:00

Artificial ecosystems provide an additional experimental tool to support laboratory work, field work, and theoretical development in competitive exclusion research. A novel application of a spatiotemporal agent based model is presented which simulates two foraging species of different intrinsic growth rates competing for the same replenishing resource in stable and seasonal environments. This experimental approach provides precise control over the parameters for the environment, the species, and individual movements. Detailed trajectories of these non-equilibrium populations and their characteristics are produced. Narrow zones of potential coexistence are identified within the environmental and intrinsic growth parameter spaces. An example of commensalism driven by the local spatial dynamics is identified. Results of these experiments are discussed in context of modern coexistence theory and research in movement-mediated community assembly. Constraints on possible origination scenarios are identified.
Improved Wavelets for Image Compression from Unitary Circuits
Authors: James C. McCord, Glen Evenbly
Date: 2022-03-04 00:00:00
Tasks: Image Compression, MS-SSIM, SSIM

We benchmark the efficacy of several novel orthogonal, symmetric, dilation-3 wavelets, derived from a unitary circuit based construction, towards image compression. The performance of these wavelets is compared across several photo databases against the CDF-9/7 wavelets in terms of the minimum number of non-zero wavelet coefficients needed to obtain a specified image quality, as measured by the multi-scale structural similarity index (MS-SSIM). The new wavelets are found to consistently offer better compression efficiency than the CDF-9/7 wavelets across a broad range of image resolutions and quality requirements, averaging 7-8% improved compression efficiency on high-resolution photo images when high-quality (MS-SSIM = 0.99) is required.