Reviewerly/exHarmony huggingface dataset:
Application of eXplainable AI and causal inference methods to estimation algorithms in networks of dynamic systems

While continuous progress in the area of machine learning is producing algorithms capable of achieving better and better decision and predictive performance, the way such algorithms operate is also becoming more and more inscrutable. When an increasing amount of decisions is being ceded to often inexplicable algorithms which are not susceptible to any form of human supervision or scrutiny, it is just natural to start raising doubts about their fairness, soundness, and reliability. This has motivated a growing need for tools capable of disentangling and explaining the mechanisms behind AI based decisions, creating a new field of research referred to as eXplainable AI (XAI). Given the significant impact that machine learning is having also on the area of estimation and control, this article advances the idea of borrowing and adapting methodologies from the area of XAI and apply them to estimation and control algorithms involving networks of dynamic processes. Specifically, we translate the methodology known as Local Interpretable Model-Agnostic Explanations (LIME) in order to explain the mechanisms behind a black-box estimation algorithm processing time-series. Furthermore, we find that LIME can be extended using notions of causal inference to detect cause-effect relations among the features that the estimation algorithm takes as inputs. This causal inference procedure provides LIME with additional explanatory power.
Transmissibility-based Kalman Filtering For Systems With Non-Gaussian Process Noise

The concept of transmissibility operators refers to the mathematical relationships between system outputs. They can be used to estimate the independent output of a system based on sensor measurements only. In this case, the output estimation is independent of the process noise or unmodeled dynamics. This allows for the estimation of process noise regardless of its probability distribution. The proposed approach takes into account the possibility of using the Kalman filter theme in the filtering of output noise regardless of the process noise distribution. The proposed approach does not require the covariance estimation of the process noise. Since the proposed approach considers the ability to formulate unmodeled dynamics or parameter uncertainties as non-Gaussian process noise, it can handle both. The potential of this approach is demonstrated by implementing it in a group of connected autonomous robots.
Performance of noisy higher-order accelerated gradient flow dynamics for strongly convex quadratic optimization problems

We study performance of momentum-based accelerated first-order optimization algorithms in the presence of additive white stochastic disturbances. For strongly convex quadratic problems with a condition number κ, we determine the best possible convergence rate of continuous-time gradient flow dynamics of order n. We also demonstrate that additional momentum terms do not affect the tradeoffs between convergence rate and variance amplification that exist for gradient flow dynamics with n = 2.
Adversarial Tradeoffs in Robust State Estimation

Adversarially robust training has been shown to reduce the susceptibility of learned models to targeted input data perturbations. However, it has also been observed that such adversarially robust models suffer a degradation in accuracy when applied to unperturbed data sets, leading to a robustness-accuracy tradeoff. Inspired by recent progress in the adversarial machine learning literature which characterize such tradeoffs in simple settings, we develop tools to quantitatively study the performance-robustness tradeoff between nominal and robust state estimation. In particular, we define and analyze a novel adversarially robust Kalman Filtering problem. We show that in contrast to most problem instances in adversarial machine learning, we can precisely derive the adversarial perturbation in the Kalman Filtering setting. We provide an algorithm to find this perturbation given data realizations, and develop upper and lower bounds on the adversarial state estimation error in terms of the standard (non-adversarial) estimation error and the spectral properties of the resulting observer. Through these results, we show a natural connection between a filter’s robustness to adversarial perturbation and underlying control theoretic properties of the system being observed, namely the spectral properties of its observability gramian.
Control-Minimal Time-Assigned Path-Constrained Trajectory Optimization

Path-constrained trajectory optimization research normally focuses on time or energy optimality. However, some applications seek reference trajectories that satisfy other constraints. In this paper, we formulate a control-minimal time-assigned path-constrained trajectory optimization problem: a mobile ground robot must traverse a given path in a specific amount of time using minimal control effort. Through a nonlinear change of variables, we solve this problem using convex optimization. We evaluate our solution with an intelligent transportation scenario where an autonomous vehicle must cross an intersection in a specific amount of time while following the turn lane’s geometric center.
RRT Guided Model Predictive Path Integral Method

This work presents an optimal sampling-based method to solve the real-time motion planning problem in static and dynamic environments, exploiting the Rapid-exploring Random Trees (RRT) algorithm and the Model Predictive Path Integral (MPPI) algorithm. The RRT algorithm provides a nominal mean value of the random control distribution in the MPPI algorithm, resulting in satisfactory control performance in static and dynamic environments without a need for fine parameter tuning. We also discuss the importance of choosing the right mean of the MPPI algorithm, which balances exploration and optimality gap, given a fixed sample size. In particular, a sufficiently large mean is required to explore the state space enough, and a sufficiently small mean is required to guarantee that the samples reconstruct the optimal control. The proposed methodology automates the procedure of choosing the right mean by incorporating the RRT algorithm. The simulations demonstrate that the proposed algorithm can solve the motion planning problem for static or dynamic environments.
An Almost Sure Convergence Analysis of Zeroth-Order Mirror Descent Algorithm

In this paper, we show almost sure convergence of zeroth-order mirror descent algorithm. The algorithm admits non-smooth convex functions and assumes only an estimate of the gradient is available, obtained using Nesterov’s Gausssian Approximation technique (NGA). We establish that under suitable condition of step-size, the function value of the iterates of the algorithm converge to a neighborhood of the optimal function value almost surely. We extend the analysis to the distributed implementation of the zeroth-order mirror descent algorithm.
Transfer Learning-Based Modeling and Predictive Control of Nonlinear Processes

This work develops a transfer learning (TL) framework for modeling nonlinear dynamic systems using recurrent neural networks (RNNs). The TL-based RNN models are then incorporated into the design of model predictive control (MPC) systems. Specifically, transfer learning uses a pre-trained model developed based on a source domain as the starting point, and adapts the model to a target domain with similar data distribution. The generalization error for TLbased RNNs (TL-RNNs) that depends on model capacity and discrepancy between source and target domains is first derived to demonstrate the generalization capability on target process. Subsequently, the TL-RNN model is utilized as the prediction model in MPC for the target process. Finally, a chemical process example is used to demonstrate the benefits of transfer learning.
Decentralized Multi-Agent Motion Planning in Dynamic Environments

In this paper we present a decentralized multi-agent motion planning algorithm for navigation in dynamic environments. Each agent constructs a graph of boundary value problems in the environment considering their own kinodynamic constraints using a learning-based motion planning framework. A game-theoretic approach is then used by each agent to select their individual path through the environment while considering the planned motion of other agents. This path is updated online to ensure collisions are avoided, and to provide a method of counteracting the freezing robot problem. The effectiveness of the algorithm is illustrated in simulations.
CHAMP: Integrated Logic with Reinforcement Learning for Hybrid Decision Making for Autonomous Vehicle Planning

A Cognitive Hybrid Autonomous Motion Planner (CHAMP) is developed for autonomous driving applications in challenging driving scenarios. The proposed hybrid planner unifies a hierarchical rule-based decision-making architecture with Reinforcement Learning (RL). For challenging intersection scenarios, RL agents are trained to replace a subset of the rules in the logical planner. The hybrid planner is systematically tested and benchmarked to demonstrate its effectiveness in handling challenging road scenario with congested and chaotic traffic conditions.
Closed-Form Hilbert Projection for Quantum State Observers

Designing an observer of an unknown quantum density operator is difficult because the operator must be Hermitian positive semidefinite with unit trace. In this paper, we derive a closed-form solution for projecting an arbitrary matrix onto the set of valid density operators. This allows us to design linear quantum state observers and retract the observer’s state back to this set while retaining the exponential convergence rate of the linear observer. The derived closed-form projection can be used alongside any quantum state estimation technique to produce a valid state estimate without increasing the state estimation error.
A Smoothing Algorithm for Minimum Sensing Path Plans in Gaussian Belief Space

This paper explores minimum sensing navigation of robots in environments cluttered with obstacles. The general objective is to find a path plan to a goal region that requires minimal sensing effort. In [1], the information-geometric RRT* (IG-RRT*) algorithm was proposed to efficiently find such a path. However, like any stochastic sampling-based planner, the computational complexity of IG-RRT* grows quickly, impeding its use with a large number of nodes. To remedy this limitation, we suggest running IG-RRT* with a moderate number of nodes, and then using a smoothing algorithm to adjust the path obtained. To develop a smoothing algorithm, we explicitly formulate the minimum sensing path planning problem as an optimization problem. For this formulation, we introduce a new safety constraint to impose a bound on the probability of collision with obstacles in continuous-time, in contrast to the common discrete-time approach. The problem is amenable to solution via the convex-concave procedure (CCP). We develop a CCP algorithm for the formulated optimization and use this algorithm for path smoothing. We demonstrate the efficacy of the proposed approach through numerical simulations.
Equivariant Reinforcement Learning for Quadrotor UAV

This paper presents an equivariant reinforcement learning framework for quadrotor unmanned aerial vehicles. Successful training of reinforcement learning often requires numerous interactions with the environments, which hinders its applicability especially when the available computational resources are limited, or when there is no reliable simulation model. We identified an equivariance property of the quadrotor dynamics such that the dimension of the state required in the training is reduced by one, thereby improving the sampling efficiency of reinforcement learning substantially. This is illustrated by numerical examples with popular reinforcement learning techniques of TD3 and SAC.
Comparative analysis of a nonlinear observer and nonlinear Kalman filters for magnetic position estimation

This paper analyzes state estimation for nonlinear systems in the presence of sensor noise and process disturbances. A class of systems involving nonlinear functions of vector arguments in the output equations is considered. A nonlinear observer that satisfies a H <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</inf> disturbance rejection constraint in addition to providing asymptotic stability in the absence of disturbances is developed using Lyapunov analysis. The observer is shown analytically to provide a guaranteed upper bound on the norm of the estimation error. The performance of the nonlinear observer is compared with the performance of two of the most popular and powerful methods for estimation in nonlinear systems - the extended Kalman filter (EKF) and the unscented Kalman filter (UKF). A magnetic position estimation problem is utilized as the real-world application for the evaluation. In the case of the disturbances being gaussian noise, the UKF and the nonlinear observer provide approximately the same level of performance and they both surpass the performance of the EKF. However, in the case of 2-norm-bounded non gaussian noise such as spikes/ pulses, the nonlinear observer is shown to significantly outperform both the UKF and the EKF.
OGDM: An Observability Guaranteed Distributed Edge Sensing Method for Industrial Cyber-Physical Systems

The new generation of industrial cyber-physical systems (ICPS) supported by the edge computing technology enables efficient distributed sensing under massive data volumes and frequent transmissions. Observability is essential to obtain good sensing performance, and most of existing sensing works directly assume that the system is observable. However, it is difficult to satisfy the assumption with the increasingly expanded network scale and dynamic scheduling of devices. To solve this problem, we propose an observability guaranteed distributed method (OGDM) for edge sensing with the cooperation of sensors and edge computing units (ECUs). We analyze the relationship between sensor scheduling and observability based on the network topology and graph signal processing (GSP) technology. In addition, we transform the observability condition into a convex form and take into account sensing error and energy consumption for optimization. Finally, our algorithm is applied to estimate the slab temperature in the hot rolling process. The effectiveness is verified by simulation results.
GrAVITree: Graph-based Approximate Value Function In a Tree

In this paper, we introduce GrAVITree, a tree- and sampling-based algorithm to compute a near-optimal value function and corresponding feedback policy for indefinite time-horizon, terminal state-constrained nonlinear optimal control problems. Our algorithm is suitable for arbitrary nonlinear control systems with both state and input constraints. The algorithm works by sampling feasible control inputs and branching backwards in time from the terminal state to build the tree, thereby associating each vertex in the tree with a feasible control sequence to reach the terminal state. Additionally, we embed this stochastic tree within a larger graph structure, rewiring of which enables rapid adaptation to changes in problem structure due to, e.g., newly detected obstacles. Because our method reasons about global problem structure without relying on (potentially imprecise) derivative information, it is particularly well suited to controlling a system based on an imperfect deep neural network model of its dynamics. We demonstrate this capability in the context of an inverted pendulum, where we use a learned model of the pendulum with actuator limits and achieve robust stabilization in settings where competing tree-based and derivative-based techniques fail.
Statistical Verification of Traffic Systems with Expected Differential Privacy

Traffic systems are multi-agent cyber-physical systems whose performance is closely related to human welfare. They work in open environments and are subject to uncertainties from various sources, making their performance hard to verify by traditional model-based approaches. Alternatively, statistical model checking (SMC) can verify their performance by sequentially drawing sample data until the correctness of a performance specification can be inferred with desired statistical accuracy. This work aims to verify traffic systems with privacy, motivated by the fact that the data used may include personal information (e.g., daily itinerary) and get leaked unintendedly by observing the execution of the SMC algorithm. To formally capture data privacy in SMC, we introduce the concept of expected differential privacy (EDP), which constrains how much the algorithm execution can change in the expectation sense when data change. Accordingly, we introduce an exponential randomization mechanism for the SMC algorithm to achieve the EDP. Our case study on traffic intersections by Vissim simulation shows the high accuracy of SMC in traffic model verification without significantly sacrificing computing efficiency. The case study also shows EDP successfully bounding the algorithm outputs to guarantee privacy.
Opportunistic Qualitative Planning in Stochastic Systems with Incomplete Preferences over Reachability Objectives

Preferences play a key role in determining what goals/constraints to satisfy when not all constraints can be satisfied simultaneously. In this paper, we study how to synthesize preference-satisfying plans in a stochastic system modeled as an MDP, given a (possibly incomplete) combinative preference model over temporally extended goals. We start by introducing new semantics to interpret preferences over infinite plays of the stochastic system. Then, we introduce a new notion of ‘improvement’ to enable comparison between two prefixes of an infinite play. Based on this, we define two solution concepts called Safe and Positively Improving (SPI) and Safe and Almost-Sure Improving (SASI) that enforce improvements with a positive probability and with probability one, respectively. We construct a model called an improvement MDP, in which the synthesis of SPI and SASI strategies that guarantee at least one improvement, reduces to computing positive and almost-sure winning strategies in an MDP. We present an algorithm to synthesize the SPI and SASI strategies that induce multiple sequential improvements. We demonstrate the proposed approach using a robot motion planning problem.
The Unscented Transform Controller: a new model predictive control law for highly nonlinear systems

The Unscented Transform which is the basis of the Unscented Kalman Filter, UKF, is used here to develop a novel predictive controller for non-linear plants, called the Unscented Transform Controller, UTC. The UTC can be seen as the dual of the UKF, the same way as the LQG regulator and the Kalman Filter are related. The UTC is demonstrated on the control of complex maneuvers in free fall of a virtual skydiver the model of which was verified in wind tunnel and free fall experiments.
Activity Recognition Using a High Gain Observer and Spectrograms

This paper proposes a novel algorithm for human activity recognition that is a combination of a high-gain observer and deep learning-based classification algorithms. The nonlinear high-gain observer designed using Lyapunov analysis accurately estimates the attitude of the chest of a human subject using measurements from a single Inertial Measurement Unit (IMU). The signals processed by the observer are then converted into spectrograms to obtain “images” of the frequency response of the signals. The images for activities from a dataset of 7 human subjects are annotated and used for training/ fine-tuning of several well-known deep learning algorithms for image processing. The results from the best combination of our algorithms shows an exceptional accuracy of 98% for activity recognition.
Decentralized Multi-Agent Reinforcement Learning for Continuous-Space Stochastic Games

Stochastic games are a popular framework for studying multi-agent reinforcement learning (MARL). Recent advances in MARL have focused primarily on games with finitely many states. In this work, we study multi-agent learning in stochastic games with general state spaces and an information structure in which agents do not observe each other's actions. In this context, we propose a decentralized MARL algorithm and we establish the near-optimality of its policy updates. Furthermore, we study the global policy-updating dynamics for a general class of best-reply based algorithms and derive a closed-form characterization of convergence probabilities over the joint policy space.
Parameterized input inference for approximate stochastic optimal control

Probabilistic inference approaches to stochastic optimal control have attracted significant interest from researchers in the past decade. Existing inference-based optimal control approaches are limited to linear controllers in a finite-horizon model-based setting. Since nonlinear systems typically admit nonlinear optimal controllers, linear controllers may yield sub-optimal trajectories when applied to nonlinear systems. In this paper, we propose a new Expectation-Maximization (EM) based inference algorithm for stochastic optimal control. The algorithm employs nonlinear basis functions to infer nonlinear controllers. We formulate the estimation problem of optimal control as a parameter inference problem. We demonstrate the effectiveness of the algorithm on a simulated nonlinear oscillator system for nonlinear control and a linear thermal system for structured control.
Resilient Distributed Parameter Estimation in Sensor Networks

In this paper, we study the problem of parameter estimation in a sensor network, where the measurements and updates of some sensors might be arbitrarily manipulated by adversaries. Despite the presence of such misbehaviors, normally behaving sensors make successive observations of an unknown d-dimensional vector parameter and aim to infer its true value by cooperating with their neighbors over a directed communication graph. To this end, by leveraging the so-called dynamic regressor extension and mixing procedure, we transform the problem of estimating the vector parameter to that of estimating d scalar ones. For each of the scalar problem, we propose a resilient combine-then-adapt diffusion algorithm, where each normal sensor performs a resilient combination to discard the suspicious estimates in its neighborhood and to fuse the remaining values, alongside an adaptation step to process its streaming observations. With a low computational cost, this estimator guarantees that each normal sensor exponentially infers the true parameter even if some of them are not sufficiently excited.
Risk-Awareness in Learning Neural Controllers for Temporal Logic Objectives

In this paper, we consider the problem of synthesizing a controller in the presence of uncertainty such that the resulting closed-loop system satisfies certain hard constraints while optimizing certain (soft) performance objectives. We assume that the hard constraints encoding safety or mission-critical specifications are expressed using Signal Temporal Logic (STL), while performance is quantified using standard cost functions on system trajectories. To ensure satisfaction of the STL constraints, we algorithmically obtain control barrier functions (CBFs) from the STL specifications. We model controllers as neural networks (NNs) and provide an algorithm to train the NN parameters to simultaneously optimize the performance objectives while satisfying the CBF conditions (with a user-specified robustness margin). We evaluate the risk incurred by the trade-off between the robustness margin of the system and its performance using the formalism of risk measures. We demonstrate our approach on challenging nonlinear control examples such as quadcopter motion planning and a unicycle.