Reviewerly/exHarmony huggingface dataset:
Using Circulation to Mitigate Spurious Equilibria in Control Barrier Function

Control Barrier Functions and Quadratic Programming are increasingly used for designing controllers that consider critical safety constraints. However, like Artificial Potential Fields, they can suffer from the stable spurious equilibrium point problem, which can result in the controller failing to reach the goal. To address this issue, we propose introducing circulation inequalities as a constraint. These inequalities force the system to explicitly circulate the obstacle region in configuration space, thus avoiding undesirable equilibria. We conduct a theoretical analysis of the proposed framework and demonstrate its efficacy through simulation studies. By mitigating spurious equilibria, our approach enhances the reliability of CBF-based controllers, making them more suitable for real-world applications.
Resilient Federated Learning Under Byzantine Attack in Distributed Nonconvex Optimization with 2-$f$ Redundancy

We study the problem of Byzantine fault tolerance in a distributed optimization setting, where there is a group of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$N$</tex> agents communicating with a trusted centralized coordinator. Among these agents, there is a subset of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$f$</tex> agents that may not follow a prescribed algorithm and may share arbitrarily incorrect information with the coordinator. The goal is to find the optimizer of the aggregate cost functions of the honest agents. We will be interested in studying the local gradient descent method, also known as federated learning, to solve this problem. However, this method often returns an approximate value of the underlying optimal solution in the Byzantine setting. Recent work showed that by incorporating the so-called comparative elimination (CE) filter at the coordinator, one can provably mitigate the detrimental impact of Byzantine agents and precisely compute the true optimizer in the convex setting. The focus of the present work is to provide theoretical results to show the convergence of local gradient methods with the CE filter in a nonconvex setting. We will also provide a number of numerical simulations to support our theoretical results.
META-SMGO-$\Delta$: Similarity as a Prior in Black-Box Optimization

When solving global optimization problems in practice, one often ends up repeatedly solving problems that are similar to each others. By introducing a rigorous definition of similarity to exploit priors obtained from past experience to efficiently solve new (similar) problems, in this work we incorporate the META-learning rationale into SMGO <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$-\Delta$</tex> , a global optimization approach recently proposed in the literature. Through a benchmark numerical example we show the practical benefits of our META -extension of the baseline algorithm, while providing theoretical bounds on its performance.
Linear Speedup of Incremental Aggregated Gradient Methods on Streaming Data

This paper considers a type of incremental aggregated gradient (IAG) method for large-scale distributed optimization. The IAG method is well suited for the parameter server architecture as the latter can easily aggregate potentially staled gradients contributed by workers. Although the convergence of IAG in the case of deterministic gradient is well known, there are only a few results for the case of its stochastic variant based on streaming data. Considering strongly convex optimization, this paper shows that the streaming IAG method achieves linear speedup when the workers are updating frequently enough, even if the data sample distribution across workers are heterogeneous. We show that the expected squared distance to optimal solution decays at <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathcal{O}((1+T)/(nt))$</tex> , where <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$n$</tex> is the number of workers, <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$t$</tex> is the iteration number, and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$T/n$</tex> is the update frequency of workers. Our analysis involves careful treatments of the conditional expectations with staled gradients and a recursive system with both delayed and noise terms, which are new to the analysis of IAG-type algorithms. Numerical results are presented to verify our findings.
NMPC Strategy for Safe Robot Navigation in Unknown Environments Using Polynomial Zonotopes

This work proposes a nonlinear model predictive control (NMPC) strategy for robot navigation in cluttered unknown environments using polynomial zonotopes. The information provided by a laser sensor is used in the computation of the collision-free area. The procedure splits the area into convex subregions which are converted into polynomial zonotopes (PZs) to generate constraints for the NMPC optimal control problem. The PZ is a set representation that can describe polytopes using fewer constraints than conventional half-space representations, thus being more efficient while maintaining the accuracy equivalent to the polytopic case. Numerical experiments demonstrate the advantages of the proposed strategy.
Counter-Adversarial Learning with Inverse Unscented Kalman Filter

In counter-adversarial systems, to infer the strategy of an intelligent adversarial agent, the defender agent needs to cognitively sense the information that the adversary has gathered about the latter. Prior works on the problem employ linear Gaussian state-space models and solve this inverse cognition problem by designing inverse stochastic filters. However, in practice, counter-adversarial systems are generally highly nonlinear. In this paper, we address this scenario by formulating inverse cognition as a nonlinear Gaussian state-space model, wherein the adversary employs an unscented Kalman filter (UKF) to estimate the defender's state with reduced linearization errors. To estimate the adversary's estimate of the defender, we propose and develop an inverse UKF (IUKF) system. We then derive theoretical guarantees for the stochastic stability of IUKF in the mean-squared boundedness sense. Numerical experiments for multiple practical applications show that the estimation error of IUKF converges and closely follows the recursive Cramer-Rao lower bound.
Impact of Relational Networks in Multi-Agent Learning: A Value-Based Factorization View

Effective coordination and cooperation among agents are crucial for accomplishing individual or shared objectives in multi-agent systems. In many real-world multiagent systems, agents possess varying abilities and constraints, making it necessary to prioritize agents based on their specific properties to ensure successful coordination and cooperation within the team. However, most existing cooperative multi-agent algorithms do not take into account these individual differences, and lack an effective mechanism to guide coordination strategies. We propose a novel multi-agent learning approach that incorporates relationship awareness into value-based factorization methods. Given a relational network, our approach utilizes inter-agents relationships to discover new team behaviors by prioritizing certain agents over other, accounting for differences between them in cooperative tasks. We evaluated the effectiveness of our proposed approach by conducting fifteen experiments in two different environments. The results demonstrate that our proposed algorithm can influence and shape team behavior, guide cooperation strategies, and expedite agent learning. Therefore, our approach shows promise for use in multi-agent systems, especially when agents have diverse properties.
Supervisor Fortification Against Covert Actuator Attacks

This work considers the supervisor fortification problem against covert actuator attacks. A supervisor <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$S^{\prime}$</tex> is said to fortify the supervisor <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$S$</tex> , the latter of which is non-resilient against covert actuator attacks, if <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$S^{\prime}$</tex> satisfies two conditions: 1) any covert actuator attack cannot cause damage infliction against <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$S^{\prime}$</tex> , and 2) <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$S^{\prime}$</tex> is control equivalent to <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$S$</tex> . The key result of this work is that we show the problem of determining the existence of a fortified supervisor to defend against any covert actuator attack, an “exist-for all” decidability question, is decidable. To show the decidability result, we provide a complete and sound procedure that ensures to synthesize a fortified supervisor as long as there exists one.
Initial State Interventions for Deconfounded Imitation Learning

Imitation learning suffers from causal confusion. This phenomenon occurs when learned policies attend to features that do not causally influence the expert actions but are instead spuriously correlated. Causally confused agents produce low open-loop supervised loss but poor closed-loop performance upon deployment. We consider the problem of masking ob-served confounders in a disentangled representation of the observation space. Our novel masking algorithm leverages the usual ability to intervene in the initial system state, avoiding any requirement involving expert querying, expert reward functions, or causal graph specification. Under certain assumptions, we theoretically prove that this algorithm is conservative in the sense that it does not incorrectly mask observations that causally influence the expert; furthermore, intervening on the initial state serves to strictly reduce excess conservatism. The masking algorithm is applied to behavior cloning for two illustrative control systems: CartPole and Reacher.
Simulator-Driven Deceptive Control via Path Integral Approach

We consider a setting where a supervisor delegates an agent to perform a certain control task, while the agent is incentivized to deviate from the given policy to achieve its own goal. In this work, we synthesize the optimal deceptive policies for an agent who attempts to hide its deviations from the supervisor's policy. We study the deception problem in the continuous-state discrete-time stochastic dynamics setting and, using motivations from hypothesis testing theory, formulate a Kullback-Leibler control problem for the synthesis of deceptive policies. This problem can be solved using backward dynamic programming in principle, which suffers from the curse of dimensionality. However, under the assumption of deterministic state dynamics, we show that the optimal deceptive actions can be generated using path integral control. This allows the agent to numerically compute the deceptive actions via Monte Carlo simulations. Since Monte Carlo simulations can be efficiently parallelized, our approach allows the agent to generate deceptive control actions online. We show that the proposed simulation-driven control approach asymptotically converges to the optimal control distribution.
On the Robustness of Stability for Quantum Stochastic Systems

We investigate the impact of undesired Markovian couplings to external systems on the stabilization towards pure states or subspaces of continuously monitored quantum systems under open-loop and feedback control protocols. In particular, we provide sufficient conditions on perturbations to maintain stability of the target and demonstrate the boundedness in mean of the solutions of perturbed systems under open-loop protocols. Effect on the stability of feedback protocols is also analyzed. This work is a step toward a comprehensive robustness analysis of measurement-based control of quantum systems with respect to model perturbations.
Episodic Logit-Q Dynamics for Efficient Learning in Stochastic Teams

We present new learning dynamics combining (independent) log-linear learning and value iteration for stochastic games within the auxiliary stage game framework. The dynamics presented provably attain the efficient equilibrium (also known as optimal equilibrium) in identical-interest stochastic games, beyond the recent concentration of progress on provable convergence to some (possibly inefficient) equilibrium. The dynamics are also independent in the sense that agents take actions consistent with their local viewpoint to a reasonable extent rather than seeking equilibrium. These aspects can be of practical interest in the control applications of intelligent and autonomous systems. The key challenges are the convergence to an inefficient equilibrium and the non-stationarity of the environment from a single agent's viewpoint due to the adaptation of others. The log-linear update plays an important role in addressing the former. We address the latter through the play-in-episodes scheme in which the agents update their Q-function estimates only at the end of the episodes.
Emulation Learning for Neuromimetic Systems

Building on our recent research on neural heuristic quantization systems, results on learning quantized motions and resilience to channel dropouts are reported. We propose a general emulation problem consistent with the neuromimetic paradigm. This optimal quantization problem can be solved by model predictive control (MPC), but because the optimization step involves integer programming, the approach suffers from combinatorial complexity when the number of input channels becomes large. Even if we collect data to train a neural network, collection of training data and the training itself are still time-consuming. Therefore, we propose a general Deep Q Network (DQN) algorithm that not only learns trajectories but also exhibits the advantages of resilience to channel dropout. Furthermore, to transfer the model to other emulation problems, a mapping-based transfer learning approach can be used directly on the current model to obtain the optimal trajectory steps for the new emulation problems.
On the Effects of Data Heterogeneity on the Convergence Rates of Distributed Linear System Solvers

We consider the fundamental problem of solving a large-scale system of linear equations. In particular, we consider the setting where a taskmaster intends to solve the system in a distributed/federated fashion with the help of a set of machines, who each have a subset of the equations. Although there exist several approaches for solving this problem, missing is a rigorous comparison between the convergence rates of the projection-based methods and those of the optimization- based ones. In this paper, we analyze and compare these two classes of algorithms with a particular focus on the most efficient method from each class, namely, the recently proposed Accelerated Projection-Based Consensus (APC) [1] and the Distributed Heavy-Ball Method (D-HBM). To this end, we first propose a geometric notion of data heterogeneity called angular heterogeneity and discuss its generality. Using this notion, we bound and compare the convergence rates of the studied algorithms and capture the effects of both cross-machine and local data heterogeneity on these quantities. Our analysis results in a number of novel insights besides showing that APC is the most efficient method in realistic scenarios where there is a large data heterogeneity. Our numerical analyses validate our theoretical results.
Efficient Off-Policy Algorithms for Structured Markov Decision Processes

Reinforcement Learning (RL) algorithms help in training an autonomous agent to learn the optimal actions in an unknown environment. RL, in conjunction with neural networks, known as deep RL, has achieved remarkable success in many real-life practical applications. However, most of these algorithms are complex to train and require a lot of data to learn optimal decisions. Hence, it is imperative to develop RL algorithms that are simple and data efficient. In this work, we propose off-policy RL algorithms that can exploit special structures present in the optimal policy of the underlying Markov Decision process. The off-policy learning enables us to make use of the available data efficiently. To this end, we first propose an off-policy algorithm that estimates the value function of only those policies with the optimal policy structure to determine the best policy. We then propose two novel off-policy algorithms utilizing Upper Confidence Bound (UCB) with less time and space complexity than our first algorithm. Through extensive experimental evaluations on RL benchmark tasks, we illustrate the efficacy of the proposed algorithms.
Reinforcement Learning-Guided Quadratically Constrained Quadratic Programming for Enhanced Convergence and Optimality

In the context of Quadratically Constrained Quadratic Programming (QCQP) with dynamic parameters, the effectiveness of various optimization approaches is heavily influenced by the quality of the initial guess. To address this challenge, this paper proposes a novel approach that leverages reinforcement learning (RL) to generate high-performing initial guesses for iterative algorithms, with the dynamic parameters serving as inputs. Our approach aims to accelerate convergence and improve the objective value, thereby enabling efficient and effective solutions to the QCQP problem under variability. In this study, we evaluate the proposed approach by applying it to an iterative algorithm, specifically the Iterative Rank Minimization (IRM) algorithm. Our empirical evaluations demonstrate the efficacy of the proposed approach in solving QCQP problems with dynamic param-eters. The RL-guided IRM algorithm yields high-quality solutions, as evidenced by significantly improved optimality and faster convergence when compared to the original IRM algorithm.
Communication-Efficient Local SGD for Over-Parametrized Models with Partial Participation

We analyze the convergence rate of Local Stochastic Gradient Descent (SGD) for over-parameterized models, which is at the core of federated learning. In this model, we allow the server to randomly select a subset of agents and communicate with them at each communication round to optimize a global objective function. This captures the realistic scenarios where the communication link between the server and the agents may break down due to random link failures or adversarial attacks. We establish convergence guarantees for smooth objective functions without the convexity assumption that is the first for the regime. We also consider an extension of our results under a different random participation setting over general network structures (rather than a star network) in which an agent participates in the local optimization steps of its neighbors by some edge-dependent probability. We characterize the convergence rate of the proposed algorithm in terms of the number of communication rounds, which confirms the communication efficiency of our methods, and justify our results through a numerical experiment.
Causal Discovery in Electronic Circuits and Its Application in Fault Diagnosis

This article demonstrates that causal discovery approaches can be applied to analog electronic circuits made of Bipolar Junction Transistors (BJTs) to find out the causal relationships among different variables of the circuit. Moreover, the obtained causal relationship structure in the form of a Directed Acyclic Graph (DAG), can be used for diagnosis and analysis of such circuits. First, it is shown that the operation process of a transistor has an inherent notion of causality, which is then exploited to show that the various parameters of a BJT circuit can be expressed in the form of Structural Equation Models (SEM). The results demonstrated using data generated using LTspice establishes that the causal structure of a BJT circuit can be retrieved using data driven causal discovery algorithms. This opens new horizons for analysis and diagnosis of BJT circuits. An example case study of circuit diagnosis is presented to showcase the capability and efficiency of the proposed method.
Responsible and Effective Federated Learning in Financial Services: A Comprehensive Survey

The financial sector is increasingly leveraging Artificial Intelligence (AI) to deliver intelligent, automated, and personalized services. However, it encounters significant data privacy challenges due to the dispersion of financial data across various entities. Federated Learning (FL) offers a potential solution by facilitating AI model training at the source of data, albeit with certain challenges. Irresponsible utilization of FL can compromise stakeholder interests, and the prevalent heterogeneity in data spaces in numerous financial FL scenarios can impede FL's performance. These complications necessitate the development of a Responsible and Effective Federated Learning (RE-FL) system in finance. In this paper, we explore the interdisciplinary field of RE-FL in finance and guide readers to understand this area thoroughly. We present a taxonomy of RE-FL approaches that address the concerns of stakeholders in FL-based financial services and identify six major dimensions: accountability, controllability, fairness, privacy, security, and effectiveness. We also propose potential directions for future research. To our understanding, this is the first literature review conducted on RE-FL in the financial sector.
Inverse Kalman Filtering for Systems with Correlated Noises

This paper focuses on two inverse problems of the Kalman filter in which the process and measurement noises are correlated. The unknown covariance matrix in a stochastic system is reconstructed from observations of its posterior beliefs. For the standard inverse Kalman filtering problem, a novel duality-based formulation is proposed, where a well-defined inverse optimal control (IOC) problem is solved instead. Identifiability of the underlying model is proved, and a least squares estimator is designed that is statistically consistent. The time-invariant case using the steady-state Kalman gain is further studied. Since this inverse problem is ill-posed, a canonical class of covariance matrices is constructed, which can be uniquely identified from the dataset with asymptotic convergence. Finally, the performances of the proposed methods are illustrated by numerical examples.