A Bayesian approach to breaking things: efficiently predicting and repairing failure modes via sampling
Authors: Charles Dawson
Chuchu Fan
Venue: CoRL 2023 Poster (2023)
Keywords: Automatic design tools
root-cause failure analysis
optimization-as-inference

Abstract: Before autonomous systems can be deployed in safety-critical applications, we must be able to understand and verify the safety of these systems. For cases where the risk or cost of real-world testing is prohibitive, we propose a simulation-based framework for a) predicting ways in which an autonomous system is likely to fail and b) automatically adjusting the system's design to preemptively mitigate those failures. We frame this problem through the lens of approximate Bayesian inference and use differentiable simulation for efficient failure case prediction and repair. We apply our approach on a range of robotics and control problems, including optimizing search patterns for robot swarms and reducing the severity of outages in power transmission networks. Compared to optimization-based falsification techniques, our method predicts a more diverse, representative set of failure modes, and we also find that our use of differentiable simulation yields solutions that have up to 10x lower cost and requires up to 2x fewer iterations to converge relative to gradient-free techniques.

A Bayesian approach to breaking things: efficiently predicting and repairing failure modes via sampling
Charles Dawson
Chuchu Fan
Department Aeronautics and Astronautics
Department of Aeronautics and Astronautics
MIT United States
MIT United States chuchu@mit. edu
cbd@mit.cm edu
Abstract: Before autonomous systems can be deployed in safety-critical appli- cations, we must be able to understand and verify the safety of these systems. For cases where the risk or cost of real-world testing is prohibitive, we propose a simulation-based framework for a) predicting ways in which an autonomous sys- tem is likely to fail and b) automatically adjusting the system's 's design to preemp- tively mitigate those failures. We frame this problem through the lens of approxi- mate Bayesian inference and use differentiable simulation for efficient failure case prediction and repair. We apply our approach on range of robotics and control problems, including optimizing search patterns for robot swarms and reducing the severity of outages in power transmission networks. Compared to optimization- based falsification techniques, our method predicts a more diverse, representative set of failure modes, and we also find that our use of differentiable simulation yields solutions that have up 10x lower cost and requires up to 2x fewer iter- ations to converge relative gradient-free techniques. Accompanying code and video can be found at https http://itttrraamm.eeeeeee github iaclub.ioo//ii seweeaaiinnnttnngge
Keywords:
Automatic design tools, root-cause failure analysis, optimization-as.
inference
Introduction
From power grids to transportation and logistics systems, autonomous systems play a central, and often safety-critical, role in modern life. Even as these systems grow more complex and ubiquitous, we have already observed failures in autonomous systems like autonomous vehicles and power net- works resulting in the loss of human life [1]. Given this context, important that we be able to verify the safety of autonomous systems prior to deployment; for instance, by understanding the different ways in which system might fail and proposing repair strategies.
Human designers often use their knowledge of likely failure modes to guide the design process; indeed, systematically assessing the risks of different failures and developing repair strategies is an important part of the systems engineering process [2]. However, as autonomous systems grow more complex, it becomes increasingly difficult for human engineers to manually predict likely failures.
In this paper, we propose an automated framework for predicting, and then repairing, failure modes in complex autonomous systems. Our effort builds on a large body of work on testing and veri- fication of autonomous systems, many of which focus on identifying failure modes or adversarial examples [3, 4, 5, 6, 7, 8], but we identify two major gaps in the state of the art. First, many existing methods [4, 5, 9, 7] use techniques like gradient descent search locally for failure modes; how- ever, in practice
are more interested in characterizing the distribution of potential failures, which requires global perspective. Some methods exist that address this issue by taking probabilistic approach to sample from an (unknown) distribution of failure modes [6, 10]. However, these meth- ods suffer from a second major drawback: although they can help the designer predict a range of
7th Conference Robot Learning (CoRL. 2023), Atlanta, USA.

---

gradient- based using
mitigate predicted modes
optimize initaa solution
x
x
X
X
Repest predict-mtiggte nydle until desired soution quality schieved
An overview of our method for predicting and repairing failure modes in autonomous systems, shown here handling connectivity failures in a drone swarm.
Figure 1:
failure modes, they do not provide guidance on how those failure modes may be mitigated; they are also inefficient due to their use of gradient-free inference methods.
We address all of these drawbacks develop framework, shown in Fig. 1, for predicting and repair- ing failure modes in autonomous systems. Taking inspiration from inference-basee based methods [10, 6). we make three novel contributions:
1. We reframe the failure prediction problem as a probabilistic sampling problem, allowing us to avoid local minima and quickly find high likelihood, high severity failure modes.
2. We exploit the duality between failure prediction and repair not only predict likely failure modes but also suggest low-cost repair strategies.
3. We employ automatic differentiation to take advantage of fast gradient- based sampling algorithms, substantially improving performance relative to the state of the art.
We demonstrate our approach on several large-scale robotics and control problems: swarm for- mation control with up to 10 agents, multi-robot search with up to 32 agents, an electric power transmission network with up to 57 nodes and 80 transmission lines. We compare our approach with baselines both failure mode prediction and repair, showing that our framework outper- forms the state-of-the-ae and scales well beyond the capabilities of existing tools, converging to solutions that up to 10x lower cost while requiring less than half many iterations. We also demonstrate that the repair strategies developed using our approach can be deployed hardware for the multi-robot search example, and software implementation can be found at https://mit-reall githmn.it.......... io/ breekinngthhnne
2
Related Work
Model-based verification Early approaches to model-based verification and fault identification used symbolic logical models of the system under test to formally reason about failures using (com- putationally expensive) satisfiability (SAT) solvers or search [11, 12]. More recent approaches to model-based failure mode identification have used mathematical models of the system dynamics to frame the problem as a reachability [13] or optimal control [14] problem. The challenge in applying these methods is that it may be difficult impossible construct symbolic model for the sys- tem under test. In this work, seek to retain the interpretability of model- techniques while eliminating the requirement for a fully symbolic model, using automatically differentiable com- puter programs instead. Such models are comparatively easy to construct [8] and can even include implicitly differentiable components such as the solutions to optimization problems [15].
Adversarial testing Verification using adversarial optimization has been applied in both model- based [7, 5, 9] and model-free [3] contexts. Generally speaking, model-based adversarial techniques use gradient- based optimization to locally search for adversarial examples that cause system fail- ure, then use gradient- based optimization to locally repair those failures [7, 5]. The drawback of these methods is that they are inherently local and typically yield only single adversarial coun-

---

terexample. Model-free approaches [3] can avoid the issue of local minima by using zero cero-order black-box optimization techniques but incur additional computational cost as a result. In contrast, we take a probabilistic approach where sample-efficient gradient-based sampling algorithms can be used to escape local minima and efficiently generate multiple potential failure cases [16].
Inference Ours is not the first work to take probabilistic approach to failure mode prediction. O' Kelly et al. develop an end-to-end verification pipeline for autonomous vehicles based on adap- tive importance sampling [10], and Zhou et al. develop a failure mode prediction system based on gradient-free Markov Chain Monte Carlo (MCMC) [6]. We take inspiration from these works and make two key improvements. First, these existing works focus exclusively on predicting likely failure modes- - they do not include a method for mitigating these failure modes once discovered while we combine failure mode prediction with repair by recognizing the duality between these problems. Second, we use differentiable simulation to replace inefficient zero-order MCMC algo- rithms with fast gradient-based samplers, resulting in substantial performance improvement.
There is also a complimentary body of work on algorithms for rare-event simulation [17, 18] that provide extensions to MCMC-based sampling algorithms that perform well even when we seek to simulate extremely rare failure cases. Our framework completely compatible with rare-event sim- ulation strategies commonly used in Sequential Monte Carlo (SMC), and we view the incorporation of these methods into our framework as a promising direction for future work.
3 Assumptions and Problem Statement
At the heart of our approach is a simulation model of the system under test, parameterized by two distinct sets of parameters. The design parameters x E X C R" are those parameters that the system designer may vary, while the exogenous parameters y E y C E are those that may vary uncontrollably (due to environmental variation, adversarial disturbance, the actions of other agents, etc.). Together, x and y define the behavior of the system
E (e.g. trace of all relevant states and observables) through the simulator function, denoted Ş Szz, y). In addition, cost function J ({) is known; i.e. J reflects the property that the system designer wishes to verify. A summary of our notation is provided in Table 1 in the appendix; we will use "designs" and "failure modes" interchangeably with "design parameters" and "exogenous parameters", respectively.
assume that
Assumption 1: S and J are programs that we can automatically differentiate almost everywhere. This setting is more general than the case when an explicit mathematical model is known, but less general than black-box setting (although many black-box systems in robotics can be automatically differentiated [19]). Assumption 2: and are continuous random variables with known, automat- ically differentiable, and potentially unnormalized prior probability densities P2,0(z) and (y)) It may be counter intuuiivi to model the design parameters as random variables, but this choice allows us to capture constraints on the design space by assigning low probability to infeasible designs. The prior distribution for y can be either estimated from historical data or constructed to reflect con- straints on the operational domain. We restrict our focus to the paraamete case, but our approach can be extended to handle mixed discrete parameters using block-resampling [20].
In this context, failure prediction entails finding exogenous parameters y" that, for some given r, lead to a high cost. To ensure that predicted failures are plausible, we must also find values for y" with high prior likelihood. To achieve this balance, we define the metric of risk-adjusted cost
Jr( = o S(z, y) + log lyg
where denotes function composition. Failure prediction is thus the problem of finding parameters y that lead to high risk-adjusted cost; moreover, since it is likely that J, will have multiple local minima with respect to y (i.e. multiple likely failure modes), we wish to sample set { ........ Vhy) of such failures. To generate this set, we replace deterninistic optimization y" = arg min, Jr( y) with sampling from the unnormalized pseudo-posterine (in the sense defined in [21]).
8" ~ ply" x py,o(" ) le (SS(z.0")
(2)
3

---

Likewise, the failure repair problem seeks to find design parameters x" that both have high prior likelihood (thus respecting the designer's prior beliefs about the design space) and result in low cost across a range of anticipated failure modes; i.e. sampling from the unnormalized neeudo-posterion
x²
p(a"lvi vnn)) Pa,u(=") D. JoS(x" NoS(""))t)v)) wt)/ng
(3)
4 Approach: Adversarial Inference
The primary challenge in sampling from these failure and repair distributions is that they will natu- rally shift as the design changes. Once the design has been updated to account for the current set of predicted failures, those failures will likely be out of date. To address this issue, we define a novel adversarial sampling method to altemate between sampling improved failure modes {yr ... . V } and then sampling improved design parameters x" to repair those failure modes, thus improving the robustness of the design while maintaining an up-to-date set of failure modes.
Our algorithm (detailed in Algorithm 1) proceeds the style of a sequential Monte Carlo algo- rithm [18]. We begin by initializing ny potential failure modes and n, candidate designs sampled from their respective prior distributions. In each of K rounds, we first sample ny new candidate designs from distribution (3) repair the current set of predicted failure modes. We then select the design that performs best against all currently-predict failures and sample Tyy new sets of ex- ogenous parameters (each representing a potential failure mode) from distribution (2). To sample from distributions (2) and (3), we use n, and n, parallel executions of a Markov chain Monte Carlo (MCMC) sampler. order to handle potential multimodality in the design and failure space, we include optional tempering to interpolate between the prior and target distributions [18).
Our proposed adversarial inference algorithm can accept any MCMC sampling algorithm sub- routine, either gradient-free or trudient-basedd In our experiments, we compare the results of using both gradient-free (random-war walk Metropolis-Hastings. or RMH) and gradient- based (Metropolis- adjusted Langevin algorithm, or MALA [22]); both of these are included the appendix. Empiri- cally, gradient-basee samplers typically converge faster, particularly on high-dimensional problems, but in cases where differentiable simulator is not available, gradient-free free sampler will suffice. We use MCMC for the sampling subroutine in all of our experiments, but our framework is also compatible with other approximate inference methods (e.g. variational inference).
Algorithm 1: Failure prediction and repair using gradient- based sampling
Input: Population sizes f,,, Ny: rounds K; substeps M; stepsize т; tempering Ai,..., λK- Output: Robust design x* and a set of failures ......., ......y,,)) with high risk-adjusted cost. Initialize candidate designs [x]o {I1,... Ina to sampled from (z))(1))
Initialize candidate failures [y]o {vi..... Un Jo sampled from py,o(y)
for i I,..., K do
(z) == pz,( (xe/=>)ee>//_-------a >w/n, 2 - Jo5(2.9)
[z]; (ample([<))--)) M, Pz.i)
Update candidate designs using predicted failures
(y))(u)= == (y) A min JoS(a.)) & Update failure predictions for new best design [y]: Sample(
P
return [y],, x² arg max (z)
Choose best design
On a theoretical level, any MCMC sampler will be sound so long as the resulting Markov chain is ergodic and satisfies detailed balance [23]. Unfortunately, there can be a large gap between asymp- totic theoretical guarantees and practical performance. First, if the target distribution multimodal and the modes are well-separated, then MCMC algorithms may be slow to move between modes, yielding a biased sampling distribution. To mitigate this effect, we include a tempering schedule S A S ... & >x < 1 to interpolate between the prior and target distributions and run multiple MCMC instances in parallel from different initial conditions. Empirically, find that tempering is not always needed, but include it for completeness.

---

The second potential practical challenge arises from the continuity and differentiability (or lack thereof) of the simulator and cost function J o S. Although gradient-based MCMC samplers like MALA remain sound so long as the target distribution is continuously differentiable almost every- where (i.e, discontinuous or non-differentiable on set of measure zero), in practice performance may suffer when the target distribution has large discontinuities. Because of these issues, we design method be compatible with either gradient-based gradient-fres sampling algorithms, and compare the results of using both methods in Section 6.
The final practical consideration that although the stochasticity in our sampling-based approach can help us explore the design and failure spaces, we incur a degree of sub-optimality as a result. When using gradient-based sampling, we have the option to reduce this sub-optimality by "quench- ing" the solution: switching simple gradient descent (e.g. using MALA for the first 90 rounds and then gradient descent the last 10 rounds). In practice, find that quenching can noticeably improve the final cost without compromising the diversity of predicted failure modes.
Theoretical Analysis
Our prediction-and-reaarr framework can work with both gradient-free or gradient-based sampling subroutines, but important to note that gradients, when available, often accelerate convergence. To support this observation, we provide non-asymptotic convergence guarantees for the gradient- based version of our algorithm, drawing on recent results in Ma al. [16]. To make these guarantees, first assume that J is L-Lipschitz smooth. Second, assume that the log prior distributions Py,o Py.0 and log are m-strongly convex outside a ball of finite radius R. The first assumption is hard to verify in general and does not hold in certain domains (e.g. rigid contact), but true for most of our experiments in Section 6. The second is easy to verify for common priors (e.g. Gaussian and smoothed uniform). Let d max (dim æ, dim y) be the dimension of the search space and E (0, 1) be convergence tolerance in total variation (TV) distance.
Theorem 5.1. Consider Algorithm with the stated assumptions on smoothness and log- eeecccncvvtyy. If m L and = Ō ((d'm (d L/(m L) + in 1/c)= 1/2 d d-1/2), then sampling each round with TV enror < requires at most M Ö (d In 2) steps.
Since convergence time for each round of prediction and mitigation scales only polynomially with the dimension of the search space, method is able to find more accurate failure predictions (and thus better design updates) than gradient-free methods with the same sample budget.
Proof sketch Ma al. [16] show that gradient-based MCMC enjoys fast convergence on non- convex likelihoods long the target likelihood is strongly log-concave in its tails (i.e. outside of a bounded region). It would be unrealistic to assume that the cost J(z, y) is convex, but we can instead rely on the strong log-concavity of the prior to dominate sufficiently in the tails and regularize the cost landscape. A formal proof is included in the appendix.
6 Experimental Results
There are two questions that we must answer in this section: first, does reframing this problem from optimization to inference lead to better solutions (i.e. lower cost designs and predicted failures that accurately cover the range of possible failures)? Second, does sellientt-bsedd MCMC with differentiableesimmlatie offer any benefits over gradient-free MCMC when used in our approach?
We benchmark our algorithm on a range of robotics and industrial control problems. We compare against meviously-publishe adversarial optimization methods [7, 5] and compare the results of using gradient-based and gradient-free MCMC subroutines in our approach. We then provide demonstration using method to solve multi-robot planning problem in hardware. The code used for our experiments can be found at https: httm/mmitttaaalae gwt..ggit.bb...... i sareakinn-thingeee
5

---

Figure 2: Environments used in our experiments. (Left to right) Multi-agent search-evasion, forma- tion control, power dispatch, aircraft ground collision avoidance, and manipulation by pushing.
Baselines We compare with the following baselines. DR: solving the design optimization problem with domain randomization ming (x; y)l. GD: solving the adversarial optimization problem miny max, (x, y) by alternating between optimizing a population of nt designs and My failure modes using local gradient descent, as in [7, 5, 24]. We also include two versions of our method, using both gradient-free (RMH) and gradient-base bbased (MALA) MCMC subroutines. All methods are given the same information about the value and gradient (when needed) of the cost and prior likeli- hoods. The gradient-free free version of our approach implements quenching for the last few rounds.
Environments We use three environments for our simulation studies, which are shown in Fig. and described more fully in the appendix. Multi-robot search: set of seeker robots must cover search region to detect set of hiders. and define trajectories for the seekers and hiders, respectively; failure occurs if any of the hiders escape detection. This environment has small (6 seeker vs. 10 hider, dim x = 60, dim y = 100) and large (12 seeker vs. 20 hider, dim x = 120, dim y = 200) versions. Formation control: swarm drones fly to a goal while maintaining full connectivity with a limited communication radius. x defines trajectories for each robot in the swarm, while y parameterizes an uncertain wind velocity field. Failure occurs when the second eigenvalue of the graph Laplacian is close zero. This environment has small (5 agent, dim 30, dim 1280) and large (10 agent, dim x 100, dimy 1280) versions. Power grid dispatch: electric generators must be scheduled to ensure that the network satisfies voltage and maximum power constraints in the event of transmission line outages. I specifies generator setpoints and specifies line admittances; failures occur when any of the voltage or power constraints are violated. This environment has small (14-bus, dim = 32, dim y = 20) and large (57-bus, dim x = 98, dimy = 80) versions. F16 GCAS: a ground collision avoidance system (GCAS) must be designed to prevent jet aircraft, modeled with aerodynamic effects and engine dynamics, from crashing into the ground. I defines control policy neural network (dim x & 1.8 10²) and defines the initial conditions (dim 5). Pushing: robot manipulator must push an object out of the way to reach another object. Failure occurs if the object is knocked over while pushing. & defines a planning policy network (dim æ 1.2 x 108) and y defines the unknown incrtial and frictional properties of the object being pushed, as well as measurement noises (dim y = 7). We implement our method and all baselines in Python using JAX. All methods were run with the same population sizes and total sample budget, using hyperparameters given in the appendix.
Solution quality For each environment, we first solve for an optimized design and a set of pre- dicted failure modes using each method. We then compare the performance of the optimal design on the predicted failure modes with the performance observed sampled exogenous parameters. The results of this experiment are shown in Fig. 3.
large test set of 10 randomly
We find that both DR and GD often fail to predict failure modes that accurately cover the tail of worst-case behaviors: in the formation and power grid examples, both DR and GD falsely indicate that all predicted failures have been successfully repaired, despite a long tail of possible failures in both cases. In the search example, adversarial GD is able to predict set of useful failure modes, but DR fails to do so. Only method (with both frree and gradient- based MCMC) accurately predicts the worst-case performance of the optimized design.

---

Jmn (MB MU-
(a) Formation, 5 agents
(b) Search, vs. 10
(c) Power grid, 14-bus
(d) F16 GCAS
(e) Formation, 10 agents
(f) Search, 12 vs. 20
(g) Power grid, 57-bus
(h) Pushing
Figure 3: A comparison of the cost of the optimal design on the predicted failure modes (red) and 10a randomly sampled test cases (blue).
Sompirga
(a) Formation, agents irl periieee (ppornetioo
(b) Search, vs.
(c) Power grid, 14-bus
(d) F16 GCAS
fards
(e) Formation, 10 agents
(0) Search, 12 vs. 20
(g) Power grid, 57-bus
(h) Pushing
Figure 4: Convergence rates of tradient-based (orange) and gradient-free (blue) MCMC samplers when used as subroutines for Algorithm 1. Shaded areas show min/max range over random seeds.
In addition to comparing the quality of the predicted failure modes, we can also compare the per- formance and robustness of the optimized design itself. On the search problem, our method finds designs with slightly improved performance relative to GD (but not relative to DR, since DR is not optimizing against challenging set of predicted failure modes). On the formation problem, our method is able to find substantially gigher-performance designs than either baseline method. On the power grid problem, our method finds designs that incur higher best-case cost, since this problem includes tradeoff between economic cost and robustness, but our method's designs are substantially more robust than the baselines, with much lighter tails in the cost distribution.
We observe that DR sometimes finds solutions that achieve lower average cost than those found by our method. We believe that this is due to DR optimizing against a less challenging failure population. This suggests the possibility of combining a failure dataset (predicted using our method) with an average-case dataset (sampled randomly from the prior) during repair; we hope explore this and other adaptive strategies in future work.
Benefits of differentiable simulation Although we have designed our method to be compatible with either gradient- tradient-based or gradient- free MCMC subroutines, we observe that gradient- based sam- plers tend to converge faster than their gradient-fre counterparts. Fig. 4 plots the performance the best-performing design each round against a static test set of 100 randomly sampled exogenous parameters for both gradient- based and gradient-free methods across all environments. Although these methods perform similarly on the formation problem, we see clear pattern in the formation control, search-evasion, and power grid examples where gradient-based MCMC converges much faster, and this advantage is greater on higher-dimensional problems (second row), compensating for the additional time needed to compute the gradients (typically a 2-3x increase in runtime).
Hardware experiments We deploy the optimized hider and seeker trajectories in hardware using the Robotarium multi-robot platform [25] (we use 3 seekers and 5 hiders, since we had difficulty
7

---

Figure 5: (Left) HW results for search-evasion with 5 hiders and 3 seckers, showing an initial search pattern (blue) and predicted failure modes (red). (Center) HW results for an optimized search pattern leaves fewer hiding places. (Right, top) An initial manipulation policy knocks over the object. (Right, bottom) The repaired manipulation policy pushes without knocking the bottle over.
testing with more agents in the limited space). We first hold the search pattemn (design parameters) constant and optimize evasion patterns against this fixed search pattern, yielding the results shown on the left in Fig. 5 where the hiders easily evade the seekers. We then optimize the search patterns using our approach, yielding the results on the left where the hiders are not able to evade the seekers.
We also deploy an optimized policy for the pushing problem to a Franka Research 7-DoF robot arm. Fig. 5 shows failure when the unoptimized policy fails account for the uncertain center of mass of the bottle, as well successful execution with the repaired policy. Videos of all experiments are provided in the supplementary materials.
7 Discussion and Conclusion
Before sending any autonomous system out into the real world, it is important to understand how it will behave in range of operational conditions, including during potential failures. In this paper, have presented a tool to allow the designers of autonomous systems to not only predict the ways in which a system is likely to fail but also automatically adjust their designs to mitigate those failures.
We apply our framework in simulation studies a range of robotics and industrial control problems, including multi-robot trajectory planning and power grid control. Our results show that, relative existing adversaria optimization methods, our novel sampling-based approach yields better pre- dictions of possible failure modes, which in turn lead to more robust optimized designs. We also show empirically that, when it is possible to define a differentiable simulator, gradient- MCMC methods allow our method to converge more than twice as fast as gradient-free methods.
7.1 Limitations
Since it would be prohibitively costly to search for failure cases in hardware experiments (especially if failures resulted in damage to the robot), our method is restricted to searching for failures in simulation. As such, it is limited to predicting only failures that are modeled by the simulator, excluding failures that could arise due to unmodeled effects. Practically, our method could be used in conjunction with hardware testing by catching some failures carlier in the development process and reducing the cost of eventual hardware testing.
A notable limitation of our approach is that it requires knowledge of the prior distribution of the exogenous disturbances y. Although this can be estimated in some cases (as in experiments), in practice there may be uncertainty about the nature of this distribution. To address this, future works might investigate distributionally robust extensions of Algorithm 1 (akin to distributionally robust optimization methods [26]). Additional limitations are discussed in the appendix.

---

Acknowledgments
C. Dawson is supported by the NSF GRFP under Grant No. 1745302. This work was partly sup- ported by the National Acronautics and Space Administration (NASA) ULI grant 80NSSC22M0070, Air Force Office of Scientific Research (AFOSR) grant 199550-33-1-0999 and the Defense Science and Technology Agency in Singapore. Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the authors and do not necessarily reflect the views of the sponsors,
References
[1] University of Texas at Austin. The Timeline and Events of the February 2021 Texas Electric Grid Blackouts. Technical report, University of Texas at Austin, July 2021.
[2] R. Shishko. NASA Systems Engineering Handbook. Number 6105 in NASA SP. National Aeronautics and Space Administration, Washington, D.C.?, 1995.
[3] A. Corso, R. Moss, M. Koren, R. Lee, and M. Kochenderfer. Survey of Algorithms for Black-Box Safety Validation of Cyber-Physical Systems. Journal of Artificial Intelligence Research, 72:377-428, Oct. 2021. ISSN 1076-9757. 1oi:10.1613/jarr.
[4] A. Corso and M. J. Kochenderfer. Interpretable Safety Validation for Autonomous Vehicles. In 2020 IEEE 23rd International Conference Intelligent Transportation Systems (ITSC), pages 1-6, Sept. 2020. doi: 1101100000/TTS445002200000000000
[5] Donti, A. Agarwal, N. V. Bedmutha, L. Pileggi, and Z. Kolter. Adversarially robust learn- ing for security-conssrained optimal power flow. In Advances in Neural Information Processing Systems, volume 34, pages 28677-28689, Curran Associates, Inc., 2021.
[6] Y. Zhou, S. Booth, N. Figueroa, and Shah. RoCUS: Robot Controller Understanding via Sampling. In Sth Annual Conference on Robot Learning, Nov. 2021.
[7] C. Dawson and C. Fan. Robust Counterexample-guided Optimization for Planning from Dif- ferentiable Temporal Logic. 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 7205-7212, Oct. 2022. doi: 1099/R0047612.200222 9901222.9998888
[8] C. Dawson and C. Fan. Certifiable Robot Design Optimization using Diffcrentiable Program- ming. In Robotics: Science and Systems XVIII, volume 18, June 2022. ISBN 778-0-9923747- 8-5.
[9] S. Yaghoubi and G. Fainekos. Gray-box Adversarial Testing for Control Systems with Ma- chine Learning Component. HSCC 2019 Proceedings of the 2019 22nd ACM International Conference on Hybrid Systems: Computation and Control, pages 179-184, Dec. 2018. doi: 10.485500/rxi..1112..111
[10] M. O' Kelly, A. Sinha, H. Namkoong, R. Tedrake, and J. C. Duchi. Scalable End-to-Ent tn- Autonomous Vehicle Testing via Rare-event Simulation. In Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
[11] J. de Kleer and B. C. Williams. Diagnosing multiple faults. Artificial Intelligence, 32(1):
97-130, Apr. 1987. ISSN 0004-3702.
[12] D. Benard, G. A. Dorais, E. Gamble, B. Kanefsky, J. Kurien, W. Millar, N. Muscettola, P. Nayak, N. Rouquette, K. Rajan, and P. Norvig. Remote Agent Experiment. Jan. 2000.
[13] Y. Annpureddy, C. Liu, Fainekos, and Sankaranarayanan. S-TaLiRo: tool for temporal logic falsification for hybrid systems. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 6605 LNCS: 254-257, 2011. ISSN 03029743. doi: 10i:1001007/978-- 19835-9 2./COVER.

---

[14] G. Chou, Y. E. Sahin, L. Yang, K. J. Rutledge, P. Nilsson, and N. Ozay. Using control syn- thesis to generate corner cases: case study on autonomous driving. IEEE Transactions on Computer-Aide Design of Integrated Circuits and Systems, ISSN 02780070. doi:10. 100011////AAD 28.288444..
Nov. 2018.
[15] B. Amos and J. Z. Kolter. OptNet: Differentiable optimization as a layer in neural net- works. In Proceedings of the 34th International Conference on Machine Learning Volume 70, ICML''17 '7, pages 136-145, Sydney, NSW, Australia, Aug. 2017. JMLR.org.
[16] Y. A. Ma, Y. Chen, C. Jin, N. Flammarion, and M. I. Jordan. Sampling can be faster than optimization. Proceedings of the National Academy of Sciences of the United States of Amer- ica, 116(42)2288112200
Oct. 2019. ISSN 10916490.
DCSUPPLEMENTAL
[17] G. Rubino and B. Tuffin. Introduction to Rare Event Simulation. Rare Event Simulation using Monte Curlo Methods, pages 1-13, Jan. 2009. doi: 10. 10.1000/97804777444....
[18] N. Chopin and O. Papaspiliopoulos. An Introduction to Sequential Monte Carlo. Springer International Publishing, Cham, 2020. ISBN 978-3-030-47844-3
[19] T. Howell, S. Le Cleac'la h, Z. Kolter, M. Schwager, and Z. Manchester. Dojo: A differentiable simulator for robotics. arXiv preprint 2tt::2003000006 2022.
[20] D. van Ravenzwaaij, P. Cassey, and S. D. Brown. A simple introduction to Markov Chain Monte Monte-Carlo sampling. Psychonomic Bulletin & Review, 25(1):143-154, Feb. 2018. ISSN 1531-5320. doi:
[21] P. Alquier, J. Ridgway, and N. Chopin. On the properties of variational approximations of Gibbs posteriors. Journal of Machine Learning Research, 17(236):1-411 2016. ISSN 1533- 7928.
[22] G. O. Roberts and O. Stramer. Langevin Diffusions and Metropolis-Hastingg Algorithms. Methodology And Computing In Applied Probability, 4(4):337-357, Dec. 2002. ISSN 1573-
7713. 1o.: 10.22////102336411113333
[23] C. Geyer. Introduction to Markev Chain Monte Carlo. Chapman and Hall/CRC, May 2011. ISBN 978-0-429-13850-8, doi: 100.12011/10000
[24] H. Xu, Ma, H.-. Liu, D. Deb, H. Liu, J.-L. Tang, and A. K. Jain. Adversarial Attacks and Defenses in Images, Graphs and Text: A Review. International Journal of Automation and Computing. 17(2):111-77, Apr. 2020. ISSN 1751-8520. doi:10.1 1007/0 111--019-221--
[25] S. Wilson, P. Glotfelter, L. Wang. S. Mayya, G. Notomista, M. Mote, and M. Egerstedt. The Robotarium: Globally Impactful Opportunities, Challenges, and Lessons Learned in Remote- Access, Distributed Control of Multirobot Systems. IEEE Control Systems Magazine, 40(1): 26-44, Feb. 2020. ISSN 1941-000X. doi: 109/MCCS
[26] H. Rahimian and S. Mehrotra. Distributionally Robust Optimization: A Review. Open Journal of Mathematical Optimization, 3:1 8 2022. ISSN 2777-5860. 1oo:10.8802/jmm.... 15.
[27] M. B. Cain, R. P. O'Neill, and A. Castillo. History of Optimal Power Flow and Formulations. Technical report, Federal Energy Regulatory Commission, 2012.
[28] Capitanescu, Martinez Ramos, Panciatici, D. Kirschen, A. Marano Marcolini, L. Plat- brood, and L. Wehenkel. State-of-the-arr, challenges, and future trends in security constrained optimal power flow. Electric Power Systems Research, 1731)1171111741, Aug. 2011. ISSN 0378-7796. doi: 10.:11..11110
[29] U.S. Department of Energy.
Grid
Optimization
Competition.
http://ggooompttiiooo.nnrgg/.//
10

---

[30] P. L. Donti, D. Rolnick, and J. Z. Kolter. DC3: A learning method for optimization with hard constraints, Apr: 2021.
[31] R. D. Zimmerman, C. E. Murillo-Sánchez, and R. J. Thomas. MATPOWER: Steady-State Operations, Planning, and Analysis Tools for Power Systems Research and Education. IEEE Transactions on Power Systems, 26(1):12-19, Feb. 2011. ISSN 1558-0679. doi:10.1 109/ TPWRS.2010.20511 168.
[32] Heidlauf, A. Collins, M. Bolender, and S. Bak. Verification Challenges in F-16 Ground Col- lision Avoidance and Other Automated Maneuvers. In EPiC Series in Computing, volume 54, pages 208-217 EasyChair, Sept. 2018. doi:10.29007/91xx
[33] O. So and C. Fan. Solving Stabilize-Avoid via Epigraph Form Optimal Control using Deep Reinforcement Learning. In Robotics: Science and Systems XIX, volume 19, July 2023, ISBN 978-0-9923747-9922

---

Table 1: Summary of notation
x E X' C
R2
Design parameters (controlled by system designer)
yey C RR4
Exogenous parameters (not controlled by designer)
{EEC C Rdc
Behavior of system (e.g. simulation trace)
8:Xx H
Simulator model of the system's 's behavior given design and exogenous parameters Cost function
J: R J x R
Risk-adjusted cost function
Pa.u (2); Py,u(y)
Prior probability distributions for design and exogenous parameters
Summary of notation
Table provides a summary of notation used in this paper.
Sampling algorithms
Algorithm relies MCMC subrouting for sampling from probability distributions given normalized likelihood. Algorithms and 3 provide examples of gradient- based (Metropolis-adjuue Langevin, or MALA) and gradient-free (random-wal walk Metropolis Hasttigg,, or RMH), respectively.
Langevin Algorithm 2: Metropolis- Langevi algorithm (MALA, [16, 22))
Input:
Initial To. steps K, stepsize т, density p(x).
Output: A sample drawn from p(x:).
for
1...., K do
Sample 17 (0, 2rI)
Gaussian noise
1i+1 + 2, + v log (2;) +
b Propose next state
Paccept p(æ: 1)e 11xe *4+1 lou 21
p(z,)e 11 - v lor ple
With probability min(1, Paccept):
Fi+1 3;;
Accept/reject proposal
return ZK
Algorithm 3: Random- walk Metropolis Hastings (RMH, [23])
Input: Initial Fo. steps K, stepsize т, density p(x).
Output: A sample drawn from p(x).
for
1...., do
Sample N00 2r
> Gaussian noise
Fi+1 + 24 +
Propose next state
Paccept +-
p(a,)e 10141-01
With probability min(1, Paccept): Zi+1 t
> Accept/reject proposal
7 return PK
12

---

Proof of Theorem 5.1
We will show the proof for sampling from the failure distribution with likelihood given by Eq. (2); the proof for repair follows similarly. The log-likelihood for the failure distribution is
logPy.oly) + [ v)
The authors of [16] show that MALA enjoys the convergence guarantees in Theorem 5.1 so long as the target log likelihood is strongly convex outside of ball of finite radius R (see Theorem 1 in [16]). More precisely, Ma et al. [16] give the bound
M S O
E
40LR
(m - L)5/2
1
d1/2
dln
(m - L) In !)
L
3/2
when step size is T = O 8LN (m - 1) 3/2 L/ (m - L) + In 1/e) d d-1/2).
Since loggv.((y) is assumed to be strongly m-convex, it is sufficient to show that as ||l||| + co, the strong convexity of the log-prior dominates the non-convexity in J [z, ((zy) y).
For convenience, denote f(y) [zz y) and g(v) Py,oo. We must first show that f(y) + g(v) is (m - -tstonggy convex, for which it suffices to show that f(y) + g(v) (m L) yl is convex. Note that
((V) + g(v)
m 1 2 f(v) + 1b/l g
m |
(5)
2
g(v) 2 2 is convex by m-stong convexity of g. so we must show that the remaining term, fly) + 1/2 is convex. Note that the Hessian of this term is v²f(v) + LI. Since we have assumed that J is T.-Lipschitz smooth (i.e. its gradients are I.-L.iscchtzz continuous), follows that the magnitudes the eigenvalues of f be positive semi-definite completing the proof.
f are bounded by L, which is sufficient for V (z) + LI
AC Power Flow Problem Definition
The design parameters x (Pg. Vlg. Pt, Qi) include the real power injection Pa and AC voltage amplitude [Vlg at each generator in the network and the real and reactive power draws at each load Pr, Qr; all of these parameters are subject to minimum and maximum bounds that we model using uniform prior distribution Pz.0- The exogenous parameters are the state y E R of each transmission line in the network; the admittance of each line is given by (B::) where a is the sigmoid function and Yi.om is the nominal admittance of the line. The prior distribution Pv.0 is an independent Gaussian for each line with mean chosen so that f Py. (y.)dy is equal to the likelihood of any individual line failing (e.g. specified by the manufacturer; we use 0.05 in our experiments). The simulator S solves the nonlinear AC power flow equations [27] to determine the state of the network, and the cost function combines the economic cost of generation eg (a quadratic function of Pg, Pt, Q)) with the total violation of constraints on generator capacities, load requirements, and voltage amplitudes:
c + v(Pg. g. P Pa.mins Pa P.maz) ) + v(Qgs Qg.min Qg.maz) + v (it, Pi,min Pl,aa) v(Qi, Qtimin, Qt.max))
(6)
(7)
+ v (IV), Mmin V Vlma))
(8)
where u(x, Emin, )max) ) = L ([x Imazl + + [xmin - x]]+), L is penalty coefficient (L = 100 in
our experiments), and [o]+ = max(o, 0) is a hinge loss.
Efficient solutions to SCOPF are the subject of active research [28] and an ongoing competition run by the U.S. Department of Energy [29]. In addition its potential economic and environmental impact [27]. SCOPF is also useful benchmark problem for reasons: 1) it is highly non-convex, 2) it has a large space of possible failures, and 3) it can be applied to networks of different sizes
13

---

to test an algorithm's 's scalability. We conduct our studies on one network with 14 nodes and 20 transmission lines (32 design parameters and 20 exogenous parameters) and one with 57 nodes and 80 lines (98 design parameters, 80 exogenous parameters)
The simulator S solves the nonlinear AC power flow equations [5, 30] for the AC voltage ampli- tudes and phase angles (IVI, and the net real and reactive power injections (P, Q) at each bus (the behavior Ş is the concatenation of these values). We follow the 2-step method described in [30] where we first solve for the voltage and voltage angles at all buses by solving system of nonlinear equations and then compute the reactive power injection from each generator and the power injec- tion from the slack bus (representing the connection to the rest of the grid). The cost function J is combination of the generation cost implied by Pg and hinge loss penalty for violating constraints on acceptable voltages each bus exceeding the power generation limits of any generator, as specified in Eq. 8. The data for each test (minimum and maximum voltage and power limits, demand characteristics, generator costs, etc.) are loaded from the data files included in the MAT- POWER software [31].
This experiment can be run with the solve_scacopf py script experiments/poxer..o wwt.ttttttteeeessstsssssseee systemms directory.
the
Search-Evasion Problem Definition
This problem includes Raeck seeker robots and Thide hider robots. Each robot is modeled using Lingle-integratr dynamics and tracks pre-planned trajectory using proportional controller with saturation at maximum speed chosen to match that of the Robotarium platform [25]. The trajectory (t) for each robot is represented as a Bezier curve with control points X,,j
x() = Σ 4 a 1=0
(1-)
The design parameters are the 2D position of the control points for the trajectories of the seeker robots, while the exogenous parameters the control points for the hider robots. The prior dis- tribution for each set of parameters is uniform over the width and height of the Robotarium arena (3.2m m X 2m).
We simulate the behavior the robots tracking these trajectories for 100s with a discrete time step of 0.1 (ineluding the effects of velocity saturation that are observed on the physical platform), and the cost function is
Whide
min Illo.....f
min j-l. j=ll....eeek
[Pride, (t) P P (t) (t))l
where r the sensing range seekers (0.5 in the Nseck = 2 case and 0.25 in for the Nsack 3 case); min(o) = -loguuuexp(-- o) is a smooth relaxation of the clement-wise minimum function where b controls the degree smoothing (b = 100 in our experiments); to...., tn are the discrete time steps of the simulation; and (lide,(()) and (t) are the (æ, y) position of the i-th hider and j-th seeker robot at time 1, respectively.
plain language, this cost is equal to the sum of the minimum distance observed between each hider and the closest seeker over the course of the simulation, adjusted for each secker's search radius.
This experiment can be run with the experiments/hidee and seek directory.
py script in the
Formation Control Problem Definition
This problem includes n drones modeled using double-integrator dynamics, each tracking pre- planned path using proportionallderivttiv controller. The path for each drone is represented as Bezier, in the pursuit-evasion problem.
14

---

The design parameters are the 2D position of the control points for the trajectories, while the ex- ogenous parameters include the parameters of a wind field and connection strengths between each pair of drones. The wind field is modeled using a 3-layer fully-connected neural network with tanh saturation at a maximum speed that induces 0.5 of drag force on each drone.
We simulate the behavior of the robots tracking these trajectories for 30s with a discrete time step of 0.05s, s, and the cost function is
1
J 10 CO || max X2(2:) 10-2
where COM indicates the center of mass of the formation and X2 (qt)) is the second eigenvalue of the Laplacian of the drone network in configuration q-- The Laplacian = D
defined in terms of the adjacency matrix A {atz), J. where dij = 81z0 (20 R - dly)). dej the distance between drones and j. R is the communication radius, and Siz is the connection strength (an exogenous parameter) between the two drones. The degree matrix D diagonal matrix where each entry is the sum of the corresponding row of A.
This experiment can be run with the solve. script in the experimeents/forrmtttnn22 directory.
F16 GCAS Problem Definition
This problem based on the ground collision avoidance system (GCAS) verification problem orig- inally posed in [32], where the challenge is to design a controller for an F16 jet aircraft that avoids collision with the ground when starting from range of initial conditions. We use the JAX imple- mentation [33] of the original F16 dynamics model published in [32]. This model has 15 states and 4 control inputs, and it includes nonlinear engine model and an approximate aerodynamics model. The original model was published with reference GCAS controller that is not able to maintain safety over the desired range of initial conditions (given in Table 4 in [32]). We supplement this reference controller with a neural network controller that only activates below specified altitude threshold; the parameters of this network and the value of the altitude threshold represent our design parameters (approximately 1,800 total parameters, with a uniform prior over these parameters). The exogenous parameters include the initial altitude, roll, pitch, roll rate, and pitch rate (h, o, 0, p, Q). drawn from Gaussian distributions:
h (1500,,200 ft
6 ~ ~/))
7((8/55,/88)
P ~ (=0,0
q q~ 7(0,
The cost function is
J = [200- min 110
1 T
Σ !=
④
2
+
+
2 +
9
where / is altitude in feet, & is roll, 0 is pitch, is angle of attack, 8 is sideslip angle, [++ is the exponential linear unit and min is a soft log-sum -xum-exppa minimum. Empirically. J 2 15 indicates that the aircraft has crashed (h = 0). The simulation is run for 15s with timestep 0.01s but stopped early if the aircraft crashes or leaves the flight envelope where the aerodynamic model accurate (-10- < a < 45" and < 30%).
Pushing Problem Definition
In this problem, model the task of pushing an obstructing object out of the way so that the robot can grasp another object. The robot receives noisy observations of the height, radius, and position of
15

---

the obstructing object and uses this information to plan a pushing action (push height and force) that can move the object to the side. The robot must ensure, without knowing the frictional or inertial properties the object, that its push just forceful enough to move the object without knocking over.
The design parameters include 1,200 parameters of neural network used predict push height and force given noisy observations about the object. The exogenous parameters include the true height h, true radius mass m, center-of-mass height heom, and friction coefficient between the object and the ground, plus the noisy observations of the height and radius h and f. We use a uniform parameters over the design parameters and the following priors for the exogenous parameters:
~ (0.11, 0.2) m r U(0.1, 0.25)
(7(01,1100) kg
Jr 1.0.
cco U(0.1h,0.9h)
8 0(1,00.11)
The cost function is
J
Σ
7 + F1+
where [:] is the ReLU function, Σ r is the net moment applied to the object about its tipping point (defined as positive in the direction of tipping, so that negative moments are stable), and Σ the net force in the pushing direction.
The simulator that we use in this example is fairly simple. It models the object as cylinder resting on a flat plane, and we make the assumption that successful push (i.e. no tipping) is quasi-static. With this assumption, we can detect a successful push by computing the net force and moment applied to the object (including static friction). The object will tip (failure) the net moment about the edge of the cylinder is positive, and the object will not move (also failure) the net force is simplified model can predict and repair
The point of this experiment is to show how certain failures, and that these repairs can transfer to hardware. There are certainly failures that this simulator will not catch (e.g. surface irregularities in the table might catch the edge of the object and cause it to flip), and there are false positives that our simulator will detect (e.g. a near-failure where the transition from static to dynamic friction reduces the net moment and prevents tipping), but we hope that this shows that our method can still be useful with a low-fidelity model.
Hyperparameters
Table 2 includes the hyperparameters used for each environment.
Details on Hardware Experiments
Seareh-evasion
The search-evasion hardware experiment implemented on the Robotarium platform, open- access multi-agent research platform [25]. Trajectories for the hiders and seekers were planned offline using our method (with K = 100 rounds and M = 10 substeps per round, taking 41 s) and then tracked online using linear trajectory-trackin controllers.
Pushing
The pushing experiment implemented using a Franka Research 3 7-DOF robot arm and an Intel RealSense D415 RGBd camera. The camera positioned over the robot's workspace, and
16

---

the depth image was used to segment the target and obstructing objects, well as estimating the height and radius of each object. These estimates were passed to the planning neural network, using weights trained using our method (K = 100, M 10). The planning network predicts push height and force; the pushing action executed using a Cartesian impedance tracking controller.
Further Limitations
In this paper, we restrict our attention to problems with continuous design and exogenous param- eters, since gradient-based inference methods realize the greatest benefit on problems with a con- tinuous domain. It is possible extend method to problems with mixed continuous-discra domains by using gradient-based sampling algorithm for the continuous parameters and gradient- free sampler for the discrete parameters; we hope to explore the performance implications of this combination in future work.
Although the sampling methods we use in this paper remain theoretically sound when the simulator and cost landscape discontinuous (as is the case in manipulation problems, for example), remains to be seen what practical effects discontinuity might have convergence rate and solution quality.
Finally, although we include details on how to use tempering with our approach, we found that tempering was not needed for convergence on any of the problems we studied; more work needed understand when tempering necessary for convergence of MCMC-base based algorithms on various robotics problems.
Table 2: Hyperparameters used for each environment. ng is the number of quenching rounds; denotes the round number in Alg. Environment
K
M
f2
A
Formation (5 agents)
5
10
50
5
Formation (10 agents)
10-3
50
5
-
Search-evasion (6 seekers, 10 hiders)
10
10
10-2
100
10
25
5
Search-evasion (12 seekers, 20 hiders)
10
10
10-2
100
10
25
-5i
Power grid (14-bus)
10
10-6 6 for a
100
10
e5
10-2 for y
Power grid (57-bus)
10
10
10 for
100
10
10
51
10 for y
F16
10
10 for ar
100
0
10v
10 for y
Pushing
10
10
10
10
10
0
10k
17


Reviews:
[{"review_id": "sZYRDp-aTsx", "reviewer": "Reviewer_GT1s", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper proposes a reframing of adversarial optimization as a Bayesian inference problem, by sampling system control parameters and failure modes from probability distributions. The proposed approach relies on the availability of differentiable simulators describing both the control behavior of the autonomous systems in question, as well as the exogenous parameters affecting the states of the agents. The paper demonstrates this approach on three different simulated domains, as well as a simple experiment on a real-world environment.", "summary_of_recommendation": "While the paper proposes a simple and novel idea and demonstrates its usability on multiple simulated domains, some concerns are present regarding usability to real-world robotic scenarios. Were I to be convinced of the applicability and scalability of this approach, I would be happy to raise my score accordingly. \n\nEDIT: The authors have addressed my concerns in their rebuttal, and have spent considerable time and effort to improve their paper in the given time, including an additional hardware experiment. I believe the updated version of the paper would be a valuable addition to the robot learning community. Therefore, I have updated my score to \"weak accept\". ", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "Are there any restrictions on the structure of the distributions $p_{x,i}, p_{y,i}$? While they should clearly allow easy sampling and inference of likelihood, would the proposed method be compatible with high-dimensional, complex distributions such as ones represented by deep neural networks?\n\nSome details are unclear regarding the real hardware experiment - is it run for two iterations of only (i.e. $K=2$)? Or does it use trajectories obtained from the simulated hide and seek environment? \n\nWhile it is not the focus of the paper, it is interesting to note that in some of the tasks, the DR approach performs better on average than the proposed algorithm on the sampled test set of exogenous parameters (Figure 3). While it makes sense that DR is not optimizing against challenging failure cases and therefore performs better, is there a way to avoid the loss of performance in the average case while optimizing for the worst case using the authors’ proposed method?\n\nRegarding convergence rates (Figure 4): it is clear that the gradient-based sampling method converges faster in terms of sampling rounds. However, how long does it take in practice to perform these iterations, and how does the dimensionality of the problem affect this duration?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper is very well written, ideas are conveyed concisely and it is very easy to follow along. The contributions of the paper are stated clearly and explained well.\n\nWhile I am not entirely familiar with the previous literature on failure mode prediction, it seems that the main ideas of the paper are novel and constitute a meaningful contribution to the field of autonomous system verification. \n\nThe experiments presented in the paper showcase the generality of this method by demonstrating its usability on a variety of robotic domains, some of which may have real-world impact. \n\nDespite its versatility displayed by the tested tasks, I am concerned that the usability of this approach to real world robotic scenarios is limited to those where good simulators are available. \n\nAll of the domains shown in the paper enjoy simulators which model both the design parameters and the failure modes well. However, for many robotic tasks (such as object manipulation or quadrupedal navigation) designing good simulators which can predict failure modes well is an extremely complex task: consider, for instance, failures arising from friction between objects. \n\nAnother concern is related to the complexity of the computation required for the sampling procedure. The experiment shown on real hardware is relatively small, with low-dimensional spaces for both the design and exogenous parameter spaces. While setting design parameters offline for a single scenario may work for many domains, calculating trajectories repeatedly may be computationally prohibitive, depending on the dimensionality of the problem.\n\nI believe an important addition to this paper is a discussion of the types of tasks and domains to which the approach proposed in the paper is applicable. At the very least, this issue should be mentioned in the limitations section. ", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}, {"review_id": "lx0uyBf57O", "reviewer": "Reviewer_meR7", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper presents a Bayesian inference-based method capable of automatically predicting potential system failures and generating corresponding repair solutions. The proposed algorithms for automatic failure detection and repair are designed in an evolutionary optimization manner, utilizing both gradient-based and sampling-based methods to find solutions. Extensive testing on three real-world robotics and control problems demonstrates promising results in terms of speed and performance.\n", "summary_of_recommendation": "the paper introduces an interesting approach to automatic failure detection and repair in robotics and control using Bayesian inference. However, addressing the aforementioned weaknesses is crucial to enhancing the paper's scientific contribution and practical relevance. Providing theoretical insights, addressing assumptions, and conducting more robust and fair comparisons would significantly strengthen the paper and make it a valuable addition to the field of automatic failure detection and repair in robotics and control. As such, I recommend rejection for the current version. ", "technical_quality": "Fair: Significant additional evidence required to support the claims.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "1. Can you clarify the novel aspects of the Bayesian inference-based method and how it extends beyond conventional optimization approaches?\n2. How do you plan to provide theoretical insights on the algorithm's convergence, optimality, and stability under specific assumptions?\n3.  How will you address concerns about the method's reliance on strong assumptions, such as the known cost function and the distribution of x and y variables, to ensure adaptability in scenarios with uncertain or incomplete information?\n4. How will you ensure a more balanced and unbiased comparison with other optimization approaches, considering the advantage provided by known distributions in the proposed method?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Strengths:\n1. The paper's comprehensive evaluation on diverse real-world robotics and control problems provides valuable insights into the method's practical applicability and versatility.\n\nWeaknesses:\n1. The methodological contribution appears limited, as the MCMC algorithm and the proposed sampling or gradient-based optimization techniques are well-known and seem to be straightforward applications of standard optimization approaches.\n2. The absence of theoretical results and analysis leaves important questions unanswered, such as the algorithm's convergence, optimality, and the underlying assumptions regarding exogenous parameters.\n3. The reliance on strong assumptions, such as the assumed known cost function and explicit knowledge of the distribution of x and y variables, raises concerns about the method's generalizability and adaptability to various real-world scenarios, particularly in settings with uncertain or incomplete information.\n4. The fairness of the comparison in the experiments is questionable. The advantage provided by known distributions in the proposed method, compared to the adversarial optimization approach, may skew the results in favor of the former, potentially affecting the scalability and performance assessment.", "extra_scores": {"impact": "2: Unsure, difficult to assess."}, "extra_text": {}}, {"review_id": "liG4_8Jms49", "reviewer": "Reviewer_nExT", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper presents an approach for predicting failures for autonomous systems and then repairing the failures by re-designing the system. Failure prediction and repair are achieved by Bayesian inference.", "summary_of_recommendation": "My current recommendation is a Weak Reject because the presented approach suffers from a few challenges that make me wonder about its scalability beyond academic examples.\n\n_Edit after rebuttal_\n\nScore updated to Weak Accept.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Relevant but unlikely to deploy to hardware in near future", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "Please provide clarifications for my comments above.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Simultaneous stress-testing and improvement of autonomous systems is an interesting topic, and I do like the idea of leveraging Bayesian inference for achieving this. But, I have some reservations about the algorithm presented in this paper:\n 1. **Prior Selection:** The approach is sensitive to the choice of the priors, used in lines 4 and 7 of algorithm 1. A good prior would be very difficult to prescribe for a complex autonomous system. The paper does not discuss in much detail how a prior can be chosen. The experimental section also does not provide any prior selection details for the presented examples.\n2. **Catastrophic Forgetting:** The design distribution $p_{x,i}$ at iteration $i$ is based on the failure samples from the previous iteration $[y]_{i-1}$. How is it ensured that the new designs are still able to address the failure modes from all iterations prior to $i-1$?\n3. **Simulator:** As the approach relies on a simulator, how do the authors envision addressing the inevitable sim-to-real gap? Additionally, I don't completely understand what Assumption 1 (almost everywhere differentiable simulator) entails.\n4. **Bayesian RL**: Another aspect that I would like to understand better is how this algorithm compares to Bayesian RL approaches. I see some strong parallels, for instance failure prediction can be viewed as exploration while design parameter selection can be viewed as exploitation.\n5. **Experimental Evaluation:** Although the environments presented in the paper demonstrate the algorithm, they seem more academically inclined and do not demonstrate the algorithm's real-world significance. Furthermore, the third environment, power grid dispatch, is completely disconnected from robotics. It would have been great to see some application where a robot's autonomy stack is stress tested for failures and then repaired using this algorithm.\n\n_Edit after rebuttal_\n\nI am still trying to convince myself that the paper has sufficient novelty. I viewed the ability of this paper to provide useful insights about what is breaking the system to be a big draw. Turns out that is not really the case. Without it, I am having a hard time understanding how this approach stands apart from the adversarial attack literature in ML where it is fairly common to search for examples that result in failures and then training with them to robustify the network. Indeed, there has been work on attacks with Bayesian optimization as well.\nHowever, the authors did a stellar job during the rebuttal phase and have introduced a hardware demo for their approach along with a theoretical extension, so I am adjusting my score to Weak Accept.", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}, {"review_id": "Rc1gY1KHPHQ", "reviewer": "Reviewer_fucE", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper uses optimization-as-inference in an adversarial setting to optimize against worst-case failure modes.\nThis is done using MCMC methods. \nThe method is evaluated on a multi-agent hide-and-seek task and power network control.", "summary_of_recommendation": "On the whole I liked the idea and it felt like an interesting way of doing robustness and optimization-as-inference. ", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "1) How well did MALA perform as an optimizer compared to DR and GD? Figure 3 combines different optimization methods and robustness strategies, so its hard to know what is significant. Is the poor performance in 3c) and f) due to the robustness being too strong or the optimizer finding worse minima? \n\n2) What role did the temperature play in the optimization? How did the schedule need to be tuned? ", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This is not the typical sort of paper you see at CoRL, but i thought it was an interesting idea and the paper presentation was nice and clear. \n\n**Pseudo-posteriors / likelihoods.**\nI think this paper would be improved by framing equations 2 and 3 as 'pseudo'-posteriors. These methods are seen across machine learning under different names, and I think pseudo-posteriors captures the Bayesian inference aspect while acknowledging there is no proper likelihood function. Pseudo posteriors are derived from KL regularized optimization problem, so they are a nice way of framing optimization-as-inference methods. Pseudo posteriors also make the 'tempering' problem explicit, since the temperature is part of the objective. The paper only mentions the temperature tuning problem in passing.\n\nSome relevant references \n\nO. Catoni. Statistical learning theory and stochastic optimization\n\nOn the properties of variational approximations of Gibbs posteriors\nP Alquier, J Ridgway, N Chopin - The Journal of Machine Learning 2016\n\nA. S. Dalalyan and A. B. Tsybakov. Aggregation by exponential weighting, sharp PAC-\nBayesian bounds and sparsity\n\nInferring Smooth Control: Monte Carlo Posterior Policy Iteration with Gaussian Processes\nJ Watson, J Peters - Conference on Robot Learning, 2023\n\n**Different approximate inference methods.** I would have liked to have seen some more exploration of different approximate inference techniques, especially since JAX was used. I imagine methods like stein variational gradient descent, the Laplace approximation and automatic differentiation VI would be straight forward to integrate with the JAX code.\n\n**Sim-to-real RL.** It would have been nice to have seen a sim-to-real deep RL experiment to assess that this method can scale.  I imagine this would need a more scalable approximate inference method and probably some integration with entropy regularized RL.\n\n**Experiment seeds and uncertainty intervals.**\nFigure 4 is missing different random seed runs and uncertainty intervals. It also looks like Figure 3 has many random test cases but only for a single run? rather than different seeds? I would have also been good if Figure 4 captured the learning speed of the baselines, \n", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper frames the problem of discovering failure modes of a system as optimization with uncertain variables. A stochastic optimization problem is formulated, and solved using a form of coordinate descent (for the design parameters and the adversarial disturbance parameters), and each coordinate optimization step is done by casting the optimization problem as an inference problem, and using MCMC. 

The initial reviews were mostly negative, with concerns about the novelty of the approach, the soundness of the methodology, the relevance of the experiments, and the relevance to CoRL. The authors managed to pull off a commendable rebuttal, including some theoretical analysis and real robot results on simple object manipulation. Following the rebuttal, most reviewers expressed a positive view of the paper, and agreed to accept it.

I find the empirical finding that the MCMC variants work well in this problem formulation on a variety of tasks to be interesting, and therefore I agree with the reviewers and recommend acceptance. 
However, the theoretical analysis feels rushed. The current results are for optimizing each coordinate (x and y), but there is no result on the complete optimization - what guarantees that the optimization doesn’t oscillate? The authors should clearly discuss this in their final version of the paper.
Decision: Accept (Poster)