Safe Nonlinear Control Using Robust Neural Lyapunov-Barrier Functions
Anonymous Author(s)
Af ﬁ liation
Address email
Abstract: Safety and stability are common requirements for robotic control systems; however, designing safe, stable controllers remains dif ﬁ cult for nonlinearand uncertain models.
We develop a model-based learning approach to synthesizerobust feedback controllers with safety and stability guarantees.
We take inspiration from robust convex optimization and Lyapunov theory to de ﬁ ne robust control
Lyapunov barrier functions that generalize despite model uncertainty.
We demonstrate our approach in simulation on problems including car trajectory tracking,nonlinear control with obstacle avoidance, satellite rendezvous with safety constraints and ﬂ ight control with a learned ground effect model.
Simulation resultsshow that our approach yields controllers that match or exceed the capabilities ofrobust
MPC while reducing computational costs by an order of magnitude
Introduction
[Figure]
considered in Section 6.
Robot control systems are challenging to design, not least because of the problems of task complexity and model uncertainty .
Robotics control problems like those in Fig.
1 ofteninvolve both safety and stability requirements, where the controller must drive the system towards a goal state while avoiding unsafe regions.
Complicating matters, the model used todesign the controller is seldom a perfect representation of thephysical plant, and so controllers must account for uncertaintyin any parameters (e.g. mass, friction, or unmodeled effects)that vary between the engineering model and true plant.
Automatically synthesizing safe, stable, and robust controllers fornonlinear reach-avoid tasks is a long-standing open problem in controls.
In this paper, we addressthis problem with a novel approach to robust model-based learning.
Our work presents a uni ﬁ edframework for handling both model uncertainty and complex safety and stability speci ﬁ cations.
Over the years, several approaches have been proposed to solve this problem.
In one view, reachavoid can be treated as an optimal control problem and solved using model predictive control (MPC)schemes and their robust variants.
Robust MPC promises a method for general-purpose controllersynthesis, ﬁ nding an optimal control signal given only a model of the system and a speci ﬁ cation ofthe task.
However, there are a number of recognized disadvantages of robust MPC.
First, there arecurrently no techniques for guaranteeing the safety, stability, or recursive feasibility of robust MPCbeyond the linear case [1].
Second, many sources of model uncertainty (e.g. mass or friction) aremultiplicative in the dynamics, but robust MPC algorithms are typically limited to additive uncertainty [1, 2].
Finally, MPC is computationally expensive, making it dif ﬁ cult to achieve high controlfrequencies in practice [3].
An alternative method for synthesizing safe, stable controllers comes from Lyapunov theory, throughthe use of control Lyapunov and control barrier functions (resp., CLFs and CBFs, [4]) — certi ﬁ catesthat prove the stability and safety of a control system, respectively.
CLFs and CBFs are similar tostandard Lyapunov and barrier functions, but they can be used to synthesize a controller rather than just verifying the performance of a closed-loop system.
Unfortunately, CLF and CBF certi ﬁ catesare very dif ﬁ cult construct in general, particularly for systems with nonlinear dynamics [5].
The most recent set of methods promising general-purpose controller synthesis come from the ﬁ eldof learning for control; for instance, using reinforcement learning [6, 7] or supervised learning[8, 9, 10, 11].
However, the introduction of learning-enabled components into safety-critical controltasks raises questions about soundness, robustness, and generalization.
Some learning-based controltechniques incorporate certi ﬁ cates such as Lyapunov functions [8], barrier functions [12, 10, 13],and contraction metrics [9, 11] to prove the soundness of learned controllers.
Unfortunately, thesecerti ﬁ cates’ guarantees are sensitive to uncertainties in the underlying model.
In particular, if themodel used during training differs from that encountered during deployment, then guarantees onsafety and stability may no longer hold.
Our main contribution is a learning-based framework for synthesizing robust nonlinear feedbackcontrollers from safety and stability speci ﬁ cations.
This contribution has two parts.
First, we provide a novel extension of control Lyapunov barrier functions to robust control, de ﬁ ning a robustcontrol Lyapunov barrier function (robust CLBF).
Second, we develop a model-based approach tolearning robust CLBFs, which we use to derive a safe controller using techniques from robust convex optimization.
Other methods for learning Lyapunov and barrier certi ﬁ cates exist, but a keyadvantage of our approach is that we learn certi ﬁ cates with explicit robustness guarantees, enablinggeneralization beyond the system parameters seen during training.
We demonstrate our approachon a range of challenging control problems, including trajectory tracking, nonlinear control withobstacle avoidance, ﬂ ight control with a learned model of ground effect, and a satellite rendezvousproblem with non-convex safety constraints, comparing our approach with robust MPC.
In all ofthese experiments, we ﬁ nd that our method either matches or exceeds the performance of robust
MPC while reducing computational cost at runtime by at least a factor of 10.
Related Work
This work builds on a rich history of certi
ﬁ cate-based control theory, including classical Lyapunovfunctions as well as more recent approaches such as control Lyapunov functions (CLFs [14, 15]) andcontrol barrier functions (CBFs [16], a generalization of arti ﬁ cial potential ﬁ elds [17]).
The majorityof classical certi ﬁ cate-based controllers rely on hand-designed certi ﬁ cates [18, 19], but these can bedif ﬁ cult to obtain for nonlinear or high-dimensional systems.
Some automated techniques existfor synthesizing CLFs and CBFs; however, many of these techniques (such as ﬁ nding a Lyapunovfunction as the solution of a partial differential equation) are computationally intractable for manypractical applications [5].
Other automated synthesis techniques are based on convex optimization,particularly sum-of-squares programming (SOS, [20]), but are limited to systems with polynomialdynamics and do not scale favorably with the dimension of the system.
A promising line of work in this area is to use neural networks to learn certi ﬁ cate functions.
Thesetechniques range in complexity from verifying the stability of a given control system [21, 22] tosimultaneously learning a control policy and certi ﬁ cate [9, 8, 10].
Most of these works do notexplicitly consider robustness to model uncertainty, although contraction metrics may be used tocertify robustness to bounded additive disturbance [9].
Most approaches to handling model uncertainty in the context of certi ﬁ cate-guided learning forcontrol involve online adaptation.
For example, [18, 23] assume that a CLF or CBF are givenand learn the unmodeled residuals in the CLF and CBF derivatives.
When combined with a QPbased CLF/CBF controller, this technique enables adaptation to model uncertainty but relies on apotentially unsafe exploration phase.
Although safe adaptation strategies exist, the main drawbackwith these techniques is their reliance on a hand-designed CLF and CBF, which are non-trivial tosynthesize for nonlinear systems.
Additionally, combined CLF/CBF controllers are prone to gettingstuck when the feasible sets of the CLF and CBF no longer intersect.
Online optimization-based control techniques such as model-predictive control (MPC) are also relevant as a general-purpose control synthesis strategy.
However, the computational complexity of
MPC, and particularly robust MPC, is a widely-recognized issue, particularly when considering deployment to resource-constrained robotic systems such as UAVs [1, 3].
We revisit the computationalcost of robust MPC, particularly as compared with the cost of our proposed method, in Section 6.
Some approaches apply learning to characterize uncertainty in system dynamics and augment a ro- bust MPC scheme [24], but these methods do not fundamentally change the computational burdenof MPC.
Other methods rely on imitation learning to recreate an MPC-based policy online [25], butthese methods can encounter dif ﬁ culties in generalizing beyond the training dataset.
A number of techniques from classical nonlinear control also deserve mention, such as sliding modeand adaptive controllers.
These methods do not directly support state constraints and so must bepaired with a separate trajectory planning layer [26].
Another drawback is that these techniquesrequire signi ﬁ cant effort to manually derive appropriate feedback control laws, and we are primarilyinterested in automated techniques for controller synthesis.
Preliminaries and Background
We consider continuous-time, control-af ﬁ ne dynamical systems of the form ˙ x
= f θ ( x ) + g θ ( x ) u ,where x ∈ X ⊆ R n , u ∈ R � , and f θ : R n → R n and g θ : R n → R n × � are smooth functionsmodeling control-af ﬁ ne nonlinear dynamics.
We assume that f θ and g θ depend on model parametersθ ∈ Θ ⊆ R r and are af ﬁ ne in those parameters for any ﬁ xed x .
This assumption on the dynamics isnot restrictive; it covers many physical systems with uncertainty in inertia, damping, or friction (e.g.rigid-body dynamics or systems described by the manipulator equations), and it includes boundedadditive and multiplicative disturbance as a special case.
We also assume that f θ and g θ are Lipschitzbut make no further assumptions, allowing us to consider cases when components of f θ and g θ arelearned from experimental data.
For concision, we will use f and g (without subscript) to refer tothe dynamics evaluated with nominal parameters θ 0 ∈ Θ .
In this paper, we consider the followingcontrol synthesis problem:
De ﬁ nition 1 (Robust Safe Control Problem) .
Given a control-af ﬁ ne system with uncertainparameters θ ∈ Θ , a goal con
ﬁ guration x goal , a set of unsafe states X
unsafe ⊆ X , 117 and a set of safe states X safe ⊆
X (such that X safe ∩ X unsafe = ∅ and x goal ∈
X safe ), ﬁ nd a control policy u
= π ( x ) such that all trajectories x ( t ) satisfying ˙ x
=f θ ( x ) + g θ ( x ) π ( x ) and x (0) ∈
X safe have the following properties for any parameters θ :
Reachability of x goal with tolerance
[Equation]
Simply put, we wish to reach the goal x goal while avoiding the unsafe states X unsafe .
We usethe notion of reachability instead of asymptotic stability to permit (small) steady-state error; in thefollowing we will use “stable” as shorthand for reachability.
Note that we do not require X safe ∪
X unsafe =
X , as it will be made clear in the following discussion that we need a non-empty boundarylayer X \ ( X safe ∪ X unsafe ) to allow for ﬂ exibility in ﬁ nding a safety certi ﬁ cate.
Lyapunov theory provides tools that are naturally suited to reach-avoid problems: control Lyapunovfunctions (for stability) and control barrier functions (for safety [4]).
To avoid issues arising fromlearning two separate certi ﬁ cates, we rely on a single, unifying certi ﬁ cate known as a control Lyapunov barrier function (CLBF).
Our de ﬁ nition of CLBFs is related to those in [27] and [28] (differingfrom the formulation in [27] by a constant offset c , and differing from [28] where safety and reachability are proven using two separate CLBFs).
We begin by providing a standard de ﬁ nition of a CLBFin the non-robust case, but in the next section we provide a novel, robust extension of CLBF theorybefore demonstrating how neural networks may be used to synthesize these functions for a generalclass of dynamical system.
In the following, we denote L f V as the Lie derivative of V along f .
De ﬁ nition 2 (CLBF) .
A function V : X → R is a CLBF if, for some c, λ > 0 ,
[Equation]
Intuitively, we can think of a CLBF as a special case of a control Lyapunov function where the safeand unsafe regions are contained in sub- and super-level sets, respectively.
If we de ﬁ ne a set ofadmissible controls K ( x ) = { u | L f V + L g V u + λ V ≤ 0 } , then we arrive at a theorem provingthe stability and safety of any controller that outputs elements of this set (the proof is included in thesupplementary material).
Theorem 1.
If V ( x ) is a CLBF then any control policy π ( x ) ∈ K ( x ) ∀ x ∈ X will be both safeand stable, in the sense of De ﬁ nition 1.
Based on these results, we can de ﬁ ne a CLBF-based controller, analogous to the CLF/CBF-basedcontroller in [18] but without the risk of con ﬂ icts between the CLF and CBF conditions, relying onthe CLBF V and some nominal controller π nominal (e.g. the LQR policy):
[Equation]
It should be clear that π CLBF ( x ) ∈ K ( x ) ∀ x ∈ X \ x goal , so this controller will result in a system 148 that is certi ﬁ ably safe and stable (with the CLBF V acting as the certi ﬁ cate).
The nominal controlsignal π nominal is included to encourage smoothness in the solution π CLBF ( x ) , particularly near thedesired ﬁ xed point at x goal where ˙
V becomes small.
CLBFs provide a single, uni ﬁ ed certi ﬁ cate ofsafety and stability; however, some signi ﬁ cant issues remain.
In particular, how do we guaranteethat a CLBF will generalize beyond the nominal parameters?
Robust CLBF Certi ﬁ cates for Safe Control
In this section, we extend the de ﬁ nition of CLBFs to provide explicit robustness guarantees, and wepresent a key theorem proving the soundness of robust CLBF-based control.
De ﬁ nition 3 (Robust CLBF, rCLBF) .
A function V : X → R is a robust CLBF for boundedparametric uncertainty θ ∈ Θ , where Θ is the convex hull of scenarios θ 1 , θ 2 , . . .
, θ n s if the standard
CLBF conditions (1a) – (1d) hold, the dynamics f and g are af ﬁ ne with respect to θ , and ∀ x ∈
X \ x goal there exist c, λ > 0 such that
[Equation]
As in the non-robust case, we de ﬁ ne the set of admissible controls for a robust CLBF, K r ( x ) =
� u |
L f
θ i V + L g θ i
V u + λ V ≤ 0 ∀ i = 0 , . .
. , n s � , and the corresponding QP-based controller, thesoundness of which is given by Theorem 2:
[Equation]
Theorem 2.
If V ( x ) is a robust CLBF, then any control policy π ( x ) ∈ K r ( x ) ∀ x ∈ X will beboth safe and stable, in the sense of De ﬁ nition 1, when executed on a system f θ , g θ with uncertainparameters θ ∈ Θ (where Θ is the convex hull of scenarios θ 0 , . .
, θ n s ).
Proof.
See the supplementary materials.
This result demonstrates the soundness and robustness of an rCLBF-based controller, but does notprovide a means to construct a valid rCLBF.
In the next section, we will present an automated modelbased learning approach to rCLBF synthesis, yielding a general framework for solving robust safecontrol problems even for systems with complex, nonlinear, or partially-learned dynamics.
Learning Robust CLBFs
A persistent challenge in using of certi ﬁ cate-based controllers is the dif ﬁ culty of ﬁ nding valid certi ﬁ cates, especially for systems with nonlinear dynamics and complex speci
ﬁ cations of X safe and
X unsafe (e.g. obstacle avoidance).
Taking inspiration from recent advances in certi ﬁ cate-guidedlearning for control [8, 10], we employ a model-based supervised learning framework to synthesizean rCLBF-based controller.
The controller architecture is comprised of three main parts: the rCLBF V , a proof controller π NN , and the QP-based controller (rCLBF-QP).
We parameterize V : X → Rand π NN : X → R � as neural networks.
These networks are trained of ﬂ ine, where π NN is used toprove that the feasible set of (rCLBF-QP) is non-empty, then V is evaluated online to provide theparameters of (rCLBF-QP), which is solved to ﬁ nd the control input.
In the of ﬂ ine training stage,our primary goal is ﬁ nding an rCLBF V ( x ) such that the conditions of De ﬁ nition 3 are satis ﬁ ed.
To ensure (1b), we de ﬁ ne V ( x ) = w T ( x ) w ( x ) ≥ 0 , where w is the activation vector of the last hidden layer of the V neural network.
To train V such that conditions (1a), (1c), (1d), and (3a) aresatis ﬁ ed over the domain of interest, we sample N train points uniformly at random from X to yielda population of training points x , then de ﬁ ne the empirical loss:
[Equation]
where a 1 – a 3 are positive tuning parameters, � > 0 is a small parameter (typically 0 . 01 ) that allowsus to encourage strict inequality satisfaction and enables generalization claims, N safe and N unsafe arethe number of points in the training sample in X safe and X unsafe , respectively, and [ ◦ ] + = max( ◦ , 0) 189 is the ReLU function.
The terms in this empirical loss are directly linked to conditions (1a), (1c),(1d), and (3a) such that each term is zero if the corresponding condition is satis ﬁ ed at all N train training points.
For example, the ﬁ nal term in this loss is designed to encourage satisfaction of the robust
CLBF decrease condition (3a).
The factor r ( x ) in the ﬁ nal term is computed by solving (rCLBF-QP)at each training point and computing the maximum violation of constraint (4), such that r ( x ) = 0when
the QP has a feasible solution and r ( x ) > 0 otherwise.
This loss is optimized using stochasticgradient descent, alternating epochs between training the V and π NN networks.
During training,we rely on π NN to compute the time derivative of V ( x ) in the ﬁ nal term of the loss.
To provide atraining signal for π NN , we de ﬁ ne an additional loss L π = � π NN − π nominal � 2 , where π nominal is 198 a nominal controller (e.g. a policy derived from an LQR approximation).
The parameters of V andπ NN are optimized using the combined loss L =
L rCLBF + (10 − 5 )
L π .
The small weight applied to
L π ensures that the training process prioritizes satisfying the CLBF conditions.
An important detail of our control architecture is that the learned control policy π NN is used primarily to demonstrate that the feasible set of (rCLBF-QP) is non-empty.
We are not required to useπ NN at execution time; we can choose any control policy from the admissible set K r ( x ) .
In the online stage, we rely on an optimization-based controller (rCLBF-QP), which solves a small quadraticprogram with n s constraints and � variables (one for each element of u ).
To ensure that this QP isfeasible at execution, we permit a relaxation of the CLBF constraints (4) and penalize relaxation witha large coef ﬁ cient in the objective.
Once trained, V can be veri ﬁ ed using neural-network veri ﬁ cation tools [29], sampling [30], or a generalization error bound [10].
More details on data collection,training, implementation, and veri ﬁ cation strategies are included in the supplementary materials.
It is important to note that this training strategy encourages satisfying (3a) only on the ﬁ nite set oftraining points sampled uniformly from the state space; there is no learning mechanism that enforcesdense satisfaction of (3a).
In the supplementary materials, we include plots of 2D sections of the statespace showing that (3a) is satis ﬁ ed at the majority of points, but there is a relatively small violationon a sparse subset of the state space. Because these violation regions are sparse, the theory ofalmost Lyapunov functions applies [31]: small violation regions may induce temporary overshoots(requiring shrinking the certi ﬁ ed invariant set), but they do not invalidate the safety and stabilityassurances of the certi ﬁ cate.
Strong empirical results on controller performance in Section 6 supportthis conclusion, though we admit that good empirical performance is not a substitute for guaranteesbased on rigorous veri ﬁ cation, which we hope to revisit in future work.
To evaluate the performance of our learned rCLBF-QP controller, we compare against min-maxrobust model predictive control (as described in [2, 32]) on a series of simulated benchmark problems representing safe control problems with increasing complexity.
The ﬁ rst two concern trajectorytracking, where we wish to limit the tracking error despite uncertainty in the reference trajectory.
The next two benchmarks are UAV stabilization problems that add additional safety constraintsand increasingly nonlinear dynamics.
The last three benchmarks involve highly non-convex safetyconstraints.
The ﬁ rst four benchmarks provide a solid basis for comparison between our proposedmethod and robust MPC, while the last three demonstrate the power of our approach to generalizeto maintain safety even in complex environments.
In each experiment, we vary model parameters randomly in Θ , simulate the performance of thecontroller, and compute the rate of safety constraints violations and average error relative to the goal � x − x goal � across simulations.
These data are reported along with average evaluation time for 233 each controller in Table 1.
To examine the effect of control frequency on MPC performance, weinclude results for two different control periods dt for all robust MPC experiments (we also reportthe horizon length N ).
In some cases we observed that the evaluation time for MPC exceeds thecontrol period; in practice this would lead to the controller failing, but in our experiments we simplyran the simulation slower than real-time.
Our robust MPC comparison supports only linear modelswith bounded additive disturbance; we linearize the systems about the goal point and select an additive disturbance to approximate the disturbance from uncertain model parameters.
The followingsections will present results from each benchmark separately, and more details are provided in thesupplementary materials, including the dynamics and constraints used for each benchmark
[Table]
as wellas the hardware used for training and execution.
[Table]
6.1 Car trajectory tracking
First
[Table]
we consider the problem of tracking an a priori unknown trajectory using two different carmodels.
In the ﬁ rst model (the kinematic model), the vehicle state is [ x e , y e , δ , v e , ψ e ] , representingerror relative to the reference trajectory ( δ is the steering angle).
The second model (the sideslipmodel) has state [ x e , y e , δ , v e , ψ e , ˙ ψ e , β ] , where β is the sideslip angle [34].
Both models havecontrol inputs for the rate of change of δ and v e .
We assume that the reference trajectory is parameterized by an uncertain curvature: at any point the angular velocity of the reference point can vary on[ − 1 . 5 , 1 . 5] .
The goal point is zero error relative to the reference, and the safety constraint requiresmaintaining bounded tracking error.
The performance of our controller is shown in Fig.
We see that for both models, both our controllerand robust MPC are able to track the reference trajectory.
However, robust MPC was only successfulwhen run at slower than real-time speeds (with a control period dt = 0 .
1 s roughly twice as fast as theaverage evaluation time).
MPC became unstable when run at a slower control frequency dt = 0 .
25 s .
In contrast, our rCLBF-QP controller runs in real-time with a control period of ≈ 10 ms on a laptopcomputer.
This signi ﬁ cant improvement in speed is due primarily to the reduction in the size of(rCLBF-QP) relative to that of the QPs used by robust MPC.
For example, for the sideslip model, ourcontroller solves a QP with 2 variables and 2 constraints, whereas the robust MPC controller solvesa QP with 35 variables and 23 constraints (after pre-compiling using YALMIP [32]).
Because thelearned rCLBF encodes long-term safety and stability constraints into local constraints on the rCLBFderivative, the rCLBF controller requires only a single-step horizon (as opposed to the recedinghorizon used by MPC).
By comparing performance between these two models, we can discern an important feature of ourapproach.
Increasing the state dimension when moving between models does not substantially increase the evaluation time for our controller (as it does for robust MPC), but it does degrade thetracking performance, suggesting that the number of samples required to train the CLBF to anygiven level of performance increases with the size of the state space.
These examples also highlighta potential drawback of our approach, which relies on a parameter-invariant robust CLBF.
Becauseit attempts to ﬁ nd a common rCLBF for all possible parameter values, our controller exhibits somesmall steady-state error near the goal.
This occurs because there is no single control input that renders the goal a ﬁ xed point for all possible parameter values and motivates our use of a goal-reachingtolerance in De ﬁ nition 1.
6.2 UAV stabilization
[Figure]
The next two examples involve stabilizing a quadrotor near the ground while maintaining a minimumaltitude.
Relative to the previous examples, these benchmarks increase the complexity of the stateconstraints, and we consider two models with increasingly challenging dynamics.
The ﬁ rst model(referred to as the “3D quadrotor”) has 9 state dimensions for position, velocity, and orientation, withcontrol inputs for the net thrust and angular velocities [9].
The second model (the “neural lander”)has lower state dimension, including only translation and velocity, with linear acceleration as aninput, but its dynamics include a neural network trained to approximate the aerodynamic groundeffect, which is particularly relevant to this safe hovering task [35].
The mass of both models isuncertain, but assumed to lie on [1 . 0 , 1 . 5] for the 3D quadrotor and [1 . 47 , 2 . 0] for the neural lander.
Fig.
3 shows simulation results on these two models.
The trend from the previous benchmarkscontinues: our controller maintains safety while reducing evaluation time by a factor of 10 relativeto MPC.
Moreover, while the robust MPC method can achieve low error relative to the goal for thethe 3D quadrotor model, the nonlinear ground effect term prevents MPC from driving the neurallander to the goal.
In contrast, the rCLBF-QP method can consider the full nonlinear dynamics ofthe system, including the learned ground effect, and achieves a much lower error relative to the goal.
6.3 Navigation with non-convex safety constraints
The preceding benchmarks all include convex safety constraints that can be easily encoded in a linearrobust MPC scheme.
Our next set of examples demonstrate the ability of our approach to generalizeto complex environments.
These problems are commonly solved by combining planning and robusttracking control, so in our comparisons we use robust MPC to track a safe reference path througheach environment.
In contrast, our rCLBF-QP controller is not provided with a reference path andinstead synthesizes a safe controller using only the model dynamics and (non-convex) safety constraints, which is a more challenging problem than the tracking problem as in Section 6.1.
The three navigation problems we consider are: (a) controlling a Segway to duck under an obstacle to reach agoal [36], (b) navigating a 2D quadrotor model around obstacles [9], and (c) completing a satelliterendezvous that requires approaching the target satellite from a speci ﬁ c direction [37].
For (a) and(c), we conducted additional comparisons with a Hamilton-Jacobi-based controller (HJ, [38]) andpolicy trained via constrained policy optimization reinforcement learning (CPO, [39]).
Simulatedtrajectories are shown in Fig.
Note that in the Segway and satellite examples, robust MPC failsto track the reference path, while the rCLBF controller successfully navigates the environment.
HJpreserves safety in the satellite example but fails to reach the goal (which is positioned near the border of the unsafe region), while HJ controller synthesis failed in the Segway example (the backwardsreachable set did not reach the start location with a 5 s horizon).
Note that the HJ satellite controllerrequires different initial conditions, since it will fail if started outside of the safe region.
The policytrained using CPO navigates to the goal in the satellite example, but it is not safe.
In the Segwayexample, CPO does not learn a stable controller (details are given in the appendix).
[Figure]
Discussion & Conclusion
These results demonstrate two clear trends.
First, the performance of our controller (in terms ofboth safety rate and error relative to the goal) is comparable to that of MPC when the MPC controller is stable.
In some cases, our method achieves lower steady-state error due to its ability toconsider highly nonlinear dynamics, as in the neural lander example.
In other cases, the dynamicsare well-approximated by the linearization and robust MPC achieves better steady-state error, butour approach still achieves a comparable safety rate.
Second, we observe that the performance ofthe robust MPC algorithm is highly sensitive to the control frequency, and these controllers are onlystable at control frequencies that cannot run in real-time on a laptop computer.
This highlights onebene ﬁ t of our method over traditional MPC, which trades increased of ﬂ ine computation for an orderof magnitude reduction in evaluation time.
In all cases, we ﬁ nd that our proposed algorithm ﬁ nds acontroller that satis ﬁ es the safety constraints despite variation in model parameters, validating ourclaim of presenting a framework for robust safe controller synthesis.
In summary, we present a novel, learning-based approach to synthesizing robust nonlinear feedbackcontrollers.
Our approach is guided by a robust extension to the theory of control Lyapunov barrier functions that explicitly accounts for uncertainty in model parameters.
Through experiments insimulation, we successfully demonstrate the performance of our approach on a range of challengingsafe control problems.
A number of interesting open questions remain, including scalable veri ﬁ cation strategies for V , the sample complexity of this learning method, and the relative convergencerates of V , π NN , and the QP controller derived from V , which we hope to revisit in future work.
Wealso plan on exploring application to hardware systems, including considerations of delay and stateestimation uncertainty.