Neural Lyapunov Control of Unknown Nonlinear Systems with Stability Guarantees
Abstract
Learning for control of dynamical systems with formal guarantees remains a challenging task.
This paper proposes a learning framework to simultaneously stabilizean unknown nonlinear system with a neural controller and learn a neural Lyapunovfunction to certify a region of attraction (ROA) for the closed-loop system.
Thealgorithmic structure consists of two neural networks and a satisfiability modulotheories (SMT) solver.
The first neural network is responsible for learning theunknown dynamics.
The second neural network aims to identify a valid Lyapunovfunction and a provably stabilizing nonlinear controller.
The SMT solver then verifies that the candidate Lyapunov function indeed satisfies the Lyapunov conditions.
We provide theoretical guarantees of the proposed learning framework in termsof the closed-loop stability for the unknown nonlinear system.
We illustrate theeffectiveness of the approach with a set of numerical experiments.
Introduction
Finding the region of attraction (ROA) of an asymptotically stable equilibrium point for a nonlineardynamical system remains one of the most challenging tasks, especially for systems with uncertainty[25].
Even though the well-known Lyapunov theorems which use the level sets of a Lyapunovfunction to estimate the ROA were proposed more than a century ago, there is no general approachto finding a valid Lyapunov function for general nonlinear systems with provable guarantees [21].
Thanks to the development of learning-based methods, several data-driven approaches have recentlyshown their effectiveness in identifying unknown (or partly known) systems [8, 29].
However, howto simultaneously find Lyapunov functions and nonlinear controllers for systems with unknowndynamics is still an open and active problem in control and robotics applications [19].
By exploiting the expressiveness of neural networks to tackle the complex nonlinearities of dynamicalsystems, we propose a learning framework for simultaneously learning control functions and neural
Lyapunov functions.
We build upon the framework in [5] but address the more challenging problemof stabilizing a nonlinear system with unknown dynamics with formal stability guarantees.
In addition,we complement the computational framework in [5], by proving the existence of a neural networksatisfying the Lyapunov conditions, except on an arbitrarily small neighborhood of the origin, whichoffers a theoretical basis for the learning framework here and in [5].
Our proposed learning framework consists of two neural networks.
The first neural network learns theunknown dynamics, while the second one aims to identify a valid Lyapunov function and a stabilizingnonlinear controller.
After training the neural networks, we derive error bounds that can be verifieddirectly by the satisfiability modulo theories (SMT) solver.
We then use Lipschitz continuity ofthe nonlinear dynamics and its neural approximation to prove rigorous theoretical guarantees onthe closed-loop stability of the unknown system under the learned nonlinear controller.
To the best knowledge of the authors, no prior work has provided closed-loop stability analysis in such a generalsetting of simultaneously learning the dynamics, a nonlinear controller, and a Lyapunov function.
We experimented on three typical nonlinear system examples: the Van der Pol oscillator, the invertedpendulum, and the unicycle path following.
To align with the theoretical results, we carefullyestablish bounds for the Lipschitz constants, keep track of the training losses, and adjust the Lyapunovconditions to be verified by the SMT solver accordingly so that the verified ROA is valid for theunknown system.
The numerical experiments show that the ROA learned this way are comparablewith the ones computed with actual dynamics, and improve that obtained from the LQR controller.
Related work.
Artificial neural networks are capable of estimating any nonlinear function witharbitrarily desired accuracy [17].
As both the right-hand side of a differential equation and the
Lyapunov function can be regarded as nonlinear (or linear) functions, mapping from the state spacesto some vector spaces, a natural question is: can we use neural networks to approximate the dynamicsand find a Lyapunov function to attain a larger estimate of the ROA, compared with widely usedlinear–quadratic regulator (LQR) and sum of squares (SOS) methods [28, 22]?
Various applications of neural networks for system identification and Lyapunov stability have beenproposed recently [19, 14].
For instance, Wang et al.
[18] use a recurrent neural network to generatethe state-space representation of an unknown dynamical system.
A Lyapunov neural network isderived in [31] to learn a fixed quadratic format Lyapunov function for finding the largest estimated
ROA.
However, when it comes to finding a Lyapunov function with a typical feedforward neuralnetwork to certify the stability for the nonlinear systems, performing this kind of regression task isdifficult for shallow neural networks, where only equality and/or inequality Lyapunov conditions,instead of explicit expression or values of the function, are given.
Consequently, it is crucial todevelop methods to test or verify the outputs of a neural network.
There are two main approaches totackle this issue.
One is converting the neural networks into a mixed-integer programming (MIP)format which can be solved by MIP solvers.
The other is to encode the verification problem into an
SMT problem.
In the first category, [7, 6] find a robust Lyapunov function for an uncertain nonlinearsystem or a hybrid system which can be partly approximated by piecewise linear neural networks withthe help of a counterexample-guided method using mixed integer quadratic programming (MIQP),and the corresponding robust ROA is estimated.
With a similar technique, [10] synthesizes a neuralnetwork controller and Lyapunov function simultaneously to certify the dynamical system’s stabilityby converting the dynamics, the Lyapunov function, and the controller into piecewise linear functionsand solving it with mixed-integer linear programming (MILP).
On the other hand, several SMTsolvers have been developed by various scholars [27, 20], for instance, dReal [16], with which taskswith various nonlinear real functions can be elegantly handled.
This can be employed in most casesfor verifying existing neural network structures, e.g., with tanh and sigmoid activation functions.
With the help of these SMT solvers, Chang et al. propose an algorithm to find and validate a neural
Lyapunov function satisfying the Lyapunov conditions with a one-hidden-layer neural network [5].
More recent work in [11] uses the same methodology to approximate control Lyapunov barrierfunctions for nonlinear systems to guarantee both stability and safety for reach-avoid robot tasks,while [35] uses a similar framework to design stabilizing controllers for unknown nonlinear systemswith an implementation of Koopman operator theory.
Gaussian process (GP) regression is usedin [1, 2] to learn the unknown systems while providing safety and stability guarantees for the statesin the ROA with high probability, at the same time how to enlarge the estimated ROA is also wellstudied.
On top of that, a method to get a larger ROA by iteratively re-design a control Lyapunovfunction is presented in [26].
However, we have to contend with the safety-critical nature of certaincontrol and robotics applications.
As a result, worse-case bounds and performance guarantees aremore desirable.
In a similar vein, an upper bound on the infinity-norm bounds were established in thecontext of training deep residual networks [24].
We remark that none of these works analyzed theclosed-loop stability in the general setting as ours.
Contributions.
We summarize the main contributions of the paper as follows
• We propose a framework for simultaneously learning a nonlinear system, a stabilizing nonlinearcontroller, and a Lyapunov function for verifying the closed-loop stability of the unknown system.•
We provide theoretical analysis on the existence of a neural Lyapunov function that verifies theLyapunov conditions, except on an arbitrarily small neighborhood of the origin, provided that theorigin is asymptotically stable.
This provides theoretical support the proposed as well as otherneural Lyapunov framework in the literature.
• We establish closed-loop loop stability for the unknown system under the learned controller,combining classical Lyapunov analysis with rigorous error estimates for neural networks, with thehelp of SMT solvers and recent tools for estimating Lipschitz constants for neural networks.
Preliminaries
Throughout this work, we consider a nonlinear control system of the form
[Equation]
where x ∈ D is the state of the system, and D ⊆ R n is an open set containing the origin;
u ∈ U ⊆
R mis the feedback control input given by u = κ ( x ) , where κ ( x ) is a continuous function of x .
Withoutloss of generality, we assume the origin is an equilibrium point of the closed-loop system
[Equation]
When there is no ambiguity, we also refer to the right-hand side of (2) by f .
We assume that we do not have explicit knowledge of the right-hand side of the system (1).
The mainobjective is to stabilize the unknown dynamical system by designing a feedback control function κ .
Stability guarantees are established using Lyapunov functions.
We next present some preliminarieson model assumptions and Lyapunov stability analysis.
Model assumptions
Assumption 1 (Lipschitz Continuity) .
The right-hand side of the nonlinear system (1) is assumed tobe Lipschitz continuous, i.e.,
[Equation]
where L is called the Lipschitz constant; ( x, u 1 ) and ( y, u 2 ) denote the concatenation of the corresponding two vectors. We assume the Lipschitz constant L is known.
Assumption 2 (Partly Known Dynamics) .
The linearized model about the origin in the form
of˙ x = Ax + Bu , where A and B are constant matrices, is known for the nonlinear system (1).
To analyze the stability properties of the closed-loop system for (1) in the presence of uncertainty(due to the need to approximate the unknown dynamics), we introduce a more general notion ofstability about a closed set
A (see, e.g., [23], and the supplementary material for a definition).
When
A = { 0 } , this coincides with the standard notion of stability about an equilibrium point.
Intuitively,set stability w.r.t.
to A is measured by closeness and convergence of solutions to the set A .
To thisend, define the distance from x to A by ∥ x
∥ A = inf y ∈
A ∥ x − y ∥ .
Set invariance [3] plays an important role in our analysis.
Definition 1 (Forward Invariance) .
A set Ω ⊂ R n is said to be forward invariant for (2) if for x 0 ∈ Ωimplies that x ( t ) ∈ Ω for all t ≥ 0 .
Definition 2 (Region of Attraction) .
For a closed forward invariant set A that is UAS, the region ofattraction is the set of initial conditions in D such that the solution for the closed-loop system (2) isdefined for all t ≥ 0 and ∥ x ( t ) ∥
A → 0 as t → ∞ .
Remark 1.
Any set satisfying Definition 2 is called a region of attraction.
The ROA is the largest setcontained in D satisfying Definition 2.
Definition 3 (Lie Derivatives) .
The Lie derivative of a continuously differentiable scalar function
V : D → R over a vector field f and a nonlinear controller u is defined as
[Equation]
The lie derivative measures the rate of change along the system dynamics.
The following is a standard Lyapunov theorem for the UAS property of a compact invariant set.
Theorem 1 (Sufficient Condition for UAS property) .
Consider the closed-loop nonlinear system (2) .
Let A ⊂ D be a compact invariant set of this system.
Suppose there exists a continuously differentiablefunction V : D
→ R that is positive definite with respect to A , i.e.,
[Equation]
[-x ) = 
 ∀ x ∈ A and V ( x ) > 
 ∀ x ∈ D \ A, (4)-] and the lie derivative is negative definite with respect to A , i.e. [Equation]
[Equation]
Then, A is UAS for the system.
See [23, 34] for sufficiency and necessity of Lyapunov conditions for set stability under more generalsettings.
The function V satisfying (4) and (5) is called a Lyapunov function with respect to A .
From the forward invariance of level sets of a Lyapunov function it follows that these sets provide anestimate of the ROA (see [21]).
Lemma 1 (Region of Attraction with Lyapunov Functions) .
Suppose that V satisfies the conditions in Theorem 1.
Denote
[Equation]
For every c > 0 , V c is a region of attraction for the closed-loop system (2).
Learn and Stabilize Dynamics with Neural Lyapunov Functions
In this section, we show how to use two shallow neural networks to learn the dynamics and find a validneural Lyapunov function.
The first network is designed to learn the dynamics as shown in (1), andthe second network aims to identify a valid neural Lyapunov function and its corresponding nonlinearcontroller.
The algorithmic structure can be found in Fig.
1 and the pseudocode in Algorithm 1.
3.1 Learn the Unknown Dynamics
[Figure] According to the well-known universal approximation theorem [9], shallow neural networks are ableto approximate the right-hand side f ( x, u ) of the nonlinear system (1) arbitrarily well.
Here we usea one-hidden-layer feedforward neural network, denoted as ϕ ( x, u ) , to conduct this regression taskwith a mean square loss.
For notational convenience, we slightly overuse the notation of ϕ to denoteboth the the learned dynamics and the parameters that define the function.
Note that if we use thetanh or sigmoid function as activation functions, the output layer should not have such activationfunctions, as f is not bounded in most cases.
When learning the dynamics, f is regarded as a function of two variables, x and u .
The inputdimension of the neuron network is ( n + m ) , and output dimension is n .
The training data of x andu are sampled uniformly and independently over their respective spaces.
In most robotic systemsthe input is provided by motors, which is typically saturated in practice.
To reflect this, we use theactivation function tanh to simulate this property.
The format of the controller is
[Equation]
6) where C is a constant matrix, determined by the saturation property of the controller in real systems,which defines the bounds of U , K and b are the weight matrix and bias vector, respectively, whichare initialized with an LQR controller based on the linearized system ˙ x
= Ax + Bu .
The bias b ischosen such that f (0 , Cσ ( b ))
=
0 , i.e., the origin is an equilibrium point for the closed-loop system(2).
Note that the bias vector b is not updated in the learning process.
3.2 Neural Lyapunov Function and Nonlinear Controller
Following learning the dynamics using ϕ , the other neural network is designed to learn a nonlinearcontroller and a neural Lyapunov function simultaneously during the same training process.
Thedetailed structure can be found in the supplementary material.
It is worth mentioning that since wetrain the Lyapunov function with data generated by the learned dynamics, which is an approximationof the actual dynamics, we rewrite (4) and (5) as follows to accommodate for approximation errors:
V (0) = 0 , and , ∀ x ∈ D\{ 0 } , V ( x ) > 0 and ∇ ϕ V ( x ) < − β.
Here β is a positive real number, which can be determined as follows.
Assume that ( x, u ) is a pair of unsampled state and inputs, and ( y, v ) is its nearest sample used intraining the neural network for learning the dynamics.
Let δ > 0 be such that ∥ ( x, u ) − ( y, v ) ∥
≤ δholds for all such ( x, u ) and ( y, v ) and let M > 0 be such that ∥ ∂V∂x
∥ < M for all x ∈ D .
Denote αas the maximum of the 2-norm loss among all training data samples.
Then the bound on the testingerror in terms of the training error can be calculated as
[Equation]
(8) where f and ϕ are Lipschitz continuous with Lipschitz constants K f , K ϕ , respectively and β > 0 issome sufficiently large constant.
Then, choosing β to satisfy (8) implies that
[Equation]
In view of (7) and (9), we have
[Equation]
which implies that the actual dynamics is also stable with the obtained neural Lyapunov function.
Here, we calculate K ϕ by using LipSDP-network developed in [12], and M can be determined bychecking the inequality with an SMT solver.
An initial guess of β is needed according to some priorknowledge, and it will be re-computed in each epoch with (8).
A valid neural Lyapunov function can be guaranteed by the SMT solver with all the Lyapunovconditions written as falsification constraints in the form of first-order logic formula over reals [5]:
[Equation]
where ε is a numerical error parameter, which is explicitly introduced for controlling numericalsensitivity around the origin.
If the SMT solver returns either UNSAT this means that the falsificationconstraint is guaranteed not to have any solutions and confirms all the Lyapunov conditions aremet.
If the SMT solver returns δ -SAT , this means there exists at least one counterexample under theδ -weakening condition [15] that satisfies the falsification conditions.
We use θ to denote the parameter vectors for a neural Lyapunov function candidate V θ .
The parametersθ and K are found by minimizing the following cost function, which is a modification of the so-calledempirical Lyapunov risk in [5] by adding one more term ∥ ∂V∂x
∥ , as we need β to be small as well:
[Equation]
where C 1 , C 2 , C 3 and C 4 are tunable constants.
The cost function can be regarded as the positivepenalty of any violation of the Lyapunov conditions in (4) and (5).
Note that the ROA can also bemaximized by adding a L 2 -like cost term to the Lyapunov risk with L ( θ, K )
+ 1 N (cid:80)
Ni =1 ∥ x i ∥ 2 −αV θ ( x i ) , where α is a tunable parameter, as shown in the original paper [5].
Theoretical Guarantees
In this section, we analyze the theoretical guarantees of using neural networks to learn to control anunknown nonlinear system and a Lyapunov function for certifying the closed-loop stability.
1: function L EARNING D YNAMICS ( X dyn , U ( bdd ) ) 2:
Set learning rate ( γ ), input dimension ( n + m ), output dimension ( n ) 3: repeat 4: f ← NN ϕ ( x ) ▷ Output of forward pass of neural network 5:
Compute MSE L ( f, ϕ ) 6: ϕ ← ϕ + γ ∇ ϕ L ( f, ϕ )
▷ Updates Weights using SGD 7: until convergence 8: return ϕ 9: end function 10: function L EARNING L YAPUNOV ( X Lyp , f ϕ , q lqr ) 11:
Set learning rate ( α ), input dimension ( n ), output dimension (1) 12:
Initialize feedback controller u to LQR solution K lqr 13: repeat 14:
V θ ( x ) , u ( x ) ← NN θ,u ( x ) ▷ Output of forward pass of neural network 15: ∇ ϕ V ( x ) ← (cid:80) D in i =1 ∂V∂x
i [ ϕ ] i ( x ) 16:
Compute Lyapunov risk L ( θ, K ) 17: ϕ ← θ + α ∇ θ L ( θ, K ) 18:
K ← K + α ∇ K L ( θ, K )
▷ Updates Weights using SGD 19: until convergence 20: return V ϕ , u 21: end function 22: function F ALSIFICATION ( f ϕ , u, V θ , ε, δ, β ) 23:
Encode conditions from (11) 24:
Use SMT solver with δ to verify the conditions 25: return satisfiability 26: end function 27: function M AIN ( ) 28: input initial guess of bound ( β ), parameters of LQR ( q lqr ), radius ( ε ), precision ( δ ), sampled states X , sampled inputs U 29: ϕ ← L EARNING D YNAMICS ( X dyn , U ( bdd ) ) 30: while Satisfiable do 31:
Add counterexamples to X 32:
V ϕ , u ← L EARNING -L
YAPUNOV
( X Lyp , ϕ, q lqr ) 33: update β according to (8) 34:
CE ← F ALSIFICATION ( ϕ, u, V θ , ε, δ, β ) 35: end while 36: end function
4.1 Approximation guarantee of unknown dynamics
Our analysis provides stability guarantees for the unknown system by rigorously quantifying theerrors using Lipschitz constants of the unknown dynamics and its neural approximation.
To this end,we need a theoretical guarantee that extends the universal approximation theorem, stating that we canapproximate the Lipschitz constants and function values by a neural network to an arbitrary precision.
Theorem 2.
(Approximation of Lipschitz constants).
Suppose that K ⊂ R n is a compact set.
(a) If f : K → R m is L -Lipschitz in the uniform norm, i.e.
[Equation]
then for every ϵ > 0 there exists a neural network of the form ϕ ( x ) =
C ( σ ◦ ( Ax + b )) forσ ∈ C 1 ( R ) and not a polynomial, A ∈ R k × m , b ∈ R k and C ∈ R k × n for some k ∈ N such thatsup x ∈ K | f ( x ) − ϕ
x ) | <
ϵ and ϕ has a Lipschitz constant of L + ϵ in the same norms as 13).
(b) If f : K → R m is L -Lipschitz in the two norm, i.e.
[Equation]
then for every ϵ > 0 there exists a neural network ϕ of the same form such that sup x ∈ K | f
( x ) −ϕ ( x ) | <
ϵ and ϕ has a Lipschitz constant of L + ϵ (cid:18) √ n + n/ϵ +
[Equation]
(cid:19) in the same norms as (14).
The idea of the proof is to first approximate f by a smooth function F and then approximate F by aneural network ϕ
and the details can be found in the supplementary material.
Remark 2.
The equivalence of norms gives an upper bound on the Lipschitz constant for all norms.
4.2 Existence of neural Lyapunov functions
As a theoretical guarantee we show that it is possible to train a neural network as a Lyapunov function,provided that a Lyapunov function exists.
According to converse Lyapunov theorems [23, 34],
Lyapunov functions do exist when the origin is UAS.
More specifically, we show that the learnedneural network satisfies the Lyapunov conditions outside some neighborhood of the origin that canbe chosen to be arbitrarily small in measure.
The idea will be to perform an under-approximationof the domain D in a controlled way.
Let ( R n , B ( R n ) , µ ) denote the standard measure space where
B ( R n ) is the Borel σ -algebra and µ is the Lebesgue measure.
The following lemma from [32] statesthat it is possible to arbitrarily under-approximate open sets by compact sets in measure.
Lemma 2.
For every open set O ∈ B ( R n ) such that µ ( O ) < ∞ and every ϵ > 0 , there exists acompact set K such that µ ( O \ K ) <
ϵ .
By the pointwise approximation of the universal approximation theorem, it is not possible to satisfythe Lyapunov conditions on D as this set contains the origin.
However, if there is a Lyapunov functionfor (2) that a neural network could learn and if practical stability (i.e., set stability w.r.t.
a sufficientlysmall neighborhood of the origin) is sufficient, Theorem 3 states that there exists a neural networksatisfying the Lyapunov conditions on a compact set K except on a closed neighborhood B of theorigin that is UAS.
Moreover, this approximation can be controlled in measure.
The details of theproof can be found in the supplementary material.
Theorem 3.
Suppose that the origin is UAS for system (2) and I is a forward invariant set containedin the ROA of the origin.
Fix any γ 1 , γ 2 > 0 .
There exists a forward invariant and compact set
K ⊂ I satisfying the under approximation µ ( I \ K ) < γ 1 .
On K there exists a neural network V ϕthat satisfies the Lyapunov conditions on K \ A , where A is a closed neighborhood of the origin.
Theneural Lyapunov function V ϕ can certify that a closed invariant set B containing A and satisfyingµ ( B \ A ) <
γ 2 is UAS.
Furthermore, the set K is contained in the ROA of B .
An important feature of Lyapunov functions is that the level sets approach the ROA.
We can showthat the neural
Lyapunov function V ϕ has such a property.
Details can be found in the supplementarymaterial.
4.3 Asymptotic Stability Guarantees of Unknown Nonlinear Systems
With the theoretical guarantee of learning a neural Lyapunov function established above, we showthat the neural network trained with the learned dynamics is robust, that is this neural network alsosatisfies the Lyapunov conditions with respect to the actual dynamics f .
Since the SMT solververifies the Lyapunov conditions outside of some ϵ -ball which is not necessarily forward invariant,the following technical assumption helps bridge this gap.
The assumption is mild, because for thenonlinear system to be stabilizable, it is reasonable to assume that it has a stabilizable linearization.
A linear system ˙ x
=
Ax + Bu is said to be stabilizable if there exists a matrix K such that A + BKis Hurwitz, i.e., all eigenvalues of A + BK have negative real part.
If ( A, B ) is stabilizable, then thegain matrix K can be obtained by an LQR controller.
Assumption 3 (ROA of LQR Controller) .
Suppose that the linearized model ˙ x
= Ax + Bu from
Assumption 2 is stabilizable.
Consequently, an LQR controller and a quadratic Lyapunov functioncan be found such that the origin is UAS for the closed-loop system (2) .
Furthermore, we assumethat the set B ε which is not verified by a SMT solver lies in the interior of a ROA of the closed-loopsystem, provided by the quadratic Lyapunov function.
Since this ε -ball is small, we further assume that the level sets of the Lyapunov function are containedin the ROA of the LQR controller.
Assumption 4 (Controlled Level Sets) .
Denote B ε := { x : ∥ x ∥ ≤ ε } .
Let V be a continuouslydifferentiable function satisfying the Lyapunov conditions on D \ B ε .
Suppose that there existsconstants 0 < c 1 < c 2 such that the following chain of inclusions holds
[Equation]
and { x ∈ D : V ( x ) ≤ c 2 } lies in the interior of the ROA of the closed loop system provided by thequadratic Lyapunov function.
Theorem 4.
(Stability Guarantees for the Unknown System) Let ϕ be the approximated dynamicsof right-hand side of the closed-loop system (2) trained by the first neural network.
There exists aneural Lyapunov function V which is learned using ϕ and verified by an SMT solver that satisfiesthe Lyapunov conditions with respect to the actual dynamics f .
Furthermore, if the system satisfies
Assumption 3 and V satisfies Assumption 4, then the origin is UAS for the closed-loop system (2) .
The details of this proof can be found in the supplementary material.
This theorem proves that if thedynamics are approximated to sufficient precision then the neural Lyapunov function satisfies the
Lyapunov conditions on D \ B ε for the actual dynamics.
Furthermore, if the level sets of the neural
Lyapunov function are sufficiently well behaved and the set B ε excluded from SMT verification issmall then this learning framework certifies that the origin is UAS for the actual system (2).
Experiments
In this section, we demonstrate the effectiveness of the proposed algorithm on learning the unknowndynamics and finding provably stable controllers as well as neural Lyapunov functions for variousclassic nonlinear systems.
In all the following problems, we use a two-layer neural network with onehidden layer for both learning the dynamics and learning the neural Lyapunov function.
For learningthe dynamics, the number of neurons in the hidden layer varies from 100 to 200 without an outputlayer activation function as stated before, and we call this neural network FNN for convenience.
However, for learning the neural Lyapunov function, there are six neurons in the hidden layer for allthe experiments, and we name this neural network VNN in short.
Regarding other parameters, weuse the Adam optimizer for both FNN and VNN, and we use dReal as the SMT solver, setting δ as0.01 for all experiments.
All the training of FNN is performed on Google Colab with a 16GB GPU,and VNN training is done on a 3 GHz 6-Core Intel Core i5.
5.1 Van der Pol Oscillator
As a starting point, we test the proposed algorithm on the nonlinear system without input u first toshow its effectiveness in learning unknown dynamical systems and finding the valid neural Lyapunovfunction.
Van der Pol oscillator is a well-known nonlinear system with stable oscillations as a limitcycle.
The area within the limit cycle is the non-convex ROA, as shown in the supplementary material.
According to the algorithm described in Section 3, we learn the two nonlinear dynamical equationswith 100 hidden neurons.
To ensure an accurate model, we use 9 million data points sampled on( − 1 . 5 , 1 . 5) × ( − 1 .
5 , 1 . 5) , and the learning rate of the training process varies from 0.1 to 1e-5 toacquire a small enough training error
α .
With the learned dynamics, we aim to find a valid Lyapunovfunction over the domain ∥ x ∥ 2 < 1 . 2 , and the obtained neural Lyapunov function is shown in Fig.
2a.
The corresponding ROA estimate can be found in Fig. 2b.
In comparison, the ROAs found by theneural Lyapunov function and the classic LQR techniques in [21] using actual dynamics are alsoplotted, as the blue and magenta ellipses respectively.
The phase portrait of the system is given as thegrey curves with small arrows.
It is obvious that the neural Lyapunov function after tuning obtainsa larger ROA.
We also have a comparable verified ROA for the system with the one obtained withactual dynamics based on the same neural Lyapunov approach, both larger than the LQR case.
Thevalues of parameters can be found in Table 1.
The dynamics, the neural Lyapunov function, and thenonlinear controller for all experiments are described in detail in the supplementary material.
[Table]
5.2 Unicycle path following
The path following task is a typical stabilization problem for a nonlinear system.
Here, we considerpath tracking control using a kinematic unicycle [4] with error dynamics.
With the learned dynamics
[Figure]
ϕ , a neural Lyapunov function can be learned on the valid region ∥ x
∥ 2 ≤ 0 . 8 , and the neural controlleris set as u = 5 tanh
( Kx + b ) .
The ROA comparison with the LQR method can be found in Fig. 3a.
Apparently, the neural network method yields a larger estimated ROA, compared to the classic
LQR approach in which the level set is determined by considering some relaxation of the largestreasonable range of linearizatin for practical systems under a small angles assumption (< π 9 ), giventhe fact that the actual dynamics is unknown.
The details regarding the bounds and β are given in thesupplementary material, and the same for the next experiment.
[Figure]
5.3 Inverted pendulum.
The inverted pendulum is another well-known nonlinear control problem.
This system has two statevariables θ, ˙ θ and one control input u .
Here, θ and ˙ θ represent the angular position from the invertedposition and angular velocity.
The valid domain is ∥ x ∥ 2 ≤ 4 .
The similar ROA comparison is shownin Fig.
3b, where the LQR approach uses the same function as given in [5].
Conclusion
This paper explores the ability of neural networks to approximate an unknown nonlinear systemwith a sufficient precision and find a neural Lyapunov function as well as a feedback controller tostabilize the unknown dynamics.
Provable guarantees are also provided with the help of bounds onthe approximation error and partial derivatives of the Lyapunov function.
Moreover, experimentalresults show that the proposed approach outperforms the existing classic approach given part of thedynamics is known.
The algorithm is only tested on low-dimensional systems.
How to address thisissue for high-dimensional systems is an interesting future research direction.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] See Section
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]2.
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Section 2and Section 4.
(b) Did you include complete proofs of all theoretical results?
[Yes] See section 73.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] See Section 5and Section 7 where the link of the code is provided.(b)
Did you specify all the training details (e
g., data splits, hyperparameters, how theywere chosen)?
[Yes] See Section 5 and Section
(c) Did you report error bars (e
g., with respect to the random seed after running experi- ments multiple times)?
[N/A](d)
Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See Section
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] This paper follows up
on a recent work of [5]
and the code is developed based on this paper as well.
Citation
of the results from this paper
can be found in Section 3 and Section
(b) Did you mention the license of the assets?
[N
/A](c)
Did you include any new assets either in the supplemental material or as a URL?
[Yes]
The link of the code can be found in Section 7(d)
Did you discuss whether and how consent was obtained from people whose data you’reusing/curating?
[N/A]
The dataset we use is generated by ourselves(e) Did you discuss whether the data you are using/curating contains personally identifiableinformation or offensive content?
[N/A]5.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
[N/A](b)
Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A](c)
Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?
[N/A]