Veriﬁed Path Following Using Neural Control Lyapunov Functions
Abstract: We present a framework that uses control Lyapunov functions (CLFs) to implement provably stable path-following controllers for autonomous mobile platforms.
Our approach is based on learning a guaranteed CLF for path following by using recent approaches — combining machine learning with automated theorem proving — to train a neural network feedback law along with a CLF that guarantees stabilization for driving along low-curvature reference paths.
We discuss how key properties of the CLF can be exploited to extend the range of the curvatures for which the stability guarantees remain valid.
We then demonstrate that our approach yields a controller that obeys theoretical guarantees in simulation, but also performs well in practice.
We show our method is both a veriﬁed method of control and better than a common MPC implementation in computation time.
Additionally, we implement the controller on-board on a 18 -scale autonomous vehicle testing platform and present results for various robust path following scenarios.
Introduction
[Figure]
Path following is a fundamental primitive for autonomous robotic platforms, wherein the goal is to steer the platform along a desired path in the workspace.
This is important for mobile platforms since many approaches to platform design rely on the separation between highlevel planners that search over paths without considering platform dynamics and lower-level plan execution which implements path following [1, 2].
However, path following is a challenging problem since platform models of interest are nonlinear.
Popular approaches such as the “Stanley” controller (originally proposed by Hoffman et al.
for the 2005 DARPA grand challenge [3]) can become unstable depending on the choice of the controller’s parameters and the curvature of the path [4].
Similarly, feedback controllers based on linearization can also be unstable if the deviation is too large [5].
Control Lyapunov functions (CLFs) are a general approach for controller design, typically for steering a control system towards an equilibrium point [6, 7].
A CLF is a positive deﬁnite function such that for any state within a well-deﬁned basin of attraction, there exists a control input that causes its derivative to be negative.
Thus, CLFs naturally deﬁne stabilizing controllers [8, 9].
However, the generation of CLFs has remained a challenging problem for researchers, and various CLF gen- eration methods have been developed.
Neural networks have recently become a popular method for designing CLFs given their ability to converge even on systems with sophisticated dynamics.
The idea is to use machine learning algorithms to infer candidate CLFs by minimizing a special loss function called “Lyapunov risk” in combination with automated theorem-proving algorithms to verify the results of the learning algorithm.
As a result, if successful, the approach yields a CLF as well as a stabilizing controller.
Thus, neural CLF generation is a promising approach for designing controllers for nonlinear systems.
Nevertheless, its use has been limited to stabilization to ﬁxed equilibria, whereas our application seeks to stabilize along a path.
Secondly, empirical validation on an actual robotic test platform has not been publicly performed.
In this paper, we address the path-following problem for a path in the workspace whose curvature κ ( s ) varies with the distance s along the path.
For κ ( s ) ≡ 0 , which corresponds to a straight-line path, the problem is amenable to existing neural CLF synthesis tools, but it becomes progressively harder as the path curvature increases and becomes dependent on s .
To address this, we exploit key properties of the system and the generated CLF to design a formalism wherein the controller derived for a straight-line path can be used for paths with non-zero, varying curvature.
We derive bounds on the maximum path curvature for which the controller is valid and we show that we can construct complex maneuvers using this approach.
Next, we demonstrate our approach on a 1 8 -scale vehicle platform.
Our demonstrations show that (a) the model-based design approach transfers to a real robotic platform even though the model used for learning the CLF is a simpler model; and (b) the resulting approach is favorable compared to other veriﬁed and unveriﬁed path-following methods in both curved and straight path-following scenarios.
1.1 Related Work
Numerous methods have been studied for generating CLFs.
Certain nonlinear control design methods such as backstepping automatically synthesize CLFs but are not widely applicable.
Tan and
Packard studied an approach using bilinear constraints that are hard to solve in practice [10].
Majumdar et al.
propose a similar approach for path following but use a control feedback law derived through linearization as a heuristic initialization to solve bilinear constraints efﬁciently in practice [11].
Data-driven approaches for CLFs were ﬁrst studied by Khansarizadeh et al.
However, a lot of data is potentially needed to generate CLFs and the ﬁnal result needs to be veriﬁed against the system dynamics.
Previous work by Ravanbakhsh et al.
[13] showed an active approach for deriving CLFs for nonlinear systems that was adapted to path following using the idea of control funnel functions [14, 15].
Furthermore, their approach was shown to be effective on a autonomous vehicle testbed in the lab.
However, the techniques described so far focus mostly on deriving polynomial CLFs.
It is well known that CLFs can be non-polynomial even for polynomial dynamical systems.
Recently, neural networks have been used to generate CLFs for systems with complex, non-polynomial dynamics.
Introduced by Richards et al.
[16] and Chang et al.
[17], neural networks have been shown to be an effective method for generating CLFs for a variety of systems.
Majumdar et al.
[18] provide the framework of “funnel libraries” in which a set of funnels F and associated feedback controllers are generated.
This library of funnels can be sequenced together to form complex vehicle maneuvers.
Bouyer et al.
[19] showed that this approach can reduce the reach-avoid problem to that of a timed automata by switching between appropriate controllers based on state measurements.
Funnel libraries are an interesting approach towards a more rigorous plan execution that supports a motion planner which can restrict itself to maneuvers available in the library. However, maintaining such a library can be cumbersome.
In particular, a large number of controllers may be needed to cover the space of possible maneuvers.
The approach presented in this paper can simplify funnel libraries by covering a larger set of trajectories with fewer controllers.
Preliminaries: Problem Deﬁnition and CLF Learning
Consider the problem of stabilizing a dynamical system ˙ x = f ( x, u ) , wherein x ∈ D ⊆ R n and u ∈ U ⊆
R m , to the equilibrium point x = 0 without loss of generality.
This is a typical control problem, e.g., where a mobile platform has command inputs that can lead it to follow a speciﬁc trajectory which may be parametrized by a point in time.
Let V : D
→ R be a control Lyapunov function (CLF) for f on D .
Formally, a CLF is deﬁned in Deﬁnition 1.
Deﬁnition 1.
A CLF is a continuously differentiable function V : D
→ R that is positive deﬁnite, i.e., V (0) =
and V ( x ) > {+
+} for all x (cid:54) = 0 , and satisﬁes the following decrease condition
[Equation]
Rather than requiring that the system reaches an equilibrium, we are often interested in region stability, wherein we deﬁne a neighborhood I of 0 , and require that the Lyapunov condition Eq.
(1) is satisﬁed in the region D \ I , called the Lyapunov (satisﬁability) region .
Using the CLF V , control inputs can be generated for any state x that belongs to the largest level set of V contained completely within D to drive the system to the smallest level set of V that contains I , in ﬁnite time [19].
The synthesis of CLFs is known to be very challenging, and the complexity increases rapidly with that of the system dynamics and the control task.
The core idea of this work is to synthesize CLFs for simple maneuvres for the robot platform (namely, straight-path following) and then extend these to more complex maneuvres.
To synthesize the “simple-task” CLFs, we used the approach discussed in Chang et al.
[17], which is a neural network based learning solution for generating CLFs.
In this framework potential neural CLFs are trained using state-control samples from the deﬁned Lyapunov region D \ I and then veriﬁed by an SMT solver to ensure that the candidate CLF meets the conditions in Deﬁnition 1.
Details of the CLF generation process are provided in appendix A.
Region
of Attraction
While the target of CLF training is to generate a CLF that is valid inside the
Lyapunov region, an important metric is the region of attraction (ROA) [20].
Deﬁnition 2.
Let V be a CLF with domain D for a system f ( x, u ) .
Any level set of V completely contained in D deﬁnes a region of attraction.
That is, for any β > 0 , if { V ( x ) ≤ β } ⊆ D , then { V ( x ) ≤ β } is an ROA for the system.
The maximal region of attraction is the maximum level set of V contained within D .
Note that it is important to consider the largest level set contained in D .
Lyapunov conditions do not directly ensure safety (i.e., inﬁnite-time horizon containment) for any states in the veriﬁed Lyapunov region since large level sets of V may exit D while descending the gradient.
Upon exiting D there is no guarantee that actions will be available to reduce V .
To ensure safety, a system’s initial state must be contained within the maximal region of attraction.
2.0.1 Vehicle Model
To train the system we use a path-following model with a ﬁxed velocity.
The model consists of two states: d e and θ e , where d e is the distance from the line as measured from the rear axle and θ e is the theta offset from nearest point on the target line (see Figure 2).
The dynamics of this system are:
[Equation]
where θ e = θ − θ p and κ ( s ) is the curvature of the reference path at the nearest point.
Note that κ ( s ) is 0 for the straight-line path shown in Figure 2.
To learn a neural CLF using the approach in Appendix A, state-control samples are generated for the learning algorithm. To calculate u at any given sample ( d e , θ e ) , we use an LQR controller.
2.1 Straight Path
to Curved Path Following
Finding CLFs for the vehicle model described in Eq.
(2) for paths of various curvature would (a) be expensive and potentially infeasible (as the neural CLF training algorithm offers no guarantee of ﬁnding a CLF and the problem becomes more difﬁcult as the curvature increases); and (b) lead to only a ﬁnite set of curves that the vehicle can execute.
Instead, our approach consists of ﬁnding a single CLF for the straight-line path, and compute the maximum curvature for which the CLF is valid, in order to extend the CLF to paths with nonzero, possibly varying, curvature.
If the minimal required steering input for the straight-path CLF is smaller than the maximum allowed steering input u max , we can derive a curved-path CLF.
More precisely, given a CLF V for a straightline path, i.e., for κ ( s ) = [-
-]0 in Eq.
(2), with ROA contained in the region d e ∈ [ − d max , d max ] , we deﬁne the minimal required input u req for V as the smallest value of u satisfying that for all state x in the Lyapunov region, there exists a steering input u with | u | ≤ u req that decreases the value of V , i.e., ∇ V ( x ) ·
f ( x, u ) < 0 .
If u req <
u max , then one can exploit the gap tan( u max ) − tan( u req ) to follow a path with curvature κ ( s ) bounded as follows.
Proposition 1.
If | κ ( s ) | [Equation]
, then V is a CLF for Eq.
Proof.
Note that for all d e ∈ [ − d max , d max ] , | κ ( s )cos( θ e ) 1 − d e κ ( s ) | <
[Equation]
| κ ( s ) | .
Thus, if
[Equation]
tan [Equation]
L , one can use the steering input u ( s ) =
arctan(tan( u ) +
L | κ ( s ) |
[Equation]
− d max | κ ( s ) | ) , where u ∈ [ − u req , u req ] satisﬁes ∇ V ( x ) · f ( x, u ) < 0 .
It holds that | u ( s ) | ≤ u max .
Moreover, with this input the second equation of Eq.
(2) becomes ˙ θ = v tan( u ) L .
Since u is a decreasing input for V with κ ( s ) = 0 , it is clear that V is a CLF for curved-path following with input u ( s ) .
Similar to a library of funnels [18], the CLF generated for straight-path following can be used for any path with curvature | κ ( s ) | ≤ tan( u max ) − tan( u req )
L + d max per Proposition 1.
However, while previous works require switching controllers [19, 18], our solution relies on a single CLF for the duration of the maneuver.
2.2 Control Extraction
import CLF, set vehicle model parameters, generate target path, discretize steering action space, set dt (time step), set K v (control cost) while sim == true do for steer in steeringAngles do θ e , d e ← calcError ( updateState ( currentState, steer )) vxRollouts.append ( calcV x ( θ e , d e )) end for u ← steeringAngles [ argmin ( square ( steeringAngles ) ∗ K v + vxRollouts )] end while
The deﬁnition of a CLF (Deﬁnition 1) guarantees the existence, at any state in the Lyapunov domain, of a control input causing the CLF to decrease over inﬁnitesimal time horizon (negative Lie derivative). A simple control strategy is then to simply execute the control that minimizes the Lie derivative at the current state.
One can plot a theoretically executable path to descend the CLF as quickly as possible using this method.
However this approach usually results in saturation of the control surface.
In our system, this holds true as the Lie derivative is a monotonic function of u , so that the control that minimizes the Lie derivative of the CLF is always the maximum or minimum u ∈ U .
Such a bang–bang controller [21] is not desirable in practice because abrupt changes in the input can damage the platform, and requires a high actuation frequency.
A better practical method for control extraction is to simulate some actions and select the action that causes the largest reduction of V ( x ) .
Given Eq.
(1) and provided the time horizon is small enough, there always exists a control action to reduce the CLF if the vehicle is in the Lyapunov region of the CLF.
The addition of a small control cost can be included for smoother operation.
This approach allows for stabilization to the origin, as well as more realistic models to be introduced to compute realistic maneuvers.
A simple, functional implementation of this idea is outlined in Algorithm 1.
This algorithm can be expanded to operate like a traditional MPC [22] by increasing the look-ahead, and minimizing a cost function balancing future V ( x ) value, control cost, and terminal V ( x ) value. [Figure] [Figure] [Figure]
[Figure] .d e θ e (a) −− 1 −− 0 . 0 . x (m)
[Figure]
m
[Figure]
[Figure]
Implementation and Experimental Evaluation 3.1 CLF Generation
Our neural Lyapunov control framework was successful in generating CLFs for various target paths and vehicle parameters.
Figure 4(a) shows a generated CLF for a straight-line path.
The CLF is veriﬁed inside the Lyapunov region deﬁned by an inner and outer radius of 0.2 and 0.8 respectively.
Note that while the CLF is valid in those bounds, the maximal ROA is the only region in which we can guarantee safety, although in practice we observe success, in both converging to the minimum and remaining in the Lyapunov region, outside the ROA.
3.2 Controller Implementation and Evaluation
Platform To test our approach in the real world we use a 18 -scale vehicle platform in a motion capture space.
The wheel base of the vehicle is 34 cm and the velocity for all tests is ﬁxed at 0 .
5 m/s. The vehicle includes an onboard computer for real-time computation and receives pose estimates from the motion capture system through a WiFi connection.
After computation, the control action is transmitted to a control unit which controls the steering servos and motor of the vehicle.
Remarkably, each iteration of Algorithm 1 is executed in less than 150 µ s.
Control Extraction in Real Time
The test objective is to use the generated CLF to stabilize the platform to a straight-line path from x = − 2 .
5 m to x = 1 m at a constant velocity of 0 .
5 m/s. On the real-world platform the Lie derivative minimization control extraction method is unable to stabilize the platform to a straight-line path: since the control surface is always saturated, the platform continuously overshoots the distance to the target path, and therefore is unable to stabilize.
Using Algorithm 1 we simulate potential rollouts for the platform and execute the lowest-cost action.
We discretize the continuous action space [ − π 4 , π 4 ] to 31 equally-spaced steering values.
We use the kinematic bicycle model to project our current state a single time-step into the future, setting the time-step to be equal to our system average update rate dt = 150 µ s. [Figure] [Figure] [Figure]
d e
[Figure] −− 1 −− 0 . 0 . [Figure] Figure 4(a) shows the convergence to the minimum of the CLF using Algorithm 1, starting from various positions in the Lyapunov region.
In practice, while not guaranteed to converge, we see that even positions starting outside the maximal ROA converge to V ( x ) = 0 .
Given any starting point in the maximal ROA we can plot the guaranteed maximum d e and θ e by computing the level-set of V ( x start ) and taking the max θ e and max d e of the computed set.
Figure 5(a) shows a reduced invariant region based on x 0 = (0 , − 0 . 4) .
For this particular starting position we can guarantee the platform will deviate no further than [ − 48 , 44] cm from the target path (a 27% d e reduction when compared to the maximal ROA maximum d e of 0.66).
Maximum Curvature Computation In Subsection 2.1, we showed that a CLF for straight-path following can be extended to curved-path following if the range of input required for straight-path following is smaller than the maximum available steering input u max
= π 4 .
The maximum allowed curvature depends on gap =
tan( u max ) − tan( u req ) , where u req is the smallest range required for straight-path following.
Given the CLF V computed Subsection 3.1, u req can be computed as:
[Equation]
where the maximum is taken for d e , θ e in the Lyapunov region.
Eq.
(3) can be rewritten as a single maximization problem:
[Equation]
Eq.
(4) has only two variables ( d e , θ e ), so that it can be solved easily using for instance grid search, or nonlinear solvers like Ipopt [23].
We compute the value 1 L tan( u req ) = 0 .
With L = 0 .
3 , we get that gap = 0 . 8512 .
By Proposition 1, with d max = 0 .
8 , it follows that V can be extended to curved-path following for any path with curvature κ ( s ) ≤
[Equation]
[Equation] [Equation] with radius of curvature R ≥ 1 .
3 m.
Using the approach discussed in Section 2.1 we can execute a complex object avoidance maneuver. The maneuver is a curved path with varying curvature, bounded by 0 .
4 m − 1 (the minimal radius of curvature is 2 . 5 m); see Figure 6. The ﬁgure shows platform paths of various starting positions and orientations after executing Algorithm 1 on the constructed path. 3.3 Side-by-side comparison with alternative techniques
We compare our control method against two MPC implementations [22] (a non-veriﬁed controller) as well as the “deep learned” controller trained when computing the neural CLF (DLC) (a veriﬁed
−−− 2 − 1 − 0 .0 .
[Figure]
controller as discussed in Chang et al.
One MPC has a low cost of applying controls (MPCL) while the other one has a high cost of applying controls (MPCH). We compare our control method in straight- and curved-path following problems. The straight target path shown in Figure 7(a) is a straight line from 0 to 30 m, with an obstacle in the initial direction of the vehicle, just outside of the vehicle control funnel (as deﬁned by the computed CLF in Figures 4 and 5). The initial position of the vehicle is on the target path, but oriented in yaw by the largest θ e contained in the maximal
ROA in Figure 4(a). This is an adversarial condition for non-veriﬁed controllers as we are unable to calculate the maximum path deviation. Rather, the amount of deviation from the target path will depend on design parameters such as control costs, MPC horizon and tuning constants. The curved target path is a circle with a radius of 4 m. The vehicle begins at (0 , 4) and completes one full lap of the path.
Figure 7(b) shows the vehicle’s deviation from the target path as a time series.
[Figure]
For each controller, we run 50 simulations with Gaussian noise ( µ = 0 , σ = 0 . 1 ) added to the initial conditions θ e and d e .
Figure 7 shows the test setup and simulated runs.
Figure 7(a) shows the target paths with each colored area being the empirical average path (solid line in the middle) plus/minus one standard deviation.
As shown in Figure 7 and Table 1, our method is the only method that is both a veriﬁed method of control (guaranteed to avoid the obstacle outside of the control funnel) and able to stabilize to the target path (small steady state error).
In particular, one will note that the other veriﬁed control method (DLC) oscillates around the target path, resulting in a large mean steady state error.
Additionally our method provides the fastest convergence to the target, with a small maximum deviation, and is more than 2 orders of magnitude faster than the MPC implementation, making it a reasonable control choice even when veriﬁed control is not mandatory.
[Table]
Limitations
The primary limitation of this work is the method to generate veriﬁed CLFs.
Some desirable CLFs do not converge to a solution (such as CLFs with larger Lyapunov regions).
The limiting factor is likely the veriﬁcation phase, speciﬁcally the veriﬁcation of Eq.
Rather than verifying ∃ u ∈ U for which ∇ V ( x ) · f ( x, u ) < 0 , we write u in terms of our LQR controller (a function of the system state) to provide a single u for each state.
Of course this greatly constrains the problem and means that the LQR controller constants must be trained simultaneously to the CLF parameters [17].
In theory SMT solvers should be able to directly check Eq.
(1) as it is a ﬁrst-order logic inequality, however in practice the control spaces are often too large.
In future work one may be able to reduce the space by bounding U to some neighborhood of a sampled control.
Additionally these veriﬁcation techniques are conditioned on how well-represented a system is by the model f ( x, u ) .
Namely, a model which is developed to represent certain behaviors of our system limit the extent of maneuvers that are veriﬁed and may be executed by the platform.
This crucial limitation exists at the interface of model validation , wherein the adequacy of the model is evaluated.
Possible next steps to address this limitation may be to consider online model synthesis and the extension of veriﬁcation to these new models.
Conclusion
In this work we show the ability to generate CLFs using neural networks [17], extract executable controls to stabilize the system, and construct safe trajectories using a single CLF.
Future work will seek improvements to CLF generation and veriﬁcation.
Additionally, CLFs can be used inside a receding horizon formulation in Algorithm 1 to provide potentially smoother control experience and faster stabilization to the target.
Finally the feasibility of non-smooth maneuvering in this framework would provide another layer of robustness when operating in less ideal conditions. Acknowledgments
This work was funded by NSF award #1932189 and the Belgian American Education Foundation (BAEF).
All opinions expressed are those of the authors and not necessarily of the NSF or BAEF.