Control with Patterns: A D-learning Method
Authors: Quan Quan
Kai-Yuan Cai
Chenyu Wang
Venue: CoRL 2024 (2024)
Keywords: Lyapunov Methods
Reinforcement Learning
Control with Patterns
D-learning
Visual Servoing
TL;DR: The proposed control with patterns based on D-learning is a type of Lyapunov function learning methods without requring any knowledge of the system dynamics, which parallels to Q-learning in reinforcement learning.

Abstract: Learning-based control policies are widely used in various tasks in the field of robotics and control. However, formal (Lyapunov) stability guarantees for learning-based controllers with nonlinear dynamical systems are challenging to obtain.
	We propose a novel control approach, namely Control with Patterns (CWP), to address the stability issue over data sets corresponding to nonlinear dynamical systems.
	For data sets of this kind, we introduce a new definition, namely exponential attraction on data sets, to describe nonlinear dynamical systems under consideration. The problem of exponential attraction on data sets is converted to a pattern classification one based on the data sets and parameterized Lyapunov functions. Furthermore, D-learning is proposed as a method for performing CWP without knowledge of the system dynamics. 
	Finally, the effectiveness of CWP based on D-learning is demonstrated through simulations and real flight experiments. In these experiments, the position of the multicopter is stabilized using only real-time images as feedback, which can be considered as an Image-Based Visual Servoing (IBVS) problem.

Control with Patterns: A D-learning Method
Quan Quan, Kai- Yuan Cai, and Chenyu Wang
School of Automation Science and Electrical Engineering, Beihang University (BUAA, Beijing University of Aeronautics and Astronautics), Beijing, China qq-buaa, kycai, wangcy2023 @buaa. edu.
Abstract: Learning-basee control policies are widely used in various tasks in the field of robotics and control. However, formal (Lyapunov) stability guarantees for learning-basn controllers with nonlinear dynamical systems are difficult to ob- tain. We propose novel control approach, namely Control with Patterns (CWP), to address the stability issue over data sets corresponding to nonlinear dynamical systems. For such data sets, we introduce a new definition, namely exponential attraction data sets, to describe the nonlinear dynamical systems under con- sideration. The problem of exponential attraction on data sets transformed into problem of pattern classification one based on the data sets and parameterized Lyapunov functions. Furthermore, D-learming is proposed as method to perform CWP without knowledge of the system dynamics. Finally, the effectiveness CWP based on D-Jearning is demonstrated through simulations and real flight ex- periments, In these experiments, the position of the multicopter is stabilized using real-time images feedback, which can be considered as Image-Based Visual Servoing (IBVS) problem.
Lyapunov Methods, Reinforcement Learning, Control with Patterns, D-learning, Visual Servoing
Keywords:
Introduction
In data-rich age, a system is often in operation when measurements of system inputs and out- puts are accessible for collection through inex- pensive and numerous information-senninng de- vices. Based on the input and output data, direct way is often to model the dynamical sys- tem according to the first principles. Then, ex- isting methods are used to analyze the stability
cf
X f=''(x
Reforemoe
Penture Abstraction
Controller
p
Festuro Abatrattion Menserenee
Figure 1:
Closed-loop system by CWP.
or design controllers for the identified system. However, there exist two difficulties. First, is not easy to get the form of the considered system, so the approximation may not be satisfied. Sec- ond, except for only few experts, the approximated model may still be hard handle with existing model-based methods.
The development of deep learning and Reinforcement Learning(RL). [1], [2] has led to new advances in these difficulties [3], [4], [5]. The advancement deep learning and RL has contributed signifi- cantly the development of neural network controllers for robotic systems [6], [7], [8]. For further discussion of related work, please refer to Appendix A.
Despite the impressive performance of these controllers, many of these studies lack critical stability guarantees that are essential for safety-critical applications. To overcome this lack, Lyapunov sta- bility [20] in control theory provides a well- well-known framework for ensuring the closed-loop stability
This paper revised version of Quan Quan and Kai- Kai-Yuan Cai, Control with Patterns Based D-learning. https://arzis org/ ab2/ 2ttp///22.000000222222222 03809v2.
8th Conference Robot Learning (CoRL. 2024), Munich, Germany.

---

of nonlinear dynamical systems. The core concept of this theory is the Lyapunov function, a scalar function whose value decreases along the closed-loop trajectory of the system. This function repre- sents the process by which the system transitions from the system at any state within the Region of Attraction (ROA) to a stable equilibrium.
Environment
Control with Patterns : D-learning Method
Dataset
Policy Evaluation (xx)
Learn Lyapunov Function
Learn D Function
Learn Lyapunov Derivate ((x.u)
(x,u)-p -P p
Dynamic System
Policy Improvement
(x)
Controller Network
(=(((xa)
Train Controller Function
x=(xm]](x)
u x
Figure 2: Overview of our method. The main content consists of two parts. For the policy evaluation step, the Lyapunov function and the D-function are updated by solving (13). After learning the D- function, train the CWP controller by solving (14). For more details, please see Appendix E.
Previous studies [9], [10] that integrate deep learning and Lyapunov control methods have primarily provided guarantees for state feedback control based on structured information (e.g., the state of a linear time-invariant system). Our work addresses the more challenging but practically relevant problem of feedback control based on unstructured information by identifying and overcoming the limitations of existing approaches synthesize, and certify controllers for real-world applications. In order to demonstrate the effectiveness of our method on real robotic systems, we design model- free flight controller that can (I) stabilize a hovering multicopter with images as feedback, similar to visual servoing controllers; (2) outperform RL; and (3) provide Lyapunov stability guarantees.
Our key contributions are:
We propose an approach, namely Control with Patterns (CWP), to the stability problem of dynam- ical systems, which transforms the controller design problem into pattern classification problem. CWP represents a novel framework related to Lyapunov function learning, that can be used to de- velop model-free controllers for general dynamical systems.
We propose D-learning, which parallels to Q-learning [11] in RL to obtain both Lyapunov function and its derivative (see in Fig.6). Unlike existing Lyapunov function learning methods that rely on controlled models or their approximation with neural networks [12], [13], the system dynamics are encoded in the so-called D-function depending on the actions. This allows CWP to be performed without any knowledge of the system dynamics.
Results from the simulation platform and real flight experiments show that our approach can stabi- lize multicopter with real- cal-time images feedback. Furthermore, the D-learning trained controller shows superior performance to the RI. trained controller.
2 Problem Formulation
Consider the following autonomous system
X r' (x',u)= u) - (x', (x))
(m (x'))
where x' D' C R² is the original state not available for measurement, m (x') M is measurement in the form of unstructured data such as images, s (-) is feature selection function

---

designed to encode the measurement into vector x E D S R", and u = c ) is the control policy. We want to focus on x rather than x', namely, consider the following autonomous system
x Øs dm
dm 0x I' (x',u) fx, u)
where f: DD - R (0) = Xo D. From observing the system (1), we can obtain the data set
Pu =
N}.
(2)
We prepare to solve the exponential attraction (see Definition B.2) problem using the Lyapunov method. The Lyapunov function for the data set (2) is supposed to have the following form
(x) = g (x)"00 0g
33)
where (0nx1) = 0, : D/ + R. and 0g E Sy S R4, The set Sy is used to guarantee that the function (x) is Lyapunov function. The derivative of V (x) yields
(x) = 0g (x) x
T
0,.
(4)
8x
We hope that the derivative satisfies
(x) S W (x)
(5)
where W (x) is also a Lyapunov function, which be further written as
W (x) (x)* On
where On Sp S R Similarly, the set S₁, is used to guarantee that the function W (x) is Lyapunov function as well.
For the data set (2), suppose that we have
T 0, S - h (xx)) T
0g (x) 0x
X
On
(6)
where i 1, .N. Then, in the following, based on (6), we show that the equilibrium state 0,x1 exponentially attractive the data set P in Theorem 2.1. For proof, please Appendix C.2.
Theorem 2.1. Under Assumptions C. for the system (1), if there exist parameter vectors 0g Sy and 0, E Sy. such that (6) holds for the data set P, then the equilibrium state x 0,x1 is exponentially attractive on the data set P.
Consequently, according to Theorem 2.1, the exponential attraction problem is converted to make
the inequality (6) hold. The inequality (6) is rewritten
y.0 2 0,1= N
where
T
h (x,)) ]
(g(x)
*;
8x
x=
fg
S a Sg Sn.
0,
Formally, according to Theorem 2.1, we construct the CWP problem formulation represented as follows
Design u E U and find
make (7) hold on the data set (2)
This problem can be classified pattern classification [14] problem. Here, y; is the compound features which describe the stability pattern for the system (I). Consequently, f f() y can be
regarded as linear discriminant function [14]. So far, turned the stability problem (6) into the pattern classification problem (8).

---

Control with Patterns based on D-learning
After formulating the CWP problem (8), we are going to consider how to construct the model-free controller based on data sets. To this end, we will design a CWP controller based on proposed D-learning method.
3.1 Control with Patterns
In order to solve the CWP problem (8), we solve the following optimization
min
wa
n.0 S3,
T 0, -W (O 2 x; (c)
0g(x)
(9)
s.t.
xx
xx
Sa $
where i = 1, N, (.).) is constraint on fg. W (0n. x) is a Lyapunov function mentioned in (5). the following for simplicity, let W (On, x) - ||x An iterative procedure for solving the inequality (9) may be used, including policy evaluation and policy improvement.
Initialization. Select any admissible (i.e., stabilizing) control Co. k = 0.
Policy Evaluation Step. Under Ck. at state X,, the control is c₁ (xx) U, resulting in
å; (cx) D. Determine the solution 9. mp by
min n, 10,Eaa0000
wa 7
0g(x)
X; (cx)
0g 1| 20 20
(10)
8x
Fg (g)
where: 1, N
Policy Improvement Step. Determine an improved policy using
min 2,7
-1
(11)
[g(x)
x=X, Xi (c)) 0g. 11 1xx
s.t.
8x
where: i 1,... NN.
By fixing x; for every step k, the iteration can be terminated after a sufficient number of steps wan - ?h and -Ts are nearly not changed. This is due the fact that the iterative procedure is in fact used to solve the optimization (9) with the coordinate descent [15].
3.2 Control with Patterns Based on D-learning
Unfortunately, in the Policy Improvement Step (11), one requires knowledge of the system dynamics x; (c). To circumvent the necessity of understanding the system dynamics, similar to Q-learning in the field of RI., we can rewrite V (x) in (4) as
D u) = (gg (x) (xx e (08 Ox
where (1) is utilized. We call the D-function as is the derivative of the Lyapunov function and expected to be decreased. one obtains D (x, u) by learning directly, then the use of the input coupling function is avoided. In the nonlinear case, sufficiently smooth. Referring [35], according the Weierstrass higher-order approximation theorem, there exists dense basis set {sp; u)) such that
assumed that the value of (x, u)
D (x, u) = 20p (x, u) = Σ 0:p (x, u) + Σ
0.sp: (x, u) 2 0. (x, u) + +EL (x,u)

---

where basis vector 0₂ [01 82 ... 82]", Φ (x, u) = lip (x, u) ip2 (x, ... PL (x, u)] and converges uniformly to zero as the number of terms retained I. + oc.
is expected to make D (x, u) V (x) as small as possible. From mathematical standpoint, is imperative minimize the value of b while adhering the following constraint
o (x,, cb (xx)) T Og (x)
x; (ck (x,))
S
8x
where 1,... Nk, 1, ..... .M. On the other hand, (5) is rewritten
0 (x, S -1 |x
Furthermore, the inequality (7) is rewritten as
y 2 0, ,
(12)
where
y/ (x,, u,) (xx)) Oa
E Sn.
On
With the D function, iterative procedures for solving the inequality (9) should be rewritten, including policy evaluation and policy improvement.
Initialization. Select any admissible (i.e., stabilizing) control co. & = 0.
Policy Evaluation Step (Based on D-learning). Under Ck2 state x,, the control is = cy (xz) 44 resulting in &; (cx) E D. Determine the solution gg.k, dd, ak, by 0, me E R by
min 0.ERL 2,8,,,,ee
+ wia + wzb
T (x; cæ (x:)) 17 || 2 0
(13)
T (x,, cj (xx)) - T 0g(x) 8x
X1 (c, (x,))
b
Fg (0g) S a
where Wi, W2 0 weights, (c, (x:)) (xx, c(xx))(x))))))) - 0, hi, N
Policy Improvement Step (Based on D-learning). Determine an improved policy using
min c.TER
7
(14)
0 (xi, cx:)) 17 |x,1|
where i = 1,,.. N
4 Simulations and Experiments
In this section, simulations and experiments demonstrate that the CWP-based controller can stabilize a hovering multicopter with images as feedback, which can be considered as an IBVS [16] problem.
4.1
Simulations Design
4.1.1
Problem Formulation of Visual Servoing
Since the camera is fixed to the body of the multicopter, based on Semi-Autonomous Autopilots (SAAs), the plant [17] can be modeled
f = v
(15)
I = m (r)
5

---

where r, v E I' indicate the position and velocity of the multicopter, and I = (r) represents the mapping from the position r of the multicopter to the image I E I taken by the camera.
For IBVS, visual servoing can be conceptualized as minimization problem between the image features s(r) extracted from the current position and the desired features s* from the desired position r". Then, the controller is decided by
s(1) - s(I))
(16)
(e)
where s(.) E S is designed feature selection function, such as neural networks, to code the mea- surement to vector in a latent space S; c(e) represents the controller based on CWP, and its input e the feature error between the current image E I and the desired image I'.
To extract the features from the current image I, we use deep metric learning methods, suggested by the work [18]. train the feature selection function. More details about feature extraction can be obtained in Appendix D.
4.1.2 D-Jearning Controller Design
We train the D-Learning controller based on the latent space S. The Lyapunov function is designed as quadratic function, as follows:
V(e) = c c (c) 0,
(17)
where e s(I) - s(T), (())))(())))))))))))))))) ee @ el (@ represents Kronecker product), and 0g = vec (P)
.which represents the vectorization the positive definite matrix P.
Using the Lyapunov function (17), the CWP controller u c(e) is given Algorithm 1, in which the D-function ((c, u) and the CWP controller c(c) is both designed 4-layer perceptron, with ReLU activations after each hidden layer.
Algorithm 1: Control with Patterns Based on D-learning with Constraints
Input: The data set generated by any admissible (i.e., stabilizing) control = co(e) (e) 1 Initialization. Select any admissible (i.c., stabilizing) control policy parameters u = co(e), D-function parameters 0, Lyapunov function parameters 9,00. and data set
Output: CWP controller
p= {(e,, u, f.) , .N...
while stopping criterion satisfied 0 do
3
Calculate the parameter 0, of the Lyapunov function by solving optimization problem
min 0,ESp.a50
wa
(g(e To, - 0O, -n (t+41 t,) ||: 0 Fg (0,) Sa a
S.t.
where i = 1,...,,N N 1, and Fy (-) represents the constraints on the variable 0y
Estimate the Lyapunov derivative function (ee,) V(e++1) -(ee)) - f,):
Update D-function by (13), where w1, W2 > 0 are weights, 1, N 1:
Determine an improved policy ca (c) by solving optimization problem (14);
7 end
4.2 Simulations Results
To verify the effectiveness of our control algorithm in the IBVS task, we first perform experimental validation in simulation environment constructed using RffySim'.
We sample the camera position in a dimension of 22 m x 8 m centered on the desired position. Then, the multicopter departs from a set start position and arrives at the desired position by the Position- Based Visual Servoing (SBVS) [16] method based on the position information. By sampling with
'nttss//tttysim.com/

---

D-leamning
DDRG
X(0)
Current
316
Controller
Desired
S180
123
Abstrattin
-38
-23
00
23
Neçai
(
(b)
(c)
Figure 3: Simulation experiments and results. (a) The multicopter control system. Given the desired
image I' and the current image I captured by the camera, the feature error s(I) E S C R is solved by the neural network vb I + R The encoding eπror = s(I) s(I*) is used as an input the controller evaluation of the servo error Ar r r" on the simulation environment compared between the controller trained by D-learning and DDPG. The square shows that the D-learning controller has smaller error than the DDPG controller. (c) Principal Component Analysis (PCA) projection of the I. Lyppuovv function (17) learned for the system (15), overlaid with the trajectories of the system controlled by the D-learning controller and the DDPG controller, which shows that the D-learning controller has better stability guarantees than the DDPG controller.
c(e). The output of the controller is the velocity E R (b) Performance
equal spacing, 301 trajectories are captured. Based on the collected data tuple (r, u, I) E R RR I, we first train the network v I - S to obtain s (I) E R32, Then, we use Algorithm to train the controller based on D-learning. Finally, we replace the PBVS controller with the D-learning controller, the experimental results are shown in F(g.3(a). The CWP controller can stabilize the multicopter using images as feedback.
As a comparison, we also train the RL controller. We consider the collected trajectory data replay buffer, and use the Deep Deterministic Policy Gradient (DDPG) [19] algorithm to train an actor as controller.Th The comparison between the D-learning controller and the DDPG controller shown in Fig.3(b), which the DDPG controller, although can also converge the reference point, the error larger than that of the D-learning controller and the Lyapunov function fails to achieve sustained convergence. This result manifests that the D-learning controller provides more reliable stability guarantees than the RL controller.
Multicopter
Mullicopter
Image
Monocular Camera
Desired Image
(3)
TPV
FPV
(A)
(b)
Figure 4: The real flight experiments and results. (a) On the real flight experiments, DJI Tello EDU, which has a front-facing camera. The desired image I' of IBVS is centered on a (TPV) and the right the Pirst-Person-Vien
drawing. (b) On the left is the Third-Person-Vinn (FPV) of the on- bo-ddard camera. The multicopter positions tend in the order as (1) + (2) + (3), and the image from FPV converges the desired image I*.

---

4.3 Real Flight Experiment
We also deploy our method on a multicopter which features a front-facing camera. We sample the camera position in a dimension of 1.6 m x 0.8 m centered on the desired position. The target image for image servoing is a painting. The details of real flight experiments are shown Fig.4. By sampling with equal spacing, 97 trajectories are captured. Based on the collected data tuple (r, u. I) E R3 R3 I, we first train the network & I + S to obtain (I) 1R32 Then, use Algorithm to train to get the controller based D-learning. Finally, We replace the controller using position as feedback with a controller using only images as feedback. Experimental results are shown in Fig.5(a). The initial displacement Aro is (-0.60 m, - -0.02 -0.29 m). The D-learning controller can stabilize the multicopter using only images as feedback. 3D trajectory pairs based on the PBVS controller using position as feedback and the D-learning controller using only images as feedback are shown in Fig.5(b).
D-losrning using 2D image foodback
erroct))
0.00
PBVS position feedback
br
Ar,
1,30
-0.50
Ar,
1.20
P(M)
Excoding crot
1.15
0.0
1.05
1.00
0,80
-0.9
-088
0.76
-0.6
-0.
0.74
y(m)
X(m) -0.5
Dicraiinss
(a)
(b)
Figure 5: The real flight result for visual servoing. (a) The position error Ar - r", the encoding error in the latent space s{I) =[("), and the velocity of multicopter shows the effectiveness of CWP based on D-learning. (b) 3D trajectory based on the PBVS controller using 3D positions as feedback and the D-Jearning controller using 2D images as feedback.
5 Discussion
Conclusion. We propose sampling-based stability condition, exponential attraction, satisfy the Lyapunov stability for learning-bast based controller. Based the Theorem 2.1, propose CWP, which transforms the controller design problem into a pattern classification problem. Subsequently, we propose D-learning for performing CWP in the absence of knowledge regarding the system dynamics. Finally, on both the simulation platform and the real multicopter platform, we show that our approach can synthesize and verify neural network controllers for control system with images as feedback, and CWP controller has better performance than the controller trained by RL.
I Limitation. Despite the success in simulated and real flight tasks, our method has not been evalu- ated in complex practical scenarios. It is anticipated that our approach will be applicable to more complex real-world robotic control tasks, such as locomotion and navigation. To achieve this ob- jective, future work will need to make further improvements in data utilization and provide stability guarantees. Our future work needs to enhance feature extraction our work, with particular fo- cus on improving the robustness and generalizability of the feature extraction process. Moreover, the feature function, Lyapunov function, and controller, all in the form of neural networks, can be learned jointly to achieve superior performance. These will be explored in the future work.

---

Acknowledgments
This work is supported by the National Natural Science Foundation of China under Grant 61973015.
References
[1] R. Sutton, and Barto. Reinforcement Learning: An Introduction. MIT press, 2018.
[2] D. Bertsekas. Reinforcement Learning and Optimal Control. Athena Scientific, 2019.
[3] N. M. Boff, S. Tu, N. Matni, J.JJ. Slotine, and V. Sindhwani. Learning Stability Certificates from Data. In Conference on Robot Learning, pages 1341-1350. PMLR, 2020.
[4] C. Dawson, S. Gao, and C. Fan. Safe Control With Learned Certificates: A Survey of Neural Lyapunov, Barrier, and Contraction Methods for Robotics and Control. IEEE Transactions Robotics, 39(3):149--176,, 2023.
[5] Y. Chow, O. Nachum, E. A. Duenez-Guzman, and M. Ghavamzadeh. A Lyapunov- bbsed Ap- proach to Safe Reinforcement Learning. Advances in Neural Information Processing Systems, 31, 2018.
[6] M. O'Connell, G. Shi, X. Shi, K. Arizzadenesheli, A. Anandkumar, Y. Yue, and S. J. Chung. Neural-fly Enables Rapid Learning for Agile Flight in Strong Winds. Science Robotics, 7(66))ebbm6597, 2022.
[7] K. Huang, R. Rana, A. Spitzer, G. Shi, and B. Boots. DATT: Deep Adaptive Trajectory Track- ing for Quadrotor Control. In Conference on Robot Learning, pages 326-340. PMLR, 2023.
[8] Wang, L. Fengb, X. Zheng, Y. Cao, O. O. Oseni, H. Xu, T. Zhang, and Gao. Policy Op- timization Method Towards Optimal-time Stability. Conference on Robot Learning, pages 1154-1182. PMLR, 2023.
[9] I.. Yang, Dai, 7. Shi, J. Hsieh, Tedrake, and Zhang. Lyapunov-stable Neural Control for State and Output Feedback: A Novel Formulation for Efficient Synthesis and Verification. arXiv preprint 2:Xiv:2404009955 2024.
[10] Y.-.. Chang, N. Roohi, and S. Gao. Neural Lyapunov Control. Advances in Neural Information Processing Systems, 32, 2019.
[1]] J. H. Watkins, and P. Dayan. Q-learning. Machine Learning, 8:279-292, 1992.
[12] H. Dai, B. Landry, L. Yang, M. Pavone, and R. Tedrake. Lyapumov-stable Neural-network Control. In Robotics: Science and Systems 2021, pages 1-12. 2021.
[13] R. Zhou, Quartz, H. De Sterck, and Liu. Neural Lyapunov Control of Unknown Nonlinear Systems with Stability Guarantees. Advances in Neural Information Processing Systems, 35, 2022.
[14] R. O. Duda, E. H. Peter, and P. E. Hart. Patlern Classification. John Wiley & Sons, 2006.
[15] Y. Nesterov. Efficiency of Coordinate Descent Methods on Huge-scale Optimization Problems. SIAM Journal on Optimization, 22(2):341-36 2012.
[16] F. Chaumette, and S. Hutchinson. Visual Servo Control. I. Basic Approaches. IEEE Robotics
& Automation Magazine, 1344)822-90, 2006.
[17] Q Quan. Introduction to Multicopter Design and Control. Springer, 2017.
[18] S. Felton, E. Fromont, and E. Marchand. Deep Metric Learning for Visual Servoing: when Pose and Image Meet in Latent Space. In 2023 IEEE International Conference Robotics and Automation, pages 741-747. IEEE, 2023.

---

[19] T. P. Lillicrap. J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra. Continuous Control with Deep Reinforcement Learning. arXiv preprint arXiv: 1509.0297, 2013.
[20] A. M. Lyapunov. The General Problem of the Stability of Motion. International Journal of Control, 55(3)5531-53, 1992.
[21] R. Bellman. Dynamic Programming. Science .133(3331):34 1966.
[22] E. D. A. Sontag. A 'Universal' Construction of Artstein's Theorem on Nonlinear Stabilization. Systems & Control Letters, 13(2):117-123, 1989.
[23] C. Dawson, Z. Qin, S. Gao, and C. Fan. Safe Nonlinear Control Using Robust Neural Lyapunov-barrin barrrir Functions. In Conference on Robot Learning, pages 1724-1735. PMLR, 2022.
[24] R. C. B. Rego, and F. M. U. De Araujo. Learning-based Robust Neuro-control: A Method to Compute Control I Lyauuovv Functions. International Journal of Robust and Nonlinear Control, 32(5):2644-26611 2022.
[25] K. Neumann, A. Lemme, and J. Steil. Neural Learning of Stable Dynamical Systems Based on Data-Driven Lyapunov Candidates. In 2013 IEEE/RSJ International Conference on Intelli- gent Robots and Systems, pages 2216-1222. IEEE, 2013.
[26] H. Ravanbakhsh, and Sankaranarayanan. Learning Control Lyapunov Functions from Coun- terexamples and Demonstrations. Autonomous Robots, 43:275-307, 2019.
[27] A. J. Taylor, V. D. Dorobantu, H. M. Le, Y. Yue, and A. D. Ames. Episodic Learning with Control Lyapunov Functions for Uncertain Robotic Systems. In 2019 JEEE/RSJ International Conference on Intelligent Robots and Systems, pages 6878-6884. IEFE, 2019.
[28] S. Tesfazgi, A. Lederer, and S. Hirche. Inverse Reinforcement Learning a Control Lyapunov Approach. In 202/ 60th IEEE Conference on Decision and Control, pages 3627-3632. IEEE, 2021.
[29] C. Cortes, and V. Vapnik. Support-vector Networks. Machine learning, 20:273-297, 1995.
[30] L. Breiman. Random Forests. Machine learning, 45:5-32, 2001.
[31] Kotsiantis. Decision Trees: Recent Overview. Artificial Intelligence Review, 39:261-
283, 2013.
[32] Vanabi-Sharin and W.J. Wilson. Automatic Selection Image Features for Visual Servoing. IEEE Transactions on Roboties and Automation, 13(6):890-99) 1997.
[33] S.M. Richards, F. Berkenkamp, and A. Krause. The Lyapunov Neural Network: Adaptive Sta- bility Certification for Safe Learning of Dynamical Systems. In Conference on Robot Learning, pages 466-476. PMLR, 2018
[34] H. Ravanbakhsh, and S. Sankaranarayanan. Learning Control Lyapunov Functions From Coun- terexamples and Demonstrations. Autonomous Robots, 43:275-307, 2019.
[35] F. L. Lewis, and D. Vrabie. Reinforcement Learning and Adaptive Dynamic Programming for Feedback Control. IEEE Circuits and Systems Magazine, 93):322-50, 2009.
[36] K. He, X. Zhang, S. Ren, and J. Sun. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pages 770- 778. IEEE, 2016.
[37] S. Felton, P. Brault, E. Fromont and E. Marchand, Visual Servoing in Autoencoder Latent Space. IEEE Robotics and Automation Letters, 7(2)33334-241,, 2022.
10

---

[38] E. Marchand, Subspace-Band Based Direct Visual Servoing. IEEE Robotics and Automation Letters, 4(3):2699-776,, 2019.
[39] C. Collewet, E. Marchand and F. Chaumette, Visual Servoing Set Free from Image Processing. In 2008 IEEE International Conference on Robotics and Automation, pages 81-86. IEEE, 2008.

---

A Related Work
RL [1], [2] and Lyapunov function learning (or certificate learning further, including barrier function and contraction metrics learning) [3], [4], have the potential to handle control problems of compli- cated systems with big data. Q-learning is RL method. D-learning, which parallels to Q-learning in RL aims to obtain both Lyapunov function and its derivative. The similarities and differences between the two are illustrated in Fig.6.
Q-leaming
Dikumnng
Reinforoement Learting
Goak Minimum Cost Function
1 yopunov Function Loorming
Goal: System Stability
Dyzamii Progeamming
Cost function basad Defized Revard
Lyspunoy's 's Method
A set of Lyapunor Funchion with Detinet Form
Value Function determined by Function
Lyspunuw Detivale prodicted by D function
Costmuller traindd by Q leomning
Controller trined. by D loammng
Richard E. Bellman
A Aleksandr Lyapunov
Figure 6:
The similarities and differences between Q-learning and D-learning, Q-learning is RL method, while D-learning a Lyapunov function learning method.
A.1 Reinforcement Learning and Lyapunov Function Learning
As solution optimal control problems forward-in-time RL often focuses optimization based on the Bellman equation [21]. In the traditional control field, the Bellman equation is often used as analysis tool rather than a direct design tool in optimization control. Nevertheless, the hand- design I Lyapunov method, which relies on pseudo-energy functions, is the most prevalent tool in both the analysis and design of control systems. Its objective is to reduce the pseudo-energy functions over time, thereby causing the state to converge to a fixed point. From a technical standpoint, RL is required to define the rewards function and compute the value function (optimal objectives defined priors). In contrast, Lyapunov function learning requires training parameterized Lyapunov function to match the data set (concrete Lyapunov functions are NOT defined as priors). Therefore, they are different in both application and design. RL with Lyapunov functions, where certificates are used to ensure safety or stability, has also been proposed recently [8]. A commonly used certificate is the sum of cost over limited time horizon as valid Lyapunov candidate [5]. In comparison RL, Lyapunov function learning offers greater flexibility candidate selection.
RL based on the Bellman equation is prevalent in the field of computer science due its model-free characteristics. More importantly, it can solve very complicated control problems. Compared with RL, Lyapunov's methods' achievements on complicated problems with big data are less. Because of the gap between developments by the Lyapunov's 's method and the Bellman equation, it is hypoth- esized that there is an increasing focus on Lyapunov function learning from data. The expectation is to unveil its enormous potential, which also the major motivation of this paper.
The objective of Lyapunov function learning is to construct Lyapunov functions from data. This process can be approached in two main ways:
Construct a Control Lyapunov Functions (CLF) [22] in formal methods. Lyapunov-stable neural- network control [12], learning-basee robust control Lyapunov barrier function [23], neural Lyapunov control [10], and learning-bass robust neuro-control [24] employ neural networks to construct both Lyapunov functions and controller simultaneously. These formal methods, which synthesize and verify controllers and Lyapunov functions together, formulate the Lyapunov certification problem
12

---

as proof that certain functions (the Lyapunov function itself, together with the negation of its time derivative) are always non-negative over a domain.
Learn certificate in deep learning methods. Demonstration learning [25], [26], episodic learning [27], and imitation learning [28] aim only searching for certificate from given control policy data. Notwithstanding the impressive performance of these controllers, many of these controllers require a sufficiently large amount of data to learn semiglobal stabilization. The data collected from actual robotic systems is expensive, which presents a challenge for these controllers.
A.2 Pattern Classification
Pattern classification fundamental process in various fields, including machine learning, data analysis, and computer vision [14]. The objective of pattern classification is to assign a label to a given input based on its characteristics. This process involves categorizing data into predefined classes or categories, making it essential for applications such as speech recognition, image analysis, and bioinformatics. As for the CWP approach, more pattern classification methods and ideas applicable:
(i) Methods. Through the CWP approach, the process of learning model-free controllers with Lya- рunov stability guarantees from data can be framed as pattern classification problem. A variety of pattern classification methods, including support vector machines [29], random forests [30], and decision trees [31], can be employed for the construction of CWP controllers.
(ii) Features. CWP employs pattern classification techniques extract pertinent features from data and construct data-driven controllers. The expression of unstructured data, such as images, videos, and sounds, is challenging due to the difficulty in defining clear physical variables. Pattern classification enables the extraction of low-dimensional structured features from unstructured, high- dimensional data. As result, features are abstracted from patterns in the form of state or set as feedback [32].
(iii) Negative samples (or counterexamples). To date, a number of existing methodologies have employed counterexamples derived from model-based optimization to transform the construction of the Lyapunov function into a pattern-laasified problem [33], [34]. In the novel method, it is anticipated that a greater number of negative samples will be generated from data that adheres to the concept of rewards in RL. To illustrate, consider scenario which a drone encounters an obstacle reaches incorrect equilibrium point through sequence of actions and states. In such an instance, the series of actions and states will be assigned negative weight within the range of [-1, 0]. As a result of the aforementioned characteristics, greater number of pattern classification methods are now applicable.
B Preliminary Remarks
B.1 Exponentially Stability and Exponentially Attraction
In this part, some definitions about stability are given related the system (I) and the data set (2). Definition B.1 (Exponentially Stable). For the system (I), an equilibrium state x O,x is expo- nentially stable if there exist a, R4 such that | (r; 0, x S a [|x e Ar in some neighbor- hoods around the origin. Global exponential stability independent of the initial state Xo-
Here, & (T; 0, Xo) represents the solution starting at Xg, t 2 should noted that we can only use the data set (2). So, a new definition related stability, especially for the data set proposed in the following.
Definition B.2 (Exponentially Attractive on P). For the dynamics (1), an equilibrium state = O is exponentially attractive on the data set P with a, X, & E RR if [ (7; x)]] S a | VT 0, Vx B (x,,)) for any x, P, where B (x1,e) denotes neighborhood around x, with radius
13

---

Definition B.2 is to describe the trajectory of the system (1) starting from the state. It is hard or impossible to get the exponential stability only based on the data set (2) except for more information on f (x) obtained further. So, the definition of exponential attraction especially for the data set can be served as an intermediate result for classical stability results. For some special systems, we can build the relationship between the exponential stability and exponential attraction.
Theorem B.3, For the autonomous dynamics = Ax, suppose i) the equilibrium state X = 0xx1 exponential attractive on the data set P with e, ii as shown Fig.7, 31 4. Ci S U,B (x,,), X; P, where {x D x 1] for positive-defmite matrix 0 P p' Ruxn
Then the equilibrium state 0,x1 is globally exponential stability.
3 xx x
G
B 8
Figure 7: C belongs the collection of B (x1,e).
Proof. For any x" # 0nx1 R" since x Txx due to being positive-definite matrix, we have
x 0x
where
Px* Since (T;0,x) = A for x E Ci, the solution is (7;00,,,) x*) satisfying
① (T;0,x**) = 0Φ (T; 0,xx)
On the other hand, since the equilibrium state = Onx is exponential attractive on the data set P with e, there exist a, X, E R, such that || (T: 0, x || I a [ ~>r Vr 2 0. Therefore,
1
|10(1;0,x)))
|10 (T;0,x*))
< xx ->x
= a e
V7 0 for any X; P. Therefore, the equilibrium state Onx1 is globally exponentially stable.
Remark 2. Theorem B.3 implics that, for autonomous linear dynamics, exponential attraction is equivalent to exponential stability if the data set covers the boundary of an ellipsoid. For general dynamics, the least amount of data required for the equivalence worth studying. Some research has applied statistical learning theory to provide probabilistic upper bounds the generalization error, but these bounds tend to be overly cautious [3].
14

---

C Details of Theoretical Analysis
C.1 Assumptions of Theorem 2.1
d, where IR.. Assumption C.2. For x E D, the function satisfies ||ā (x) /0x| S ly, where Ly 1...
Assumption C.1. For x E D, ||x||
Assumption C.3. For XI, E D, there exists lg E R2+ such that
|| 0g (x) /0x
x=xI x=x]x=x
0g (25xx/00 /0x x=xxa ||
S la ||| X211
Assumption C.4. For E D, there exist k1, 12 E R4 such that h1
||x||² - [g (x) 0
- k2 ||x
Assumption C.5. For x E D there exists a ky E R4 such that k3 ||x|| x h (x)² On-
Assumption C.6. ||x,]], ||x||| # 0 for V (x;; x;) E P.
C.2 Proof of Theorem 2.1
Proof. This proof consists of three steps.
Step 1. || || S ly ||Ax;|l-
For any E B (x;,)), it can be written
x= x; + Ax;
where x,X; E D and Ax; E B (). Then Ax S e. this case, we have
x = x; + Ax; = f (x, + Ax;)
AN; = f fx: + Ax,) fxx).
Under Assumption C.2, further have A < 1 |xxl|-
Step 2. || (r; 0, x)|| S a ||x|| - for Vx E B (x; c)
For any x E B (x,, e) according to the definition of V (x) in (3), we have
Ög (x) Ox
(x)
x 0,
0g (x)
(x: Ax;,)
0g
dx
+
T
0g (x)
0g (x)
0g (x)
x;+
(X; + AX;)
x; 0,
0x
Ox
8x
x=
x=x:+Ax,
x==x
T
Og (x)
*; 0g (x)
0g (x)
0g (x)
*;
Ax;
Og
0x
0x
0x
8x
x
xx +Ax;
x=
xx=x+Axx;
S h (x,) T 0, + 1 |Ax|| || |
|0 + lg |X, + Ax;
| |1yl| (From Assumption C.3)
< h (x,) 1 0,, 1gl/ 0 ||| |x||| /g// 0 (1|x;| + E)
< h (x,) 0, + / |0, = 4 100,1
< ks |x] + 2 |0, |x + 4y41 |10 2. (From Assumption C.5)
Then there exists 0 < ha ka and let
4131? 110,11 2 4(k3 ka)los 110, |x||| v
|x|||
>
21,1, 100,11
15

---

such that V. (x) I -ks ||x, 12 Then, by Assumption C.I and Assumption C.4, there exists Ks ky ||xx||/ / such that
(x) S -k4 |x:112
S ks |x||² (From Assumption C.I)
S 2AV (x), Vx B (x;,5) (From Assumption C.4)
where À = ks /2k2 Consequently, for x B (x,,E), we have
v ((((T;0,))) - VV (x) -AA
where 2 0.
As result, by Assumption C.4, we have
10 (T:00xx s a |
for x E B (x;, 8), where a v hi2 /ki.
With Steps 1,2, there exist ||x|| e Ar vr 2 0, Vx E B Ex) Moreover, A, are independent of x,, so the result is applicable to any X; P. Therefore, the equilibrium state = 0,x1 is exponentially attractive on the data set P.
/ka /k/ 0 < X < 13 /2kg E > 0, such that | (T; 0) <
D Details of Feature Extraction
In order to extract the features from the current image I, ResNet-18 [36] model, trained using metric learning, employed. The principal objective of metric learning is develop a novel metric that diminishes the inter-sample distances within a given class and amplifies those between distinct classes.
In order to more accurately represent the feature in question, it is proposed that a multimodal latent space, designated as S, be created in which both position and image representations are mapped. The interrelationship between the latent space, images, and positions is illustrated in Fig. 8(a). The position maps to feature embedding Sr, and the image I acquired the position noted as st.
| = 0.8m
MLP
0.4m
Position
0.2m
S
ML.P
Camera m0mma.
I
CNN
CNN
(a)
(b)
Figure 8: (a) The proposed latent space for visual servoing. Both images and positions are projected in the feature space S, where they can be compared. (b) PCA projection of trajectories in the latent space, for 2D motions. Circles show the error for the position embeddings s, S,- for various distances.
16

---

We argue that for the optimal policy of visual servoing, the distance between two embeddings should be equal to the distance between their underlying positions: d5(s1,,,11) s1,) = ||r rxll2 where ds represents the Euclidean distance:
ds(s), 8c) = ||s| 8x|l-
(18)
To learn the space S, we propose to use two distinct, parallel neural networks. The first is Φ R S that maps a position to embedding image I to its latent representation SI (b(
((r) The second model s/ I + S, maps an
In order to train & and 1h, we devise our loss function Ls, which is based on the distances between the latent representation of the camera tuple (ry, I2) and the camera tuple (rk Ix) by
Ls Rs Lag + L
(19)
where La.a is the loss function to train R + S, fa,ga is the loss function to train vb : I S and L is the loss function to shape the feature space S in the following
Laye MSELoss (I]r; rull, |sr, Sr,
(20a)
yy. MSELosS (I|sT, - Sre ll.. [|s,, Srx Il)
(20b)
Ls.e MSELoss (|sı,, s1 1|s,, Sri
(20c)
By comparing representation with each specific tuple, we ensure that single iteration guides the encoding towards a more stable location. As illustrated in Fig.8(b), the minimization of e in the latent space results in the emergence of nearly straight lines in the latent space. The error between position embeddings also correlates well with the error from image representations.
In [37], the authors propose the use of autoencoder visual servoing as method for performing visual servoing in the latent space of an autoencoder. An autoencoder is neural network architecture comprising an encoder and a decoder. Together, these components learn to project images into low- dimensional representations and then reconstruct them back to the original space. This approach analogous to that of PCA-based visual servoing [38]. In consideration two encodings, $1, s1- extracted from unstructured data, specifically current image I and desired image I', the neural network-based IBVS control law is expressed follows:
= - (SI S1-)
(21)
where L,g is computed analytically by applying the chain rule:
Ost OI
Laj ØI
and L, represents the Moore-Penrone pseudoinverse of the matrix L,, While this process is typ- ically applied images, the same rationale can be extended an encoder for any type of inputs, provided that the interaction matrix of the input be defined. Neural networks built with py Torch2 possess automatic differentiation capabilities, ensuring that the features extracted from images continuous. This continuity permits their utilization in visual servoing and facilitates the extraction of structured features from unstructured data.
E Details of Figure 2
In 2, CWP constructs the model-free controller based on the data set F. Here, (x, u, x') F represents the collected data tuple of the unstructured data x the policy u. and the structured feature encoding x' from the unstructured data x. V(x) represents the Lyapunov function (3), and (xx) the D-function in (13). To update the control policy c(x), CWP computes the gradient V.(x)
0V f (x, u) represents the derivative of Lyapunov function in (4). D(x, u) represents
to minimize the optimization objective in (14).
hatps://pytorch.orr/
17

---

Details of Simulations and Real Flight Experiments
Table.1 presents the configuration information a custom-built multicopter in simulations and Ta- ble.2 presents the configuration information of a Tello EDU multicopter' real flight experiments. We use the custom built multicopter for data collection and validation simulations, and use the Tello EDU multicopter for data collection and validation in the real flight experiments. For commu- nication,
DJTTelloPy' as the upper-leves flight control communication program.
For data collection, we use motion capture system to obtain the position E R and velocity V E R of the multicopter. For real flight experiments, we set Remote Controller(RC) to control the velocity IR of the multicopter.
To ensure real-time control, use CUDA speed up the inference process. In simulations ducted platform with Intel 17-13700K CPU and an RTX 4070 Ti GPU, the neural network inference frequency exceeds 100 Hz, thereby meeting the requisite standards for real-time image feedback control.
custom built multicopter
Weight Airframe radius Diagonal size
1.5 kg
0.225 mm
450 mm
30 Fps 720 x 405 X4 CopterSim running PX4 Table 1: Configurations of the custom built multicopter in simulations.
Camera frame rate
Camera resolution
Configuration
Flight controller in simulations
Tello EDU multicopter
Weight Size
0.08 kg
mm 92.5 mm 41 mm
Propeller size
3 inches
Camera frame
30 Fps
Camera resolution
960 x 720
Table 2: Configurations of the Tello EDU multicopter in real flight experiments.
18


Reviews:
[{"review_id": "6AfXkTffFn", "reviewer": "Reviewer_FHr9", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 5, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper addresses the challenge of ensuring formal (Lyapunov) stability guarantees for learning-based controllers in nonlinear dynamical systems. It introduces a novel control approach called Control with Patterns (CWP) to tackle this issue. The key contributions are:  Exponential Attraction on Data Sets: A new definition to describe nonlinear dynamical systems is proposed. Pattern Classification: The problem of exponential attraction is converted into a pattern classification problem using parameterized Lyapunov functions. D-Learning Method: This method allows the implementation of CWP without prior knowledge of system dynamics. Empirical Validation: The effectiveness of CWP and D-learning is demonstrated through simulations and real flight experiments, particularly in stabilizing the position of a multicopter using real-time images for feedback, akin to an Image-Based Visual Servoing (IBVS) problem.", "summary_of_recommendation": "Recommendation for acceptance, supported by its unique idea and evaluation.", "technical_quality": 4, "clarity_of_presentation": 3, "potential_impact": 4, "robotics_focus": 4, "quality_of_limitations": 3, "questions_for_rebuttal": "The integration of Lyapunov theory with deep learning presented in this paper is both fascinating and innovative. The authors have successfully combined a robust theoretical foundation with substantial experimental validation. Here are a few comments and suggestions for further improvement:\n\n1. Figure 2 Clarification: In Figure 2, it would be beneficial to include definitions and explanations for the variables in the caption, such as x, u, x', V(x), etc. This will enhance the clarity and understanding of the figure for readers.\n\n2. Pattern Classification Background: Providing a more comprehensive introduction to pattern classification would be appreciated. This additional background will help readers who may not be familiar with this concept to better grasp the significance and application of pattern classification in the context of this research.\n\n3. System Architecture: Including a detailed system architecture diagram would significantly improve the paper. This diagram should illustrate the parameters outputted by the model and how these parameters are used to control the drone effectively. A clear visual representation of the system’s workflow will aid in understanding the overall approach.\n\n4. Technical Details: Incorporating technical details, such as the camera framerates (in Hz) and model computation times (in milliseconds), is crucial. These details provide insight into the practical implementation and performance of the system, which is essential for readers looking to replicate or build upon this work.\n\n5. Real-Time Control: It is important to elaborate on the measures taken to ensure that the model output is fast enough to meet real-time control requirements. This could include techniques for optimizing computation speed or strategies for handling delays. Providing this information will address potential concerns about the system’s responsiveness and reliability.\n\n6. Training Data: Detailed information about the type of data used to train the model should be included. This could cover aspects such as the nature of the data, the diversity of the scenarios covered, and the preprocessing steps taken. Understanding the training data is crucial for evaluating the model’s robustness and generalizability.\n\nIn summary, the paper presents a unique and compelling integration of Lyapunov theory with deep learning. By incorporating these additional details and clarifications, the paper would provide a more comprehensive and accessible presentation of the research, ultimately benefiting a wider audience.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "513MuS77vz", "reviewer": "Reviewer_5T6L", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The authors introduce a novel algorithm to learn stable policies via pattern classification. To learn under unknown dynamics, they propose a method based on D-Learning.", "summary_of_recommendation": "The paper presents an interesting problem and interesting solution. Nevertheless, there is some missing information regarding the feature selection that authors should explain in more detail.", "technical_quality": 3, "clarity_of_presentation": 3, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 2, "questions_for_rebuttal": "- What are the properties the features should satisfy? Are they required to be smooth?\n- Is the controller guaranteed to perform well out of demonstrations area? If the dynamics are unknown, how is possible to train a model in regions where u lack demonstrations?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5tAWPIw9Ji", "reviewer": "Reviewer_6cXy", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 4, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The purpose of the paper is twofold, at first, it proposes a novel method for control, namely Control with Patterns - CWP, to address the stability issues over datasets corresponding to non-linear dynamical systems, to mitigate the challenges of Lyapunov methods employed in learning-based controllers applied to such datasets. For this kind of datasets, a new definition is formulated, namely, \"exponential attraction\" to describe such non-linear dynamical systems.  In second place, a D-learning algorithm is proposed to perform CWP, when the underlying dynamics knowledge of the system is not given. Experiments are showcased - both in a simulated and a real-world scenario.", "summary_of_recommendation": "Given the clarity of the theoretical presentation, the novelty and the originality of the topic, and the interesting - although irriproducibile - experimental section, the reviewer suggests to accept the paper for publication.", "technical_quality": 3, "clarity_of_presentation": 4, "potential_impact": 2, "robotics_focus": 4, "quality_of_limitations": 2, "questions_for_rebuttal": "An extension of the experimental result sections is strongly recommended, in particular: in terms of showcasing and commenting the experimental results - for both simulated and real-flight experiments - and in terms of the describing the testbed used for real flight experiments - and, if possible, motivating the choice of the latter with respect to, for example, a different setup.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper was evaluated positively by the reviewers. Raised concerns were addressed in an updated version of the paper.

Here is a high level overview of the reviews. (before rebuttal)
### Strengths:
- The integration of Lyapunov theory with deep learning is both fascinating and innovative, combining a robust theoretical foundation with substantial experimental validation. (Reviewer FHr9)
- The problem addressed is highly relevant for robotics, particularly learning stable controllers from images, which is crucial for robot learning. (Reviewer 5T6L)
- The paper is well-organized, easy to read, and comes with a clear flow and coherent structure, including a mathematically detailed description of the method. (Reviewer FHr9, Reviewer 6cXy)
- The theoretical contributions are novel and interesting, presenting a new control approach (CWP) and a D-learning method to perform CWP without prior knowledge of system dynamics. (Reviewer FHr9, Reviewer 6cXy)

### Weaknesses:
- The paper lacks detailed explanations for the features extracted from images, which are crucial for the controller's performance. (Reviewer 5T6L)
- There is insufficient detail on the experimental setup, particularly for real-life applications. (Reviewer 6cXy)
- The paper requires additional technical details such as camera framerates, model computation times, and measures taken to ensure real-time control. (Reviewer FHr9)
- The training data and its diversity, as well as the preprocessing steps, are not adequately described, which is important for evaluating the model's robustness and generalizability. (Reviewer FHr9)

The weaknesses have been mostly addressed in the rebuttal.
Decision: Accept