Online Dynamics Learning for Predictive Control with an Application to Aerial Robots
Authors: Tom Z. Jiahao
Kong Yao Chee
M. Ani Hsieh
Venue: CoRL 2022 Poster (2022)
Keywords: Online Learning
Model Learning
Model Predictive Control
Aerial Robotics

Abstract: In this work, we consider the task of improving the accuracy of dynamic models for model predictive control (MPC) in an online setting. Although prediction models can be learned and applied to model-based controllers, these models are often learned offline. In this offline setting, training data is first collected and a prediction model is learned through an elaborated training procedure. However, since the model is learned offline, it does not adapt to disturbances or model errors observed during deployment. To improve the adaptiveness of the model and the controller, we propose an online dynamics learning framework that continually improves the accuracy of the dynamic model during deployment. We adopt knowledge-based neural ordinary differential equations (KNODE) as the dynamic models, and use techniques inspired by transfer learning to continually improve the model accuracy. We demonstrate the efficacy of our framework with a quadrotor, and verify the framework in both simulations and physical experiments. Results show that our approach can account for disturbances that are possibly time-varying, while maintaining good trajectory tracking performance.

Online Dynamics Learning for Predictive Control with an Application to Aerial Robots
Tom Z. Jiahao", Kong Yao Chee", M. Ani Hsieh GRASP Laboratory
University of Pennsylvania United States {zjh, ckongyao, m. w.hsiehJe Sriihh@@eese @aeas upenn. edu
Abstract: In this work, we consider the task of improving the accuracy of dy- namic models for model predictive control (MPC) in an online setting. Although prediction models can be learned and applied to model-based controllers, these models often learned offline. In this offline setting. training data is first col- lected and prediction model is learned through an elaborated training procedure. However, since the model is learned offline, it does not adapt to disturbances model eπrоrs observed during deployment. To improve the adaptiveness of the model and the controller, we propose an online dynamics learning framework that continually improves the accuracy of the dynamic model during deployment. We adopt knowledge-based neural ordinary differential equations (KNODE) as the dynamic models, and use techniques inspired by transfer learning to continually improve the model accuracy. We demonstrate the efficacy of our framework with a quadrotor, and verify the framework in both simulations and physical exper- iments. Results show that our approach can account for disturbances that are possibly time-varying, while maintaining good trajectory tracking performance.
Keywords: Online Learning, Model Learning, Model Predictive Control, Aerial Robotics
Introduction
In recent years, model predictive control (MPC) has shown significant potential in robotic applica- tions. As an peiiiization-base approach that uses prediction models, the MPC framework lever- ages readily available physics models or accurate data-driven models to allow robotic systems to achieve good closed-loop performance for variety of tasks. However, the reliance accurate dynamic models makes it a challenging task for the controller to adapt to system changes or en- vironmental uncertainties. In the case where the robot dynamics change during deployment in the presence of disturbances, it becomes essential for the controller to update and adapt its dynamic model in order maintain good performance. Recent advancements in deep learning techniques have demonstrated promising results in using neural networks to model dynamical systems. One advantage of neural networks over physics models is that they alleviate the need for bottom-up con- struction of dynamics, which often requires expert knowledge or physical intuition. Neural networks also increasingly faster to optimize, owing to the development of optimization algorithms. This makes the application of neural networks to model predictive controllers more amenable, which al- lows the controllers adapt to disturbances or system changes by refining its dynamic model. In this work, we propose a novel online dynamic learning algorithm using deep learning method, knowledge-based neural ordinary differential equations. Through both simulations and physical ex- periments, we show that our framework, which comprises of suite of algorithms, can improve the adaptiveness of MPC by continually refining its dynamic model of quadrotor system, and in turn maintains desirable closed-loop performance during deployment.
Related Work. Traditional supervised machine learning paradigms consist of two phases training and inference. The training phase requires data to be collected in advance and then takes place in
* Equal contribution, co- first authors.
6th Conference on Robot Learning (CoRI. 2022), Auckland, New Zealand.

---

an offline fashion. After the model is trained, it is then deployed for inference. Online learning, on the other hand, performs learning with data arriving in a sequential order. Notably, online learning has been used to transfer existing knowledge to assist training [1, 2}, predict general time series data [3, 4], and learn inverse dynamics in a derivative free manner [5].
Learning dynamic models has gathered increasing attentions because of the development in scientific machine learning. A spectrum of techniques has emerged ranging from blackbox approaches [6. 7J, to methods that require some known structures [8, 9]. In the middle of this spectrum, techniques have been developed to combine physics knowledge with machine learning [10. 11, 12, 13], and physical priors have also been incorporated into neural networks [14]. A library based on continuous time deep learning techniques have been developed for scientific machine learning [15]. Online model learning is increasingly being used by model-based control methods for their adaptability to chang- ing dynamics. It has been used to learn dynamies models for variety of robots including quadru- ple robots, robotic arms, and autonomous racing vehicles [16, 17, 18, 19]. However, using neural networks to model dynamies faces several challenges in robotic applications. Optimizing neural networks online can be slow for real-time robotic applications [20]. Hence, many existing works use other parametric models such as matrices [16] and Gaussian mixture models [17]. Within our framework, we make use of lightweight neural networks to learn residual dynamics, which allow the neural networks to be optimized online. In addition, a limited amount of data may be available for training during robot deployment. To address this problem, neural networks have been boot- strapped using previously recorded data for initial training [17]. and local approximators are used to in parallel to generate data for training neural networks. In model-basee reinforcement learning (MBRL), the control policy and system dynamics are often jointly optimized [21]. However, the on- line updates can be slow, and therefore real-time application robotic platforms with fast dynamics may be challenging. An application of MBRL to train a quadcopter requires three minutes of flight time [22], which may not be available during deployment. For our framework, through simulations and physical experiments, we show that our proposed framework only requires a small amount of data to train each model and it achieves good trajectory tracking performance.
There are a number of works that use data-driven models within an MPC framework. In [23, 24], Gaussian processes
used to characterize dynamics models for MPC. In [25]. the authors use model with a neural network. Our work is hugely
hybrid model that combines a first-principle inspired by [25], but instead of learning the model offline, we devise framework that allow the dynamic model to be continually improved during deployment. The motivation for online dynamics learning is so that the model is able account for uncertainty and disturbances that are not captured by the training data. To the authors' best knowledge, there has yet been online learning techniques specifically developed for learning dynamical systems and being applied to model predictive control schemes. There are works in the literature that promote adaptiveness for MPC. In [26), the authors propose updating the state and control weights in the MPC optimization problem an online set- ting. The authors in [27] propose an adaptive cruise control scheme that includes a real-time weight tuning strategy, where the weights in the optimization problem are adjusted based on various operat- ing conditions. In our work, instead of optimizing for the cost weights or other parts of the objective function during deployment, we use deep learning to compensate for the uncertain dynamics that are not accounted for in the dynamics constraints within the MPC framework. In [28), nonlinear MPC scheme augmented with a LI adaptive component proposed. The errors between the state estimator and LI observer are incorporated into the control scheme through adaptation law. In similar vein, the authors in [29] proposed a linear MPC scheme combined with a LI adaptive mod- ule for quadrotor trajectory tracking. Joshi et al. [30] use learning rule to update weights of the outermost layer of the neural network. This network is then used in the adaptive term in the pro- posed controller for quadrotor. In [31]. a LI adaptive control augmentation is incorporated into geometric controller. In contrast to the above works, instead of augmenting existing controllers with an adaptive module, our proposed framework directly updates the dynamic constraints within the model predictive controller and the control inputs are obtained by solving the optimization problem.
2 Problem Formulation
Given a robot that uses a model predictive controller, we aim to use data to continually improve its closed-loop performance in an online setting. Specifically, we seek to refine the dynamics model within the MPC framework during robot deployment. We consider a robot with the following dy-

---

ref
TRAINING
bb)
[(x, u)
f
f(x. u
fa
CONTROLLER
fo(x, u)
f(x, u)
[x u
(x, u)
ROBOT
ONLINE DATA
Figure I: Framework overview. (a) The training process runs in parallel with the robot carrying some tasks. The robot states and control inputs are collected for training. The trained model used to update the dynamic model in the controller. (b) the neural network consists of a nominal model and repeatedly added neural networks. Each neural network and the nominal model run in parallel with each other, with their output coupled to give the state derivative.
namics,
x = xx,
(1)
where the function f represents the true dynamics of the robot and the vectors x and u are the state and the control input of the robot. As the robot is deployed for a specified task, a streaming sequence of data samples that consist of states and control inputs is collected. This is denoted by S : ((())) u(la)), (x(12), ((11))....) where {4a, (1....) } are the time stamps in which the data is collected. With S, we seek to generate accurate estimates of the robot dynamics denoted by
x = fx,
(2)
where i represents updated estimate of the dynamics model. This updated model f is then integrated back into the MPC framework to provide more accurate predictions of the dynamics, which in turn improves closed-loop performance. In particular, to construct ff accurately, we utilize knowledge-base based neural ordinary differential equations (KNODE) and tools from online learning.
3
Online Dynamics Learning
3.1 Knowledge-based Neural Ordinary Differential Equations
Neural ordinary differential equations (NODE) was first proposed approximate continuous-depth residual networks [6]. It has since been used in scientific machine learning to model a wide variety of dynamical systems [1 Knowledge-basse
neural ordinary differential equations (KNODE) an extension of NODE that couples physics knowledge with neural networks by leveraging its compat- ibility with first principles knowledge, and has demonstrated learning continuous- time models with improved generalizability. Three aspects of KNODE make it suitable candidate for our online learning task. First, it requires less data for training, which means each data collection period during robot deployment can be short, and thus improving the adaptiveness of the MPC. Second, KNODE continuous-tinn time dynamic model, which compatible with many existing MPC frameworks. Third, many robotic systems have readily available physics models that can be used as knowledge. The original KNODE was applied only to non-controlled dynamical systems, but variants of have been developed to incorporate control inputs to dynamic models [25]. In this work, approach as [25], where we concatenate the state and control as z [x" u²'T. During training, the control is simply ignored when computing loss.
a similar
Using KNODE model, the dynamics in (2) is expressed as f(z, = (f 1), fo(, ()), where j the physies knowledge, fo a neural network parametrized with 0, and Mr a selection matrix parametrized with vi, which couples the neural network with knowledge. A loss function is then
3

---

given by
[(,, v) = m Σ de+1
- x(T)] 8(4,-7)(x(7)
R(0,w).
(3)
where m the number of points the training trajectory. 5 is the Dirac delta function, is T is any sampling time in T, and R is the regularization on the neural network and coupling matrix parameters. The estimated state ((T) in (3) comes from 2(T). which given by
z(T) z((4)) + JT (z(W), dw.
(4)
Intuitively, (T) is the state at generated using with the initial condition z(t;), and the loss function (3) computes the mean squared error between the one-step-ahead estimates of the states and the ground truth data. The integration in (4) is computed using numerical solvers in practice.
The optimization of the neural network parameters in KNODE can be done with either backpropa- gation the adjoint sensitivity method, which is memory efficient alternative to backpropagation. In this work, the adjoint sensitivity method is used, similar to [6]. In the following sections, the dynamic models all based on KNODE.
3.2 Online Data Collection and Learning
Algorithm Data collection and model updates
A Algorithm 2 Online dynamics learning
1: Initialize the current time, save total duration, and the
1:
Initialize
toral duration and while No new data available do
collection interval as f,. lxx A. and beot
2:
(, +- O)
2:
f, +-
3:
while f; < fvv
3: OnlineData <-
4:
4: while fxx do
5:
if New model is available then Controller updates new model 4;
6:
end while
6:
7:
TTrin new model with the newest dala
7:
8:
Save the trained model
8:
end if
9:
ta <- current time
9:
if
t;
foot
10:
10:
Save OnlineData 1
12:
OnlineData +-
end if
14:
Robot updates state using control input
15:
Append new zobot state and control input to OnlineData
16:
f.
time
17:
end while
The basic features of the proposed online dynamics learning algorithm include the dynamic model update logic and trainer that runs in parallel with the robot, which are described in Alg. and respectively.
For data collection, the robot state is repeatedly appended to data array during deployment. This data array is saved and reset (Line and 10 of Alg. ) at regular time intervals (Line 8 of Alg. 1). A key hyperparameter of data collection is the collection interval fcol, which dictates how much data in batch (measured seconds) to be collected for training. With longer collection interval, the adaptiveness of the algorithm will decrease because it not only takes longer to collect data but also takes more time to train a new model. This leads to less frequent model updates for the controller, and therefore less adaptiveness to systen/environment changes. With shorter collection interval, while the adaptiveness improves, a model is more likely to overfit during training due to the smaller training data size.
When new data becomes available, a model is trained as described in Alg. 2. A challenge for online dynamic model learning is how to continually improve the model with newly collected data. When the model gets updated, the closed-loop trajectory of the controller changes accordingly and any new data collected will reflect the updated controller. As result, the previous dynamic model needs to be preserved, and training the neural network already onboard will not serve the purpose as alters the previous controller. To tackle this problem, we use fixed-sized queue, and repeatedly add new neural networks to the queue. We weigh the neural networks with exponential forgetting factors, and discard the oldest neural network to maintain the queue size illustrated in Fig. I (b). This

---

design achieves constant computation and memory cost. Mathematically, given nominal model f as the first dynamic model estimate, the approximate model f is recursively constructed by
f(itt) Mo0+1) (f e (f for < p,
(5)
= f()=
where p is the queue size, the index (i + 1) denotes the (i + 1)th update to dynamic model, and fo is the ith neural network added to the queue, parametrized by ((i+1)- In this work, Mg. is formed by stacking two n by n identity matrices, where n is the dimension of the state. For the (i + 1)th update, only (1++1) are trained, while other existing neural networks are frozen.
Also note that after each controller update, the next set of data for training must be entirely produced using the updated controller. If a set of data is collected using a mix of old and new controllers, it must be discarded so that the next set of training data represented the most up-to-date controller. This requirement is illustrated by the update in Alg. Line 7 and the loop guard in Alg. 2 Line 4.
3.3 Applying Learned Models in MPC
Inspired by the framework in [25], we apply the updated dynamic model f in MPC framework. Specifically, we solve the following constrained optimization problem in a receding horizon manner,
N
minimize Ro MY
Σ i=1
(xf Qz; + ut Rus) + P
(6a)
subject to
F++1 = f(x; u:),
Vi= 0...., N -
(6b)
5i X, ui EUU, To x(), IN E Xt:
Vi 0, .....: N
(6c)
(6d)
where F;, uy are the predicted states and control inputs, N is the horizon and X, U, X' are the state, control input and terminal state constraint sets. f(..-) in (6b) is discretized version of the learned KNODE model, as described in Sections 3.1 and 3.2. This model is used to predict the future states within the horizon. precise model gives more accurate predictions of the states, which in turn provides more effective control actions. The weighting matrices Q and R are designed to penalize the states and control inputs within the cost function and P the terminal state cost matrix. xx()) is the state obtained at time step f, which acts as an input to the optimization problem (6). Upon solving (6). the first element the optimal control sequence un is applied to the robot as the control action. The robot moves according to this control action and generate new state measurements æ(t + 1), which will be used to solve (6) at the next time step. The optimization problem is implemented and solved using CasADi [32]. An interior-point method IPOPT [33] within the library is used to solve the problem. The solver is warm-started at each time step by providing it with an initial guess of the solution, which is based on the optimal solution, both primal and dual variables, obtained from the previous time step.
Simulations
To demonstrate the efficacy of the framework in real-world robotic applications. we apply to quadrotor system and conduct tests in both simulations and physical experiments.
4.1 Dynamics of Quadrotor System
To apply the KNODE-MPC-ONE
framework, we first construct a KNODE model, by combining nominal model derived from physies, with neural network. For the quadrotor, the nominal model can be derived from its equations of motion,
mř mg + Roy, I I== w Iws
where r and w are the position and angular rates of the quadrotor, 7), T are the thrust and moments generated by the motors of the quadrotor. g is the gravity vector and R is the transformation matrix
5

---

that maps 7) to the accelerations. m and I are the mass and inertia matrix of the quadrotor. Further- more, by defining the state as := rr' r q w " and control imput as u == [n T TJ where q denotes the quaternions representing its orientation, the nominal component of the KNODE model can then be expressed as f(x, u). The KNODE model is then constructed using the algorithms described in Section 3.2.
4.2 Simulation Setup
simulation environment for the quadrotor system is implemented based on the equations of motion given in (7). An explicit Runge- Kutta method (RK45) with a sampling time of 2 milliseconds is used for numerical integration to simulate dynamic responses of the quadrotor. The predictive controller described in Section 3.3 is assumed to have full measurements of the dynamics of the quadrotor. The solution of (6) provides control commands to the quadrotor model to generate the closed- -ooo responses. In these simulations, we consider circular trajectories, with range of target speeds of 0.8, 1.0 (nd 1.2 m/s and radii of 2, 3 and 4m. Each simulation run lasts 8 seconds. To test the adaptiveness of our framework, of the quadrotor flight. At time = 2 seconds, the mass of the quadrotor reduced by 50% and at time 5 seconds, it is increased to 133% of the original mass. In simulation, set data collection interval tcol = 0.15s and neural network queue size p = 3. An implementation of our proposed framework can be found in this repository: https: // ////ttttuubb www/Tommm/. 0n1 weweiinnnen------e KNODE-MPC.
consider two time instances during the simulation where change the mass
Benchmarks. We verify the performance of our proposed framework, KNODE-MPC-OnIle comparing against three benchmarks. First, we consider a standard nonlinear model predictive con- trol (MPC) framework, where we use discretized version of the nominal model f(x, u) as the prediction model in (6b). Comparing against this benchmark provides insights on the role of the neural network, in the presence of unknown residual dynamics. As second benchmark, we con- sider the approach taken [25], where the KNODE model is learned offline. This approach consists of two phases; data is first collected using nominal controller and KNODE model is trained using this collected data. The KNODE model is then deployed as the prediction model in (6b). We denote this approach as KNODE-MPC. By comparing tiveness of online learning against residual dynamics and uncertainty that are possibly time-varying. In partieular, to highlight the adaptive and generalization abilities of our approach, for the KNODE- MPC approach, we collect training data such that it only accounts for the first mass change at time- = 2 seconds, but not the second mass change time 5 seconds. As third benchmark, we consider a nonlinear geometric controller [34] to demonstrate the efficacy of the MPC algorithms.
by
approach to KNODE-MPC we show the effee-
4.3 Simulation Results
Table 1: Overall trajectory tracking mean squared errors (MSF) of our proposed framework against various baselines, across trajectories of different radii and speeds. For each speed and radius, the lowest MSEs are marked in bold.
Radius [m]
2.0
3.0
4.0
Speed [m//]]
0.8
1.0
1.2
0.8
1.0
1.2
0.8
1.0
1.2
MPC
0.0904
0.1280
0.1705
0.0949
0.1371
0.1861
0.0967
0.1412
0.1937
KNODE-MPC [25]
0.1222
0.1945
0.2555
0.1974
0.1769
0.2098
0.5303
0.4175
0.3418
Geometric Control [34]
0.2168
0.2572
0.3253
0.2067
0.2267
0.2606
0.2046
0.2194
0.2416
KNODE-MPC-Online (ours)
0.0660
0.1113
0.1678
0.0657
0.1043
0.1554
0.0709
0.1092
0.1571
Table I gives a comparison of our framework, KNODE--PPCOOline, against the benchmarks, MPC, KNODE-MPC and nonlinear geometric control. The plotted overall mean squared errors (MSE) are calculated by considering the difference between the time histories of the reference trajectory and the quadrotor position for each simulation run, and along each axis. They are calculated element- wise and indicate the overall trajectory tracking performance for each run. As shown in the ta- ble, KNODE-MPC-Onlin provides the best overall performance across different target speeds and radii. Notably, by considering the average across the runs, KNODE-MPC and geometric control by 18.6%, 58.8% and 53.3% respectively. KNODE-MPC- Online performs well as it is able to account for the mass changes that occur during mid-flight. On the other hand, KNODE-MPC, with the KNODE model trained offline, only able account for the first mass change, but it is unable to account for the second change, as the effects of the second
outperforms MPC,

---

Top View of Trajectories
Time history of Altiuude
1.5
KNODE Online
1.0
KNNIDE-MPC
Rafoeance
0.5
AUSRU30 (6)
-0.5
-1.0
3.5
x [md
tim [5]
Figure 2: Left panel: Trajectory plot for MPC, KNODE-MPC and WWODE-MPC-OREE (ours), from the top view. The reference trajectory is shown in red and the quadrotor position is initialized (z, y, 2) (3, 0, 0). Right panel: Time history of the altitude under the different frameworks.
mass change not observed from its training data. Figure 2 illustrates the trajectories taken by the quadrotor when the MPC frameworks are deployed. The quadrotor is required to track a circular reference trajectory of radius 3m, shown in red. Evidently, from both the top view and time his- framework allows the quadrotor to track the reference trajectory
tories, the KNODE-MPC-OnIin more closely, compared the two frameworks.
Physical Experiments
5.1 Experimental Setup
The open-source Crazyflie 2.1 quadrotor [35] is used as the platform for our physical experiments. The Crazyflie has mass of 32g and has size of cm² A computer running on Intel i7 CPU is used as the base station and communication with the Crazyflie is established with the Crazyradio PA and at nominal rate of 400 Hz. The software architecture implemented using the CrazyROS package [36]. Position measurements of the quadrotor are obtained with a VICON motion capture system which communicates with the base station. Linear velocities are estimated from these po- sitions, while accelerations and angular velocities are measured from the onboard accelerometers and gyroscope sensors. For these experiments, we implement a hierarchical control scheme where the nonlinear MPC scheme generates acceleration commands and pass them to controller [34] to generate angle and thrust control commands, which are then sent to the lower-level controllers within the Crazyflie firmware. During each run, the Crazyflie is commanded at target speed of 0.4m/s and to track a circular trajectory of radius 0.5m. We conduct 10 runs for each method, with total of 40 runs. Similar to the mass changes done in simulations, proposed framework by attaching an object of mass 1.3g (4. % of the mass of Crazyflic) mid-flight. We compare our approach with the benchmarks described in Section 4.2. For the KNODE-MPC approach, training data is collected with the object attached, so that its effects can be accounted for in the model that is trained offline. In physical experiments, set data collection interval fcol 2s and neural network queue size p = 3.
test the adaptiveness of the
5.2 Results and Discussion
Results from the physical experiments are illustrated in Figure 3. The MSEs shown are defined in the same way, as described in Section 4.3. Statistics for each run are computed using 30 seconds of flight data, approximately starting from the time when the object attached. As shown in the figure, our proposed approach, KNODE-MPC-Online outperforms the benchmarks in terms of the overall MSE. In particular, the overall median MSE for KNODE--PPC-Oline improves by 55.1 %, 21.3% and 43.2%, as compared to nominal MPC, KNODE-MPC, and the geometric controller re- spectively. Furthermore, it is observed that results of KNODE-MPC-Onliee have a smaller variance as compared to the benchmarks. This implies that the approach is consistent in terms of control
7

---

Trajectory Tracking MSE [m]
0.1
0.08
0.06
0.04
0.02
0
MPC
KNODE MPC
Nonlinear Geometric
KNODE MPC Online
(ours)
0.1
0.1
0.2
MSE. |m]
MSE, [m]
0.16
0.06
0.06
0..22
0.04
0.04
0.08
0.02
0.02
0.04
Figure 3: Performance of KNODE-MPCC-nling in physical experiments. Statístics for trajectory tracking mean squared errors (MSE) for the benchmarks and our approach, KNODE-MPC-Onlin are shown. The top of the bars denote the median, while the ends of the error bars represent the 25th and 75th percentiles. The top plot depicts the overall MSE and the bottom three plots shows the MSEs in separate axes.
performance across all runs. Since the nominal MPC and geometric control frameworks are unable account for the mass perturbation mid-flight, the MSEs observed be generally larger, es- pecially in the z axis. More notably, from the MSEs across each of the axes, we observe that the offline KNODE-MPC approach has larger errors in the and y axes and smaller errors in the axis. This suggests that the KNODE model trained offline is able to compensate for the mass perturbation induced in flight, since its effect is present the training data. However, it is unable to compen- sate for сттогs in the x-y plane, which are likely not reflected in the training data. In contrast, the approach achieves more consistent MSEs in all three axes and is able to adapt to uncertainty during flight.
6 Conclusion
In this work, we propose a novel and sample-efficiett framework, KNODE-MPC-Online, that learns the dynamics of a quadrotor robot in online setting. We then apply the learned KNODE model in an MPC scheme and adaptively update the dynamic model during deployment. Results from simulations and real-world experiments show that the proposed framework allows the quadrotor to adapt and compensate for uncertainty and disturbances during flight and improves the closed-loop trajectory tracking performance. Future work includes applying this framework to other robotic applications where dynamic models can be learned to achieve enhanced control performance.
7 Limitations
fundamental assumption our framework the continuous-tims nature of system dynamics. This means our framework has limited applicability to stochastic systems. However, there has been variants of NODE which models stochastic differential equations. For future work, we hope to extend our algorithm to incorporate stochastic models to improve its applicability.

---

Acknowledgments
This work supported by NSF IIS 1910308 and DSO National Laboratories, 12 Science Park Drive, Singapore 118225. The authors would also like to thank all reviewers and the area chair for their reviews and comments.
References
[1] Zhao, S. C. Hoi, J. Wang. and B. Li. Online transfer learning. Artificial Intelligence, 216:76-102, 2014. ISSN 0004-3702. nttps://www. sciencedirect.
URL
https://ww..scienceereeee http:///w....ccccec//../c/c////a
[2] S. C. Hoi, D. Sahoo, J. Lu, and P. Zhao. Online learning: A comprehensive sur- vey. Neurocomputting 459:249-289, 2021. ISSN 0925-2312. doihhtps://bi.oog//000111.
neucom.2021.04.111 URL https: http://wsienceiireeeee//// hct///cieeece///ttiicc////ppapp $0925231221006706.
[3] O. Anava, E. Hazan, S. Mannor, and O. Shamir. Online learning for time series prediction, 2013. URL /ttp://aaxvvvv.e org/ Serrg/bbs/3000 abs/1302.
[4] V. Kuznetsov and M. Mohri. Time series prediction and online learning. In V. Feldman, A. Rakhlin, and O. Shamir, editors, 29th Annual Conference on Learning Theory, volume 49 of Proceedings of Machine Learning Research, pages 1190-1213, Columbia University, New York, New York, USA, 23-26 Jun 2016. PMLR. URL https: / /proceee/nn/c/s mlr press/ v49 l199/unnnttov166 html.
[5] D. Romeres, M. Zorzi, R. Camoriano, S. Traversaro, and A. Chiuso. Derivative online learning of inverse dynamics models. IEEE Trunsactions on Control Systems Technology, 28 (3):816-830. 2020. 1. 191111TTTTT1121192999
[6] R. T. Q. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud. Neural or- dinary differential equations. In Advances in Neural Inf. Process. Syst., pages 6571-6583. Curran Associates, Inc., 2018. URL http: //ppprr/pprrs../ee mips. cc/paper/ 7892-neural dffeenar-fffeeeeniiilll---------
[7] J. Pathak, B. Hunt, M. Girvan, Z. Lu, and F. Ou.. Model-free prediction of large spatiotem- porally chaotic systems from data: A reservoir computing approach. Phys. Rev. Lett., 120: 024102, Jan 2018. doi: 10. 1.. 103/PhysRevLeett 120.024102. URL https://link: aps. org/ doi/10. 1003/hhyseeeLett....mmmmmm.m.... 120. 024102.
[8] S. Brunton, J. Proctor, and J. Kutz. Discovering governing equations from data: Sparse iden- tification of nonlinear dynamical systems. Proceedings of the National Academy of Sciences, 113:3932-399 -9377, 09 2015. 1o.:
[9] A. A. AlMomani, J. Sun, and E. Bollt. How entropic regression beats the outliers problem in nonlinear system identification. Chaos, 30:013107, 01 2020.
[10] A. Wikner, J. Pathak, B. Hunt, M. Girvan, T. Arcomano, I. Szunyogh, A. Pomerance, and E. Ott. Combining machine learning with knowledge- based modeling for scalable forecasting and subgrid- scale closure of large, complex, spatiotemporal systems. Chaos: An Interdisci- plinary Journal of Nonlinear Science, 30:053111, 05 2020. doi:1001063/5.000555
[11] T. Z. Jiahao, M. A. Hsieh, and E. Forgoston. Knowledge-basee learning of nonlinear dynamics and chaos. Chaos: An Interdisciplinary Journal Nonlinear Science, 31(11):11110, 2021. doi: 10..1063/500666/1
[12] Greydanus, M. Dzamba, and J. Yosinski. Hamiltonian neural networks, 2019. URL https: / //arxiv..n wwwg/bs./aa...o.oo..ooo.o.ooo.ee ba0b/ 01563.
[13] M. Cranmer, S. Greydanus, Hoyer, Battaglia, Spergel, and S. Ho. Lagrangian neural networks, 2020. URL https:/ /tte://arxiii org/ abs/2003 04630.

---

[14] M. Finzi, G. Benton, and A. G. Wilson. Residual pathway priors for soft equivariance con- straints. In M. Ranzato, A. Beygelzimer, Y. Dauphin, Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 30037-30049, Cur- ran Associates, Inc., 2021. httpbtttt:::///rreededgg http://ppppppegggeeeeeeeecc//app f
[15] C. Rackauckas, Y. J. Martensen, C. Warner, K. Zubov, R. Supekar, D. Skinner, A. Ramad- han, and A. Edelman. Universal differential equations for scientific machine learning, 2020. URL https: http:////aaaaviorraabbbseeeee
04385.
[16] Y. Sun, W. L. Ubellacker, W.-L. Ma, X. Zhang. C. Wang, N. V. Csomay-Shamm M. Tomizuka, K. Sreenath, and A. D. Ames. Online learning of unknown dynamics for model- based controllers in legged locomotion. IEEE Robolics and Automation Letters, 6(4):8442- 8449, 2021.
[17] J. Smith and M. Mistry, Online simultaneous semi-parametric dynamics model learning. IEEE Robotics and Automation Letters, 5(2):2039-2045 2020.
[18] I. Georgiev, C. Chatzikomis, T. Völk1, J. Smith, and M. Mistry. Iterative semi-parametric dynamics model learning for autonomous racing. arXiv preprint arXiv:201 2020.
[19] G. Williams, B. Goldfain, J. M. Rehg, and E. A. Theodorou. Locally weighted regression pseudo-rehearsal for online learning of vehicle dynamies. arXiv preprint urXiv:/905.m 162, 2019.
[20] I. Lenz, R. A. Knepper, and A. Saxena. Deepmpe: Learning deep latent features for model predictive control. In Robotics: Science and Systems, volume 10. Rome, Italy, 2015.
[21] L. Dong, Y. Li, X. Zhou, Y. Wen, and K. Guan. Intelligent trainer for dyna-style model-basee deep reinforcement learning. IEEE Transactions Neural Networks and Learning Systems, 32(6):2758-277 2771, 2020.
[22] N. O. Lambert, D. S. Drew, J. Yaconelli, S. Levine, R. Calandra, and K. S. Pister. Low-level control of quadrotor with deep model-based reinforcement learning. IEEE Robotics and Automation Letters, 4(4):4224-4230, 2019.
[23] J. Kabzan, L. Hewing, A. Liniger, and M. N. Zeilinger. Learning-based model predictive control for autonomous racing. 4(4):3363-3370, 2019.
[24] G. Torrente, Kaufmann, Föhn, and D. Scaramuzza. Data-driven mpe for quadrotors. (2):3769-3776, 2021.
[25] K. Y. Chee, T. Z. Jiahao, and M. A. Hsieh. Knode-mpe: A knowledge- data-driven predictive control framework for aerial robots. IEEE Robotics and Automation Letters, 7(2): 2819-2826, 2022. doi: 10. A.........222....2
[26] D. Kostadinov and D. Scaramuzza. Online weight-adaptivy nonlinear model predictive control. In 2020 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1180-1185. IEEE, 2020.
[27] R. Zhao, P. K. Wong. Z. Xie, and J. Zhao. Real-time weighted multi-objective model predictive controller for adaptive cruise control systems. International journaa of automotive technology. 18(2):279-292. 2017.
[28] D. Hanover, P. Foehn, S. Sun, E. Kaufmann, and D. Scaramuzza. Performance, precision, and payloads: Adaptive nonlinear mpc for quadrotors. IEEE Robotics and Automation Letters, (2):690-697, 2021.
[29] K. Pereida and P. Schoellig. Adaptive model predictive control for hich-accuracy trajectory tracking in changing conditions. In 20/8 JEEE/RSI International Conference on Intelligent Robots and Systems (IROS), pages 7831-7837. IEEE, 2018
[30] G. Joshi, J. Virdi, and Chowdhary. Asynchronous deep model reference adaptive control. arXiv preprint ar 2r 1.02920, 2020.
10

---

[3] Z. Wu, S. Cheng. K. A. Ackerman, A. Gahlawat, A. Lakshmanan, P. Zhao, and N. Hov- akimyan. L I adaptive augmentation for geometric tracking control of quadrotors. 2022 International Conference on Robotics and Automation (ICRA), pages 1329-1336. IEEE, 2022.
[32] J. A. E. Andersson, J. Gillis, G. Horn, J. B. Rawlings, and M. Diehl. CasADi A framework for nonlinear optimization and optimal control. Math. Program. Computation, (1):1-36, 2019.
A software
[33] A. Wächter and L. T. Biegler. On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Mathematical programming, 106(1):25-57, 2006.
[34] D. Mellinger and V. Kumar. Minimum snap trajectory generation and control for quadrotors. In 2011 IEEE international conference on robotics and automation, pages 2520-2522 IEEE,
2011.
[35] Bitcraze. Crazyflie 2.1. URL htt.:///r/////a....aaa...../aa/r
[36] W. Hönig and N. Ayanian. Flying Multiple UAVs Using ROS, pages 83-118. Springer Int. Publishing, 2017. ISBN 978-3-319-54927-9,
3.


Reviews:
[{"review_id": "wCbpvCfeGEi", "reviewer": "Reviewer_sQ7A", "summary": "This paper developed a framework for online updating of the dynamics modeled by KNODE for MPC.\nThis ensured adaptability to changes in the environment, and even when unknown weights were actually attached to the drone, that change was reflected appropriately in the dynamics to achieve optimal control.", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The online adjustment of dynamics enhanced adaptability to the environment.\n- The proposed method introduced a trick to avoid forgetting the past, which is an important point in online learning.\n\n\nWeaknesses:\n- Online dynamics learning itself is not new.\n- In avoiding past forgetting, the append-type method diverges in both memory and computational cost.\n- The adaptability of the proposed method is not well demonstrated with only one example.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "In the context of model-based RL, it is not uncommon to update dynamics online, which is not consistent with the claims of this paper.\nAlso, if online learning is anticipated, a structure such as the proposed method, which ultimately requires infinite memory and computational cost, is not appropriate.\nAs mentioned in the Limitations section, I cannot find the value of this method unless the proposed method incorporates tricks to keep the computational cost constant.", "technical_quality": "Fair: Significant additional evidence required to support the claims.", "clarity_of_presentation": "Fair: Major revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Limitations are not well addressed", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "2: Unsure, difficult to assess.", "reviewer_expertise": "3: The reviewer is fairly confident that the evaluation is correct"}, "extra_text": {"issues": "I would like to see a proper survey of previous studies, as online methods for updating dynamics are not uncommon.\n\nThe specific MPC solver was unknown unless I am missing something.\nSome solvers are computationally very expensive, and optimization should not be possible on time for methods such as the proposed method, where the computational cost of the dynamics explodes.\nWith this in mind, please clarify the solver and discuss the realtimeness of the optimization."}}, {"review_id": "leXMoUCt-A-", "reviewer": "Reviewer_GTzB", "summary": "This paper presents an online learning framework in the context of model predictive control. The method includes a joint, parallel framework where the control scheme is updated based on new, incoming models; and the models are trained and updated based on the acquisition of new data in an online fashion. The models are based on the now-well-known neural ODE framework. The paper evaluates the method with a series of simulated quadrotor experiments w/r/t tracking error. ", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- The justification for KNODE presented in lines 82-92 is appreciated.\n- The limitations section discusses several of my concerns, indicating that the the authors seem to anticipate several questions about the work that might arise\n\nWeaknesses\n- Maintaining a potentially large number of neural networks seems burdensome and possibly unnecessary\n- There is a lack of a discussion or theory w/r/t convergence guarantees. The paper seems to imply that \"more data is better\" but this is not always the case, both in terms of the data that is generated itself and/or in terms of the algorithms that are using these data", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The paper presents an interesting idea and a relevant topic for robotics (and other applications). The paper would be significantly improved with a broader set of evaluation baselines and/or more exposition on theoretical guarantees in the absence of more rigorous experimental (simulation) evalutions. ", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "reviewer_expertise": "4: The reviewer is confident but not absolutely certain that the evaluation is correct"}, "extra_text": {"issues": "Line 85: \"First, it requires less data for training...\". Less data than *what*? GPs? Standard neural networks? System identification?\n\nI am a bit confused by equations (3) and (4). What is the time step in equation (3), i.e the difference between t_i and t_{i+1}? Is this a sort of design parameter, based on the dynamics of the system under consideration? Also, t_s is in T, i.e. any sampling time in a set. But how is T defined? Is it based on m, since we are averaging over a bunch of different start times? I conceptually understand what is going on here but the terms and nomenclature are slightly confusing. \n\nHow sensitive is this approach, generally speaking, to the numerical integration of equation (4). Numerical solvers / integrators are not always stable; I am curious to see if there are practical issues here. \n\nLines 113-118: I appreciate the discussion of t_{col} and the implications of longer or shorter collection intervals. However, it would be nice to see a discussion of how to characterize this tradeoff or identify the optimal tradeoff (perhaps this is coming later). \n\nIn equation (5) and the surrounding discussion, it is proposed to keep all (I think???) of the neural network-based models. This seems burdensome in terms of both time and memory. Can the authors comment on this choice? Why is it necessary to keep all of the preceding models? The authors make both an explicit claim and seem to have the implicit assumption that \"more data means better models\". I can think of counterexamples. Perhaps the authors feel the same way. The paper also provides no stability or asymptotic guarantees - i.e. that over time the model does indeed get better. Furthermore, the system needs to handle \"jumps\" in the model, which might cause locally \"strange\" behavior. i.e. the system experiences something it has not seen before and therefore the NN updating might not be smooth. In short: this type of behavior could be unsafe, even if the overall trend of the iteration is one of improvement. \n\nLine 124: \"to tackle this problem, we repeatedly add new neural networks in parallel\" but it is unclear what is done with them. Does the selection matrix, M_phi, continue to adapt? Is this learned or learnable? Is this the mechanism that provides stability for learning? \n\nThe end of section 3.2 discusses the coupling between generating new data, training, and what model(s) the controller might be using In lines 130-133, it states that new data must be produce by an updated controller, and then refers to Alg. 2 Line 4. It is unclear how *waiting* (what the while loop does on line 4) implements this requirement. \n\nIt is still unclear why all the neural networks, f_theta, are needed when the recursion only depends on hat{f} and phi. Isn't it ok to discarbe and of the i^th or earlier neural networks? I am sorry I do not have a specific suggestion here, other than to say there is some confusion here, as well as some broader questions. \n\nI appreciate the experimental setup and evaluation in section 5. However, the evaluation seems to be too \"inward facing\". As the authors acknowledge in the related works subsection, there is a lot going on in this area right now. And yet the evaluation includes MPC, KNODE-MPC, and the paper's online variant. As I mentioned earlier, I appreciate the authors' justification for choosing KNODE; however, it is still unclear whether this method would outperform other model-predictive RL (or control) methods that do or do not use machine learning. This question is especially concerning given that KNODE-MPC-Online seems to barely outperform vanilla MPC. \n\nLines 245-248: I agree that the continuous time nature of the dynamics could be a limitation. However, I was thinking more about mode switching and hybrid systems in this context, not necessarily the stochastic nature of the problem. Can't the uncertainty / stochastic systems still be modeled in a (discretized) variant of continuous dynamics? I feel like hybrid systems is the real problem here. \n\nLines 249-252, the authors acknowledge one of my earlier concerns. One thing that is unclear is how substantial this growth in computational load is: is it exponential in the amount of time under experiment? And then, if we do network pruning or reduced order modeling, do we reduce the accuracy of the models in such a way to render the method ineffective? Furthermore, I had an earlier comment about the tradeoffs in increasing or decreasing collection time. Perhaps I missed this, but a more rigorous assessment of this tradeoff would be nice. "}}, {"review_id": "LlXxmi2qZf1c", "reviewer": "Reviewer_Jxoq", "summary": "The major contribution of this paper is to augment the existing work [20] to be implemented for online training. For the learning of dynamics to execute online, neural networks are stacked with weighted outputs to progressively capture the new features in the vehicle dynamics. Experiments are conducted to show the capability of the proposed method to adapt to changing dynamics of a quadrotor.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The major strength of the paper is to design an iterative update scheme to incorporate newly learned NN to describe the dynamical model from online data better. The more precise dynamics will yield better knowledge in the MPC controller and hence improve tracking performance subject to changes in the dynamics.\n\nThe major weakness of the paper is the lack of description/explanation of the selection mechanism of the stacked NN. Specifically, the authors mentioned the selection matrix M_\\psi, which “couples the neural network with knowledge.” However, there’s no description of how the selection matrix is formed or designed. The ambiguity introduces further questions to the proposed framework. For example, what if a newly introduced NN contains the information captured by a previously stacked NN? This question tailors more towards the efficiency of stacking NN’s where it is possible that the NN’s contain repeated knowledge.\n\nAnother weakness is that the paper frequently refers to the adaptiveness of the proposed framework. Yet the literature review covers no results from the adaptive control community. However, there are recent adaptive control designs that work fairly well for quadrotor control. The authors should include it in the literature review and cite related works, e.g., \n[1] Kostadinov, D. and Scaramuzza, D., 2020, August. Online weight-adaptive nonlinear model predictive control. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 1180-1185). IEEE.\n[2] Hanover, D., Foehn, P., Sun, S., Kaufmann, E. and Scaramuzza, D., 2021. Performance, precision, and payloads: Adaptive nonlinear mpc for quadrotors. IEEE Robotics and Automation Letters, 7(2), pp.690-697.\n[3] Joshi, G., Virdi, J. and Chowdhary, G., 2020. Asynchronous deep model reference adaptive control. arXiv preprint arXiv:2011.02920.\n[4] Wu, Z., Cheng, S., Ackerman, K.A., Gahlawat, A., Lakshmanan, A., Zhao, P. and Hovakimyan, N., 2022, May. L 1 Adaptive Augmentation for Geometric Tracking Control of Quadrotors. In 2022 International Conference on Robotics and Automation (ICRA) (pp. 1329-1336). IEEE.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "I recommend this paper to be accepted, given that the authors can clearly explain how the selection matrix M_\\psi is designed and include the references of adaptive control for quadrotors.  \n", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Excellent: Enjoyable to read.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "5: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature"}, "extra_text": {"issues": "A couple of minor issues are listed below in addition to the major ones summarized in the “Strengths And Weaknesses” section:\n1. In line 49, towards the end, there is a typo ‘hysical.’\n2. In line 81, a period is missing between [11] and knowledge-based\n3. In figure 2, given that the heatmap does not show a clear tendency in the 3x3 grids, it might be more straightforward to turn the MSE into a table with numbers.\n4. In figure 3, the plot on the right can be more informative if the horizontal axis is time instead of y position (to show the altitude holding capability).\n5. Please remove the last paragraph in Section 7 as it is not related to the paper's focus.\n"}}, {"review_id": "1yVgm0ZNJqIv", "reviewer": "Reviewer_YuBR", "summary": "In this paper, the authors propose KNODE-MPC-Online, an MPC that relies on Knowledge Based Neural Ordinary Differential Equations as dynamic models and updates the model online using neural networks. Experiments are conducted both in simulation and on a real robot.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:\n* The paper is well written and well structured\n* Related work is sufficiently addressed\n* Real world experiments are conducted and limitations are discussed\n* The approach is interesting and pseudocode is provided in the form of algorithm\n\nCons:\n* Some clarifications should be provided about the approach which is currently not reproducible\naccording to how it is described:\n  - How do you compute the M matrix?\n  - It is unclear how you use learned model in the MPC (Although there is a section about that in the paper). Specifically, given you learn several models, how do you 'choose' (or mix) them in order to obtain your prediction?\n* It would be interesting to compare against other learning methods (even if not online) with MPC\nto have a better sense of how it performs", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The paper is well written and well structured, but it lacks some clarity in the approach which make it currently difficult to re-implement.\n\nAlthough the paper is somewhat incremental, some clarity would make it much better and potentially useful. Experiments are interesting,\nbut it would have been nice to have a larger set of baselines to compare against that make use of learning with MPC (even if not online).", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Fair: Major revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "issues": "* Clarity\n* Some more experiments missing (minor)", "reviewer_expertise": "4: The reviewer is confident but not absolutely certain that the evaluation is correct"}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)