Feedback Favors the Generalization of Neural ODEs
Authors: Jindou Jia
Zihan Yang
Meng Wang
Kexin Guo
Jianfei Yang
Xiang Yu
Lei Guo
Venue: ICLR 2025 Oral (2025)
Keywords: Neural ODEs
feedback
generalization
learning dynamical systems
model predictive control
Primary area: learning on time series and dynamical systems

Abstract: The well-known generalization problem hinders the application of artificial neural networks in continuous-time prediction tasks with varying latent dynamics. In sharp contrast, biological systems can neatly adapt to evolving environments benefiting from real-time feedback mechanisms. Inspired by the feedback philosophy, we present feedback neural networks, showing that a feedback loop can flexibly correct the learned latent dynamics of neural ordinary differential equations (neural ODEs), leading to a prominent generalization improvement. The feedback neural network is a novel two-DOF neural network, which possesses robust performance in unseen scenarios with no loss of accuracy performance on previous tasks. A linear feedback form is presented to correct the learned latent dynamics firstly, with a convergence guarantee. Then, domain randomization is utilized to learn a nonlinear neural feedback form. Finally, extensive tests including trajectory prediction of a real irregular object and model predictive control of a quadrotor with various uncertainties, are implemented, indicating significant improvements over state-of-the-art model-based and learning-based methods.

Published as a conference paper at ICLR 2025
FEEDBACK FAVORS THE GENERALIZATION OF NEURAL ODES
Jindou Jia!.2; Zihan Yang' ' Meng Wang', Kexin Guo
Jianfei Yang". Xiang Yu Lei Guo
'Beihang University "Hangzhou Innovation Institute of Beihang University Nanyang Technological University
ABSTRACT
The well-known generalization problem hinders the application of artificial neural networks continuous- time prediction tasks with varying latent dynamics. In sharp contrast, biological systems can neatly adapt to evolving environments benefiting from real-time feedback mechanisms. Inspired by the feedback philosophy, we present feedback neural networks, showing that a feedback loop can flexibly correct the learned latent dynamics of neural ordinary differential equations (neural ODEs), leading to prominent generalization improvement. The feedback neural network is a novel two-DOF neural network, which possesses robust performance in unseen scenarios with no loss of accuracy performance on previous tasks. A linear feedback form is presented to correct the learned latent dynamics firstly, with convergence guarantee. Then, domain randomization is utilized to learn a nonlinear neural feedback form. Finally, extensive tests including trajectory prediction of real irregular object and model predictive control of quadrotor with various uncertainties, are implemented, indicating significant improvements over state-of-ther thetthrrrt model-base and learning-bast based methods, [
INTRODUCTION
Stemming from residual neural networks (He et al. 2016), neural ordinary differential equation (neural ODE) (Chen et al. 2018) emerges as a novel learning strategy aiming at learning the latent dynamic model of an unknown system. Recently, neural ODEs have been successfully applied various scenarios, espe- cially continuous-tinn
Neural ODE
Feedback neural network
(
a (4)
O O 00
missions (Liu & Stacey 2024
Verma al. 2024 Greydanus al.) 2019] Cran-
Sucural
f fnearee
mer et al. 2020). However, like traditional neural networks, the generalization problem limits the appli- cation of neural ODEs in real-world applications.
nneurol
Prediction
Prediction
1=1 + A/)
#t Af)
Traditional strategies like model simplification, fit coarsening, data augmentation, and transfer learning have considerably improved the generalization per- formance of neural networks on unseen tasks (Rohlfs 2022). However, these strategies usually reduce the accuracy performance on previous tasks, and large- scale training data and network structures are often required approximate previous accuracy. The ob- jective of this work is to develop a novel network architecture, acquiring the generalization improvement while preserving the accuracy performance.
# (x)
Figure Neural network architectures. Left: Neural ODE developed in Chen et al. (2018). Right: Proposed feedback neural network.
"Equal contribution.
CCoreesonding authors ((xxquo, x allannyybbua hataaggy_buaaaaauuaeee.e. wdw.aaaee...c.........oo Codes available at attttps:// // tes .oo.le.oool...o..oooooooooooooo. www/viee//e/eeeeooooo feedbacknn

---

Published as a conference paper at ICLR 2025
Living beings can neatly adapt to unseen environments, even with limited neurons and computing power. One reason can be attributed to the existence of internal feedback (Aoki et al. 2019). Internal feedback has been shown exist in biological control, perception, and commumccttio systems. handling external disturbances, internal uncertainties, and noises (Sarma et al. 2022 Markov et al. 202T). In neural circuits, feedback inhibition is able regulate the duration and magnitude of excitatory signals (Luo) 2021). In engineering systems, internal feedback indicates impressive effects across filtering and control tasks, such Kalman filter (Kalman 1960), Luenberger observer [Luenberger] 1966), extended state observer (Guo et al. 2020), and capoortioonl-innggrra derivvivveevv control (Ang et al. (205). The effectiveness of feedback lies in its ability to harness real-time deviations between internal predictions/estiaatoo uncertainties. The cognitive corrections are then performed timely. However, existing neural networks rarely incorporate such real-time feedback mechanism.
and external measurements to infer dynamical
In this work, we attempt to enhance the generalization of neural ODEs by incorporating the feedback scheme. The key idea is to correct the learned latent dynamical model of a Neural ODE according to the deviation between measured and predicted states, as illustrated in Figure T We introduce two types of feedback: linear form and nonlinear neural form. Unlike previous training methods that compromise accuracy for generalization, the developed feedback neural network is a two-DOF framework that exhibits generalization performance unseen tasks while maintaining accuracy on previous tasks. The effectiveness of the presented feedback neural network is demonstrated through several intuitional and practical examples, including trajectory prediction of spiral curve, trajectory prediction of an irregular object and model predictive control (MPC) of a quadrotor.
2 NEURAL ODES AND LEARNING RESIDUES
significant application of artificial neural networks centers around the prediction task., azt) H x(t + At). Note that t indicates the input æ evolves with time. Chen et al. (2018) utilize neural networks to directly learn latent ODEs of target systems, named Neural ODEs. Neural ODEs greatly improve the modeling ability of neural networks, especially for continuous-time dynamic systems (Massaroli et al., 2020), while maintaining constant memory cost. The ODE describes the instantaneous change of a state xt)) E E"
dxx(t) dt
(a(t),
where f(:): R" x Rm x R - IR" represents a latent nonlinear mapping, and I(1) E I denotes external input. Note that compared with Chen et al. (2018), we further consider [(t) that can extend the ODE to controlled dynamics. The adjoint sensitive method is employed in [Chnn et al. (2018) to
train neural ODEs without considering (t). Appendix A.1] we provide an alternative training
strategy in the presence of (t), from the view of optimal control.s
Given the ODE ① and an initial state st)), future state can be predicted as an initial value problem
++At
x<( + At) =x))=
s(x (T) dr.
The workflow of neural ODEs is depicted in Figure However, like traditional learning methods, generalization is a major bottleneck for neural ODEs (Marion 2024). Learning residuals will appear if the network has not been trained properly (e.g., underfitting and overfitting) or the applied scenario has a slightly different latent dynamic model. Take a spiral function as an example (Appendix A.3. When network trained from a given training set (Figure (a)) transferred to a new case (Figure (b)), the learning performance will dramatically degrade (Figure (d)). Without loss of generality, the learning residual error is formalized as
f (ar(() (L), T(I),)) = fncural (ar(1), ((),4 + Af
where fneural (.) R" x RM x R + R² represents the learned ODE model parameterized by 0, and At( R denotes the unknown learning residual error. In the presence of A (t),, the prediction error of (2) will accumulate over time. The objective of this work is to improve neural ODEs with as few modifications as possible suppress the effects of A (()..
2

---

Published as a conference paper at ICLR 2025
3
NEURAL ODES WITH A LINEAR FEEDBACK
3.1 CORRECTING LATENT DYNAMICS THROUGH FEEDBACK
Even though learned experiences are encoded by neurons in the brain, living organisms can still adeptly handle unexpected internal and external disturbances with the assistance of feedback mech- anisms (Aoki et al. 2019, Sarma et al. 2022). The feedback scheme has also proven effective in traditional control systems, facilitating high-performance estimation and control objectives. Exam- ples include Kalman filter (Kalman 1960), Luenberger observer (Luenberger, 1966), extended state observer (Guo et al. 2020), and aeapoortiool--nttegal-deeiiaiiii control (Ang et al. 2005).
We attempt to introduce the feedback scheme into neu- ral ODEs, named feedback neural networks, shown in Figure Neural ODEs have exploited latent dynamical models (t) of target systems in training set. The key idea of feedback neural networks is to further cor- rect [neurat(() according to state feedback. Denote as the historical evaluation moment satisfying 4 S I. At current moment we collect & + 1 state measurements {z())) x(tt).............)....) x(t))} }. in which fy. = t. As portrayed in Figure 2 f newral (t) modified by historical evaluation errors to approach its truth dynamics f(t), i.e.
(1
F(t)
SA/())
[(a(ti) 23))
nneurai
Marl())) #352))
[(a(ts) 233)
30
12
1g
Figure 2: The learned latent dynamics modified through accumulative eval- uation errors to approach the truth one.
fneural(t) =f fnearal (t) + Σ (t.) - (t.)) (4)
where L Ruxn represents the positive definite matrix and E (t,) E IK" represents the predicted state from the last evaluation moment, e.g., an Euler integration
x (1,) = (f4-1) + aturraatttt)))))
(5)
with the prediction step T, E R. To avoid storing more and more historical measurements over time, define an auxiliary variable
# (t) - (t)
(a (4) 2 (f:))
(6)
1=0
where & (t) IR can be regarded estimation of (t). Combining 4 and 66. can lead to
fneural (1) = [nlurat(() + ((x(() - +())).
From 5 and 6 can be further rendered that
(tx) (tx-1) + T. Tsfreural (tx-1).
(8)
By continuating the above Euler integration, it can be seen that (t) is the continuous state of the modified dynamics, i.e., à(()= - f f (()). (L). Finally, fneurat(() can be persistently obtained through
7 and (8) recursively, instead of 4 and (5) aacummuattvvel..
3.2 CONVERGENCE ANALYSIS
In this part, the convergence property of the feedback neural network analyzed. The state observa- tion error of the feedback neural network is defined as W(t) ) = x(t) - @(),, and its derivative it).
i.e. the approximated error of latent dynamics is defied as f(t) = f(i) (t). Substitute
and 3 into 77 one can obtain the error dynamics
(t) Lā(() + Af()).
Before proceeding, reasonable bounded assumption the learning residual error Af(() is made.
3

---

Published as a conference paper at ICLR 2025
Assumption 1. There exists unknown upper bound such that
|| S
(10)
where ||
denotes the Euclidean norm and y E R is an unknown positive value.
Note that the above assumption can common step disturbances (Figure S12)..
Theorem 1. Consider the nonlinear system 1 Under the linear state feedback 7 and the bounded Assumption [ the state observation error Z(t) and its derivative ā(t) (i.e., f(t) can exponentially converge to bounded sets B1 {ā(t) E TR" =|(((t)]l S 7/ /m [L) and B2 =
(a(()) RR" : [+) {ndm(L)/Amn + + respectively, which can be regulated by L.
Proof. See Appendix A.2
3.3 MUL.II-TEE PREDICTION
With the modified dynamics f(t) and current xt),, the next step is to predict
x(t+ At) in (2). By defining xt) [x] (t), (t)) E R22 from (
Multi-step prediction
have =(t) (t), (T) One intuitional means obtain = At) is to solve the ODE problem with modern solvers. However, as shown in Theorem 1 the convergence of fit can only be guaranteed as current t. In other words, the one-step prediction result by solving the above ODE is accurate, while the error will accumulate in the long-term prediction. In this part, an alternative multi-step prediction strategy is developed to circumvent this problem.
anti
Feedback NN
APy)
Feedback NN
The proposed multi-step prediction strategy is portrayed in Figure which can be regarded feedback neural network is regarded as the input of the next layer. Take the first two layers as an example. The step prediction æt + T.) obtained by xtt + T.) = xt) + (t) (()), 0)T,, The second layer with the input of ætt T.) will output xtt + 2T.). In such framework, the convergence of later layers will not affect the convergence of previous layers. Thus, the prediction error will converge from top to bottom in order.
aoff - 1M.)
cascaded form one-step prediction. The output of each
Feedback NN
a(t NL.
Figure 3: The multi- step prediction.
Note that the cascaded prediction strategy can amplify the data noise in case of large L. A gain decay strategy is designed to alleviate this issue. Denote the feedback gain of i-th later as L;, which decays as increases
L L - Si
(11)
where 33 represents the decay rate. The efficiency of the decay strategy is presented in Figure ggg) The involvement of the decay factor in the multi-step prediction process significantly enhances the robustness to data noise.
3.4 ABLATION STUDY ON OBSERVER GAIN
The adjustment of linear feedback gain can be separated from the training of neural ODEs, which can increase the flexibility of the structure.
The gain adjustment strategy intuitional. Mieoremm[innicate that the prediction error will converge to bounded set as the minimum eigenvalue of feedback gain is positive. And the converged set can shrink with the increase of the minimum eigenvalue. In reality, the amplitude of Xm (L) limited since the feedback is usually noised. The manual adjustment of Am (L) needs the trade-off between prediction accuracy and noise amplification. Thus, an ablation study on (L) to show practical implications of Theorem I Jnder Assumption 9 is implemented.
Figure shows the multi-step prediction errors (N = 50) with different levels of feedback gains and uncertainties. Two phenomena can be observed from the heatmap. The one is that the prediction error increases with the level of uncertainty. The other is that the prediction error decreases with the gain at the beginning, but due to noise amplification, the prediction error worsens if the gain is set too large.
4

---

Published as a conference paper at ICLR 2025
4 NEURAL ODES WITH A NEURAL FEEDBACK
Section has shown a linear feedback form can promptly im- prove the adaptability of neu-
40
35
ral ODEs in unseen scenarios, However, two improvements could be further made. At first,
Feedback gain
5 10 15 20 8
30
30
20
Prediction
10
will be more practical if the
15
gain tniig procedure could be avoided. Moreover, the linear feedback form can be extended to a nonlinear one (tx((t) (())) : I - R' to adopt more intricate scenes, as experi- enced in the control field (Han) 2009).
Degrée uncertainty
Degree of uncertainty
Figure 4: Prediction errors of the spiral curve with different levels of feedback gains and uncertainties to show practical implications of Theorem [ udner Assumption 9 The right image enlargement of the left one. The blue star denotes the case without uncertainty, and the uncertainty increases along both the left and right directions. When the gain is set as 0, the feedback neural network will equal the neural ODE. The related simulation setup is detailed in Appendix A.3.4
partial
effectual solution to model the feedback part us- ing another neural network, i.., Aneurat (t) (t)), {) param- eterized by g. Here we design separate learning strategy learn E. At first, the neural ODE trained the nominal task without considering the feedback part. Then the feedback part is trained through domain randomization by freezing the neural ODE. In this way. the obtained feedback neural network is skillfully considered as two-DOF network. On the one hand, the original neural ODE preserves the accuracy on the previous nominal task. On the other hand, with the aid of feedback, the generalization performance available in the presence of unknown uncertainties.
4.1 DOMAIN RANDOMIZATION
The key idea of domain randomization (obin et al. 2017 Peng et al. 2018) is to randomize the system parameters, noises, and perturbations as collecting training data so that the real applied case can be covered as much as possible. Taking the spiral example as an example (Figure (a)), training with domain randomization requires datasets collected under various periods, decay rates, and bias parameters, that the learned networks are robust to the real case with a certain of uncertainty.
Two shortcomings exist when employing domain randomization.
the one hand, the existing trained network needs to be retrained and the computation burden of training is dramatically increased. On the other hand, the training objective forced to focus on the average performance among different parameters, such that the prediction ability the previous nominal task will degraded, shown in Figure (a). To maintain the previous accuracy performance, larger-scale network designs are often required. In other words, the domain randomization trades precision for robustness. In the proposed learning strategy, the generalization ability is endowed to the feedback loop independently, so that the above shortcomings can be circumvented.
4.2 LEARNING A NEURAL FEEDBACK
In this work, we specialize the virtue from domain randomization to the feedhack part h hneurat rather than the previous neural network fnearal (t). The training framework is formalized follows
E* arg min
Σ JED!
|
$
8.l. F, T. + ( (Jnearat( + hncural (I 1,)))
(12)
where Recase denotes the number of randomized cases, Dj {x. Ei x = 1,... ..... m) denotes the training set of the i-th case with m samples, x" denotes the labeled state, and x,,j denotes one-step prediction of state, which is approximated by Fuler integration method here.
5

---

Published as a conference paper at ICLR 2025
(a) The training set
[b) The test
0.1
2
0.1
Unknown
-0
0.05
|[
10
Origin
9
0.05
10
Origin
(c) Trained performance
(d) Testing performance of Neural ODE
(e) Testing performance of Feedback NN
20
10
15
-10
5
-20
-20
Truth
Truth
Truth
(f) Multi-step prediction errors testing
(5) Performance decay strategy
(h) Performance with different N
20
Prodiction error
MMM00M0M0 Noural ODE
10
Prediction
Truth
Without decay decay
Neural ODE Feedback NN
[c]
[+]
N
Figure 5: A toy example presented to intuitively illustrate the developed linear feedback. The mission is to predict the future trajectory of spiral curve with given initial state {=(t), (t)). The neural ODE is trained on a given training set (a), yielding an approving learning result (c). Note that the pentagrams denote start points. The trained network is then transferred to test set (b), which model is significantly different from the training one. With the linear feedback mechanism, the feedback neural network achieve better approximated accuracy of the change rate (e), in comparison with the neural ODE (d). As result, a smaller multi-step prediction error (f) can be attained by benefiting from the feedback neural network. (g) shows that the noise amplification issue in multi-step prediction can be alleviated by the gain-decay strategy. (h) further presents the 50.
prediction results with different prediction steps N. N in (f)-(g)
The learning procedure of the feedback part hneurai (t) is summarized as Algorithm 1 the neural ODE fnearat (t) on the nominal task, the parameters of simulation model are randomized to produce Neasse cases. Subsequently, the feedback neural network is implemented in these cases and the training set Dire of each case is constructed. The training loss is then calculated through (12). which favors the update of parameter & by backpropagation. The above steps repeated until the expected training loss is achieved or the maximum number iterations was reached.
After training
Algorithm Learning neural feedback through domain randomization
Input: Randomize parameters to produce Yease cases; trained neural ODE fncural on nominal task.
Result: Neural feedback hneurai- Initialize: Network parameter E: Adam optimizer.
repeat
2: Run feedback neural network among Plaase cases to produce i;.;i
3: Construct datasets Deraa
4: Evaluate loss through [12 randomly selected mini-batch data;
5: Update g by backpropagation;
6: until convergence

---

Published as a conference paper at ICLR 2025
Training Neural through domain randomization
(b) Training feedback through domain randomization
(c) Evaluation training loss
10
10
0,49783
Truth
Truth
-10 0
10
40
80
120
160
200
Epochs
Figure 6: Learning with domain randomization. (a): Train the neural ODE through domain random- ization. It can be seen that the learning performance of latent dynamics on the nominal task (Figure 5 (a)) degrades
inducing domain randomization, in comparison with ( Previous works usually try to scale up neural networks to approach the previous performance. (b): Freeze the neural ODE after training on the nominal task and train the feedback part through domain randomization. The feedback neural network maintains the previous performance on the nominal task. (c) The training loss the feedback part. Note that the neural ODE employed (a) and (b) have the same architectures as the one in Figure (c).
For the spiral example, Figure 0 (b) presents the learning performance of the feedback neural network the nominal task. can be seen that the feedback neural network can precisely capture the latent dynamics, maintaining the previous accuracy performance of Figure (c). Moreover, the feedback neural network also has the generalization performance randomized cases, shown in Appendix Figure STO Figure (c) further provides the evolution of training loss of the feedback part on the spiral example. More training details are provided in Appendix A.3.3
EMPIRICAL STUDY
5.1 TRAJECTORY PREDICTION OF AN IRREGULAR OBJECT
Precise trajectory prediction of a free-flying irregular ob- ject challenging task due to the complicated aero- dynamic effects. Previ- ous methods can be mainly classified into model-based scheme (Frese et al.) 2001 Müller et al. 2011 Bouffard al. 2012 and learning- based scheme Kim et al. 2014 Yu al.] 2021). With historical data, model-based methods aim accurately fitting the drag coefficient of an analytical drag model, while learning-basse -aased ones try to directly learn an accelera- tion model using specific ba- sis functions. However, the above methods lack of online adaptive ability employing. Benefiting from the feedback mech- anism, our feedback neural network can correct the learned model in real time, leading to a more generalized performance in cases out of training datasets.
Prediction [m]
0.08
0.06
An Irregular bottle
0.04
Model based Neural ODE
0.02
Feedback
0.0
0.2
0.4
t [s)
Figure 7: Trajectory prediction results of an irregular bottle. Left: The irregular bottle is thrown out by hand and performs an approxi- mate parabolic motion. Right: The prediction errors with different methods. The prediction horizon is set as 0.5 s. The colored shaded area represents the standard deviations of all 9 test trajectories.
We test the effectiveness the proposed method on an open-soures dataset (Ja et al. 2024). in comparison with the model-based method (Frese et al. 2001 Müller al. 2011 Bouffard al.

---

Published as a conference paper at ICLR 2025
results
12.5
Trial
10.0
Trial
Trial
Loss
7.5
5.0
1mm]
7mm
2
10
20
40
-4
Epochs
Figure 8: Training sets and convergence procedures. Left: Collected trajectories used for training. We first randomly sample positional waypoints in a limited space, followed by optimizing polynomials that connect these waypoints through the minimum snap method (Mellinger & Kumar 2011). Then the quadrotor with the baseline controller from Jia et al. (2022) is commended to follow planned trajectories, yielding real fly results the training set. 40 trajectories
collected with the length of 200 discrete nodes each. Right: Training curves of 6 random trials. All training trials converged rapidly thanks to stable integration and end-to-end analytic gradients.
2012) and the learning- bearning-based method (Chen et al. 2018). The objective of this mission is to accurately predict the object's position after 0.5 8, as is thrown by hand. 21 trajectories are used for training, while 9 trajectories are used for testing. The prediction result is presented in Figure 7 It can be seen that the proposed feedback neural network achieves the best prediction performance. Moreover, the predicted positions and Iearned latent accelerations of all test trajectories provided in Figure S2 and Figure S3 respectively. Implementation details are provided in Appendix AA.4
5.2 MODEL PREDICTIVE CONTROL OF QUADROTOR
MPC works in the form of receding- -horiinn trajectory optimizations with dynamic model, and then determines the current optimal control input. Approving optimization results highly rely on accurate dynamical models. Befitting from the powerful representation capability of neural networks
for complex real-world physics, noticeable works (Torrente et al., 2021 Salzmann et al.] 2023
Sukhija et al., 2023) have demonstrated that models incorporating first principles with learning-busee components can enhance control performance. However, as the above models are offline-lcarned within fixed environments, the control performance would degrade under uncertainties in unseen environments.
In this part, the proposed feedback neural network is employed the quadrotor trajectory tracking scenario concerning model uncertainties and external disturbances, to demonstrate its online adaptive capability. In offline training, neural ODE is augmented with the nominal dynamics firstly account for aerodynamic residuals. The augmented model is then integrated with an MPC controller. Note that parameter uncertainties of mass, inertia, and aerodynamic coefficients, and external disturbances are all applied in tests, despite the neural ODE only capture aerodynamic residuals in training. For the feedback neural network, the proposed multi-step prediction strategy embedded into the model prediction process in MPC. Therefore, the formed beedback-enhancee hybrid model can effectively improve prediction results, further leading to precise tracking performance. More implementation details refer to Appendix A.5.3
5.2.1 LEARNING AERODYNAMIC EFFECTS
While learning the dynamics, the augmented model requires the participation of external control inputs, i.e., motor thrusts. Earning a quadrotor model augmented with neural ODE could be tricky with end-to-end learning patterns since the open-loop model are intensively unstable, leading to the diverge of numerical integration. To address this problem, baseline controller from Jia et al. (2022) is applied to form a stable closed-loop system. The adjoint sensitive method is employed in Chen et al. (2018) to train neural ODEs without considering external control inputs. We provide an alternative training strategy concerning external inputs in Appendix A.I from the view of optimal control. Figure 8 shows training trajectories and convergence procedures. trials of training
8

---

Published as a conference paper at ICLR 2025
Nomi-MPC
Neural-MPC
MLP-MPC
--1.0
0.05
3
-00055
0.0
0.0
-2.5
-2.5
-2.5
-0.6
Tracking error (m)
2/20
4 0.0 (m)
-2.5
-25
225
2.5
2.5
0.0
[m)
2.5
0.0
(m)
2.5
0.0
x [m)
0.4
MMC
AdapNN-MPP MPC
FNN- MUN-MPC
0..
-0.5
1.0
3 0.5
2.5
1100
0.0
0.0
0.0
0.0
-2.5
-2.5
-2.5
$000
0.0
-2.5
-2.5
-2.5
2.5
2.5
(m)
2.5
[(m)
2.5
[m)
Nomi-MPC
Neural-MPC
MLP-MPC
FB-MPC
AdapNN-MPC
FNN-MPC
RMSE [m]
0.248
0.167
0.182
0.203
0.151
0.093
Figure 9:
Tracking the Lissajous trajectory using MPC with different prediction models.
carried out, each with distinct initial values for network parameters. The trajectory validations are carried out using 3 randomly generated trajectories (Figures S7). More learning details refer to Appendix
A.5.2 A.5.2
5.2.2 FLIGHT TESTS
In tests, MPC is implemented with six different models: the nominal model (27). the neural ODE aug- mented model (Section A.5.2). the feedforward neural network augmented model (Saviolo & Loianno 2023). the feedback enhanced nominal model, the adaptive neural network augmented model (Cheng al. 2019) and the proposed feedback neural network, abbreviated as Nomi-MPC, Neural-MPC, MLP-MPC, FB-MPC, AdapNN-MPC, and FNN-MPC, for the sake of simplification. More details of all compared methods refer to Section A.5.4. Moreover, 37.6% mass uncertainty. [40%, 40%, 0] inertia uncertainties, [14.3%, 14.3%, 25.0% drag coefficient uncertainties, and [0.3, 0.3, 0.3] trans- lational disturbances are applied. The flight results on a Lissajous trajectory (out of training set) are presented in Figure 9
The tracking performance is evaluated by root mean square error (RMSE).
can be seen the Neural-MPC outperforms the Nomi-MPC since intricate acrodynamic effects are captured by the neural ODE. Moreover, the performance of MLP-MPC is relatively unsatisfactory compared with the Neural-MPC. The reason can be attributed to its single-step training manner instead of the multi-step one of the Neural-MPC, leading poor multi-step prediction. However, because unseen parameter uncertainties and external disturbances are not involved in the training set, the Neural-MPC still has considerable tracking errors. Due to the adaptive ability of the last layer, AdapNN-MPC can handle a certain level of uncertainty. In contrast, FNN-MPC achieves the best tracking performance. The reason can attributed to the multi-step prediction of the feedback neural network improves the prediction accuracy subject to multiple uncertainties, as shown in Figure S8
6 RELATED WORK
6.1 NEURAL ODES
Most dynamical systems can be described by ODEs. The establishments of ODEs rely on analytical physics laws and expert experiences previously. To avoid such laborious procedures, Chen et al. (2018) propose to approximate ODEs by directly using neural networks, named neural ODEs. The prevalent residual neural networks (He et al. 2016) can be regarded
Euler discretization of neural ODEs Marion et al. (2024). The universal approximation property of neural ODEs has been studied theoretically (Zhang et al. 2020; Teshima al.) 2020 Li al. (022), which show the sup-universality for C² diffeomorphisms maps Teshima et al. 2020) and L u-uiiierrsality for general continuous maps (Li et al. 2022 Marion (024) further provides the generalization bound (i.e.,
9

---

Published as a conference paper at ICLR 2025
upper bound on the difference between the theoretical and empirical risks) for wide range of parameterized ODEs.
6.2 GENERALIZATION OF NEURAL NETWORKS
In classification tasks, neural network models face the generalization problem across samples, distributions, domains, tasks, modalities, and scopes (Rohlfs] 2022). Plenty of empirical strategies have been developed to improve the generalization of neural networks, such as model simplification. fit coarsening, and data augmentation for sample generalization, identification of causal relationships for distribution generalization, and transfer learning for domain generalization.
Domain randomization (Tobin et al. 2017 Peng et al., 2018) has shown promising effects to improve the generalization for sim-to-real transfer applications, such as drone racing (Kaufmann et 2023)
. quadrupedal locomotion (Choi et al. 2023), , and humanoid locomotion (Radosavovic et al. 2024).
The key idea is to randomize the system parameters, noises, and perturbations in simulation so that the real-world case can be covered as much as possible. Although the system's 's robustness can be improved, there are two costs to pay. One is that the computation burden in the training process is dramatically increased. The other is that the training result has a certain of conservativeness since the training performance is an average of different scenarios, instead of a specific case.
6.3 REAL-TIME RETRAINING AND ADAPTATION
Recently, online continual learning (Ghunaim al. 2023) and test-time adaptation (Liang al. 2024) have emerged as promising solutions to handle unknown test distribution shifts. Online continual learning focuses on the reduction of real-time training load, aiming at generalizing across new tasks while maintaining performance previous tasks. Test-time adaptation tries to utilize real-time unlabeled data to obtain self-adapted models. For example, an extended kalman filter-baser adaptation algorithm with a forgetting factor is developed by /Abuduweili & Liu (2020) to generalize neural network-basee models. Moreover, in order to improve the flexibility of neural networks, the last layer of networks can be regarded as weighted vector, which can be adjusted adaptively according to real-time state feedback (Cheng et al., 2019 O O'Connell et al.) 2022] Richards et al.. 2023 Saviolo et al. 2024). The training for separating the last layer and front structure can be carried out within a bi-level optimization framework. In such a paradigm, the uncertainty out of training sets is reflected
the last layer of networks, which can be online adjusted in control-oriented
(Richards et al., (023) or regression-oriente (Cheng et al. 2019 O' CoCnnell et al.) (0022) fashion.
Patil al. (2022) further develops real-time weight adaptation laws for all layers of feedforward
neural networks, with stability guarantees.
Different from the above retraining or adaptation strategy, the presented method directly corrects the learned latent dynamics of neural ODEs with real-time feedback, yielding a two-DOF network structure. Moreover, the feedback can be learned in neural form. Integrating adaptive neural ODEs with the developed feedback mechanism may be valuable research direction (Section A.8)
7 CONCLUSION
Inspired by the feedback philosophy in biological and engineering systems, we proposed to incorpo- rate feedback loop into the neural network structure for the first time, as far as we known. In such way, the learned latent dynamics can be corrected flexibly according to real-time feedback, leading to better generalization performance in continuous-time missions. The convergence property under linear feedback form was analyzed. Subsequently, domain randomization was employed to learn a nonlinear neural feedback, resulting in a two-DOF neural network. Finally, applications on trajectory prediction of irregular objects and MPC of robots were shown.
Limitations. First, the feedback gain and decay rate for the linear feedback neural network need to be tuned manually. Future work will try to build bi-level optimization framework to train neural ODE while searching the optimal gains. Such joint optimization manner can also capture the coupled information between feedforward neural ODE and feedback network. Morevoer, the presented nonlinear neural form is preliminarily tested Section Future work will pursue to exploit its potential in more complex tasks.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Abulikemu Abuduweili and Changliu Liu. Robust online model adaptation by extended kalman filter with exponential moving average and dynamic multi-epoch strategy. Learning for Dynamics and Control, pp. 65-74, 2020.
Kiam Heong Ang. Gregory Chong, and Yun Li. PID control system analysis, design, and technology. IEEE Transactions Control Systems Technology, 13(4):559-576, 2005.
Stephanie K Aoki, Gabriele Lillacci, Ankit Gupta, Armin Baumschlager, David Schweingruber, and Mustafa Khammash. A universal biomolecular integral feedback controller for robust perfect adaptation. Nature, 570(7762):533-5533 2019.
Patrick Bouffard, Anil Aswani, and Claire Tomlin. Learning -aaned model predictive control quadrotor: Onboard implementation and experimental results. In Proceedings of IEEE International Conference on Robotics and Automation, pp. 279-284, 2012.
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. In Proceedings of Advances in Neural Information Processing Systems, volume 31, 2018.
Yujiao Cheng, Weiye Zhao, Changliu Liu, and Masayoshi Tomizuka. Human motion prediction using semi adaptalae neural networks. In Proceedings of American Control Conference, pp. 4884 4890, 2019.
Suyoung Choi, Gwanghyeon Ji, Jeongsoo Park, Hyeongjun Kim, Juhyeok Mun, Jeong Hyun Lee, and Jemin Hwangbo. Learning quadrupedal locomotion on deformable terrain. Science Robotics, 8(74))eade2256, 2023.
Miles Cranmer, Sam Greydanus, Stephan Hoyer, Peter Battaglia, David Spergel, and Shirley Ho. Lagrangian neural networks. arXiv preprint arXhv:2003.04630 2020.
Matthias Faessler, Antonio Franchi, and Davide Scaramuzza. Differential flatness of quadrotor dynamics subject rotor drag for accurate tracking of high-speed trajectories. IEEE Robotics and Automation Letters, 3(2):620-626, 2017.
U. Frese, B. Bauml, S. Haidacher, G. Schreiber, I. Schaefer, M. Hahnle, and G. Hirzinger. Off-the- shelf vision for a robotic ball catcher. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1623 1629 vol.3, 2001.
Yasir Ghunaim, Adel Bibi, Kumail Alhamoud, Motasem Alfarra, Hasan Abed Al Kader Hammoud, Ameya Prabhu, Philip HS Torr, and Bernard Ghanem. Real-time evaluation in online continual learning: A new hope. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1888 11897, 2023.
Samuel Greydanus, Misko Dzamba, and Jason Yosinski. Hamiltonian neural networks. In Proceedings of Advances Neural Information Processing Systems, 2019.
Kexin Guo, Jindou Jia, Xiang Yu, Lei Guo, and Lihua Xie. Multiple observers based anti- disturitturb control for quadrotor UAV against payload and wind disturbances. Control Engineering Practice, 102:104560, 2020.
Jingqing Han. From PID to active disturbance rejection control. IEEE Transactions on Industrial Electronics, 56(3);900-906, 2009.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, 2016.
Jindou Jia, Kexin Guo, Xiang Yu, Weihua Zhao, and Lei Guo. Accurate high- gigh-maneuvering trajectory tracking for quadrotors: drag utilization method. IEEE Robotics and Automation Letters, 7(3): 6966-6973, 2022.
11

---

Published as a conference paper at ICLR 2025
Jindou Jia, Wenyu Zhang, Kexin Guo, Jianliang Wang, Xiang Yu, Yang Shi, and Lei Guo. EVOLVER: Online learning and prediction of disturbances for robot control. IEEE Transactions on Robotics, 40:382-400 2024.
Rudolph Emil Kalman. A new approach linear filtering and prediction problems. 1960.
Elia Kaufmann, Leonard Bauersfeld, Antonio Loquercio, Matthias Müller, Vladlen Koltun, and Davide Scaramuzza. Champion-les drone racing using deep reinforcement learning. Nature, 620 (7976)9882 987, 2023.
Seungsu Kim, Ashwini Shukla, and Aude Billard. Catching objects in flight. IEEE Transactions Robotics, 3065):0049-1065 2014.
Qianxiao Li, Ting Lin, and Zuowei Shen. Deep learning via dynamical systems: An approximation perspective. Journal of the European Mathematical Society, 25(5):1671-1099 2022.
Jian Liang, Ran He, and Tieniu Tan. A comprehensive survey test-time adaptation under distribu- tion shifts. International Journal of Computer Vision, pp. 1-34, 2024.
Zefang Liu and Weston M. Stacey. Application of neural ordinary differential equations for tokamak plasma dynamics analysis. In ICLR 2024 Workshop ALDDifeernntalEqqutiins In Science, 2024.
David Luenberger. Observers for multivariable systems. IEEE Transactions Automatic Control, 11(2):90-197, 1966.
Liqun Luo. Architectures of neuronal circuits. Science, :ab6655 2021.
Pierre Marion, Generalization bounds for neural ordinary differential equations and deep residual networks. In Proceedings of Advances in Neural Information Processing Systems, volume 36, 2024.
Pierre Marion, Yu-Han Wu, Michael Eli Sander, and Gérard Biau. Implicit regularization of deep residual networks towards neural ODEs. In Proceedings of International Conference on Learning Representations, 2024.
Daniil A Markov, Luigi Petrucco, Andreas M Kist, and Ruben Portugues. A cerebellar internal model calibrates feedback controller involved in sensorimotor control. Nature Communications, 12(1): 1-21, 2021.
Stefano Massaroli, Michael Poli, Jinkyoo Park, Atsushi Yamashita, and Hajime Asama. Dissecting neural odes. In Proceedings of Advances in Neural Information Processing Systems, volume 33, pp. 3952-3963, 2020.
Daniel Mellinger and Vijay Kumar. Minimum snap trajectory generation and control for quadrotors. In Proceedings of IEEE/RSJ International Conference Robotics and Automation, pp. 2520-2525, 2011.
Mark Müller, Sergei Lupashin, and Raffaello D' Andrea. Quadrocopter ball juggling. Proceedings of IEEE/RSJ International Conference Intelligent Robots and Systems, pp. 5113-5120, 2011.
Michael O'Connell, Guanya Shi, Xichen Shi, Kamyar Azizzadenesheli, Anima Anandkumar, Yisong Yue, and Soon-Jo Chung. Neural-fly enables rapid learning for agile flight in strong winds. Science Robotics, 7666)ceb 2022.
Omkar Sudhir Patil, Duc M. Le, Max L. Greene, and Warren E. Dixon. Lyapunov-derived control and adaptive update laws for inner and outer layer weights of a deep neural network. IEEE Control Systems Letters, 6:185 $:1855-1980 2022.
Xue Bin Peng, Marcin Andrychowicz, Wojeiech Zaremba, and Pieter Abbeel. Sim-to-real transfer of robotic control with dynamics randomization. Proceedings of IEEE International Conference Robotics and Automation, pp. 3803-3810. 2018.
Ilija Radosavovic, Tete Xiao, Bike Zhang, Trevor Darrell, Jitendra Malik, and Koushil Sreenath. Real-world humanoid locomotion with reinforcement learning. Science Robotics, (89))cadi9957, 2024.
12

---

Published as a conference paper at ICLR 2025
Spencer M. Richards, Navid Azizan, Jean-Jacques Slotine, and Marco Pavone. Control-oriented meta-learning. The International Journal of Robotics Research, 42(10):777-797, 2023, doi: 10.117702885492331665588
Chris Rohlfs. Generalization in neural networks: broad survey, arXiv preprint arXiv:2209.01610, 2022.
Tim Salzmann, Elia Kaufmann, Jon Arrizabalaga, Marco Pavone, Davide Scaramuzza, and Markus Ryll. Real-time neural MPC: Deep learning model predictive control for quadrotors and agile robotic platforms. IEEE Robotics and Automation Letters, 8(4):2397-24044 2023.
Anish A Sarma, Jing Shuang Lisa Li, Josefin Stenberg, Gwyneth Card, Elizabeth S Heckscher, Narayanan Kasthuri, Terrence Sejnowski, and John Doyle. Internal feedback in biological control: Architectures and examples. In Proceedings of American Control Conference, pp. 456- 461, 2022.
Alessandro Saviolo and Giuseppe Loianno. Learning quadrotor dynamics for precise, safe, and agile flight control. Annual Reviews in Control, 55:45-60 60, 2023.
Alessandro Saviolo, Jonathan Frey, Abhishek Rathod, Moritz Diehl, and Giuseppe Loianno. Ac- tive learning of discrete-time dynamics for uncertainty-awart model predictive control. IEEE Transactions on Robotics, 40: 1273-1199,,, 2024, doi:
Jean-Jacques Slotine, Weiping Li, et al. Applied nonlinear control, volume 199, Prentice hall Englewood Cliffs, NJ, 1991.
Bhavya Sukhija, Nathanael Köhler, Miguel Zamora, Simon Zimmermann. Sebastian Curi, Andreas Krause, and Stelian Coros. Gradient baase trajectory optimization with learned dynamics. In Proceedings of IEEE International Conference on Robotics and Automation, pp. 1011-1018, 2023.
Takeshi Teshima, Koichi Tojo, Masahiro Ikeda, Isao Ishikawa, and Kenta Oono. Universal approxima- tion property of neural ordinary differential equations. In NeurIPS 2020 workshop Differential Geometry meets Deep Learning., 2020.
Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, and Pieter Abbeel. Do- main randomization for transferring deep neural networks from simulation the real world. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 23-30, 2017.
Guillem Torrente, Elia Kaufmann, Philipp Föhn, and Davide Scaramuzza. Data-driven MPC for quadrotors. IEEE Robotics and Automation Letters, 6(2):3769-3776, 2021.
Yogesh Verma, Markus Heinonen, and Vikas Garg. ClimODE: Climate and weather forecasting with physics-informed neural ODEs. In Proceedings of International Conference on Learning Representations, 2024.
Yunda Yan, Xue-Fang Wang, Benjamin James Marshall, Cunjia Liu, Jun Yang, and Wen-Hua Chen. Surviving disturbances: A predictive control framework with guaranteed safety. Automatica, 158: 111238, 2023.
Hongxiang Yu, Dashun Guo, Huan Yin, Anzhe Chen, Kechun Xu, Zexi Chen, Minhang Wang, Qimeng Tan, Yue Wang, and Rong Xiong. Neural motion prediction for in-flight uneven object catching. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4662 4669, 2021.
Han Zhang, Xi Gao, Jacob Unterman, and Tom Arodz. Approximation capabilities of neural ODEs and invertible residual networks. In Proceedings of International Conference on Machine Learning, pp. 11086-11095, 2020.
13

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.1 TRAINING NEURAL ODES WITH EXTERNAL INPUTS
Firstly, we formulate the learning problem as optimization problem:
min
L. xi;", 0) + LN (#N.
(13)
1=1
s.t. 2 fheural (z,, L 0)
(14)
where æ; E 1 and I; R denotes the model rollout state and the real sample at time t, respectively,
( (Zg. Ii, 1,, 0) refers to the discretized integration of fneural (z(t). [(t), f, 0) with fixed discrete step since real-world state trajectories æ;" E IR"" are sequentially recorded with fixed timestep based on the onboard working frequency. 1:(-) R, (x(-) E IR are defined to quantify the
Ti+1 = fhe nearaa
state differences between model rollout a; and real-world state æ;;" In this article, we select functions in weighted quadratic form, i.e., (zz a.,) L, .r,).
By utilizing the optimal control theory and variational method, the first-order optimality conditions of the learning problem could be derived as
N-1 Σ
==
X Fieural (2; I t,, 0)
(15)
2141 = V H fneural (Ei, I, ti, 0), = zz(0)
(16)
X V H Vali Onnewrat T
0l
Ait1. AN DEN
(17)
dx
0H
Σ ==1
Vol. + A VoS VeSraalal
(18)
00
where H R stands for the Hamiltonian of this problem, J is the objective function in Solving (18) could be done by applying gradient descent on 0. The gradient is analytic and available (summarized Algorithm P by sequentially doing forward rollout (16) of x and backward rollout (17) of X, where the latter one is also known as the term adjoint solve reverse-mode differentiation.
13
Algorithm 2 Analytic gradient computation
Input: Learning objective 44()), (x()); model fd fheurat continuous trajectories (x" (1), Result: Gradient 0H
/00. /80.
I: æ Forward rollout of fá Faeeraa using (6);
2: Compute V 24 V F V VayIN IN. Vel, Vefa feeuraaii
3: À + Reverse rollout of H using 17
4: OH 0H/000 Compute gradient using
18
Algorithm 3 Training neural ODEs with external inputs
Input: Learning objective (x (-); mini-batch size s; trajectories Diro = (x) t).
Result:
Neural ODE fd aenural. Initialize: Network parameters 0: slice Dera into M segments (Dfra,.., Mr with length each.
repeat
2: for I:e } in (Dima,.. .M. do
3: Compute analytic gradient 0H /00 using Algorithm 2
4:
Compute learning rate a using Adam or other methods;
5:
O - 0 a 0H /00:
6:
end for
7:
until convergence
The gradient computing only supports for a single continuous state trajectory, and the computational complexity scales linearly with the trajectory length. However, in real-world applications, multiple
14

---

Published as a conference paper at ICLR 2025
trajectory segments with long horizon might be produced. We introduce mini- mini-batching well as stochastic optimization methods to deal with the drawback, as summarized Algorithm 3 The learning rate could be determined using Adam or other stochastie gradient descent-related methods.
A.2 PROOF OF THEOREM 1
A.2.1
CONTINUOUS-TIME STABILITY
The proof procedure requires the Lyapunov stability analysis arising from the traditional control field (Sotinee al. 1991). At first, define Lyapunov function
((t) = 1
(t))
(19)
Differentiate V(t) E IR, yielding
(t) = ē(t) 8 (t)
5(1)" (
-a((t) Lā(t) + +0(1) Af(t)
Am (L) a(t)² a(t) 22m (L)
1
(20)
2
where (a) and (b) are driven by substituting
p and using Young's inequality T ATAf(t) (t) s
Am [m(L]]] (m S Am be rendered that
[
27 L) respectively. By combing (19 and 20 it can
2
-m 2Am (L) (I)
(21)
By solving the first-order ordinary differential inequality, one can achieve 0 V(t) S AAm( (V(0) - 5] + 5
(22)
with [2 (272 E It can be further implied that
lim | S
7
(23)
Xm,
converge bounded set B, = (m(t) E I= ||ā(t)] I /m (L)) with the feedback modification. can be seen that the upper bound be regulated arbitrarily small by increasing Am (L).
which shows that even with learning residuals, the state observation
Finally, from 9 it can be concluded that the derivative of the state observation error can also converge bounded set B- (t())) IR" | $ (A) (L) + 7) with the maximum
eigenvalue of feedback gain Am (L).
Figure ST shows the convergence of the state observation error in the spiral curve example. Related simulational setup the same Figure It can be that the theoretical bounded set is relatively conservative, as the result of the sufficiency of Lyapunov theorem.
As for unbounded learning residuals violating Assumption I we think it is still a major challenge in learning fields. It reveals that neural networks have completely lost the representational ability to target uncertainties. The best strategy may be retraining the networks based on fresh datasets, like an online continual learning mission (Ghunaim et al. 2023).
A.2.2 DISCRETE- TIME STABILITY
As the developed procedure is discrete-time, we further provide the convergence analysis in discrete-time form.
15

---

Published as a conference paper at ICLR 2025
Convergence of state observation error
0.25
Ama L
Transient
0.10
0.00
02
06
16
Feedback NN
15
[s]
Figure S1: In the spiral curve example, the state observation error can converge the theoretical bounded set.
Discretizing (2. it can be obtained that
xxtx) xt (ta 1) + +T. (ta((t-))
(24)
Define state observer error Z(ta) = x((x) (txx). By making a difference between 24 and (8) one can achieve
((x)) ((4=11) 1) + (f (4x-rat(4x-1)
(c) ((t+-1) + Tá neurat (turai(tx-1) - fneurai(ta-1) + Af(t))
(I T, (, + (T,A(())
(25)
where (c) and (d) are driven by substituting (3) and 7 respectively. With the bounded Assumption
I if the observer gain L makes
D) stable, i.e., p( T)L) 1, system (25) is input-to-state
stable (ISS) (Yan et al. 2023). p(-) denotes the spectral radius.
A.3
IMPLEMENTATION DETAILS OF SPIRAL CASE
A.3.1 SPIRAL DYNAMICS
The adopted spiral model is formalized as
d (1)
w
x (1) +
(26)
w
7/
with period w E R, decay rate R, and bias E R.
In tests, the initial value is set as (0) = 19, 0 For the nominal task, w, n, and e are set as 2, 0.1, and 0, respectively.
A.3.2 TRAINING DETAILS OF NEURAL ODE
The adopted MIP for training ODE has layers with 50 hidden units and ReLU activation functions. The training datasets consist of 1000 samples, discretized from 0 s to 10 8 with 0.01 8 step size. In training, RMSprop optimizer with the default learning rate of 0.001. The network trained with batch size of 20 for 400 iterations.
A.3.3 TRAINING DETAILS OF FEEDBACK PART
As for the feedback part, we adopt MLP with hidden layers with 50 hidden units each and ReLU activation functions. The training datasets are collected through domain randomization, with 20 randomized cases, i.e., w = {0.8 +0.12 3.08]. 7 = {0.04 +0.005 0.135), 5 = { (-24 +2.4 21.6). Each case consists of 1000 samples, discretized from 0 to 20 with 0.02 step size. In training, we use RMSprop optimizer with the learning rate of 0.01. The network is trained with batch size of 100 for 2000 iterations.
16

---

Published as a conference paper at ICLR 2025
A.3.4 SETUP OF GAIN ADJUSTMENT TEST
Figure shows the ablation study on linear feedback gain and degree of uncertainty. In this test, feed- back gain is selected from {0 +5 45] in order, and uncertainties are set as w {0.8 +0.4 : 4.4), 7 {0.04: +0.02 0.22]. E {-24 +8 96) in order. The prediction step is set as 50. The prediction results are evaluated using the means of 2-norm prediction errors.
A.4 IMPLEMENTATION DETAILS OF TRAJECTORY PREDICTION OF IRREGULAR OBJECTS
The input state neural ODE consists of position and velocity. The adopted MLP for training latent ODE has 3 hidden layers with 100 hidden units each and ReLU activation functions. The training datasets consist of 21 trajectories, with 1058 samples each. The step size is 0.001 s. In training, we Adam optimizer with the default learning rate of 0.001. The network is trained with batch size of 20 for 1000 iterations.
Different from the one-step prediction strategy utilized in Jia al. (2024) (modeled dynamical systems concerning attitude), this work predicts future states in a forward-rolling way, learning a more precise result. For the compared drag model-based method, the drag coefficient comes from Jia et al. (2024) fitted by least squares. The prediction error in Figure 7 evaluated by the 2-norm of position prediction error.
A.5 IMPLEMENTATION DETAILS OF MODEL PREDICTIVE CONTROL A QUADROTOR
A.5.1 QUADROTOR PRELIMINARIES
A quadrotor dynamics be defined as state-space model with 12-dimensional state vector a = [p. v, θ w) E IR 12 and 4-dimensional input vector u = [7,, 72, T3, T4] E IR of motor thrusts. Two coordinate systems are defined, the earth-fixed frame E {X E Y E. Zæ} and the body-fixed frame B = { X B, YB, }.}. The position p E R$ and the velocity E R while the body rate w R is defined in B. The relationship between & and decided by the Euler angle θ E R The translational and rotational dynamics can be formalized as
are defined in
p = TT
T + gZg
é W W [7. T = C(Ti, T2, Ts, Ta]
Jō - (Jw) +T
(27)
where g stands for the magnitude of gravitational acceleration, W(-) refers the rotational mapping matrix of Euler angle dynamics and C is the control allocation matrix. We note the nominal dynamics of quadrotor as = f(x u..
Next, differential flatness-based controller (DFBC) (Mellinger & Kumar 2011 for the quadrotor is introduced, which adopted here to form closed-loop system for end-to-end learning that remains stable and differentiable numerical integration. By receiving the flat outputs V [p. v, a,j] E 1 the positional signal and its higher-order derivatives, as the command signal, DFBC computes the desired motor thrusts for the actuators under the 12-dimensional state feedback. By virtue of the differential flatness property of the quadrotor, one can covert the flat outputs into nominal states æ and inputs u using related differential flatness mappings if the yaw motion remains zero. We note this controller as [ ui] π 22 ær, ) where z is auxiliary state of controller for the expression integrators and approximated derivatives in the rotational controller.
A.5.2 IMPLEMENTION OF LEARNING AERODYNAMICS EFFECTS
In training, no external wind and parameter uncertainties exist, and the aerodynamic drag is modeled as RDR v (Faessler et al. 2017), where R refers to the current rotational matrix that maps the frame 13 to the frame ε, and D d0.6[[0.6, 0.6, 0.1]} is coefficient matrix which is fitted by real flight data from Dia et al. (2022).
A neural ODE f feeurat (with parameters 0) is augmented with the nominal dynamics to capture the acrodynamic effect, i.c., = a Z + gZ + frcaral (v. O) A MLP with hidden layers with 36 hidden units is adopted.
17

---

Published as a conference paper at ICLR 2025
End-to-ent learning of fneural could be done using the algorithm 3 but a stable numerical integration is necessary. A closed loop system of the augmented dynamics using DFBC is employed, noted as [tt =] ((xx, =] D ). In the proposed algorithm, (at =] turns out to be the new state and Tp becomes the auxiliary input instead of the input of the augmented dynamics ra.
We generate 40 T trajectories with the discrete nodes of 200 each for learning by randomly sampling the positional waypoints in a limited space, followed by optimizing polynomials that connect these waypoints, shown in Figure 8 For validations of the learned neural ODE, we generate another random & trajectories 2.5x longer than that used in training, the result illustrated in Figures S4 S7 indicates good prediction on all 12 states.
A.5.3 IMPLEMENTATION OF MPC WITH FEEDBACK NEURAL NETWORKS
MPC works in the form of trajectory optimization (88) with receding-horizon N with discrete dynamic model fa. obtain the current optimal control input uo, while maintaining feasibility constraints u, U, E X, i.e.,
N Σ (N xTy)
min mnN Mo N-]
1 (2, xt] + (u u!))
i=1
(28)
s.t. 0++1 fa(æ,, u:), 3o x00)
u, E U, 2; E E
where the objective functions (-) E (.),(() E (., E R penalize the tracking error between model predicted trajectory {Zi.N, M1:N) } and the up-comming reference trajectory { wi where quadratic loss are often adopted. In this application, we make (.) (x(+) (x-x") x:
h
x²), (.) = (u x²) R(u u"), where Q diag[[100g [1000x11 50 13x1]} R 12x12 and R R= [1411
{ E 2244 The feasibility constraints u, E U and æ; E X are normally designed using box constraints. We make 04x1 I 44x1 to avoid control saturation and | < π 23x1 to avoid singularities while using Euler angle-bate attitude representation. The receding horizon length N is set to be 10.
The key idea of using a feedback neural network augmented model is to apply the multi-step prediction mechanism to the model prediction process in MPC. The multi-step prediction algorithm requires the current feedback state Eo and current input u1 to update the sequence of .i.N- The updated 1 can be directly applied for the next receding horizon optimization. We choose a linear feedback gain of L- (3iag(32xx) E IR 12x12 with decay rate of 0.1.
A.5.4 BENCHMARK COMPARISONS
In the quadrotor example, in order show the effectiveness of the proposed feedback neural network, five other models are compared: the nominal model (27). the neural ODE augmented model (Section A.5.2), the feedforward neural network augmented model (Saviolo Loianno 2023), the feedback enhanced nominal model, and the adaptive neural network augmented model (Cheng et al. 2019). abbreviated as Nomi-MPC NeUral-MPC, MLP-MPC, FB-MPC, and AdapNN-MPCC for the sake of simplification.
The MLP augmented model employs the fully connected neural network to learn aerodynamic drag
(Saviolo & Loiianoo 2023). The feedback enhanced nominal model refers to the analytic model (27) strengthened by proposed feedhack mechanism. The adaptive neural network augmented model
(Cheng et al.) 2019) uses the feedforward neural network to learn acrodynamic drag in which the
last layer is regarded as a weighted vector, being adjusted adaptively according to real-time state feedback. Similar idea is also proposed in Connell et al. (2022); Richards et al. (2023); Saviolo et al. (024). In tests, all learning- based methods have the same hidden layers, and the parameters of the AdapNN-MPC are adjusted for optimal performance.
The training loss on the training set, the validation set, and the test set of MLP augmented model is provided in Figure S13
18

---

Published as a conference paper at ICLR 2025
A.5.5 TEST RESULTS
A periodic 3D Lissajous trajectory is used for comparative tests, where variety of attitude-velciity combination is exploited. The position trajectory can be written as p(t) = (rxsin(2xt/Ta). /T T rysin(2nt/Tp))
+ r=cos((xx/TT)) where the parameters are 12,,"', T2. Tz, Ty. T., h] = [3.0, 3.0, 0.5, 6.0, 3.0, 0.5). Tracking such trajectory requires con- version of the flat outputs to the nominal 12-dimensional state a of the quadrotor using differential flatness-based mapping.
During trajectory tracking, it could be seen from Figure S8] that the prediction accuracy of latent dynamics at the first step improved significantly under the multi-step prediction. Although the learning-basse model provides more solid results on dynamics prediction than just using the nominal model, with the help of feedback, convergence property of prediction error can be achieved, leading to better tracking performance (Figure
A.6 ABLATION STUDY
Section 3.4 has analyzed the sensitivity of observer gain at different levels of uncertainties. In this part, we further conduct the ablation studies on linear and nonlinear neural feedback units, and decay rate.
A. LINEAR FEEDBACK UNIT
We test the performance of correcting the latent dynamics of spiral curves at 12 different levels of learning residuals, with or without linear feedback unit. The parameter uncertainties cover
Aw {-0.72 +0.12 0.6), An {-0.03 +0.005 0.025), As {-14.4 +2.4 12). All compared results are summarized in Figure S9 which indicates the effectiveness of the linear feedback unit.
A.6.2 NEURAL FEEDBACK UNIT
Similar to the last test, we further test the performance of the neural feedback unit at 12 different levels of learning residuals. All compared results are summarized in Figure STO] It can be found that the developed feedback neural network shows better generalization performance by enabling the neural feedback unit.
It can be seen from Figure $9 and Figure S10 both methods can achieve comparative learning performance. Compared with the linear feedback, no prior gain tunning is required for the neural feedback at the cost of training cost.
A.6.3 DECAY RATE
Ablation study on decay rate: The performance of the decay rate is examined in the spiral curve example. In tests, the decay rate is set as {0 +0.01 0.06) in sequence, and the multi-step prediction errors (Figure can be seen that the prediction error decreases with the increase of 3 at the beginning due noise mitigation. However, as 3 continues to increase, the convergence time becomes slower, leading to gradual increase prediction error.
are calculated in RMSE The test results are shown in the Figure STT
In practice, the tunning of feedback gain and decay rate is very intuitive. They can be increased slowly from small value until the critical value with the best estimation performance is reached.
A.7 TRAINING COST
For the training of neural ODEs, two strategies are employed in this work: the adjoint sensitive method developed in Chen et al. (0018) without considering external inputs, and the alternative training method developed in Appendix A.I concerning external inputs. The adjoint sensitive method utilized in the spiral curve and irregular object examples, and its computational resource and training time the same as Chen al. (018).. The alternative training method concerning external inputs is employed the quadrotor example to learn residual dynamics. takes around 30 mins to run 50 epochs laptop with 13th Gen Intel(R) Core(TM) 19- 99-13900H. The alternative training
19

---

Published as a conference paper at ICLR 2025
method is derived from the view of optimal control, and its computational resource and training time are comparable to the adjoint sensitive method theoretically.
Two feedback forms presented. No prior training is required for the linear feedback form, showing its advantage over traditional learning-bast generalization methods. Moreover, the linear feedback consists of several analytic equations, consuming almost computing resources. As for the neural feedback form, due to the optimization problem being non-convex, a satisfactory result usually takes 10 mins hour of training time on laptop with Intel(R) Core(TM) Ultra 185H 2.30 GHz.
A.8 COMBINATION WITH ADAPTIVE NEURAL ODE
In this part, we further explore the combination potential of the developed linear feedback neural network with the test-time adaptation (Cheng et al. 2019 O' O'Connell et al. 2022 Richards et al.
2023
Saviolo et al. X R T X R' represents front layers of neural network, and x E R' denotes the
).24). Let f neuratt (x)(t) [(t), 1, 0)
E (()()). [()), L, 0)x. where E (-))
weighted vector of the last layer of neural network, which is constant in test case and can be adjusted
adaptively according to real-time state feedback. Integrating with the adaptive scheme, can be adjust to
fncurat (1) = = + L(x((1) #()))
(29)
where X is updated through an adaptive law
X TE (t)ā(t)
(30)
with positive definite observer gain r
E R x TR.
Theorem 2. Consider the nonlinear system (.). Under the linear state feedback (29.. the adaptive
law (30). and the bounded Assumption [ the state observation error Ø(t) and its derivative (t))
(i.e., (() can exponentially converge to bounded sets B1 = (m((t E IR" ( S 7/>m (L)} and
B2 (t)) E 12". | S 7)mr(L)/) (L) + 7. respectively, which can be regulated by L.
Proof. Define the estimation error x = X x and a Lyapunov function
1 V(t)= a(t) a(t) x(t) + 2 x
1
(31)
Differentiate V()) E R. yielding
À (1) = a()) (#) (1) (1)) T &
(c) -(1)" --(() ((((()==(1)((()))) (E(I + AS(() -X=="
--1" Lā(() + 5(1) Af()
(32)
where (e) is driven by substituting (20). which is omitted here.
29 and 30 The following proof process consistent with
further increases the flexibility of the neural network by inducing the adaptive mechanism. We further test this scheme in the quadrotor example, as shown in [S1] Note that the feedback gain set the same as that of the previous feedback neural network. can seen that the adaptation- -enaanccd feedback neural network (abbreviated as AdapFNNN achieves performance comparable the previous feedback neural network, with a slightly larger RMSE.
Compared
We think that the possible reason why the AdapFNN does not bring significant performance im- provement that the last layer of the neural network not trained analytically. In other words, the uncertainty of the test scenario is not reflected in the last layer. The bilevel training strategy (O' Connell et al. 2022 Richards et al. 2023) may help improve AdapFNN's performance.
20

---

Published as a conference paper at ICLR 2025
2.0
[m]
10
Truth
15
1.0
0
0.5
Feedback NN
0.3
1.0
y(m)
0.5
0.5
/mm
0.0
0.5
0.5
y1m)
1.0
[m]
100
[m]
1.0
[m]
0.0
2
[mm)
2mm]
2
1.0
10
1.5
0.5
v/ml
0.5
fm]
0.5
1.0
0.5
y/m)
[m]
* [m]
1.0
[m]
1.5
2
x(m]
[m))
2
[(m]
1.0
100
0.5
[(m]
1.0
0.5
1mml
0.5
* x/m)
1.0
0.5
vlm)
0.0
[mm]
0.5
Y/m]
Figure S2: Trajectory prediction results of all test trajectories in Section predicted trajectories almost overlap with the truth ones.
5.1
It can be seen that the
21

---

Published as a conference paper at ICLR 2025
0.0
Auzeleration (m
-2.5
Acceleration Im Vs)
-25
Truth
Accelers Acceleration Ietaattion (m
-5.0
Learnty
Truth
-7.5
-7.
Truth
10.0
10.0
Learntz
0.00
0.25
0,50
0.00
0.25
0.0
0.00
0.25
0.50
[[s]
[[5]
[[5]
0.0
0.0
Accelrrttoo [m*1s]
Acceleratee ration [m²]s]]
-2.5
-2.5
Acceleration Im
-5.0
5.00
-7.5
-7.5
-10.0
-10.0
0.00
0.50
[[5]
[[s]
[[5]
Acceleration [m+]s]
-2.5
[m3
Accelee niceleration Im²
-5.0
-10.0
-12.5
0.00
0.25
0.50
0.00
0.25
0.50
0.00
0.25
0.50
[[s]
Figure S3: The learning performance of latent accelerations of all 9 test trajectories in Section 5.1 It can be seen that the feedback neural network accurately capture the latent dynamics of test trajectories out of the training set.
-1
3
1
-2
7m/l
0
7/ml
0
[m]
o
x [m]
2
[(m]
x [m]
Reference
Prediction
Truth
Figure S4: random trajectories generated for validations of the learned neural ODE, named traj-#1, traj-#2, and traj-#3. All trajectories show well-predicted motions pose and attitude. Detailed results on all 12 states are provided in Figures S7
22

---

Published as a conference paper at ICLR 2025
0.2
0.10
Pos
Vel
Pitch
Omogax
0.0
-2
-0.05
-0.2
100
100
0.2
0.5
0.5
0.0
Omegay
Poxy
-0.5
Vely
Pitch
0.0
-0.2
-1.0
100
0.00
0.02
Omrgaz
-0.01
-1
0.00
-0.02
-2
Prediction
Truth
Figure S5: Validation of learned neural ODE. Prediction on all 12 states of traj-#1.
W
0.2
Omega
Pos
Pitch
0.0
0.0
-0.2
100
100
100
0.2
W
W
Omegay
vely
Pitch
0.0
0.0
-1
-02
-0.5
-2
100
0.04
m
2minga?
pasz
Wal
Yaw
0.02
0.000
-0025
-1
Prediction
Truth
Figure S6: Validation of learned neural ODE. Prediction on all 12 states of traj-#2.
23

---

Published as a conference paper at ICLR 2025
N
0.2
W
0.2
Omega
Piss
velk
Pitch
0.0
0.0
-0.2
-0.2
100
100
100
100
0.2
Omoga
Pos
well
Pitch
0.0
0.0
0.1
-0.2
100
100
100
100
M
0.04
Omoga
Pos
Vall
Yaw
0.00
0.02
0.00
100
0
100
100
100
Prediction
Truth
Figure S7: Validation of learned neural ODE. Prediction on all states of traj-
(a) Nomi-MPP
(b) NeUral-MPC
(c) FNN-MPC
Acceleration [m²/s] Acceleration y [m/ [m3/s] /s] Acceleration (m3 /s]
2.5
-2.5
-5
200
200
200
20
10-
-10
-20
200
200
0
200
-5
-5
-5
-10
-10
200
0
200
0
200
Prediction
Truth
Figure S8: Test on the Lissajous trajectory. Prediction on the translational latent dynamics (i.e., acceleration) at the first step using different prediction models. The feedback neural network augmented model achieve the best prediction performance.
24

---

Published as a conference paper at ICLR 2025
Feedback
Feedback
Feedback
20
-0.72, Arg
-0.08
-144
-0A, dap
-0,005
-0.48, = -000,
Neural
Feedback neural
Neural
Feedback network
Neural Feedback neural network
10
-0.360
Neural ODE
Feedback neural network
Neural ODE
Feedback neural network
Neural ODE
Feedback neural network
0.4
02
de
0.24,
Neural ODE
Feedback neural network
Neural ODE
Feedback neural network
Neural ODE
Feedback neural network
10
10
0.86 0.015,
0.48, dap
(16, Aay 0.025,
Figure S9: Test the linear feedback form on 12 randomized cases in the spiral curve example. can be found that the developed feedback neural network shows better generalization performance as enabling the lincar feedback unit.
Feedback neural network
Neural
Feedback neural network
Feedback network
20
10
-0.72, Ag
-06,4a
-0.40
-0.0,
Neural ODE
Feedback neural network
Neural ODE
Feedback neural network
Neural ODE
Feedback neural network
10
5.0
Neural ODE
Feedback neural network
Neural ODE
Feedback neural network
Neural ODE
Feedback neural network
10
Au 0,
Ag
Neural ODE
Feedback neural network
Neural ODE
Feedback neural network
Neural ODE
Feedback neural network
10
0.65,
0.6,
Figure S10: Test the neural feedback form on randomized cases in the spiral curve example.
The developed feedback neural network with the neural feedback unit shows better generalization performance than the neural ODE.
25

---

Published as a conference paper at ICLR 2025
Ablation study on decay rate
4,0
3.5
Prediction error- RMMSE
3.0
2.0
1.5
1.0
0.5
0.0
0.00
0.01
0.02
0.03
0.04
0,05
Decay rate
Figure S11: Ablation study on decay rate.
Testing performance Neural ODE
(b)
Testing performance Feedback NN
20
15
15
10
10
deep banc
Step disturbance
5
Truth
Truth
-40
-10
0
20
-40
-20
20
X
Figure S12: Test on step disturbance in the spiral curve example. The test setup is the same as Figure 5 eexcept for the induce of step disturbance. As f 7 (denoted by a yellow diamond symbol), the latent dynamics is changed suddenly (w : 3 + 1. v -0.05 + -0.12, & : 10 > 5). It can be seen that the proposed feedback neural network can attenuate the step disturbance quickly.
0.3
Train
Val
Test
0.2
Loss
0.1
0.0
100
200
300
400
500
Epochs
Figure S13: The training loss on the training set, the validation set, and the test set of the compared MLP augmented model.
26

---

Published as a conference paper at ICLR 2025
FNN-MPC
AdapFNN-MPC
Tracking error [m]
1.0
-10
0.5
3
005
005
002
0.0
-2
7/m)
(m)
-2
Ami -2
2
[mm
0
2
FNN-MPC
AdapFNN-MPE
RMSE [m]
0.093
0.108
Figure S14: Tracking the Lissajous trajectory using MPC with adaptation-enhanced feedback neural network.
27


Reviews:
[{"review_id": "Wcrg0CYq1E", "reviewer": "Reviewer_q2Ng", "summary": "This paper studies the challenge of enhancing generalizability and adaptability in Neural ODEs for continuous time-series prediction tasks. It introduces feedback mechanisms into the Neural ODE framework, which dynamically adjust model parameters in response to errors in predictions. The study begins with a theoretical analysis, providing a convergence analysis of linear feedback, and extends to a practical implementation where a neural network learns the feedback mechanism. Experimental results highlight the improved performance of the neural feedback mechanisms over traditional ODE models.", "questions": "1) Review Related Work on Feedback Adaptation in Time-Series Prediction [1,2].  \n2) Explore the Relationship with Continual Learning [2,3,4].  \n3) Perform Experiments on Linear Feedback.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1) The paper studies the critical issue of adaptability and generalizability in continuous-time prediction tasks using Neural ODEs, which is essential for real-world applications.\n2) By incorporating feedback mechanisms, the approach potentially enhances the adaptability and generalizability of Neural ODEs, making it a contribution to the field.\n3) By conducting several experiments, the proposed method demonstrates superior performance across various trajectory and dynamics prediction tasks.", "weaknesses": "1) **Comparison with Existing Feedback Mechanisms**: While feedback mechanisms are commonly employed in other neural network-based time-series prediction tasks, this paper does not compare its approach to these established methods. For example, studies such as [1,2] use feedback mechanisms inspired by the Kalman filter and Extended Kalman Filter (EKF) to adapt neural networks for online time-series prediction. Including a comparison with these relevant works would be.\n\n2) **Connection to Continual Learning**: The claims about generalization across new tasks while maintaining performance on previous tasks touch upon the principles of continual learning. A discussion of related works in continual learning, particularly focusing on online continual learning [3] and test-time adaptation [2,4], would provide a richer context.\n\n3) **Evaluation of Theoretical Claims and Linear Feedback**: The paper presents a theorem on the convergence of linear feedback to a bounded error but does not empirically evaluate this claim. Designing a simple toy experiment to test this theorem and discussing the practical implications of the assumptions would enhance the credibility and thoroughness of the research. In addition, reporting the results of Linear feedback on several trajectory prediction tasks in the experiment will improve the quality of the paper.\n\n4) There are minor typos and notation errors that need correction for clarity. Line 167: $\\dot{z}=[\\hat{f}^T,\\hat{f}^T]^\\top$ -> $\\dot{z}=[f^T,\\hat{f}^T]^\\top$ ; Line 257：  'sate'  ->  'state'.\n\n[1] Y. Cheng, et al. \"Human motion prediction using semi-adaptable neural networks.\" 2019 American Control Conference (ACC). IEEE, 2019.  \n[2] A. Abuduweili,  et al. \"Robust online model adaptation by extended kalman filter with exponential moving average and dynamic multi-epoch strategy.\" Learning for Dynamics and Control, 2020.  \n[3] Y. Ghunaim, et al. \"Real-time evaluation in online continual learning: A new hope.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023.  \n[4] J. Liang, et al. \"A comprehensive survey on test-time adaptation under distribution shifts.\" International Journal of Computer Vision (2024): 1-34.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1HfUUA81j5", "reviewer": "Reviewer_nZLP", "summary": "The paper introduces *feedback neural networks*, enhancing the generalization of neural ODEs in continuous-time tasks by adapting learned dynamics through real-time feedback. Inspired by biological systems, the network employs a two-DOF structure with linear feedback (whose convergence is theoretically analyzed) and nonlinear feedback learned via domain randomization. Unlike prior methods, it achieves robust generalization without sacrificing accuracy. Experiments, including trajectory prediction and quadrotor control, demonstrate its improvements over existing methods.", "questions": "See the weaknesses part.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Clear and promising motivation and concise writing that trace the development of integrating Neural ODEs with feedback structures, progressing from linear to nonlinear feedback.\n2. Theoretical analysis is presented for the linear feedback case, demonstrating that prediction error converges to a bounded set with an appropriately chosen feedback gain.\n3. Extensive illustrations on simple cases effectively demonstrate their method’s performance compared to standard Neural ODEs.\n4. In addition to toy experiments on trajectory prediction, they conduct practical quadrotor tracking experiments, incorporating learned prediction models within an MPC controller, where the proposed FNN-MPC shows superior performance over alternative approaches.", "weaknesses": "I have limited experience with Neural ODEs, so I’m not fully confident in assessing the contribution to this area. My main concerns, however, relate to the experimental section. While the current experiments effectively showcase the improvements of FNN, the experimental setup is relatively simple and lacks key baselines in learning dynamics models beyond Neural ODEs, such as [1-3]. Furthermore, it is unclear how flight trajectories were collected and how tracking performance was evaluated. Given the complexity of quadrotor dynamics and aerodynamics, these aspects significantly impact the difficulty of the task and the validity of the results. Although a preliminary overview of quadrotor dynamics is provided in the appendix, additional discussion on environmental conditions, such as wind fields and airflows [4], would help clarify the experimental setting.\n\n[1] Leonard Bauersfeld, et al. NeuroBEM: Hybrid Aerodynamic Quadrotor Model. RSS 2021.\n\n[2] Alessandro Saviolo and Giuseppe Loianno. \"Learning quadrotor dynamics for precise, safe, and agile flight control.\" *Annual Reviews in Control* 55 (2023): 45-60.\n\n[3] Pratyaksh Prabhav Rao et al. \"Learning Long-Horizon Predictions for Quadrotor Dynamics\". IROS 2024 \n\n[4] Leonard Bauersfeld, et al. \"Robotics meets Fluid Dynamics: A Characterization of the Induced Airflow below a Quadrotor as a Turbulent Jet\".", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7BV4a7FdMM", "reviewer": "Reviewer_a2wg", "summary": "This paper introduces closed-loop feedback neural ODEs which offer better generalization performance compared to open-loop neural ODEs. A linear state-error feedback term is introduced to compensate for prediction inaccuracies in the closed-loop error mechanism. This feedback term is shown to improve the generalization performance which was poor in the previous open-loop neural ODE formulations.", "questions": "I do not have further questions or suggestions beyond the comments I made above. The authors are suggested to carefully examine all of my above comments. I commend the authors for this interesting work and look forward to reading their responses.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "I believe this paper makes a strong contribution. The paper is well-written and easy to follow. The technical development with the use of closed-loop state error feedback term is well-motivated. I found the paper an interesting read.", "weaknesses": "Despite the significant contributions, there are quite a few mathematical errors and a general lack of mathematical rigor in the paper. These errors might be minor and easy to resolve, I suggest the authors address the following issues.\n\n1. Equation (7) appears erroneous. Combining (4) and (6) should actually give the term $L(\\bar{x}(t)-\\hat{x}(t))$, not $L(x(t)-\\hat{x}(t))$. This error gets carried forward to Eq. (9) of the convergence analysis, thus potentially invalidating Theorem 1. As far as I can help, this issue can be avoided redefining Eq. (6) in terms of $x(t)$ instead of $\\bar{x}(t)$. In that case, Eq. (8) needs to be adjusted accordingly. Alternatively, the authors can add and subtract $Lx(t)$ term in the derivation of Eq. (9), which should result in an extra $L(x-\\bar{x})$ term in (9). This term will need to be bounded and that bound would need to be considered in the ensuing Lyapunov-based stability analysis to obtain the correct error bounds. Either way, the mathematical analysis needs revisions to address the error in Eq. (7).\n\n2. The development in Eq. (4)-(8) is in a discrete-time setting but the convergence analysis in Section 3.2 is in a continuous-time setting. To be consistent, the authors are suggested to either reformulate the development in Section 3.1 using continuous-time integrals instead of discrete-time summations, or rederive the convergence analysis in Section 3.2 using discrete-time Lyapunov theorem or contraction analysis. I believe this should be easy to address.\n\n3. Although the proof of Theorem 1 is correct, it appears the analysis is not performed in the most efficient manner. Specifically, the use of Young's inequality in Eq. (2) is not the most efficient because it results in the $\\lambda_m(L)-\\frac{1}{2}$ term which carries forward through the analysis. The problem with this term is that the gain condition $\\lambda_m(L)>frac{1}{2}$ would need to be imposed to enure an exponential decay. Such a gain condition is missing in the paper. However, changing the analysis in Eq. (20) would remove the need for a gain condition. I suggest the authors use the following form of Young's inequality instead\n\n$$ \\tilde{x}^T \\Delta f(t) = \\sqrt{\\lambda_m(L)}||\\tilde{x}||\\frac{\\gamma}{\\sqrt{\\lambda_m(L)}}=\\frac{\\lambda_m(L)||\\tilde{x}||^2}{2}+\\frac{\\gamma^2}{2\\lambda_m(L)}.$$\n\nThis would result in \n\n$$\\dot{V} \\leq -\\frac{\\lambda_m(L)||\\tilde{x}||^2}{2}+\\frac{\\gamma^2}{2\\lambda_m(L)}.$$\n\nThis formulation would eliminate the need of an extra gain condition, while also obtaining a tighter bound in the $\\delta$ term.\n\n4. The purely time-dependent exponentially decaying gain schedule for the feedback gain in Eq. (11) can be problematic in terms of the performance, because the theoretical error bounds in Theorem 1 contain $\\lambda_m(L)$ in the denominator. The bounds get enormous with the exponential decay, thus essentially making the neural ODE open-loop with time. There are better ways to schedule the gain using closed-loop feedback. The authors are recommended to look into recursive least squares algorithms that are popular in the adaptive control or Kalman filtering literature, where the gain decreases as the estimator gains more information instead of scheduling it to always decay with time. \n\n5. This is minor but the technical writing throughout the paper lacks mathematical rigor. Specifically, the mathematical entities (states, functions, constants etc.) are not formally defined. For example, the domain of x can be precisely defined, i.e., $x(t)\\in\\mathbb{R}^n$. The function $f$ can be defined formally as $f:\\mathbb{R}^n\\times\\mathbb{R}^m\\times\\mathbb{R}\\to \\mathbb{R}^n$. The notations $f_\\{neural}(t)$ and $\\hat{f}_\\{neural}(t)$ (which are not formally defined) are confusing because they take only time as the argument. However, they also seem to depend on x and \\hat{x}. The term $L$is defined to be a constant, but it should actually be a positive-definite matrix. The concatenated state $z$ needs to be defined similarly with an appropriate domain. In the statement of Theorem 1, instead of stating \"the state observation error and its derivative can converge to bounded sets exponentially, which upper bounds can be regulated by the feedback gain L\", the authors should explicitly state the obtained exponentially convergent bounds.\n\n6. Minor issue, but Figure 1 seems to indicate the error is being passed through another neural network instead of a linear feedback gain, which is not the case in this paper. The figure needs to be revised replacing this second neural network with a -L gain block.\n\nEDIT: Increasing the score to 8 after author responses.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "exMV3oUKPJ", "reviewer": "Reviewer_Lpz2", "summary": "This paper successfully enhanced the generalization capability of Neural ODEs by introducing feedback mechanisms, demonstrating significant performance improvements across multiple practical applications. Specifically, the authors proposed a Two-Degree-of-Freedom (Two-DOF) neural network architecture that combines linear and nonlinear feedback mechanisms, effectively correcting prediction errors and exhibiting stronger adaptability and generalization capability when dealing with complex and nonlinear dynamical systems. Overall, it provides a new perspective and solution for the generalization problem of neural networks in continuous-time prediction tasks.", "questions": "1. **Automated Design of Feedback Gain and Decay Rate**:\n    - Could an adaptive mechanism be developed to dynamically adjust the feedback gain matrix \\(L\\) and decay rate \\(\\beta\\) based on real-time prediction errors or system states? This would reduce the need for manual parameter tuning and enhance the method's flexibility and adaptability. Alternatively, provide more detailed documentation on the specific effects of feedback gain matrix \\(L\\) and decay rate \\(\\beta\\).\n\n2. **Comprehensive Ablation Studies and Sensitivity Analysis**:\n    - Suggest adding ablation experiments on the independent effects of each component (such as neural feedback network, linear and nonlinear feedback), along with detailed sensitivity analysis of the feedback gain matrix \\(L\\) and domain randomization parameters. This would help better understand each part's contribution to overall performance.\n\n3. **Compare with More Baseline Models**:\n    - To more comprehensively demonstrate the advantages of the proposed method, it is recommended to compare with a wider range of baseline models, including other improved Neural ODE variants or feedback-based models, to validate its effectiveness across diverse scenarios.\n\n4. **Detailed Description of Computational Cost and Training Time**:\n    - Suggest adding a dedicated section in the paper to thoroughly discuss the performance of the proposed method in terms of computational resources and training time, including comparisons with traditional methods. This would help readers better evaluate its practical feasibility and efficiency.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 2, "presentation": 4, "contribution": 3, "strengths": "- By combining linear and nonlinear (neural feedback) forms of feedback, an innovative network architecture is proposed that enhances generalization capability in unseen scenarios while maintaining accuracy on previous tasks. This design demonstrates innovation and practical value in current research. The experimental section covers trajectory prediction and quadrotor control, validating the method's superior performance.\n\n- The paper provides theoretical analysis, proving system stability and convergence under linear feedback conditions. This establishes a solid theoretical foundation for the method's reliability and robustness. Additionally, the paper is well-structured with rigorous logic and clear writing. The figures and tables are intuitive. Detailed algorithm descriptions enhance the method's reproducibility.\n\n- Through domain randomization techniques, the nonlinear neural feedback component is learned independently, demonstrating an effective way to improve generalization without significantly increasing training burden. This method effectively extends the applicability of feedback mechanisms when facing unknown uncertainties.\n\n- The feedback loops can be easily integrated into existing trained Neural ODEs without requiring retraining, reducing implementation difficulties in practical applications and enhancing the method's feasibility and scalability. Furthermore, the method shows potential for online adjustment, further increasing its practical value. \n\n- Generalization ability is a key challenge for neural networks in practical applications, and the paper effectively improves Neural ODE's generalization performance through feedback mechanisms, addressing the performance degradation issue of traditional methods in unseen tasks. This research has significant implications for neural network architecture design and practical engineering applications, showing broad application potential.", "weaknesses": "1. **Oversimplified Convergence Analysis (Section 3.2)**:\n    - The convergence analysis is solely based on bounded learning residual error (Assumption 1), assuming an upper bound γ. However, in practical applications, system dynamics can be highly complex, making residual errors difficult to strictly bound and potentially time-varying or having more complex characteristics (such as unbounded growth or sudden disturbances). This idealized assumption limits the applicability of theoretical analysis in real scenarios.\n\n2. **Manual Tuning of Feedback Gain Matrix \\(L\\)**:\n    - The selection of feedback gain matrix \\(L\\) relies mainly on manual tuning, based on theoretical analysis and experimental results. This approach is both time-consuming and difficult to precise in complex systems, especially in high-dimensional systems or real-time changing environments where manual selection may not meet dynamic adjustment needs.\n\n3. **Training Strategy Issues in Neural Network Feedback Section**:\n    - The authors adopt a two-stage training method: first training the Neural ODE, then fixing parameters to train the feedback part. This approach may not be optimal as parameters of both networks might be coupled, preventing the overall model from achieving optimal performance under joint optimization. Could different training approaches be discussed?\n\n4. **Insufficient Ablation Studies**:\n    - Although the paper includes some ablation studies, they lack comprehensiveness and depth. For instance, there's insufficient in-depth analysis of individual components' impacts (like neural feedback network, linear and nonlinear feedback), and detailed sensitivity analysis of feedback gain matrix \\(L\\) and domain randomization parameters. Additionally, the limited variety of baseline models fails to cover more different types or improved Neural ODE variants, affecting the comprehensive demonstration of the method's advantages.\n\n5. **Lack of Detailed Discussion on Computational Cost and Training Time**:\n    - The paper's discussion of computational cost and training time is insufficient, failing to comprehensively address the proposed method's performance in terms of computational resources and training time compared to traditional methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)