Feedback Favors the Generalization of Neural ODEs

## Abstract

Abstract The well-known generalization problem hinders the application of artificial neural networks in continuous-time prediction tasks with varying latent dynamics. In sharp contrast, biological systems can neatly adapt to evolving environments benefiting from real-time feedback mechanisms. Inspired by the feedback philosophy, we present feedback neural networks, showing that a feedback loop can flexibly correct the learned latent dynamics of neural ordinary differential equations (neural ODEs), leading to a prominent generalization improvement. The feedback neural network is a novel two-DOF neural network, which possesses robust performance in unseen scenarios with no loss of accuracy performance on previous tasks. A linear feedback form is presented to correct the learned latent dynamics firstly, with a convergence guarantee. Then, domain randomization is utilized to learn a nonlinear neural feedback form. Finally, extensive tests including trajectory prediction of a real irregular object and model predictive control of a quadrotor with various uncertainties, are implemented, indicating significant improvements over state-of-the-art model-based and learning-based methods 3 3 3 The project codes will be made available upon publication. .

## 1 Introduction

Figure: Figure 1: Neural network architectures. Left: Neural ODE developed in Chen et al. (). Right: Proposed feedback neural network.
Refer to caption: extracted/5924549/Figs/diagram.png

Stemming from residual neural networks (He et al., ), neural ordinary differential equation (neural ODE) (Chen et al., ) emerges as a novel learning strategy aiming at learning the latent dynamic model of an unknown system. Recently, neural ODEs have been successfully applied to various scenarios, especially continuous-time missions (Liu & Stacey, ; Verma et al., ; Greydanus et al., ; Cranmer et al., ). However, like traditional neural networks, the generalization problem limits the application of neural ODEs in real-world applications.

Traditional strategies like model simplification, fit coarsening, data augmentation, and transfer learning have considerably improved the generalization performance of neural networks on unseen tasks (Rohlfs, ). However, these strategies usually reduce the accuracy performance on previous tasks, and large-scale training data and network structures are often required to approximate previous accuracy. The objective of this work is to develop a novel network architecture, acquiring the generalization improvement while preserving the accuracy performance.

Living beings can neatly adapt to unseen environments, even with limited neurons and computing power. One reason can be attributed to the existence of internal feedback (Aoki et al., ). Internal feedback has been shown to exist in biological control, perception, and communication systems, handling external disturbances, internal uncertainties, and noises (Sarma et al., ; Markov et al., ). In neural circuits, feedback inhibition is able to regulate the duration and magnitude of excitatory signals (Luo, ). In engineering systems, internal feedback indicates impressive effects across filtering and control tasks, such as Kalman filter (Kalman, ), Luenberger observer (Luenberger, ), extended state observer (Guo et al., ), and proportional-integral-derivative control (Ang et al., ). The effectiveness of feedback lies in its ability to harness real-time deviations between internal predictions/estimations and external measurements to infer dynamical uncertainties. The cognitive corrections are then performed timely. However, existing neural networks rarely incorporate such a real-time feedback mechanism.

In this work, we attempt to enhance the generalization of neural ODEs by incorporating the feedback scheme. The key idea is to correct the learned latent dynamical model of a Neural ODE according to the deviation between measured and predicted states, as illustrated in Figure 1. We introduce two types of feedback: linear form and nonlinear neural form. Unlike previous training methods that compromise accuracy for generalization, the developed feedback neural network is a two-DOF framework that exhibits generalization performance on unseen tasks while maintaining accuracy on previous tasks. The effectiveness of the presented feedback neural network is demonstrated through several intuitional and practical examples, including trajectory prediction of a spiral curve, trajectory prediction of an irregular object and model predictive control (MPC) of a quadrotor.

## 2 Neural ODEs and learning residues

A significant application of artificial neural networks␣‌centers around the prediction task., $\bm{x}(t)\mapsto\bm{x}(t+\Delta t)$. Note that $t$ indicates the input $\bm{x}$ evolves with time. Chen et al. () utilize neural networks to directly learn latent ODEs of target systems, named Neural ODEs. Neural ODEs greatly improve the modeling ability of neural networks, especially for continuous-time dynamic systems (Massaroli et al., ), while maintaining a constant memory cost. The ODE describes the instantaneous change of a state $\bm{x}(t)$

$$ $\displaystyle\frac{{d\bm{x}}(t)}{{dt}}=\bm{f}\left({\bm{x}(t),\bm{I}(t),t}\right)$ (1) $$

where $\bm{f}(\cdot)$ represents a latent nonlinear mapping, and $\bm{I}(t)$ denotes external input. Note that compared with Chen et al. (), we further consider $\bm{I}(t)$ that can extend the ODE to non-autonomous cases. The adjoint sensitive method is employed in Chen et al. () to train neural ODEs without considering $\bm{I}(t)$. In Appendix A.1, we provide an alternative training strategy in the presence of $\bm{I}(t)$, from the view of optimal control.

Given the ODE (1) and an initial state $\bm{x}(t)$, future state can be predicted as an initial value problem

$$ $\displaystyle\bm{x}(t+\Delta t)=\bm{x}(t)+\int_{t}^{t+\Delta t}{\bm{f}\left({ \bm{x}\left(\tau\right),\bm{I}\left(\tau\right),\tau}\right)d\tau}.$ (2) $$

The workflow of neural ODEs is depicted in Figure 1. However, like traditional learning methods, generalization is a major bottleneck for neural ODEs (Marion, ). Learning residuals will appear if the network has not been trained properly (e.g., underfitting and overfitting) or the applied scenario has a slightly different latent dynamic model. Take a spiral function as an example (Appendix A.3.1). When a network trained from a given training set (Figure 5 (a)) is transferred to a new case (Figure 5 (b)), the learning performance will dramatically degrade (Figure 5 (d)). Without loss of generality, the learning residual error is formalized as

$$ $\displaystyle\bm{f}\left({\bm{x}(t),\bm{I}(t),t}\right)=\bm{f}_{neural}\left({ \bm{x}(t),\bm{I}(t),t,\bm{\theta}}\right)+\Delta\bm{f}(t)$ (3) $$

where $\bm{f}_{neural}\left(\cdot\right)$ represents the learned ODE model parameterized by $\bm{\theta}$, and $\Delta\bm{f}(t)$ denotes the unknown learning residual error. In the presence of $\Delta\bm{f}(t)$, the prediction error of (2) will accumulate over time. The objective of this work is to improve neural ODEs with as few modifications as possible to suppress the effects of $\Delta\bm{f}(t)$.

## 3 Neural ODEs with a linear feedback

### 3.1 Correcting latent dynamics through feedback

Even though learned experiences are encoded by neurons in the brain, living organisms can still adeptly handle unexpected internal and external disturbances with the assistance of feedback mechanisms (Aoki et al., ; Sarma et al., ). The feedback scheme has also proven effective in traditional control systems, facilitating high-performance estimation and control objectives. Examples include Kalman filter (Kalman, ), Luenberger observer (Luenberger, ), extended state observer (Guo et al., ), and proportional-integral-derivative control (Ang et al., ).

Figure: Figure 2: The learned latent dynamics are modified through accumulative evaluation errors to approach the truth one.
Refer to caption: extracted/5924549/Figs/sketch.png

We attempt to introduce the feedback scheme into neural ODEs, named feedback neural networks, as shown in Figure 1. Neural ODEs have exploited latent dynamical models $\bm{f}_{neural}(t)$ of target systems in training set. The key idea of feedback neural networks is to further correct $\bm{f}_{neural}(t)$ according to state feedback. Denote $t_{i}$ as the historical evaluation moment satisfying $t_{i}\leq t$. At current moment $t$, we collect $k$ historical state measurements $\left\{\bm{x}(t_{0}),\bm{x}(t_{1}),\cdots,\bm{x}(t_{k})\right\}$, in which $t_{k}=t$. As portrayed in Figure 2, $\bm{f}_{neural}(t)$ is modified by historical evaluation errors to approach the truth dynamics $\bm{f}(t)$, i.e.,

$$ $\displaystyle\bm{\hat{f}}_{neural}(t)=\bm{f}_{neural}(t)+\sum\limits_{i=0}^{k} {\bm{L}\left({\bm{x}\left({{t_{i}}}\right)-\bm{\bar{x}}\left({{t_{i}}}\right)} \right)}$ (4) $$

where $\bm{L}$ represents the positive definite gain and $\bm{\bar{x}}\left({{t_{i}}}\right)$ represents the predicted state from the last evaluation moment, e.g., an Euler integration

$$ $\displaystyle\bm{\bar{x}}\left({{t_{i}}}\right)=\bm{x}\left({{t_{i-1}}}\right) +T_{s}\bm{\hat{f}}_{neural}(t_{i-1})$ (5) $$

with the prediction step $T_{s}$.

To avoid storing more and more historical measurements over time, define an auxiliary variable

$$ $\displaystyle\bm{\hat{x}}\left({{t}}\right)=\bm{\bar{x}}\left({{t}}\right)- \sum\limits_{i=0}^{k-1}{\left({\bm{x}\left({{t_{i}}}\right)-\bm{\bar{x}}\left( {{t_{i}}}\right)}\right)}$ (6) $$

where $\bm{\hat{x}}\left({{t}}\right)$ can be regarded as an estimation of $\bm{{x}}\left({{t}}\right)$. Combining (4) and (6), can lead to

$$ $\displaystyle\bm{\hat{f}}_{neural}(t)=\bm{f}_{neural}(t)+\bm{L}(\bm{x}(t)-\bm{ \hat{x}}(t)).$ (7) $$

From (5) and (6), it can be further rendered that

$$ $\displaystyle\bm{\hat{x}}\left({{t_{k}}}\right)=\bm{\hat{x}}\left({{t_{k-1}}} \right)+T_{s}\bm{\hat{f}}_{neural}(t_{k-1}).$ (8) $$

By continuating the above Euler integration, it can be seen that $\bm{\hat{x}}(t)$ is the continuous state of the modified dynamics, i.e., $\bm{\dot{\hat{x}}}(t)=\bm{\hat{f}}_{neural}(t)$. Finally, $\bm{\hat{f}}_{neural}(t)$ can be persistently obtained through (7) and (8) recursively, instead of (4) and (5) accumulatively.

### 3.2 Convergence analysis

In this part, the convergence property of the feedback neural network is analyzed. The state observation error of the feedback neural network is defined as ${\bm{\tilde{x}}}(t)=\bm{{x}}(t)-\bm{\hat{x}}(t)$, and its derivative ${\bm{\dot{\tilde{x}}}}(t)$, i.e., the approximated error of latent dynamics is defied as ${\bm{\tilde{f}}}(t)=\bm{{f}}(t)-\bm{\hat{f}}_{neural}(t)$. Substitute (1) and (3) into (7), one can obtain the error dynamics

$$ $\displaystyle{\bm{\dot{\tilde{x}}}}(t)=-\bm{L}{\bm{{\tilde{x}}}}(t)+\Delta\bm{ f}(t).$ (9) $$

Before proceeding, a reasonable bounded assumption on the learning residual error $\Delta\bm{f}(t)$ is made.

###### Assumption 1 .

There exists an unknown upper bound such that

$$ $\displaystyle\|\Delta\bm{f}(t)\|\leq\gamma$ (10) $$

where $\|\cdot\|$ denotes the Euclidean norm and $\gamma$ is an unknown positive value.

Then the following convergence theorem can be established.

###### Theorem 1 .

Consider the nonlinear system (1), Under the linear state feedback (7) and the bounded Assumption 1, the state observation error ${\bm{\tilde{x}}}(t)$ and its derivative ${\bm{\dot{\tilde{x}}}}(t)$ (i.e., ${\bm{\tilde{f}}}(t)$) can converge to bounded sets exponentially, which upper bounds can be regulated by the feedback gain $\bm{L}$.

###### Proof.

See Appendix A.2.
∎

Figure: Figure 3: The multi-step prediction.
Refer to caption: extracted/5924549/Figs/multi_prediction.png

###### Assumption 1 .

###### Theorem 1 .

###### Proof.

### 3.3 Multi-step prediction

With the modified dynamics ${\bm{\hat{f}}}(t)$ and current $\bm{x}(t)$, the next step is to predict $\bm{x}(t+\Delta t)$ as in (2). By defining $\bm{z}(t)=\left[\bm{x}^{T}(t),{\bm{\hat{x}}}^{T}(t)\right]^{T}$, from (8), we have $\bm{\dot{z}}(t)=\left[{\bm{\hat{f}}}^{T}(t),{\bm{\hat{f}}}^{T}(t)\right]^{T}$. One intuitional means to obtain $\bm{z}(t+\Delta t)$ is to solve the ODE problem with modern solvers. However, as shown in Theorem 1, the convergence of $\bm{\tilde{f}}(t)$ can only be guaranteed as current $t$. In other words, the one-step prediction result by solving the above ODE is accurate, while the error will accumulate in the long-term prediction. In this part, an alternative multi-step prediction strategy is developed to circumvent this problem.

The proposed multi-step prediction strategy is portrayed in Figure 3, which can be regarded as a cascaded form of one-step prediction. The output of each feedback neural network is regarded as the input of the next layer. Take the first two layers as an example. The first-step prediction $\bm{x}(t+T_{s})$ is obtained by $\bm{{x}}(t+T_{s})=\bm{{x}}(t)+\bm{\hat{f}}(\bm{{x}}(t),\bm{\hat{x}}(t),\theta)
T_{s}$. The second layer with the input of $\bm{{x}}(t+T_{s})$ will output $\bm{{x}}(t+2T_{s})$. In such a framework, the convergence of later layers will not affect the convergence of previous layers. Thus, the prediction error will converge from top to bottom in order.

Note that the cascaded prediction strategy can amplify the data noise in case of large $\bm{L}$. A gain decay strategy is designed to alleviate this issue. Denote the feedback gain of $i$-th later as $\bm{L}_{i}$, which decays as $i$ increases

$$ $\displaystyle\bm{L}_{i}=\bm{L}\odot e^{-\beta i}$ (11) $$

Figure: Figure 4: Prediction errors of the spiral curve with different levels of feedback gains and uncertainties. The right image is a partial enlargement of the left one. The blue star denotes the case without uncertainty, and the uncertainty increases along both the left and right directions.
Refer to caption: extracted/5924549/Figs/ablation_gain_comp.png

where $\beta$ represents the decay rate. The efficiency of the decay strategy is presented in Figure 5(g). The involvement of the decay factor in the multi-step prediction process significantly enhances the robustness to data noise.

### 3.4 Gain adjustment

The adjustment of linear feedback gain $\bm{L}$ can be separated from the training of neural ODEs, which can increase the flexibility of the structure. In other words, the feedback loop can be easily embedded into existing trained neural ODEs, without retraining.

The gain adjustment strategy is intuitional. Theorem 1 indicates that the prediction error will converge to a bounded set as the minimum eigenvalue of feedback gain is greater than $1/2$. And the converged set can shrink with the increase of the minimum eigenvalue. In reality, the amplitude of $\lambda_{m}(\bm{L})$ is limited since the feedback $\bm{x}$ is usually noised. The manual adjustment of $\lambda_{m}(\bm{L})$ needs the trade-off between prediction accuracy and noise amplification.

Figure 4 shows the multi-step prediction errors ($N$ = 50) with different levels of feedback gains and uncertainties. When the gain is set as 0 0, the feedback neural network will equal the neural ODE. The related simulation setup is detailed in Appendix A.3.4. Two phenomena can be observed from the heatmap. The one is that the prediction error increases with the level of uncertainty. The other is that the prediction error decreases with the gain at the beginning, but due to noise amplification, the prediction error worsens if the gain is set too large.

## 4 Neural ODEs with a neural feedback

Section 3 has shown a linear feedback form can promptly improve the adaptability of neural ODEs in unseen scenarios. However, two improvements could be further made. At first, it will be more practical if the gain tuning procedure could be avoided. Moreover, the linear feedback form can be extended to a nonlinear one $\bm{h}(\bm{x}(t)-\bm{\hat{x}}(t))$ to adopt more intricate scenes, as experienced in the control field (Han, ).

An effectual solution is to model the feedback part using another neural network, i.e., $\bm{h}_{neural}(\bm{x}(t)-\bm{\hat{x}}(t),\bm{\xi})$ parameterized by $\bm{\xi}$. Here we design a separate learning strategy to learn $\bm{\xi}$. At first, the neural ODE is trained on the nominal task without considering the feedback part. Then the feedback part is trained through domain randomization by freezing the neural ODE. In this way, the obtained feedback neural network is skillfully considered as a two-DOF network. On the one hand, the original neural ODE preserves the accuracy on the previous nominal task. On the other hand, with the aid of feedback, the generalization performance is available in the presence of unknown uncertainties.

### 4.1 Domain randomization

The key idea of domain randomization (Tobin et al., ; Peng et al., ) is to randomize the system parameters, noises, and perturbations as collecting training data so that the real applied case can be covered as much as possible. Taking the spiral example as an example (Figure 5 (a)), training with domain randomization requires datasets collected under various periods, decay rates, and bias parameters, so that the learned networks are robust to the real case with a certain of uncertainty.

Two shortcomings exist when employing domain randomization. On the one hand, the existing trained network needs to be retrained and the computation burden of training is dramatically increased. On the other hand, the training objective is forced to focus on the average performance among different parameters, such that the prediction ability on the previous nominal task will degraded, as shown in Figure 6 (a). To maintain the previous accuracy performance, larger-scale network designs are often required. In other words, the domain randomization trades precision for robustness. In the proposed learning strategy, the generalization ability is endowed to the feedback loop independently, so that the above shortcomings can be circumvented.

### 4.2 Learning a neural feedback

In this work, we specialize the virtue from domain randomization to the feedback part $\bm{h}_{neural}(t)$ rather than the previous neural network $\bm{f}_{neural}(t)$. The training framework is formalized as follows

$$ $\displaystyle{\bm{\xi}^{*}}$ $\displaystyle=\mathop{\arg\min}\limits_{\bm{\xi}}\sum\limits_{i=1}^{{n_{case}} }{\sum\limits_{j\in\mathcal{D}_{i}^{tra}}{\left\|{\bm{x}_{i,j}^{*}-\bm{x}_{i,j }}\right\|}}$ $\displaystyle s.t.\quad{\bm{x}_{i,j}}$ $\displaystyle={\bm{x}_{i,j-1}}+{T_{s}}\left({{\bm{f}_{neural}}({\bm{x}_{i,j-1} })+\bm{h}_{neural}\left({{\bm{x}_{i,j-1}}-{\bm{\hat{x}}_{i,j-1}},\bm{\xi}} \right)}\right)$ (12) $$

where $n_{case}$ denotes the number of randomized cases, $\mathcal{D}_{i}^{tra}=\{{\bm{x}_{i,j-1}},{\bm{\hat{x}}_{i,j-1}},\bm{x}_{i,j}^{
*}|j=1,\dots,m\}$ denotes the training set of the $i$-th case with $m$ samples, ${\bm{x}}_{i,j}^{*}$ denotes the labeled sate, and ${\bm{x}}_{i,j}$ denotes one-step prediction of state, which is approximated by Euler integration method here.

The learning procedure of the feedback part $\bm{h}_{neural}(t)$ is summarized as Algorithm 1. After training the neural ODE $\bm{f}_{neural}(t)$ on the nominal task, the parameters of simulation model are randomized to produce $n_{case}$ cases. Subsequently, the feedback neural network is implemented in these cases and the training set $\mathcal{D}_{i}^{tra}$ of each case is constructed. The training loss is then calculated through (4.2), which favors the update of parameter $\bm{\xi}$ by backpropagation. The above steps are repeated until the expected training loss is achieved or the maximum number of iterations was reached.

Figure: Algorithm 1 Learning neural feedback through domain randomization

Figure: Figure 5: A toy example is presented to intuitively illustrate the developed scheme. The mission is to predict the future trajectory of a spiral curve with a given initial state $\{\bm{x}(t),\bm{y}(t)\}$. The neural ODE is trained on a given training set (a), yielding an approving learning result (c). Note that the pentagrams denote start points. The trained network is then transferred to a test set (b), which model is significantly different from the training one. With the feedback mechanism, the feedback neural network can achieve a better approximated accuracy of the change rate (e), in comparison with the neural ODE (d). As a result, a smaller multi-step prediction error (f) can be attained by benefiting from the feedback neural network. (g) shows that the noise amplification issue in multi-step prediction can be alleviated by the gain-decay strategy. (f) further presents the prediction results with different prediction steps $N$. $N$ in (f) and (g) is set as $50$.
Refer to caption: extracted/5924549/Figs/Spiral_curve_comp.png

Figure: Figure 6: Learning with domain randomization. (a): Train the neural ODE through domain randomization. It can be seen that the learning performance of latent dynamics on the nominal task (Figure 5 (a)) degrades as inducing domain randomization, in comparison with Figure 5 (c). Previous works usually try to scale up neural networks to approach the previous performance. (b): Freeze the neural ODE after training on the nominal task and train the feedback part through domain randomization. The feedback neural network maintains the previous performance on the nominal task. (c) The training loss of the feedback part. Note that the neural ODE employed in (a) and (b) have the same architectures as the one in Figure 5 (c).
Refer to caption: x1.png

For the spiral example, Figure 6 (b) presents the learning performance of the feedback neural network on the nominal task. It can be seen that the feedback neural network can precisely capture the latent dynamics, maintaining the previous accuracy performance of Figure 5 (c). Moreover, the feedback neural network also has the generalization performance on randomized cases, as shown in Appendix Figure S1. Figure 6 (c) further provides the evolution of training loss of the feedback part on the spiral example. More training details are provided in Appendix A.3.3.

## 5 Empirical study

### 5.1 Trajectory prediction of an irregular object

Figure: Figure 7: Trajectory prediction results of an irregular bottle. Left: The irregular bottle is thrown out by hand and performs an approximate parabolic motion. Right: The prediction errors with different methods. The prediction horizon is set as $0.5\ s$. The colored shaded area represents the standard deviations of all $9$ test trajectories.
Refer to caption: extracted/5924549/Figs/exp1_error_comp.png

Precise trajectory prediction of a free-flying irregular object is a challenging task due to the complicated aerodynamic effects. Previous methods can be mainly classified into model-based scheme (Frese et al., ; Müller et al., ; Bouffard et al., ) and learning-based scheme (Kim et al., ; Yu et al., ). With historical data, model-based methods aim at accurately fitting the drag coefficient of an analytical drag model, while learning-based ones try to directly learn an acceleration model using specific basis functions. However, the above methods lack of online adaptive ability as employing. Benefiting from the feedback mechanism, our feedback neural network can correct the learned model in real time, leading to a more generalized performance in cases out of training datasets.

We test the effectiveness of the proposed method on an open-source dataset (Jia et al., ), in comparison with the model-based method (Frese et al., ; Müller et al., ; Bouffard et al., ) and the learning-based method (Chen et al., ). The objective of this mission is to accurately predict the object’s position after $0.5\ s$, as it is thrown by hand. $21$ trajectories are used for training, while $9$ trajectories are used for testing. The prediction result is presented in Figure 7. It can be seen that the proposed feedback neural network achieves the best prediction performance. Moreover, the predicted positions and learned latent accelerations of all test trajectories are provided in Figure S2 and Figure S3, respectively. Implementation details are provided in Appendix A.4.

Figure: Figure 8: Training the neural ODE in the presence of external inputs. Left: Trajectory samples used for training. $40$ trajectories are generated with the length of $200$ discrete nodes each. Right: Training curves of $6$ random trials. All training trials converged rapidly thanks to stable integration and end-to-end analytic gradients.
Refer to caption: extracted/5924549/Figs/aero_learning.png

### 5.2 Model predictive control of a quadrotor

MPC works in the form of receding-horizon trajectory optimizations with a dynamic model, and then determines the current optimal control input. Approving optimization results highly rely on accurate dynamical models. Befitting from the powerful representation capability of neural networks for complex real-world physics, noticeable works (Torrente et al., ; Salzmann et al., ; Sukhija et al., ) have demonstrated that models incorporating first principles with learning-based components can enhance control performance. However, as the above models are offline-learned within fixed environments, the control performance would degrade under uncertainties in unseen environments.

Figure: Figure 9: Tracking the Lissajous trajectory using MPC with different prediction models. The tracking error is evaluated by root mean square error (RMSE).
Refer to caption: extracted/5924549/Figs/simulation_result.png

In this part, the proposed feedback neural network is employed on the quadrotor trajectory tracking scenario concerning model uncertainties and external disturbances, to demonstrate its online adaptive capability. In offline training, a neural ODE is augmented with the nominal dynamics firstly to account for aerodynamic residuals. The augmented model is then integrated with an MPC controller. Note that parameter uncertainties of mass, inertia, and aerodynamic coefficients, and external disturbances are all applied in tests, despite the neural ODE only capture aerodynamic residuals in training. For the feedback neural network, the proposed multi-step prediction strategy is embedded into the model prediction process in MPC. Therefore, the formed feedback-enhanced hybrid model can effectively improve prediction results, further leading to a precise tracking performance. More implementation details refer to Appendix A.5.3.

#### 5.2.1 Learning aerodynamic effects

While learning the dynamics, the augmented model requires the participation of external control inputs, i.e., motor thrusts. Earning a quadrotor model augmented with a neural ODE could be tricky with end-to-end learning patterns since the open-loop model are intensively unstable, leading to the diverge of numerical integration. To address this problem, a baseline controller is applied to form a stable closed-loop system. As shown in Figure 8, the training trajectories are generated by randomly sampling positional waypoints in a limited space, followed by optimizing polynomials that connect these waypoints (Mellinger & Kumar, ). The adjoint sensitive method is employed in Chen et al. () to train neural ODEs without considering external control inputs. We provide an alternative training strategy concerning external inputs in Appendix A.1, from the view of optimal control. $6$ trials of training are carried out, each with distinct initial values for network parameters. The trajectory validations are carried out using $3$ randomly generated trajectories (Figures S4-S7). More learning details refer to Appendix A.5.2.

#### 5.2.2 Flight tests

In tests, MPC is implemented with four different models: the nominal model, the neural ODE augmented model, the feedback enhanced nominal model, and the feedback neural network augmented model, abbreviated as Nomi-MPC, Neural-MPC, FB-MPC, and FNN-MPC, for the sake of simplification. Moreover, $37.6\%$ mass uncertainty, $[40\%,40\%,0]$ inertia uncertainties, $[14.3\%,14.3\%,25.0\%]$ drag coefficient uncertainties, and $[0.3,0.3,0.3]N$ translational disturbances are applied. The flight results on a Lissajous trajectory are presented in Figure 9. It can be seen the Neural-MPC outperforms the Nomi-MPC since intricate aerodynamic effects are captured by the neural ODE. However, due to the fact that unseen parameter uncertainties and external disturbances are not involved in training set, Neural-MPC still has considerable tracking errors. In contrast, FNN-MPC achieves the best tracking performance. The reason can be attributed to the multi-step prediction of the feedback neural network improves the prediction accuracy subject to multiple uncertainties, as shown in Figure S8.

## 6 Related work

### 6.1 Neural ODEs

Most dynamical systems can be described by ODEs. The establishments of ODEs rely on analytical physics laws and expert experiences previously. To avoid such laborious procedures, Chen et al. () propose to approximate ODEs by directly using neural networks, named neural ODEs. The prevalent residual neural networks (He et al., ) can be regarded as an Euler discretization of neural ODEs Marion et al. (). The universal approximation property of neural ODEs has been studied theoretically (Zhang et al., ; Teshima et al., ; Li et al., ), which show the sup-universality for $C^{2}$ diffeomorphisms maps (Teshima et al., ) and $L^{p}$-universality for general continuous maps (Li et al., ). Marion () further provides the generalization bound (i.e., upper bound on the difference between the theoretical and empirical risks) for a wide range of parameterized ODEs.

Recently, plenty of neural ODE variants have been developed concerning different purposes. Considering the discrete and instantaneous changes in dynamics systems (e.g., a bouncing ball), neural ODEs can be extended by employing auxiliary neural event functions to model the switching moments (Chen et al., ). From the point of view of conservation, Hamiltonian neural networks are designed to learn the Hamiltonian of a dynamics system in an unsupervised way (Greydanus et al., ). However, canonical coordinates are needed for Hamiltonian neural networks. Lagrangian neural networks are subsequently developed to circumvent the requirement of canonical coordinates (Cranmer et al., ). Concerning dynamical systems modeled by integro-differential equations (e.g., brain activity), the neural networks employed in Zappala et al. () can capture both Markovian and non-Markovian behaviors. Other variants include neural controlled differential equations (Kidger et al., ), neural stochastic differential equations (Oh et al., ), and characteristic neural ODEs Xu et al. (). Different from our feedback scheme, all of the above methods belong to the forward propagation mechanism.

### 6.2 Generalization of neural networks

In classification tasks, neural network models face the generalization problem across samples, distributions, domains, tasks, modalities, and scopes (Rohlfs, ). Plenty of empirical strategies have been developed to improve the generalization of neural networks, such as model simplification, fit coarsening, and data augmentation for sample generalization, identification of causal relationships for distribution generalization, and transfer learning for domain generalization. More details of these approaches on classification tasks can refer to Rohlfs (). Here, we mainly review state-of-the-art research related to continuous-time prediction missions.

Domain randomization (Tobin et al., ; Peng et al., ) has shown promising effects to improve the generalization for sim-to-real transfer applications, such as drone racing (Kaufmann et al., ), quadrupedal locomotion (Choi et al., ), and humanoid locomotion (Radosavovic et al., ). The key idea is to randomize the system parameters, noises, and perturbations in simulation so that the real-world case can be covered as much as possible. Although the system’s robustness can be improved through domain randomization, there are two costs to pay. One is that the computation burden in the training process is dramatically increased. The other is that the training result has a certain of conservativeness since the training performance is an average of different scenarios, instead of a specific case.

Recently, domain randomization has proven inadequate to cope with unexpected disturbances (Shi et al., ). An adversarial learning framework is formalized in Shi et al. () to exploit sequential adversarial attacks for quadrupedal robots, and further utilize them to finetune previous reinforcement learning-based controllers. Brain-inspired neural networks have shown striking generalization performance in new environments with drastic changes (Chahine et al., ; Lechner et al., ), benefiting from its attention concentration feature. By incorporating symbolic knowledge, Wang et al. () show the generalization of neural networks can be enhanced across different robot tasks.

All of the above strategies try to learn a powerful model for coping with diverse scenarios, which may be laborious and computationally intensive. In this work, it is shown that only a closed-loop feedback adjustment is sufficient to improve the generalization, without changing the original feedforward network structure or training algorithm. The proposed strategy is simple but efficient.

## 7 Conclusion

Inspired by the feedback philosophy in biological and engineering systems, we proposed to incorporate a feedback loop into the neural network structure for the first time, as far as we known. In such a way, the learned latent dynamics can be corrected flexibly according to real-time feedback, leading to better generalization performance in continuous-time missions. The convergence property under a linear feedback form was analyzed. Subsequently, domain randomization was employed to learn a nonlinear neural feedback, resulting in a two-DOF neural network. Finally, applications on trajectory prediction of irregular objects and MPC of robots were shown. Future work will pursue to test the proposed feedback neural network on more open-source datasets of continuous prediction missions.

##### Limitations.

As analyzed in Theorem 1, accurate prediction necessitates a convergence time. Specifically, the convergence time for multi-step prediction scales linearly with the prediction horizon. While experimental results on selected examples indicate that a satisfactory prediction performance can be attained after a short convergence period, future demonstrations across a broader range of cases, particularly those demanding transient performance, are necessary.

## Appendix A Appendix

### A.1 Training neural ODEs with external inputs

Firstly, we formulate the learning problem as an optimization problem:

$$ $\displaystyle\min_{\bm{\theta}}J=\sum_{i=1}^{N-1}\;l_{i}(\bm{x}_{i},{\bm{x}_{i }}^{r},\bm{\xi})+l_{N}(\bm{x}_{N},{\bm{x}_{N}}^{r})$ (13) $\displaystyle s.t.\;\;\bm{x}_{i+1}=\bm{f}_{neural}^{d}(\bm{x}_{i},\bm{I}_{i},t _{i},\bm{\theta})$ (14) $$

where $\bm{x}_{i}$ and $\bm{I}_{i}$ denotes the model rollout state and the real sample at time $t_{i}$ respectively, $\bm{x}_{i+1}=\bm{f}_{neural}^{d}(\bm{x}_{i},\bm{I}_{i},t_{i},\bm{\theta})$
refers to the discretized integration of $\bm{f}_{neural}\left({\bm{x}(t),\bm{I}(t),t,\bm{\theta}}\right)$ with fixed discrete step since real-world state trajectories ${\bm{x}_{i}}^{r}$ are sequentially recorded with fixed timestep based on the onboard working frequency. $l_{i}(\cdot)$, $l_{N}(\cdot)$ are defined to quantify the state differences between model rollout $\bm{x}_{i}$ and real-world state ${\bm{x}_{i}}^{r}$. In this article, we select the functions in a weighted quadratic form, i.e., $({\bm{x}_{i}}^{r}-{\bm{x}_{i}})^{\top}\bm{L}_{i}({\bm{x}_{i}}^{r}-{\bm{x}_{i}})$.

By utilizing the optimal control theory and variational method, the first-order optimality conditions of the learning problem could be derived as

$$ $\displaystyle\begin{aligned} H&=J+\sum_{i=1}^{N-1}\bm{\lambda}_{i}^{\top}\bm{f }_{neural}^{d}(\bm{x}_{i},\bm{I}_{i},t_{i},\bm{\theta})\end{aligned}$ (15) $\displaystyle\bm{x}_{i+1}=\nabla_{\bm{\lambda}}H=\bm{f}_{neural}^{d}(\bm{x}_{i },\bm{I}_{i},t_{i},\bm{\theta}),\;\bm{x}_{0}=\bm{x}(0)$ (16) $\displaystyle\bm{\lambda_{i}}=\nabla_{\bm{x}}H=\nabla_{x}l_{i}+(\frac{\partial \bm{f}_{neural}^{d}}{\partial{\bm{x}}})^{\top}\bm{\lambda}_{i+1},\;\bm{\lambda }_{N}=\frac{\partial{l_{N}}}{\partial{\bm{x}_{N}}}$ (17) $\displaystyle\begin{aligned} \frac{\partial{H}}{\partial{\bm{\theta}}}=\sum_{i =1}^{N-1}\;\nabla_{\bm{\theta}}l_{i}+\bm{\lambda}_{i}^{\top}\nabla_{\bm{\theta }}\bm{f}_{neural}^{d}=0\end{aligned}$ (18) $$

where $H$ stands for the Hamiltonian of this problem. Solving (18) could be done by applying gradient descent on $\bm{\theta}$. The gradient is analytic and available (summarized in Algorithm 2) by sequentially doing forward rollout (16) of $\bm{x}$ and backward rollout (17) of $\bm{\lambda}$, where the latter one is also known as the term adjoint solve or reverse-mode differentiation.

Figure: Algorithm 2 Analytic gradient computation

Figure: Algorithm 3 Training neural ODEs with external inputs

The gradient computing only supports for a single continuous state trajectory, and the computational complexity scales linearly with the trajectory length. However, in real-world applications, multiple trajectory segments with a long horizon might be produced. We introduce mini-batching as well as stochastic optimization methods to deal with the drawback, as summarized in Algorithm 3.
The learning rate could be determined using Adam or other stochastic gradient descent-related methods.

### A.2 Proof of Theorem 1

The proof procedure requires the Lyapunov stability analysis arising from the traditional control field (Slotine et al., ). At first, define a Lyapunov function

$$ $\displaystyle V(t)=\frac{1}{2}{\bm{{\tilde{x}}}}(t)^{T}{\bm{{\tilde{x}}}}(t).$ (19) $$

Differentiate $V(t)$, yielding

$$ $\displaystyle\dot{V}\left(t\right)$ $\displaystyle=\bm{{\tilde{x}}}{\left(t\right)^{T}}\bm{\dot{\tilde{x}}}\left(t\right)$ $\displaystyle\mathop{=}\limits^{(a)}\bm{{\tilde{x}}}{\left(t\right)^{T}}\left( {-\bm{L}{\bm{{\tilde{x}}}}(t)+\Delta\bm{f}(t)}\right)$ $\displaystyle=-\bm{{\tilde{x}}}{\left(t\right)^{T}}\bm{L}{\bm{{\tilde{x}}}}(t) +\bm{{\tilde{x}}}{\left(t\right)^{T}}\Delta\bm{f}(t)$ $\displaystyle\mathop{\leq}\limits^{(b)}-\left[\lambda_{m}(\bm{L})-\frac{1}{2} \right]\bm{{\tilde{x}}}{\left(t\right)^{T}}\bm{{\tilde{x}}}{\left(t\right)}+ \frac{1}{2}\gamma^{2}$ (20) $$

where $(a)$ and $(b)$ are driven by substituting (9) and (10), respectively. By combing (19) and (A.2), it can be rendered that

$$ $\displaystyle\dot{V}\left(t\right)\leq-\left[2\lambda_{m}(\bm{L})-1\right]V(t) +\frac{1}{2}\gamma^{2}.$ (21) $$

By solving the first-order ordinary differential inequality, one can achieve

$$ $\displaystyle 0\leq V(t)\leq e^{-\left[2\lambda_{m}(\bm{L})-1\right]t}\left[V( 0)-\delta\right]+\delta$ (22) $$

with $\delta={\gamma}^{2}/\left[{4\lambda_{m}(\bm{L})-2}\right]$. It can be further implied that

$$ $\displaystyle\mathop{\lim}\limits_{t\to\infty}\|\bm{\tilde{x}}(t)\|\leq{\gamma }/\sqrt{\left[{2\lambda_{m}(\bm{L})-1}\right]}$ (23) $$

which shows that even with learning residuals, the state observation error can converge to a bounded set with the feedback modification. It can be seen that the upper bound can be regulated to arbitrarily small by increasing $\lambda_{m}(\bm{L})$.

Finally, from (9), it can be concluded that the derivative of the state observation error can also converge to a bounded set, i.e.,

$$ $\displaystyle\mathop{\lim}\limits_{t\to\infty}\|\bm{\dot{\tilde{x}}}(t)\|\leq \left|\lambda_{M}(\bm{L})\right|{\gamma}/\sqrt{\left[{2\lambda_{m}(\bm{L})-1} \right]}+\gamma$ (24) $$

with the maximum eigenvalue of feedback gain $\lambda_{M}(\bm{L})$.

### A.3 Implementation details of spiral case

#### A.3.1 Spiral dynamics

The adopted spiral model is formalized as

$$ $\displaystyle\bm{\dot{x}}\left(t\right)=\left[{\begin{array}[]{*{20}{c}}{-\eta }&\omega\\ {-\omega}&{-\eta}\end{array}}\right]\bm{x}\left(t\right)+\left[{\begin{array}[ ]{*{20}{c}}\varepsilon\\ \varepsilon\end{array}}\right]$ (29) $$

with period $\omega$, decay rate $\eta$, and bias ${\varepsilon}$.

In tests, the initial value is set as $\bm{x}(0)=\left[9,0\right]^{T}$. For the nominal task, $\omega$, $\eta$, and ${\varepsilon}$ are set as $2$, $0.1$, and 0 0, respectively.

#### A.3.2 Training details of neural ODE

The adopted MLP for training ODE has $3$ layers with $50$ hidden units and ReLU activation functions. The training datasets consist of $1000$ samples, discretized from $0\ s$ to $10\ s$ with $0.01\ s$ step size. In training, we use RMSprop optimizer with the default learning rate of $0.001$. The network is trained with a batch size of $20$ for $400$ iterations.

#### A.3.3 Training details of feedback part

As for the feedback part, we adopt MLP with $2$ hidden layers with $50$ hidden units each and ReLU activation functions. The training datasets are collected through domain randomization, with $20$ randomized cases, i.e., $\omega=\left\{0.8:+0.12:3.08\right\}$, $\eta=\left\{0.04:+0.005:0.135\right\}$, ${\varepsilon}=\left\{-24:+2.4:21.6\right\}$. Each case consists of $1000$ samples, discretized from $0\ s$ to $20\ s$ with $0.02\ s$ step size. In training, we use RMSprop optimizer with the learning rate of $0.01$. The network is trained with a batch size of $100$ for $2000$ iterations.

#### A.3.4 Setup of gain adjustment test

Figure 4 shows the ablation study on linear feedback gain and degree of uncertainty. In this test, feedback gain is selected from $\left\{0:+5:45\right\}$ in order, and uncertainties are set as $\omega=\left\{0.8:+0.4:4.4\right\}$, $\eta=\left\{0.04:+0.02:0.22\right\}$, ${\varepsilon}=\left\{-24:+8:96\right\}$ in order. The prediction step is set as $50$. The prediction results are evaluated using the means of $2$-$norm$ prediction errors.

### A.4 Implementation details of trajectory prediction of irregular objects

The input state of neural ODE consists of position and velocity. The adopted MLP for training latent ODE has $3$ hidden layers with $100$ hidden units each and ReLU activation functions. The training datasets consist of $21$ trajectories, with $1058$ samples each. The step size is $0.001\ s$. In training, we use Adam optimizer with the default learning rate of $0.001$. The network is trained with a batch size of $20$ for $1000$ iterations.

Different from the one-step prediction strategy utilized in Jia et al. () (modeled as non-autonomous systems concerning attitude), this work predicts future states in a forward-rolling way, learning a more precise result. For the compared drag model-based method, the drag coefficient comes from Jia et al. () fitted by least squares. The prediction error in Figure 7 is evaluated by the $2$-$norm$ of position prediction error.

### A.5 Implementation details of model predictive control of a quadrotor

#### A.5.1 Quadrotor Preliminaries

A quadrotor dynamics can be defined as a state-space model with a $12$-dimensional state vector $\bm{x}=[\bm{p},\bm{v},\bm{\Theta},\bm{\omega}]^{\top}$ and a $4$-dimensional input vector $\bm{u}=[T_{1},T_{2},T_{3},T_{4}]^{\top}$ of motor thrusts.
Two coordinate systems are defined, the earth-fixed frame $\bm{\mathcal{E}}=[\bm{X}_{E},\bm{Y}_{E},\bm{Z}_{E}]$ and the body-fixed frame $\bm{\mathcal{B}}=[\bm{X}_{B},\bm{Y}_{B},\bm{Z}_{B}]$. The position $\bm{p}$ and the velocity $\bm{v}$ are defined in $\bm{\mathcal{E}}$ while the body rate $\bm{\omega}$ is defined in $\bm{\mathcal{B}}$. The relationship between $\bm{\mathcal{E}}$ and $\bm{\mathcal{B}}$ is decided by the Euler angle $\bm{\Theta}$. The translational and rotational dynamics can be formalized as

$$ $\displaystyle\dot{\bm{p}}=\bm{v},\quad\dot{\bm{v}}=\bm{a}=-\frac{1}{m}\bm{Z}_{ B}T+g\bm{Z}_{E}$ (30) $\displaystyle\dot{\bm{\Theta}}=\bm{W}(\bm{\Theta})\bm{\omega},\quad\bm{J}\dot{ \bm{\omega}}=-\bm{\omega}\times(\bm{J}\bm{\omega})+\bm{\tau}$ $\displaystyle[T,\bm{\tau}]^{\top}=\bm{C}[T_{1},T_{2},T_{3},T_{4}]^{\top}$ $$

where $g$ stands for the magnitude of gravitational acceleration, $\bm{W}(\cdot)$ refers to the rotational mapping matrix of Euler angle dynamics and $\bm{C}$ is the control allocation matrix. We note the nominal dynamics of quadrotor as $\bm{\dot{x}}=\bm{f}(\bm{x},\bm{u})$.

Next, differential flatness-based controller (DFBC) (Mellinger & Kumar, ) for the quadrotor is introduced, which is adopted here to form a closed-loop system for end-to-end learning that remains stable and differentiable numerical integration. By receiving the flat outputs $\bar{\bm{\Psi}}=[\bm{p},\bm{v},\bm{a},\bm{j}]$, the positional signal and its higher-order derivatives, as the command signal, DFBC computes the desired motor thrusts for the actuators under the $12$-dimensional state feedback. By virtue of the differential flatness property of the quadrotor, one can covert the flat outputs into nominal states $\bm{x}$ and inputs $\bm{u}$ using related differential flatness mappings if the yaw motion remains zero.
We note this controller as $[\bm{\dot{z}},\bm{u}]^{\top}=\bm{\pi}(\bm{z},\bm{x},\bm{\bar{\Psi}})$, where $\bm{z}$ is auxiliary state of controller for the expression integrators and approximated derivatives in the rotational controller.

#### A.5.2 Implemention of learning aerodynamics effects

In training, the aerodynamic drag can be modeled as $\bm{R}\bm{D}\bm{R}^{\top}\bm{v}$ (Faessler et al., ), where $\bm{R}$ refers to the current rotational matrix that map the frame $\bm{\mathcal{B}}$ to the frame $\bm{\mathcal{E}}$, and $\bm{D}=diag\{[0.6,0.6,0.1]\}$ is a coefficient matrix.

A neural ODE $\bm{f}_{neural}$ (with parameters $\bm{\theta}$) is augmented with the nominal dynamics to capture the aerodynamic effect, i.e., $\dot{\bm{v}}=\bm{a}=-\frac{1}{m}\bm{Z}_{B}T+g\bm{Z}_{E}+\bm{f}_{neural}(\bm{v}
,\bm{\Theta},\bm{\theta})$. A MLP with $2$ hidden layers with $36$ hidden units is adopted.

End-to-end learning of $\bm{f}_{neural}$ could be done using the algorithm 3, but a stable numerical integration is necessary. A closed-loop system of the augmented dynamics using DFBC is employed, noted as $[\dot{\bm{x}},\dot{\bm{z}}]^{\top}=\bm{\Phi}([\bm{x},\bm{z}]^{\top},\bm{\bar{
\Psi}})$. In the proposed algorithm, $[\dot{\bm{x}},\dot{\bm{z}}]^{\top}$ turns out to be the new state and $\bm{\bar{\Psi}}$ becomes the auxiliary input instead of the input of the augmented dynamics $\bm{u}$.

We generate $40$ $\bm{\bar{\Psi}}$ trajectories with the discrete nodes of $200$ each for learning by randomly sampling the positional waypoints in a limited space, followed by optimizing polynomials that connect these waypoints, as shown in Figure 8. For validations of the learned neural ODE, we generate another $3$ random $\bm{\bar{\Psi}}$ trajectories $2.5\times$ longer than that used in training, the result illustrated in Figures S4-S7 indicates a good prediction on all $12$ states.

#### A.5.3 Implementation of MPC with Feedback Neural Networks

MPC works in the form of trajectory optimization (31) with receding-horizon $N$ with a discrete dynamic model $\bm{f}_{d}$, to obtain the current optimal control input $\bm{u}_{0}$, while maintaining feasibility constraints $\bm{u}_{i}\in\mathbb{U},\bm{x}_{i}\in\mathbb{X}$, i.e.,

$$ $\displaystyle\min_{\bm{x}_{1:N},\bm{u}_{0:N-1}}\;l_{N}(\bm{x}_{N},\bm{x}_{N}^{ r})+\sum_{i=1}^{N}\;l_{x}(\bm{x}_{i},\bm{x}_{i}^{r})+l_{u}(\bm{u}_{i},\bm{u}_{ i}^{r})$ (31) $\displaystyle\begin{aligned} s.t.\;\;&\bm{{x}}_{i+1}=\bm{f}_{d}(\bm{x}_{i},\bm {u}_{i}),\;\bm{x}_{0}=\bm{x}(0)\\ &\bm{u}_{i}\in\mathbb{U},\;\bm{x}_{i}\in\mathbb{X}\end{aligned}$ $$

where the objective functions $l_{x}(\cdot),l_{u}(\cdot),l_{N}(\cdot)$ penalize the tracking error between model predicted trajectory $\{\bm{x}_{1:N},\bm{u}_{1:N}\}$ and the up-comming reference trajectory $\{\bm{x}_{1:N}^{r},\bm{u}_{1:N}^{r}\}$, where quadratic loss are often adopted. In this application, we make $l_{x}(\cdot)=l_{N}(\cdot)=(\bm{x}-\bm{x}^{r})^{\top}\bm{Q}(\bm{x}-\bm{x}^{r})$, $l_{u}(\cdot)=(\bm{u}-\bm{u}^{r})^{\top}\bm{R}(\bm{u}-\bm{u}^{r})$, where $\bm{Q}=diag\{[\bm{100}_{3\times 1},\bm{50}_{6\times 1},\bm{1}_{3\times 1}]\}$ and $\bm{R}=diag\{\bm{1}_{4\times 1}\}$. The feasibility constraints $\bm{u}_{i}\in\mathbb{U}$ and $\bm{x}_{i}\in\mathbb{X}$ are normally designed using box constraints. We make $\bm{0}_{4\times 1}\leq\bm{u}\leq\bm{4}_{4\times 1}$ to avoid control saturation and $|\bm{\Theta}|\leq\bm{{\pi}/{2}}_{3\times 1}$ to avoid singularities while using Euler angle-based attitude representation. The receding horizon length $N$ is set to be $10$.

The key idea of using a feedback neural network augmented model is to apply the multi-step prediction mechanism to the model prediction process in MPC. The multi-step prediction algorithm requires the current feedback state $\bm{x}_{0}$ and current input $\bm{u}_{1}$ to update the sequence of $\hat{\bm{x}}_{1:N}$. The updated $\hat{\bm{x}}_{1:N}$ can be directly applied for the receding horizon optimization of the next state. We choose a linear feedback gain of $\bm{L}=diag\{\bm{3}_{12\times 1}\}$ with a decay rate of $0.1$.

#### A.5.4 Test results

A periodic $3D$ Lissajous trajectory is used for comparative tests, where a variety of attitude-velocity combination is exploited. The position trajectory can be written as $\bm{p}(t)=[r_{x}sin(2\pi t/T_{x}),\;r_{y}sin(2\pi t/T_{y}),\;h+r_{z}cos(2\pi t
/T_{z})]$, where the parameters are $[r_{x},r_{y},r_{z},T_{x},T_{y},T_{z},h]=[3.0,3.0,0.5,6.0,3.0,3.0,0.5]$. Tracking such trajectory requires a conversion of the flat outputs to the nominal $12$-dimensional state $\bm{x}$ of the quadrotor using differential flatness-based mapping.

During trajectory tracking, it could be seen from Figure S8 that the prediction accuracy of latent dynamics at the first step is improved significantly under the multi-step prediction. Although the learning-based model provides more solid results on dynamics prediction than just using the nominal model, with the help of feedback, a convergence property of prediction error can be achieved, leading to a better tracking performance (Figure 9).

Figure: Figure S1: Test on $12$ randomized cases in the spiral curve example. The developed feedback neural network shows better generalization performance than the neural ODE.
Refer to caption: x2.png

Figure: Figure S2: Trajectory prediction results of all $9$ test trajectories in Section 5.1. It can be seen that the predicted trajectories almost overlap with the truth ones.
Refer to caption: x3.png

Figure: Figure S3: The learning performance of latent accelerations of all $9$ test trajectories in Section 5.1. It can be seen that the feedback neural network can accurately capture the latent dynamics of test trajectories out of the training set.
Refer to caption: x4.png

Figure: Figure S4: $3$ random trajectories generated for validations of the learned neural ODE, named traj-#$1$, traj-#$2$, and traj-#$3$. All trajectories show well-predicted motions on pose and attitude. Detailed results on all $12$ states are provided in Figures S5-S7.
Refer to caption: extracted/5924549/Figs/traj_validate.png

Figure: Figure S5: Validation of learned neural ODE. Prediction on all $12$ states of traj-#$1$.
Refer to caption: extracted/5924549/Figs/val_state_plot_1.png

Figure: Figure S6: Validation of learned neural ODE. Prediction on all $12$ states of traj-#$2$.
Refer to caption: extracted/5924549/Figs/val_state_plot_2.png

Figure: Figure S7: Validation of learned neural ODE. Prediction on all $12$ states of traj-#$3$.
Refer to caption: extracted/5924549/Figs/val_state_plot_3.png

Figure: Figure S8: Test on the Lissajous trajectory. Prediction on the translational latent dynamics (i.e., acceleration) at the first step using different prediction models. The feedback neural network augmented model achieve the best prediction performance.
Refer to caption: extracted/5924549/Figs/sim_dx_plot.png

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: 1) Review Related Work on Feedback Adaptation in Time-Series Prediction [1,2].  
2) Explore the Relationship with Continual Learning [2,3,4].  
3) Perform Experiments on Linear Feedback.
soundness: 3
strengths: 1) The paper studies the critical issue of adaptability and generalizability in continuous-time prediction tasks using Neural ODEs, which is essential for real-world applications.
2) By incorporating feedback mechanisms, the approach potentially enhances the adaptability and generalizability of Neural ODEs, making it a contribution to the field.
3) By conducting several experiments, the proposed method demonstrates superior performance across various trajectory and dynamics prediction tasks.
summary: This paper studies the challenge of enhancing generalizability and adaptability in Neural ODEs for continuous time-series prediction tasks. It introduces feedback mechanisms into the Neural ODE framework, which dynamically adjust model parameters in response to errors in predictions. The study begins with a theoretical analysis, providing a convergence analysis of linear feedback, and extends to a practical implementation where a neural network learns the feedback mechanism. Experimental results highlight the improved performance of the neural feedback mechanisms over traditional ODE models.
weaknesses: 1) **Comparison with Existing Feedback Mechanisms**: While feedback mechanisms are commonly employed in other neural network-based time-series prediction tasks, this paper does not compare its approach to these established methods. For example, studies such as [1,2] use feedback mechanisms inspired by the Kalman filter and Extended Kalman Filter (EKF) to adapt neural networks for online time-series prediction. Including a comparison with these relevant works would be.

2) **Connection to Continual Learning**: The claims about generalization across new tasks while maintaining performance on previous tasks touch upon the principles of continual learning. A discussion of related works in continual learning, particularly focusing on online continual learning [3] and test-time adaptation [2,4], would provide a richer context.

3) **Evaluation of Theoretical Claims and Linear Feedback**: The paper presents a theorem on the convergence of linear feedback to a bounded error but does not empirically evaluate this claim. Designing a simple toy experiment to test this theorem and discussing the practical implications of the assumptions would enhance the credibility and thoroughness of the research. In addition, reporting the results of Linear feedback on several trajectory prediction tasks in the experiment will improve the quality of the paper.

4) There are minor typos and notation errors that need correction for clarity. Line 167: $\dot{z}=[\hat{f}^T,\hat{f}^T]^\top$ -> $\dot{z}=[f^T,\hat{f}^T]^\top$ ; Line 257：  'sate'  ->  'state'.

[1] Y. Cheng, et al. "Human motion prediction using semi-adaptable neural networks." 2019 American Control Conference (ACC). IEEE, 2019.  
[2] A. Abuduweili,  et al. "Robust online model adaptation by extended kalman filter with exponential moving average and dynamic multi-epoch strategy." Learning for Dynamics and Control, 2020.  
[3] Y. Ghunaim, et al. "Real-time evaluation in online continual learning: A new hope." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023.  
[4] J. Liang, et al. "A comprehensive survey on test-time adaptation under distribution shifts." International Journal of Computer Vision (2024): 1-34.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: See the weaknesses part.
soundness: 3
strengths: 1. Clear and promising motivation and concise writing that trace the development of integrating Neural ODEs with feedback structures, progressing from linear to nonlinear feedback.
2. Theoretical analysis is presented for the linear feedback case, demonstrating that prediction error converges to a bounded set with an appropriately chosen feedback gain.
3. Extensive illustrations on simple cases effectively demonstrate their method’s performance compared to standard Neural ODEs.
4. In addition to toy experiments on trajectory prediction, they conduct practical quadrotor tracking experiments, incorporating learned prediction models within an MPC controller, where the proposed FNN-MPC shows superior performance over alternative approaches.
summary: The paper introduces *feedback neural networks*, enhancing the generalization of neural ODEs in continuous-time tasks by adapting learned dynamics through real-time feedback. Inspired by biological systems, the network employs a two-DOF structure with linear feedback (whose convergence is theoretically analyzed) and nonlinear feedback learned via domain randomization. Unlike prior methods, it achieves robust generalization without sacrificing accuracy. Experiments, including trajectory prediction and quadrotor control, demonstrate its improvements over existing methods.
weaknesses: I have limited experience with Neural ODEs, so I’m not fully confident in assessing the contribution to this area. My main concerns, however, relate to the experimental section. While the current experiments effectively showcase the improvements of FNN, the experimental setup is relatively simple and lacks key baselines in learning dynamics models beyond Neural ODEs, such as [1-3]. Furthermore, it is unclear how flight trajectories were collected and how tracking performance was evaluated. Given the complexity of quadrotor dynamics and aerodynamics, these aspects significantly impact the difficulty of the task and the validity of the results. Although a preliminary overview of quadrotor dynamics is provided in the appendix, additional discussion on environmental conditions, such as wind fields and airflows [4], would help clarify the experimental setting.

[1] Leonard Bauersfeld, et al. NeuroBEM: Hybrid Aerodynamic Quadrotor Model. RSS 2021.

[2] Alessandro Saviolo and Giuseppe Loianno. "Learning quadrotor dynamics for precise, safe, and agile flight control." *Annual Reviews in Control* 55 (2023): 45-60.

[3] Pratyaksh Prabhav Rao et al. "Learning Long-Horizon Predictions for Quadrotor Dynamics". IROS 2024 

[4] Leonard Bauersfeld, et al. "Robotics meets Fluid Dynamics: A Characterization of the Induced Airflow below a Quadrotor as a Turbulent Jet".

---

confidence: 4
contribution: 4
initial_rating: 6
presentation: 3
questions: I do not have further questions or suggestions beyond the comments I made above. The authors are suggested to carefully examine all of my above comments. I commend the authors for this interesting work and look forward to reading their responses.
soundness: 3
strengths: I believe this paper makes a strong contribution. The paper is well-written and easy to follow. The technical development with the use of closed-loop state error feedback term is well-motivated. I found the paper an interesting read.
summary: This paper introduces closed-loop feedback neural ODEs which offer better generalization performance compared to open-loop neural ODEs. A linear state-error feedback term is introduced to compensate for prediction inaccuracies in the closed-loop error mechanism. This feedback term is shown to improve the generalization performance which was poor in the previous open-loop neural ODE formulations.
weaknesses: Despite the significant contributions, there are quite a few mathematical errors and a general lack of mathematical rigor in the paper. These errors might be minor and easy to resolve, I suggest the authors address the following issues.

1. Equation (7) appears erroneous. Combining (4) and (6) should actually give the term $L(\bar{x}(t)-\hat{x}(t))$, not $L(x(t)-\hat{x}(t))$. This error gets carried forward to Eq. (9) of the convergence analysis, thus potentially invalidating Theorem 1. As far as I can help, this issue can be avoided redefining Eq. (6) in terms of $x(t)$ instead of $\bar{x}(t)$. In that case, Eq. (8) needs to be adjusted accordingly. Alternatively, the authors can add and subtract $Lx(t)$ term in the derivation of Eq. (9), which should result in an extra $L(x-\bar{x})$ term in (9). This term will need to be bounded and that bound would need to be considered in the ensuing Lyapunov-based stability analysis to obtain the correct error bounds. Either way, the mathematical analysis needs revisions to address the error in Eq. (7).

2. The development in Eq. (4)-(8) is in a discrete-time setting but the convergence analysis in Section 3.2 is in a continuous-time setting. To be consistent, the authors are suggested to either reformulate the development in Section 3.1 using continuous-time integrals instead of discrete-time summations, or rederive the convergence analysis in Section 3.2 using discrete-time Lyapunov theorem or contraction analysis. I believe this should be easy to address.

3. Although the proof of Theorem 1 is correct, it appears the analysis is not performed in the most efficient manner. Specifically, the use of Young's inequality in Eq. (2) is not the most efficient because it results in the $\lambda_m(L)-\frac{1}{2}$ term which carries forward through the analysis. The problem with this term is that the gain condition $\lambda_m(L)>frac{1}{2}$ would need to be imposed to enure an exponential decay. Such a gain condition is missing in the paper. However, changing the analysis in Eq. (20) would remove the need for a gain condition. I suggest the authors use the following form of Young's inequality instead

$$ \tilde{x}^T \Delta f(t) = \sqrt{\lambda_m(L)}||\tilde{x}||\frac{\gamma}{\sqrt{\lambda_m(L)}}=\frac{\lambda_m(L)||\tilde{x}||^2}{2}+\frac{\gamma^2}{2\lambda_m(L)}.$$

This would result in 

$$\dot{V} \leq -\frac{\lambda_m(L)||\tilde{x}||^2}{2}+\frac{\gamma^2}{2\lambda_m(L)}.$$

This formulation would eliminate the need of an extra gain condition, while also obtaining a tighter bound in the $\delta$ term.

4. The purely time-dependent exponentially decaying gain schedule for the feedback gain in Eq. (11) can be problematic in terms of the performance, because the theoretical error bounds in Theorem 1 contain $\lambda_m(L)$ in the denominator. The bounds get enormous with the exponential decay, thus essentially making the neural ODE open-loop with time. There are better ways to schedule the gain using closed-loop feedback. The authors are recommended to look into recursive least squares algorithms that are popular in the adaptive control or Kalman filtering literature, where the gain decreases as the estimator gains more information instead of scheduling it to always decay with time. 

5. This is minor but the technical writing throughout the paper lacks mathematical rigor. Specifically, the mathematical entities (states, functions, constants etc.) are not formally defined. For example, the domain of x can be precisely defined, i.e., $x(t)\in\mathbb{R}^n$. The function $f$ can be defined formally as $f:\mathbb{R}^n\times\mathbb{R}^m\times\mathbb{R}\to \mathbb{R}^n$. The notations $f_\{neural}(t)$ and $\hat{f}_\{neural}(t)$ (which are not formally defined) are confusing because they take only time as the argument. However, they also seem to depend on x and \hat{x}. The term $L$is defined to be a constant, but it should actually be a positive-definite matrix. The concatenated state $z$ needs to be defined similarly with an appropriate domain. In the statement of Theorem 1, instead of stating "the state observation error and its derivative can converge to bounded sets exponentially, which upper bounds can be regulated by the feedback gain L", the authors should explicitly state the obtained exponentially convergent bounds.

6. Minor issue, but Figure 1 seems to indicate the error is being passed through another neural network instead of a linear feedback gain, which is not the case in this paper. The figure needs to be revised replacing this second neural network with a -L gain block.

EDIT: Increasing the score to 8 after author responses.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 4
questions: 1. **Automated Design of Feedback Gain and Decay Rate**:
    - Could an adaptive mechanism be developed to dynamically adjust the feedback gain matrix \(L\) and decay rate \(\beta\) based on real-time prediction errors or system states? This would reduce the need for manual parameter tuning and enhance the method's flexibility and adaptability. Alternatively, provide more detailed documentation on the specific effects of feedback gain matrix \(L\) and decay rate \(\beta\).

2. **Comprehensive Ablation Studies and Sensitivity Analysis**:
    - Suggest adding ablation experiments on the independent effects of each component (such as neural feedback network, linear and nonlinear feedback), along with detailed sensitivity analysis of the feedback gain matrix \(L\) and domain randomization parameters. This would help better understand each part's contribution to overall performance.

3. **Compare with More Baseline Models**:
    - To more comprehensively demonstrate the advantages of the proposed method, it is recommended to compare with a wider range of baseline models, including other improved Neural ODE variants or feedback-based models, to validate its effectiveness across diverse scenarios.

4. **Detailed Description of Computational Cost and Training Time**:
    - Suggest adding a dedicated section in the paper to thoroughly discuss the performance of the proposed method in terms of computational resources and training time, including comparisons with traditional methods. This would help readers better evaluate its practical feasibility and efficiency.
soundness: 2
strengths: - By combining linear and nonlinear (neural feedback) forms of feedback, an innovative network architecture is proposed that enhances generalization capability in unseen scenarios while maintaining accuracy on previous tasks. This design demonstrates innovation and practical value in current research. The experimental section covers trajectory prediction and quadrotor control, validating the method's superior performance.

- The paper provides theoretical analysis, proving system stability and convergence under linear feedback conditions. This establishes a solid theoretical foundation for the method's reliability and robustness. Additionally, the paper is well-structured with rigorous logic and clear writing. The figures and tables are intuitive. Detailed algorithm descriptions enhance the method's reproducibility.

- Through domain randomization techniques, the nonlinear neural feedback component is learned independently, demonstrating an effective way to improve generalization without significantly increasing training burden. This method effectively extends the applicability of feedback mechanisms when facing unknown uncertainties.

- The feedback loops can be easily integrated into existing trained Neural ODEs without requiring retraining, reducing implementation difficulties in practical applications and enhancing the method's feasibility and scalability. Furthermore, the method shows potential for online adjustment, further increasing its practical value. 

- Generalization ability is a key challenge for neural networks in practical applications, and the paper effectively improves Neural ODE's generalization performance through feedback mechanisms, addressing the performance degradation issue of traditional methods in unseen tasks. This research has significant implications for neural network architecture design and practical engineering applications, showing broad application potential.
summary: This paper successfully enhanced the generalization capability of Neural ODEs by introducing feedback mechanisms, demonstrating significant performance improvements across multiple practical applications. Specifically, the authors proposed a Two-Degree-of-Freedom (Two-DOF) neural network architecture that combines linear and nonlinear feedback mechanisms, effectively correcting prediction errors and exhibiting stronger adaptability and generalization capability when dealing with complex and nonlinear dynamical systems. Overall, it provides a new perspective and solution for the generalization problem of neural networks in continuous-time prediction tasks.
weaknesses: 1. **Oversimplified Convergence Analysis (Section 3.2)**:
    - The convergence analysis is solely based on bounded learning residual error (Assumption 1), assuming an upper bound γ. However, in practical applications, system dynamics can be highly complex, making residual errors difficult to strictly bound and potentially time-varying or having more complex characteristics (such as unbounded growth or sudden disturbances). This idealized assumption limits the applicability of theoretical analysis in real scenarios.

2. **Manual Tuning of Feedback Gain Matrix \(L\)**:
    - The selection of feedback gain matrix \(L\) relies mainly on manual tuning, based on theoretical analysis and experimental results. This approach is both time-consuming and difficult to precise in complex systems, especially in high-dimensional systems or real-time changing environments where manual selection may not meet dynamic adjustment needs.

3. **Training Strategy Issues in Neural Network Feedback Section**:
    - The authors adopt a two-stage training method: first training the Neural ODE, then fixing parameters to train the feedback part. This approach may not be optimal as parameters of both networks might be coupled, preventing the overall model from achieving optimal performance under joint optimization. Could different training approaches be discussed?

4. **Insufficient Ablation Studies**:
    - Although the paper includes some ablation studies, they lack comprehensiveness and depth. For instance, there's insufficient in-depth analysis of individual components' impacts (like neural feedback network, linear and nonlinear feedback), and detailed sensitivity analysis of feedback gain matrix \(L\) and domain randomization parameters. Additionally, the limited variety of baseline models fails to cover more different types or improved Neural ODE variants, affecting the comprehensive demonstration of the method's advantages.

5. **Lack of Detailed Discussion on Computational Cost and Training Time**:
    - The paper's discussion of computational cost and training time is insufficient, failing to comprehensively address the proposed method's performance in terms of computational resources and training time compared to traditional methods.

Meta-review: The paper introduces feedback neural networks to enhance the generalization capabilities of Neural ODEs for continuous-time prediction tasks. The key innovation is a two-degree-of-freedom (Two-DOF) architecture that combines linear and nonlinear feedback mechanisms to correct prediction errors in learned latent dynamics.

positive:

- combining feedback with Neural ODEs to improve generalization without sacrificing accuracy (not new, but valuable to see the results).
- sound theoretical foundation with convergence analysis for the linear feedback case
- experimental validation in real-world applications like quadrotor control
- the feedback mechanism can be integrated into existing trained Neural ODEs without requiring retraining

Weaknesses:

- somewhat idealized assumptions in the convergence analysis (bounded learning residuals)
- manual tuning required for feedback gain matrix
- could benefit from more extensive comparison with state-of-the-art baselines

Authors managed to address most of reviewers concerns and acknowledged the limitations of their approach. I vote for accepting the paper.

Decision: Accept (Oral)