Zero-Shot Transfer of Neural ODEs
Authors: Tyler Ingebrand
Adam Thorpe
ufuk topcu
Venue: NeurIPS 2024 poster (2024)
Keywords: Zero-shot Transfer
Neural ODE
Model-based control
Primary area: robotics
TL;DR: We introduce a method for zero-shot transfer and dynamics prediction via function encoders using neural ODE basis functions.

Abstract: Autonomous systems often encounter environments and scenarios beyond the scope of their training data, which underscores a critical challenge: the need to generalize and adapt to unseen scenarios in real time. This challenge necessitates new mathematical and algorithmic tools that enable adaptation and zero-shot transfer. To this end, we leverage the theory of function encoders, which enables zero-shot transfer by combining the flexibility of neural networks with the mathematical principles of Hilbert spaces. Using this theory, we first present a method for learning a space of dynamics spanned by a set of neural ODE basis functions. After training, the proposed approach can rapidly identify dynamics in the learned space using an efficient inner product calculation. Critically, this calculation requires no gradient calculations or retraining during the online phase. This method enables zero-shot transfer for autonomous systems at runtime and opens the door for a new class of adaptable control algorithms. We demonstrate state-of-the-art system modeling accuracy for two MuJoCo robot environments and show that the learned models can be used for more efficient MPC control of a quadrotor.

Zero-Shot Transfer of Neural ODEs
Tyler Ingebrand, Adam J. Thorpe, Ufuk Topeu University of Texas at Austin Austin, TX 78712
Abstract
Autonomous systems often encounter environments and scenarios beyond the scope of their training data, which underscores a critical challenge: the need to generalize and adapt unseen scenarios in real time. This challenge necessitates new mathe- matical and algorithmic tools that enable adaptation and zero-shot transfer. To this end, we leverage the theory of function encoders, which enables zero-shot transfer by combining the flexibility of neural networks with the mathematical principles of Hilbert spaces. Using this theory, we first present method for learning space of dynamics spanned by set of neural ODE basis functions. After training, the proposed approach can rapidly identify dynamics in the learned space using an efficient inner product calculation. Critically, this calculation requires no gradient calculations retraining during the online phase. This method enables zero-shot transfer for autonomous systems at runtime and opens the door for new class of adaptable control algorithms. We demonstrate state-of-the-a system modeling accuracy for two MuJoCo robot environments and show that the learned models can be used for more efficient MPC control of quadrotor.
Introduction
Models that are adaptable, generalizable, and capable of learning online from minimal data are essential for autonomy. These models must adapt to unseen tasks and environments at runtime without relying upon priori parameterizations. For example, consider an autonomous UAV delivery robot navigating in dense, urban environment through varying wind patterns and carrying uncertain payloads. This scenario requires rapid adaptation to ensure safe and correct operation because conditions can change unpredictably and online model updates are impractical. While prior works can control autonomous systems in a single setting, they fail to adapt to the continuum of real-life scenarios. The key challenge is enabling zero-shot transfer of learned models, where models quickly adapt to new data provided at runtime without retraining.
We present method for modeling differential equations by learning set of basis functions pa- rameterized by neural ODEs. Our key insight is to learn a space of functions that captures feasible behaviors of the system. By focusing learning the structure of the space of differential equations, our approach implicitly learns how the dynamics change due to changes in the environment. Our approach is based on the theory of function encoders [1), framework for zero-shot transfer that has been applied to lask transfer in reinforcement learning contexts.
We formulate the space of learned functions linear space equipped with inner product (e.g. a Hilbert space), and learn a set basis functions over this space, where each basis function is represented by neural ODE. This structure offers efficient way approximate online dynamics via a linear combination of the basis functions.
By representing functions in Hilbert space and pre-training identify the basis functions' coefficients for a new dynamical system runtime using minimal data. This is useful, for instance, in scenarios where we can pre-train offline in simulation, but need to
suite of functions, we can quickly
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Monte Carlo
(f.92)
C (f-ge)
91
Σ S nz
D2
Online
Neural ODEs
Pixed Basis
9a
Duiasets
94
VoL
g8
Offine Training of (91.-...90
92
FF
Zero-Shot Prediction
span (pan.
96.
Figure 1: An illustration of our approach. The training phase uses set of datasets D to train basis functions (.......9 .. } to span F. The zero-shot phase uses online data to identify the coefficients for new function, which can be estimated linear combination of the basis functions.
quickly identify dynamics online from single trajectory in zero-shot manner. This strategy greatly reduces the computational overhead associated with adapting or re-training a neural ODE to new tasks runtime. The ability to efficiently encode the behavior of system at runtime without retraining is a key component our approach. Our approach is outlined in Figure
Our approach yields models which generalize to large set of possible system behaviors and achieves better long-term prediction accuracy than neural ODEs alone. We showcase our approach on problem of predicting the behavior of first-order ODE system with Van der Pol dynamics. We demonstrate the scalability of our approach on two MuJoCo robotics environments [28]. Finally, we test the feasibility of using the learned model for downstream tasks such as model-predictive control (MPC). Our results show that our model achieves significantly better long-horizon prediction accuracy compared the nearest baseline. Additionally, the MPC controller using model has lower slew rate, indicating that the improved model accuracy leads to more efficient control decisions.
1.1 Contributions
Representing Spaces of Dynamical Systems: We propose a novel framework for representing spaces of dynamical systems, i.e. induced by hidden system parameters, variations in the underlying physies, or changing environmental features. Using large-scale set of data collected offline, we learn collection of neural networks that act functional basis. This approach is based in the theory of function encoders [14). Yet the extension to using neural ODEs as basis functions non-trivial and poses several challenges, mainly from the need to integrate the learned model.
Method for Online Adaptation: We construct method for adapting neural ODE estimations based on online data without gradient updates, i.e. zero-shot transfer of system models. Our approach overcomes significant challenge in learning behaviors of differential equations and dynamical systems. By offloading the computational effort to the training phase, we enable rapid online identification, adaptation, and prediction without retraining.
Empirical Results: We demonstrate accurate bong-horizon predictions in challenging robotics tasks and show these models can be used for online control of quadrotor systems. We assess the quality of the approach in three areas and answer the following questions: 1) How well does our approach adapt new dynamics online? 2) How does our approach compare to existing approaches for long-horizon prediction tasks? and 3) Does our approach work for downstream tasks such as control? We show that function encoders using neural ODEs basis functions consistently outperform existing approaches.
2 Background
2.1 Neural ODEs
Consider ordinary differential equation ii() ((<(1),)), where ʃ is Lipschitz continuous and <t()) E RD is the state at time i. Given an initial condition xr(4), the ODE solution can be written as
x(ty) z(to) l" f()(T),7)dr.
(1)

---

Note that in general, the explicit dependence of f(x(t), on can be removed by augmenting the state z to include As such, we omit t throughout.
Neural ODEs parameterize the function, / as a neural network. In particular, neural ODEs solve the ODE using off-the-shelf integrator and optimize the neural network with respect prediction
loss. The training procedure requires dataset D = {ts, [(t))]= which is used to train the model via a supervised objective, such as mean squared error, back-propagated through the integrator.
Neural ODEs have demonstrated impressive accuracy for long-horizon predictions of continuous-time systems. Furthermore, they be trained on trajectories with irregular time intervals between samples [18], and generalize better than multi-layer perceptron models. However, they lack adaptability and need to be retrained for every scenario.
2.2 Function Encoders
To achieve zero-shot transfer, we employ the theory of function encoders 14] While typical machine learning approaches learn a single function, function encoders learn basis functions to span space of functions. This allows the function encoder achieve zero-shot transfer within this space by identifying the coefficients of the basis functions for any function in the space at runtime. Once the coefficients have been identified, the function can reproduced as a linear combination of basis functions. Despite the broad applicability of function encoders, the extension to capture solutions to differential equations via neural ODEs is non-trivial.
Formally, consider a function space F = {f f : -" where X' C. R". Instead of learning a single function, function encoders learn basis functions 92.92--- ......... Ik that are parameterized by neural networks in order to span F [4]. Define the inner product of F as (f.g)r r = f(f(2), (x)))r. Then, the functions f E can be represented
linear combination of basis functions,
f(x) = Σ eagi (x 0.),
(2)
where e R are real coefficients and 92 are the network parameters for gs. Let V be the volume of X. For any function f E F, an empirical estimate of the coefficients can be calculated using data via Monte-Carie estimate of the inner product.
Q: = (S59i)
m Σ (Nar)) 1.))
The basis functions are trained using set of datasets, D (D1, D2y--., where each dataset
D. = (( f( consists of input-output pairs corresponding to a single function ft E F. For each function Sa and corresponding dataset D,, first compute the coefficients {cl, c Cu via (3) and then obtain an empirical estimate of fs via (2). Then compute the error of the estimate of f. though the norm induced by the inner product of F. The loss function is simply the sum of the losses for all f., which is minimized via gradient descent. For more details, see 14]
After training, the basis functions are fixed, and the coefficients c of a new function f E gpi.ngi........ } are computed via (3) or via least-squares, using data collected online. This key for efficient, online calculations, since the approximation in is effectively a sample mean and requires no gradient calculations.
Orthogonality of the Basis Functions: Note that function encoders do not enforce orthogonality of the basis functions explicitly [4]. Using Gram-Schmidt to orthonormalize the basis functions during training can significantly increase the training time and is computationally intensive. Instead, during training the coefficients are computed using presuming that the basis functions are orthogonal. This is key. This causes the basis vectors to naturally become more orthogonal as training progresses since the loss implicitly penalizes the basis functions if they not orthonormal. See Appendix
3 Function Encoders With Neural ODEs as Basis Functions
Consider space F of Lipschitz continuous dynamical systems f X' - X'. The space of dynamical systems arise, for instance, due uncertain parameters, minor variations in first-order physics
3

---

model, or changing environmental features. Given an initial condition z(fu), our goal is to estimate the state x(t,) at future time t >> tg- The integral form of the initial value problem is given by ①
Our approach can be separated into two distinct phases: offline training and zero-shot prediction. During offline training, we presume that have access to offline dataset D (D1: D2,---)- where D. is realization of a trajectory from a function ft E F. During zero-shot prediction, we seek to predict previously unseen function f and have access a minimal trajectory D taken from f. We seek to learn set of basis functions 91.....: 91 that span F, where & is a user-specified hyper- parameter. By learning a set of basis functions that span the space F, we obtain a means to represent the behavior of any dynamical system in the space. However, we do not observe measurements of directly since we cannot typically measure the instantaneous derivative à of dynamical system. Instead, we will equivalently learn a set of neural ODE basis functions such that the underlying neural networks which are being integrated correspond to 91, Ok- We then seek to compute representation of new function using data collected online.
3.1 Computing a Set of Neural ODE Basis Functions
For every F, define the integral term in ① as a function H
X, given by,
H (z(())) = L. [(x(T))dr.
We model the dynamical system f using a function encoder as in linearity of the definite integral, we have that,
(2) Using in and by the
Σ c. (.((((())) 0.)) dr dr= Σe,
I" 9 (x(T)) 10.) Σ (G(())))
H (x(to),
where Ji is neural network parameterized by 0, and G((z(to), : Sh gx((() 0.) One interpretation of the above equation is that we can represent H as a weighted combination of basis functions G,, and the problem of learning a set basis functions gl....: can equivalently be viewed as learning set of neural ODEs Gj...., Gg- Thus, we define the Hilbert space H of functions H as in and equip it with the following inner product,
(H, G) := (HTE,,)), G(2,t)) d2,
(6)
We then learn basis functions GI,..., GR spanning H where each basis function is a neural ODE,
From (6). the coefficients of a function H H are given by c G.)H. However, from [14). computing the inner product exactly is generally intractable in high-dimensional spaces. We can empirically estimate the coefficients c; using a trajectory of (potentially irregularly) sampled states (a(t) |t tu...., tm from a dynamics function f F. Using the trajectory, we form the dataset
D = (a((t, ). ((t++1))7== We can compute c using via a Monte-Carle estimate of the inner
product in (6)
V ((tz11)=(tt)) (()=))-((().(ddtt)ttt)))))
c (H,
where V the volume of the region of integration, and following from 1..
x(()+1) - x11)) = (/rm (())r=H((<((s)))
(8)
In other words, substitute the difference between states xx(5++1) - x((,) for H(t(tt) (4+) in 7
Let D (D1, D2.... be set of datasets, where each D; [(=(tt). from a trajectory from function Je E F. For each dataset Dr, we compute the coefficients ......, Ck according to (7). The coefficients can used to approximate the corresponding He via (5). We then evaluate the error of H₃ using the dataset Dj and minimize its loss via gradient descent. This is done
is collected

---

Algorithm Training Function Encoders with Neural ODE Basis Functions
1: Input: Set of datasets D, number of basis functions k; learning rate or
2: Oufput: Neural ODE basis functions G1 G2..., G5
3: Initialize 91, .2,.... 9k as neural networks with parameters 0 = {01, 02 0k
4: while not converged do
5: Ioss L ==
6: for all De E D do
7: for E 1,.... k do
8:
(m(()))))))))) -x()) (((())) -1))xx I. L Σm - ((x((t++)) x(1,)) E GG, (2)
~
V. th
Z===
9:
end for
end for
12: 0 = aVoL
end while
for multiple functions f E F at each gradient update to ensure the basis learns the space rather than a single function. We present this Algorithm
Applying Algorithm I yields basis functions which span the space of dynamical systems, where each basis function is a neural ODE. This space describes possible behaviors of the system, where variations in environmental parameters, physics, etc. correspond a particular dynamics function within this space. Therefore, this algorithm represents complicated system behaviors simply vector within a Hilbert space. Section 3.2 shows how to use these basis functions for zero-shot dynamics prediction from small amounts of online data.
3.2 Efficient Online Transfer Without Retraining
After training, we fix the parameters of the basis functions 91, ... Sk can compute the coefficient representation e R for any function f span ..........h.. 9k via 77. If D is rich enough to capture the various behaviors the systems in F, then we can estimate the behavior of any dynamics f e F.
Given data collected online from single trajectory. we can compute the coefficients using the Monte-Carke estimate of the inner product as in (7). This approximation a crucial component of the approach. It allows the inner product to be computed from data through an operation that is effectively sample mean. Therefore, this approach can be computed online quickly even for large amounts of data. Then, given the coefficients, the future states of the system can be predicted using (5) These properties allow the neural ODE to achieve zero-shot transfer. Identifying the coefficients only requires inner product calculations, vector addition, and scalar multiplication, and so it can be computed online without any gradient updates.
The Residuals Method:
The zero vector of the coefficients space corresponds to the zero function. Since the feasible dynamics differentiated by their coefficients, it numerically convenient if the coefficients corresponding to all feasible systems are centered around zero.
re-center the space of coefficients around the center of the cluster of feasible dynamics. This is done by first modeling the average dynamics Fang in the set of datasets DD and then learning the residuals between each function and Fang- In other words, the basis functions are trained span the function space corresponding to R(x((l);(r) = xx(t)) x((o) F Fang (x((o),,)) This method can achieve better accuracy, but requires learning one additional neural ODE, F Foeg- Alternatively, an approximate dynamics model based on prior knowledge can be used as Faug. We describe the training procedure for this approach in Algorithm 2
Thus,
3.3 Incorporating Zero-Order Hold Control Inputs
We can account for zero-order hold (ZOH) control input E U C. IR with minimal modifications. ZOH control input is given by a piecewise constant function, meaning is held constant over the period of integration. Given controlled dynamics, f X' x li + X. we modify the corresponding
functions H to incorporate a constant input, H (z<(t)) u. k, Et=1 es f g, (=<())) u. 0.)dT. Then, using trajectory data that also includes the controls applied at each time interval, we can estimate
5

---

Ar1000
1-1
1P2.37
(F-2)
1-0.0
Obooad Truh.
K0.05
N0128 Km
Figure 2: The approximated dynamics for different Van der Pol systems, where the parameter is varied. This plot shows that a NODE can only fit single Van der Pol system, whereas FE + NODE + Res fit space of Van der Pol systems from 5000 example data points.
the coefficients using datasets D = (x((t))))) uy +(t++)))) substituting 9: (x(x()), u, 0:) in (5)
and 77. The remainder the training procedure is unchanged.
4 Numerical Experiments
We demonstrate the effectiveness of our approach for predicting and controlling dynamical systems through several numerical experiments. We first demonstrate that the approach can adapt different dynamics using a Van Der Pol oscillator system. We then show oong-horizon prediction accuracy on challenging MuloCo robotics experiments and compare to neural ODEs (NODE) (5) and function en- coders as in 14 using the residuals method (FE Res). Lastly, we show the learned models are suf- ficiently accurate downstream tasks on difficult control task using a quadrotor system. The source code is available at htthtts:://giieee
Current off-the-shed integrators with adaptive step sizes do support efficient batch calculations. Because this algorithm involves training numerous neural ODEs on a large amount of data, the ability to train on data batches required. Therefore, we implement an RK4 integrator since can be efficiently computed for multiple data points in parallel. The Van der Pol visualization uses 11 basis functions while the MuloCo and Drone experiments use 100. For ablations on how the hyper-parameters affect results, see Appendix G
4.1 Visualization on Van der Pol Oscillator
adapt space of dynamics that vary according nonlinear parameter. The Van der Pol dynamics are defined as, à = y, y 1(1 x²)y - J. where [x, y] I22 is the state, and μ is a hidden parameter. We collect multiple datasets Dj where μ is fixed for the duration of the trajectory, but varies between trajectories. We train basis functions using Algorithm We then compute the coefficients via 7 and approximate the dynamics via (5).
We first demonstrate that approach
We plot the results in Figure 2 As expected, we observe that our proposed approach can predict the dynamics of a space of Van der Pol systems without retraining. We can also see that single neural ODE trained on the same data can only fit a single function. Therefore, its prediction corresponds most closely with the behavior of single Van der Pol system that has the mean μ value. This illustrates that approach is capable of adapting to different dynamics runtime.

---

0.25
1.0
0,20
0.5
1.3tpp MSE
0.15
0.5
K-Step
Half Cheetah
200
400
600
800
1000
20
40
60
90
0.25
1.0
1.5kpp MSE
0.05
0.2
0.00
0.0
Ant
200
400
600
800
1000
20
40
60
80
Gmdiint Updates
Lookcohend
NODB
Rtes
PE NODE
NODE Ros
Oracle
Figure 3: Model performance on predicting the dynamics of MuJoCo robotics environments with hidden parameters. 200 example data points are given to identify dynamics. The results show that FE NODE Res. makes accurate, long-horizon predictions even in the presence of hidden parameters. Evaluation is over 5 seeds, shaded regions show the first and third quartiles around the median.
4.2 Long-Horizon Prediction MuJoCo Environments
We evaluate the performance of our proposed approach on the Half-Cheetah and Ant environments 28) shown in 3 The hidden environmental parameters are the length of the limbs, the friction coefficient, and the control authority. Of the two environments, Ant is more difficult due to its higher degrees of freedom. For training, we collect a dataset of trajectories where the hidden parameters are unobserved, but held constant throughout the duration of given trajectory. After training, we use 200 datapoints, equivalent to about seven seconds of data for system running at 30 Hertz, and use only this data to identify the dynamics. Note this online phase is computationally simple, and can be done in only milliseconds on a GPU.
Neural ODEs (NODE) perform poorly because they have no mechanism to condition the prediction on the pirdeen-araaeterss Effectively, NODE learns the mean dynamics over all dynamics functions in the training set. Function encoders using the residuals method (FE + Res) can implicitly condition their predictions on the hidden parameters through the coefficient calculation, though they are unable to achieve accurate long horizon predictions on the more challenging Ant problem. This is because it lacks the inductive bias neural ODEs, Our approach (FE NODE) can both implicitly condition the predictions on the hidden parameters through data, but also benefits from the inductive bias of neural ODEs. see that the residuals method performs best out of approaches in both environments. This is because the average model significantly reduces the epistemic uncertainty and provides meaningful baseline from which to center the training. The average model acts as a good inductive bias and makes it easier to distinguish between the learned functions during training. We additionally compare against an oracle prediction approach (Oracle), which has access to the hidden parameters additional input with neural ODE the underlying architecture. While its 1-step prediction accuracy demonstrates good empirical performance, the hong-horizon predictions are unstable. This is because Oracle is required generalize an entire space of dynamics with one NODE, which is a complex and difficult function to learn.
4.3 Realistic Robotics Experiments and Control of Quadrotor System
Lastly, we seek to test the accuracy of our approach for use on downstream tasks such as control on realistic example using robotic system. We seek to determine if the learned models are sufficiently accurate for model-based control in the presence of hidden parameters. We use simulated quadrotor system using PyBulle [II, which is a highly nonlinear control system. We use the quadrotor's 's mass hidden parameter. The goal is to predict the future state of the quadrotor system under any hidden
7

---

10
Anegge Now
64-
0.00
0.25
10
50
6/03
6:01
0.006
8.08
0.800
0.855
0.001
0:35
0.900
Lockutocll Stuge
Man
Kisa
Figure 4: Model performance on the PyBullet quadrotor environment with varying mass. Function encoders improve model performance across varying masses. Shaded region 1st and 3rd quartiles over 200 trajectories (left) and over 5 trajectories (middle, right).
100
015
Repeated MPC corrections
850
835
0:2
8.9
Accurate MPC Control
Low Mass Hrajstory
High Mass Ttojectory
0:35
NO0E
FE1 NOOE
NODE Reo.
Gaal
Figure 5: Qualitative analysis of the difference in control between NODEs and our approach. Two trajectories with the same initial position but different masses shown. NODE is unaware of the mass, and so its position requires constant correction. In contrast, FE + NODE (+Res) accounts for the mass through the coefficients, meaning it more accurate and requires fewer corrections.
parameters, using 2000 data points collected online identify the dynamics. Then, given the learned model of system behavior, we seek to control the quadrotor using gradient-based model predictive control (MPC) reach a pre-specified hover point. The results are plotted in Figure 4
The results show that FE + NODE + Res outperforms competing approaches at long- long-horizon pre- dictions. Furthermore, that FE + NODE + Res accurately predicts the system behavior across varying masses. We observe slight decay in performance for low masses which more sensitive to control inputs during simulation, which causes the simulated trajectories to diverge more from the rest of the observed data. The neural ODE (NODE) performs poorly for different masses, and its performance decays quickly as the dynamics deviate from the mean behavior.
plot the 10-step MSE function of mass in Figure 4 and observe
Lastly. see that this prediction accuracy translates the downstream performance of an MPC controller. While all approaches are sufficiently accurate for control due to the fact that MPC is partially robust model inaccuracies, the prediction accuracy has corresponding impact performance. Neural ODEs (NODE) demonstrate a high slew rate, which reflects the need for repeated positional corrections necessitated by taking bad actions. In contrast, FE NODE and FE + NODE + Res have lower slew rates as they make more accurate decisions. We plot two example trajectories that demonstrate this behavior in Figure
the task
5 Scope & Limitations
Overhead: Our approach ineurs a cost of either increased inference time or memory, depending on the basis functions are integrated sequentially or in parallel. We integrate them sequentially during

---

training to reduce memory overhead and allow for larger batch sizes, while integrating in parallel at execution to prioritize inference speed. See Appendix[] D
Data dependency: In order to make efficient, online approximations of a new function f E F without gradient calculations, the basis functions must be trained to span the space of possible dynamics. To do so, there must be sufficient example datasets of possible dynamics under fixed hidden parameters in D. This implies a larger amount of data must be collected to learn a space of dynamics then would be needed to learn a single dynamics function.
Integration: The training procedure trains the basis functions short time intervals t, tu. in which xtt) E X. The basis functions have only been trained for imputs in the space of X, where their behavior outside of X' is unpredictable. As result, it is necessary either integrate each basis function for a short time interval before calculating the state according to ( or to integrate the basis functions described in B Integrating the hasis functions over long horizons without calculating the
state of the system during intermediate steps may lead the predicted state to leave X, at which point the behavior of that basis function becomes unpredictable.
6 Related Work
Function approximation techniques often employ a linear model over predefined set of basis functions. Techniques such as Taylor series, Fourier series, and orthogonal polynomial systems utilize infinite set of basis functions, theoretically allowing perfect function representation 6 23 7) However, in high-dimensional spaces, these techniques become impractical due to the exponential growth in the number of basis functions. Additionally, many approaches depend on the choice of a feature map or kernel to define the function space (2 25). which imposes structure by selecting the class of functions learn from (e.g. using radial basis functions (3]. These design choices necessarily introduce approximation errors through the choice of function class, or may depend on prior domain knowledge, which may not always be available. Methods for identifying system dynamics that use large library of pre-defined basis functions, such as Koopman operators 22] or nonlinear system identification through sparse regression such SINDy 4 26 15) have received considerable attention. Yet these approaches typically employ a finely-cratted finite dictionary of basis functions, which requires careful choice to achieve good data-driven performance (20]. Neural network approaches such as functional-link and orthogonal networks (30 7 omit hidden layers and use gradient descent to learn a linear combination of features, encoding the function class into the network architecture, but fail to generalize well, and are not amenable to zero-shot transfer. In contrast, we compute the coefficients of the model through a well-defined inner product, which scales well with data and can be computed quickly. Furthermore, our basis functions are entirely learned from data during the training phase, similar to representation learning 1. and thus require no prior assumptions domain knowledge.
Basis Functions:
Neural ODEs: In existing work, neural ODEs have proven to be a powerful tool for modeling
dynamical systems [T0 24 22 [7] and stochastic differential equations [3], but generally require an extensive data collection and training phase. While the model training can be enhanced g and the models can incorporate prior knowledge [9] [0] to reduce the training time, they inherently
focus on a single system at a time. This inherently limits their ability to generalize across different systems without retraining. Notably, parameterized neural ODEs [9] pass the model parameters as an additional input to the neural ODE This approach has been shown to achieve form of transfer within the set of allowable parameters, but requires extensive knowledge of the system parameters and the structure of the dynamics, both at training and test time. In principle, our approach can be combined with these existing approaches to incorporate their distinct advantages, making our approach highly generalizable to different systems and modeling frameworks.
Deep Learning Techniques: Few-shot meta learning aims to solve a similar problem, where a learned model is adapted given an online dataset [12. However, meta learning requires gradient updates to the learned models, which may be too slow for real-time control. Transformers are another technique that can adapt given an online dataset by feeding that dataset as input to the encoder side of the transformer. However, transformers have long forward pass times and scale quadratically with the amount of data [6], and so they are not amenable for model-based control. Domain randomization in reinforcement learning is another technique to generate policy which is robust to large set

---

of dynamics [27] In contrast, our dynamics model adapts to the current dynamics, and thus our controller is adaptive, rather than robust.
7 Conclusion & Future Work
We introduced zero-shot neural ODEs, which accomplish both long-horizon predictions and zero-shot transfer. We demonstrated the performance of this approach on two challenging MuJoCo tasks and on the control of quadrotor system. Our approach makes significant step towards online adaptability model-basee control and has implications for the safe control of autonomous systems in the presence of uncertainty. In future work, plan to address safety during training, perhaps using the properties the Hilbert space to characterize the epistemic uncertainty. We also plan to explore theoretical extensions to stochastic differential equations and Hilbert spaces of probability measures.
Broader Impact
This approach demonstrates several clear benefits for enabling same-day adaptation, which is critical need for autonomous systems that will be deployed in new, unstructured environments. Nevertheless, this approach will require more thorough theoretical analysis before it can be deployed on actual robotics systems, e.g. to determine confidence or sample bounds to guarantee safery. Notably, this work is step toward bridging the sim-to-real gap, though it remains unclear how well real-world systems will be represented by set of basis functions learned in simulation.
Acknowledgements
Thank you to Dr. Cyrus Neary for helpful discussions. This material is based upon work supported by the National Science Foundation under NSF Grant Number 2214939. Any opinions, findings, and conclusions or recommendations expressed in this material are those the authors and do not necessarily reflect the views of the National Science Foundation. This material is based upon work supported by the Air Force Office of Scientific Research under award number AFOSR FA9550-194 0005. Any opinions, findings and conclusions or recommendations expressed in this material are those the author(s) and do not necessarily reflect the views the U.S. Department of Defense.
References
[1] Y. Bengio, A. Courville, and P. Vincent. Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8):1998-182) 2013.
[2] P. Bevanda. S. Sosnowski, and S. Hirche. Koopman operator dynamical models: Learning, analysis and control. Anmual Reviews in Control, 52:197-212, 2021.
[3] D. Broomhead and D. Lowe. Radial basis functions, and adaptive networks. Royal Signals and Radar Establishment Malvern (United Kingdom), RSRE-MEMO-4144 03 1988.
functional interpolation
[4] S. Brunton, J. L. Proctor, and J. N. Kutz. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the National Academy of 2016.
Sciences, 113(15):3932-33
[5] R. Q. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud. Neural ordinary differential equations. In Advances in Neural Information Processing Systems, volume 31, 2018
[6] C. Cheng, 7. Peng, W. Zhang, and G. Meng. Sateerra-series-basee nonlinear system modeling and its engineering applications: A state-of-the-art review. Mechanical Systems and Signal Processing. 377:3400364, 2017.
[7] S. Dehuri and S.BB. Cho. A comprehensive survey on functional link neural networks and an adaptive PSO-BP learning for CFLNN. Neural Computing and Applications, 1972)-87-2005 2010.
10

---

[8] F. Djeumou, C. Neary, E. Goubault, S. Putot, and U. Topeu. laylar- neural ordinary differential equations: Toward fast training and evaluation of neural odes. In IJCAI, 2022.
[9] Djeumou, C. Neary, E. Goubault, S. Putot, and U. Topeu. Neural networks with physics- informed architectures and constraints for dynamical systems modeling. In L4DC, 2022.
[10] F. Djeumou, C. Neary, and U. Topeu. How to learn and generalize from three minutes of data: Physics-constraiie constrained and uncertainty-awant neural stochastic differential equations. In CoRL, 2023.
[11] E. Dupont, A. Doucet, and Y. W. Teh. Augmented neural odes. In Wallach, H. Larochelle, A. Beyyellzierr d' d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
[12] C. Finn, P. Abbeel, and S. Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning, ICML, volume 70, pages 1126-1135. PMLR, 2017.
[13] L. Hodgkinson, C. van der Heide, Roosta, and M. W. Mahoney. Stochastic continuous normalizing flows: training SDEs as ODEs. In Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence, volume 161 of Proceedings of Machine Learning Research, pages 1130- 1140. PMLR, 27-30 Jul 2021.
[14] Ingebrand, A. Zhang, and U. Topcu. Zero-shot reinforcement learning via function encoders. In ICML. ICML, 2024.
[15] K. Kaheman, J. N. Kutz, and S. L. Brunton. SINDy-PI: A robust algorithm for parallel implicit sparse identification of nonlinear dynamics. CoRR, 2020.
[16] F. D. Keles, P. M. Wijewardena, and C. Hegde. On the computational complexity of self- attention. In International Conference Algorithmic Learning Theory, volume 201 of Pro- ceedings of Machine Learning Research, pages 597-619, 2023.
[17] J. Kelly, J. Bettencourt, M. J. Johnson, and D. K. Duvenaud. Learning differential equations that are easy to solve. In H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 4370-4380. Curran Associates, Inc., 2020.
[18] Kidger, J. Morrill, Foster, and T. Lyons. Neural controlled differential equations for irregular time series. In Advances in Neural Information Processing Systems, volume 33, pages 6696-6707, 2020.
[19] K. Lee and E. J. Parish. Parameterized neural ordinary differential equations: Applications to computational physics problems. Proceedings of the Royal Society A, 477(2253), 2021.
[20] Q Li, F. Dietrich, F. M. Bollt, and I. G. Kevrekidis. Extended dynamic mode decomposition with dictionary learning: A data-driven adaptive spectral decomposition of the Koopman operator. Chaos: An Interdisciplinary Journal of Nonlinear Science, 27(10)103111, 2017.
[21] X. Liu, T. Xiao, S. Si, Q. Cao, S. Kumar, and C. Hsieh. Neural SDE: stabilizing neural ODE networks with stochastic noise. CoRR, 2019.
[22] Y. Lu, A. Zhong, Q. Li, and B. Dong. Beyond finite layer neural networks: Bridging deep architectures and numerical differential equations. In ICML, 2018.
[23] J. Patra and A. Kot. Nonlinear dynamic system identification using Chebyshev functional link artificial neural networks. IEÉE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 32(4):505-511, 2002.
[24] M. Raissi, P. Perdikaris, and G. E. Karniadakis. Multistep neural networks for data-driven discovery of nonlinear dynamical systems, 2018.
[25] B. Schölkopf and A. J. Smola. Learning with Kernels: support vector machines, regularization, optimization, and beyond. Adaptive computation and machine learning series. MIT Press, 2002.
11

---

[26] J. N. K. Steven L. Brunton, Joshua L. Proctor. Sparse identification of nonlinear dynamics with control (SINDYe). IFAC-PapersOnLLie,, Volume 49, Issue 18:710-715, 2016.
[27] Tobin, R. Fong, A. Ray, Schneider, W. Zaremba, and Abbeel. Domain randomization for transferring deep neural networks from simulation to the real world. In IROS, pages 23-30. IEEE, 2017.
[28] E. Todorov, T. Erez, and Y. Tassa. MuJoCo: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 5026-5033 2012.
[29] C. K. Williams and C. E. Rasmussen. Gaussian processes for machine learning. MIT press Cambridge, MA, 2006.
[30] S.-S. Yang and C.-S. Tseng, An orthogonal neural network for function approximation. IEEE Transactions on Systems, Man, and Cyhernetics, Part B (Cyhernetics). 26(5):779-785, 1996.
[31] Z. Yuan, A. W. Hall, S. Zhou, L. Brunke, M. Greeff, Panerati, and A. P. Schoellig. Safe- control-gym: A unified benchmark suite for safe learning-base based control and reinforcement learning in robotics. IEEE Robotics and Automation Letters, 1(4::11422-1114) 2022.
12

---

A Hardware
All experiments use Intel 9th Generation i9 CPU and a Nvidia 2060 GPU with 6GB of memory.
B Faster Integration
When computing the approximate inner product between the true system and each basis function, necessary to compute G,(x(1o),4) (y) for every tuple in the dataset D. However, once the coefficients have been computed, is, in theory, no longer necessary to integrate the basis functions separately. From (5. approximate H
H(x(to),ty) ["' Σ (()(((()) 6.) dr,
by the linearity of the integral. In effect, than the basis functions themselves. As a result, inference for a specific set of coefficients can be decreased from requiring l integrations only single integration. However, we find this method to make less accurate predictions in practice, which may be due to our choice of integrator. This trick may be better suited to variable step-size integrators, which benefit more from reduced calls the integrator than RK4 does.
summing the gradients of each basis function rather
C Method of Integration
We leverage RK4 as the default integrator for this work, it can run a forward pass in milliseconds. There are more accurate integrators available, such as odeint. However, there is inherently a trade off with respect to both training time and execution time. Integrators such adaptive step size solvers can potentially make 20 or more calls to the neural ODE during a forward pass, while RK4 makes only 4. The increased number of neural ODE forward passes greatly increases memory usage and compute time. We experimented with more accurate integrators, but ultimately found this tradeoff to be unfavorable. Future work should investigate integrators that are fast, but achieve better accuracy than RK4.
D Overhead
While the function encoder algorithm alone has minimal overhead relative to MLP, this the case for neural ODEs due to the need for integration. The k neural ODEs may either be integrated sequentially or in parallel. If they are integrated sequentially, the memory overhead is lower, especially with respect to back-propagation. Thus, we find this useful for offline training, where the sequential method allows us to compute gradients for a larger batch of data at the cost of training time. In the
contrast, online execution generally favors inference speed over memory overhead. Thus, parallel method for online inference in the drone example, which requires much more memory but only small overhead of inference time relative to neural ODEs alone.
E Residuals Method Algorithm
Training a function encoder with the residuals method requires two separate loss functions. The first loss function trains Fary data from all datasets Dr. which means it effectively learns the expectation F given the training set. This loss function can be skipped Favg is a fixed function based prior knowledge.
The second loss function trains the basis functions. Unlike in Algorithm I the function being learned is =1) xx(;)) Fang (a(())))4) 1--1 - (z)). In other words, the residual between the data and the average function. This loss only used to train the basis functions, used train the average function. See Algorithm 2
13

---

Algorithm 2 The Residuals Method
1: Input: Set of datasets D, number of basis functions k
2: Output: Average function Favg and Neural ODE basis functions G1, G2 ...y G1
3: Initialize favg and - ..2. 9k neural networks with parameters 0 and 0 {01, 02,
4: while not converged do
5: // Train Average Function
6: loss L1 =0
7:
for all D; D do
8:
L1 L1 + mm
((((tt)) Fang(o(t,),t5+ -t
9:
end for
10:
0=6 VV
11:
// Train Basis Functions
12:
loss L2 =0
for all D2 D do
14:
for i E 1,...,k k do
15:
~ m-1 Σ
m j=
Coeg(x(t))))441 ( -1)1
16:
end for
La La -ド=1
(I(l)+11 x(4) Pang(x(1)))555) 4))
Eis Eizr (5t(((())),,4111--))))
18:
end for
19:
= Vo
20:
end while
Implementation Details
All baselines use the same training scheme. We use an ADAM optimizer with learning rate of le - 3, and gradient clipping with max norm of 1. NODE baselines uses 4 hidden layers of size 512, while FE NODE baselines uses hidden layers of size 51 for each basis function. Note this leads to approximately the same number of parameters for both approaches because the number of hidden parameters scales quadratically with the size of the hidden layers. All baselines train on 50 functions per gradient update via gradient accumulation. States are normalized to have O mean and unit variance.
random policy is used to collect data for the MuloCo environments. PIDD-ased exploratory policy, which moves to random nearby points, is used to collect data for the quadrotor since random policy collides with the floor. Evaluations are done on a holdout set collected through the same means.
All quadrotor baselines use the same MPC controller. The controller optimizes the actions through a combined sampling, gradient descent over 100 iterations. The episode is 100 steps, while the planning horizon steps. The controller optimizes 100 sample trajectories in parallel. and ultimately chooses the best one. Warm starting is used for following MPC calls improve performance. The cost function penalizes distance to the objective point, deviance from a stable horizontal position, velocity, and the difference between torques each rotor.
14

---

G Hyper-Parameter Ablations
G.1 Number of Basis Functions
0.25
FE NODE Bes.
0.15
0.6
1-8up
MSB
0.10
0.4
0.55
0.00
0.0
CheCChh
40
60
80
100
20
Number Basis Punccions
Time
Figure 6: We ablate the effect of the number of basis functions (k) on the performance of the learned model. Results are shown for the FE NODE Res. algorithm applied the Half Cheetah environment. The results indicate the the proposed approach is insensitive to the number of basis functions around k 100, while performance eventually decays as approaches
G.2 Number of Example Data Points
FE NODE Ro.
Exaamples
0.20
1-Sapp Lave
0.15
0.5
Half Cheetah
000
200
400
500
300
1000
0.0
20
Number of Eosmmpi Datapeists
Time Steps
Figure 7: We ablate the effect of the number of example data points on the performance of the learned model. Results are shown for the FE NODE Res. algorithm applied the Half Cheetah environment. The results indicate the the proposed approach is insensitive to increasing example dataset sizes, which suggests that 200 data points sufficient for the coefficients converge.
H Generalization
Inside Dara Regime
Figure 3: This figure shows the generalization capabilities of the proposed method. The black line indicates the ground truth Van Der Pol dy- namics, and the red line shows an approxima- tion. The model was trained on E [0.1, 3.0]. The left side of the figure shows Van Der Pol dynamics for values of that are within the dis- tribution of training environments, though each environment is unseen. The right shows the ap- proximation for μ = 4.0, which lies outside of the training distribution. The figure shows that the function encoder is able to reasonably gener- alize outside of its training set in this example.
p 20
- 30
15

---

I Orthonormality
Consider set of basis functions 91..... 9k- Suppose that : not orthonormal. Now consider function f. and suppose f happens to be in the span of 91.....9k 9k- Then f can be expressed as bbg g. where b of coefficients and g the concatenation of .r....s -k- The coefficients are calculated via the inner product,
(b 9,9)
(g).9))
(g ma)
c
=
(1.9k)
(6T g.gh)
(9k.
(gk 9h)
The loss function = - - el.. If then the loss will be Observe that = b and only if the Gram matrix is identity, and the Gram matrix is identity only for an orthonormal basis. In other words, the minimizer of the loss function is an orthonormal basis. Thus, in order for gradient descent to decrease loss, the basis functions converge towards orthonormality [14] This intuition is empirically validated [14,, Appendix A.5.
As final note, the coefficients can be computed via least squares after training. Least squares does not require an orthonormal basis uses the Gram matrix account for the inner products between basis functions.
16

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We claim to learn a space of dynamics spanned by a set of neural ODEs. These claims justified in the methods $3 validated in $4
where we show theoretical results, and empirically
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the key limitations of our approach in $5 including: computational
overhead. dependence training data, and the constraints integration.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
17

---

Answer: [NA]
Justification: Our method does not rely upon any new theorems or proofs. We reference existing results and theorems where appropriate in the text.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversell, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide a comprehensive description of the experimental results in $4 including: the architecture, integration scheme, the number of basis functions used, and descriptions of the data used for training. Further implementation details are provided in the
appendix in |
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
18

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: We provide code as zip file in the initial submission. A link to github repository will be provided in the final version.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
httppp.ccppubcccguuudeeee Codeeubmii ssi criionPooiccy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Implementation details are provided in the experimental results section and comprehensive details provided in the appendix +
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We provide confidence intervals for the first and third quartile in all figures. The significance of the error bars with respect to parameter variation and random seeds is thoroughly described in the text.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
19

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We describe the computing resources used to compute all results in the
appendix in $ Computational overhead is discussed in $
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http:://waaaapp..cpppp/pc.ccccta
Answer: [Yes]
Justification: Our work conforms with the NeurlPS guidelines and Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Our work has positive implications for the design and modeling of autonomous systems. We briefly discuss the broader impacts of our work in $8
Guidelines:
The answer NA means that there is no societal impact of the work performed.
20

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We do not provide model dataset with high risk for misuse.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
Justification: Our experiments do not rely upon any restricted licenses code, and provide citations to key libraries used.
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
21

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourced data or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: Our paper does not involve crowdsourced data or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
22

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
23


Reviews:
[{"review_id": "K2eV4hs7WH", "reviewer": "Reviewer_A5qk", "summary": "The paper aims to address the challenge of zero-shot transfer and adaptation. The authors propose tackling this challenge by learning a dynamics space spanned by neural ODE basis functions, which can then be used for rapid identification and adaptation to dynamics at inference time without additional training. The paper demonstrates the efficacy of the proposed approach, using both a simpler oscillator system and scaled up simulated robotics environments.", "questions": "See suggestions in the Weaknesses section above.", "limitations": "The authors address method limitations adequately in the paper.", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- While this is not my area of expertise, to the best of my knowledge, the method proposed in the paper and the corresponding experiments are novel.\n- The paper is well written and clear, and fast adaptation is a crucial problem to the field, especially in the field of robotics and control.\n- The experiments seem well designed and thorough, showing the contribution of each of the method components when ablated, and demonstrating the efficacy of the approach for control via MPC.", "weaknesses": "- To place the results in context with prior work, it would be additionally helpful for the robotics experiments to show comparisons to other methods that enable adaptation (e.g. training a model free method with domain randomized parameters). \n- It’s not clear how well the method will scale to more complex dynamics, or when trying to generalize beyond the learned basis functions (e.g., if they are not expressive enough or span the relevant spaces for a dynamical system, or if insufficient data is used to learn the dynamics space).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback. See below for responses to your suggestions. \n\n**To place the results in context with prior work, it would be additionally helpful for the robotics experiments to show comparisons to other methods that enable adaptation (e.g. training a model free method with domain randomized parameters).**\n\nDomain randomization allows RL algorithms to find policies which are robust to randomized parameters, however the policy does not adapt to those parameters. Our approach is inherently solving a different problem, in that our controller can adapt to the specific dynamics it is experiencing. This is done by using a small, online dataset to identify the coefficients of the current dynamics function. Then, the basis functions (in combination with the coefficients) are used to model the current system dynamics. A controller using this system model therefore adapts to the current dynamics in an online fashion.  \n\n**It’s not clear how well the method will scale to more complex dynamics, or when trying to generalize beyond the learned basis functions (e.g., if they are not expressive enough or span the relevant spaces for a dynamical system, or if insufficient data is used to learn the dynamics space).**\n\nThe ablation in the response to all reviewers, section 1a, shows that the basis functions are sufficiently expressive to span the space of dynamics present in the system. Furthermore, as $L_2$ is a Hilbert space, any function in  $L_2$ can be perfectly described as a linear combination of basis functions. In other words, basis functions are sufficiently expressive for the vast majority of problem settings. \n\nGeneralization beyond the function space in the dataset is a fascinating question. Please see the response to all reviewers, section 2 for more information.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zQxx0w4MOj", "reviewer": "Reviewer_3pa5", "summary": "This paper proposes a method to learn the dynamics of autonomous systems in a few shot manner. The core assumption is that the dynamics function dx/dt=f(x) of a new system can be modeled by a linear combination of basis dynamics functions. The method involves two stages. In the offline stage, the method learns a set of basis functions with neural ODE. In the online stage, the method uses incoming dynamics to fit the linear weights, which when combined with the learned basis, can be used to model the new system. The authors evaluated the effectiveness of the method on  Van der Pol Oscillato system (with varying parameters) and Mujoco Ant.", "questions": "1. In line 165, the authors said \"Given data collected online from a single trajectory\". I am curious to see ablations how the method improves as the amount of online data grow\n\n2. In figure 3, error goes up as the number of look-ahead steps increases. While this is partially expected due to compounding error, how much of it shall be attributed to limited expressiveness of linear basis. e.g. I can do an experiment where I train a NODE with many many data, not just 200 example points, and roll it out. You will witness how MSE loss increases and gain insights about a upper bound for prediction accuracy that's not due to limited data / limited number of basis", "limitations": "Please see my points in weakness", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "1. The writing is very clear. Readers with reasonable math background & ODE shall understand this paper reasonably well\n\n2. The core of the method is intuitive, and the method is sound. Basis function with linear weighting for quick adaptation has been grounded in many fields.\n\n3. I appreciate the fact that the authors explains the limitation and scope of the project, and there is no overclaim", "weaknesses": "1. My main criticism of the paper is the technical contribution. The problems this system can solve seem to be constrained to systems limited variation in parameter, where the offline dataset & online system share a high level of similarity. While the authors explained how they apply NODE very clearly, I think using NODE is not a fundamental contribution because you can also have other sequence models trained to predict residue. The idea of basis function has also been explored by many prior literatures. \n\n2. The title is misleading - you are doing quick online adaptation (few shot) instead of a zero-shot setting. A few trajectories of online data is required to adapt to the new system\n\n3. There are a lot of baselines the authors should look into since few shot learning / quick adaptation is a long standing topic of research. For example, other basis function methods, meta learning methods. I understand that the authors picked a setting where time is continuous, but the mujoco Ant environment is also traditionally studied as discrete too, so a meta learning + some sequence prediction model should be applicable here. If you can provide a convincing argument about why those priors works aren't applicable it's also fine.\n\n4. As I suggested in questions section, there could be many perspectives the author should dive deeper to improve the paper\n\n\nminor:\n1. In line 114, the reasoning doesn't provide grounding to the orthogonality question - the training objective make them span the space assuming diverse enough dynamic systems - but this is unrelated to orthogonality! Are you saying that non orthogonal is okay as soon as they span the space?\n\n2. It would be good to give more intuitive visualizations of estimation error in mujoco Ant. e.g. render the predicted trajectories as video\n\n3. ablation about the number basis functions used would be helpful to understanding", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**My main criticism of the paper is the technical contribution. The problems this system can solve seem to be constrained to systems limited variation in parameter, where the offline dataset & online system share a high level of similarity. While the authors explained how they apply NODE very clearly, I think using NODE is not a fundamental contribution because you can also have other sequence models trained to predict residue. The idea of basis function has also been explored by many prior literatures.**\n\n* We use hidden environmental parameters as an example problem setting because it is frequent in robotics. However, we never make use of the hidden parameters directly. As mentioned in Section 3, this algorithm can be applied to a space of dynamical systems that arises for *any* reason. \n\n* Generalization to OOD online systems is indeed an interesting direction. Please see section 2 of the response to all reviewers for a discussion. \n\n* We are interested in model-based control. Sequence prediction models, e.g. transformers, are often too computationally complex for model predictive control. SOTA control algorithms such as iCEM MPC need hundreds of forward passes per control-action. Transformers are well known to have quadratic cost with respect to input space, and their forward pass is much slower than neural ODEs, and so they are not suitable for this setting. Meta-learning algorithms can be applied under the same data assumptions, but often finetune or retrain the learned model based on new data. This training period is not amenable to real-time control. Lastly, there is indeed a plethora of analytical basis function methods. However, analytical basis functions often scale extremely poorly to high-dimensional function spaces. Other approaches like SINDy have similar scaling issues, but also require a potentially lengthy sparse coefficient identification procedure which can be too time-consuming for real time control. In contrast, our method enjoys the same mathematical interpretation as these approaches, while scaling to high-dimensional problems due to the use of a relatively small number of neural network basis functions. We have extended the related works to include information on why common sequence models and meta-learning are not applicable to this setting. \n\n**The title is misleading - you are doing quick online adaptation (few shot) instead of a zero-shot setting.**\n\nThe term “zero-shot” means different things to different communities. For example, the LLM community will often say “few-shot prompting” to mean operating a model with a few examples as input, and no finetuning. However, for ODEs, dynamics modeling, and reinforcement learning, “Zero-shot” is the ability to leverage unseen data at execution time to improve model performance without additional training, where the zero comes from zero gradient steps. See “Does Zero-Shot Reinforcement Learning Exist?” (Ahmed Touati, Jérémy Rapin, Yann Ollivier, 2022), “Unsupervised Zero-Shot Reinforcement Learning via Functional Reward Encodings“ (Kevin Frans, Seohong Park, Pieter Abbeel, Sergey Levine, 2024), “Zero-Shot Reinforcement Learning via Function Encoders” (Tyler Ingebrand, Amy Zhang, Ufuk Topcu, 2024), etc. \n\n**A few trajectories of online data is required to adapt to the new system.**\n\nFor the Mujoco examples, we only leverage 200 data points. For a system running at 30 Hz, this is equivalent to only ~7 seconds of data rather than a full trajectory. This data can then be used to identify the system dynamics immediately at runtime.  We have improved the experiment description in section 4.2 to clarify this. \n\n**In line 114, the reasoning doesn't provide grounding to the orthogonality question, ...,  Are you saying that non orthogonal is okay as soon as they span the space?**\n\nThank you for your comment, we have revised the wording of line 114 for clarity. Please see the response to all reviewers, section 3 for more information on orthonormality. \n\n**It would be good to give more intuitive visualizations of estimation error in mujoco Ant. e.g. render the predicted trajectories as video.**\n\nGood idea, we will add screenshots to the appendix.\n\n**Ablation about the number basis functions used would be helpful to understanding.**\n\nWe agree that the effect of the number of basis functions is an interesting ablation. Please see the response to all reviewers, section 1a. \n\n**In line 165, the authors said \"Given data collected online from a single trajectory\". I am curious to see ablations how the method improves as the amount of online data grow.**\n\nWe agree that examining the prediction accuracy as the amount of data increases is an interesting experiment. Please see the response to all reviewers, section 1b. \n\n**In figure 3, error goes up as the number of look-ahead steps increases. While this is partially expected due to compounding error, how much of it shall be attributed to limited expressiveness of linear basis. e.g. I can do an experiment where I train a NODE with many many data, not just 200 example points, and roll it out. You will witness how MSE loss increases and gain insights about a upper bound for prediction accuracy that's not due to limited data / limited number of basis.** \n\nPlease see the response to all reviewers, section 1, which may provide some more intuition on the effect of the example data and number of basis functions on the performance. While using more example data slightly improves performance, a key feature of our approach is that it can operate in low data settings. Additionally, the method is insensitive to the number of basis functions, which implies that the basis functions are more than sufficient to express the dynamics. Lastly, “NODE” and “Oracle” are also neural ODE models, and so they suffer from the same compounding error problems as our approach. Thus, those approaches provide a reference for how much of the error is due to the inherent difficulty of dynamics prediction.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uW8LueKVv1", "reviewer": "Reviewer_6dAJ", "summary": "The paper presents a novel framework for the zero-shot transfer of neural ODEs by leveraging function encoders to represent a space of dynamical systems. It demonstrates the method's effectiveness in adapting to unseen environments without retraining, using MuJoCo and quadrotor experiments.", "questions": "1. The computational overhead for training multiple neural ODEs might be significant. How does this affect the scalability and real-time applicability of your method?\n\n2. The paper does not enforce orthogonality explicitly for the basis functions, which might lead to suboptimal representation. Have you evaluated the levels of orthogonality (and their relationship with performance) in your method?\n\n3. The experiments lack diversity in evaluating the model's robustness to completely unseen environments or significantly different conditions. How confident are you in your model’s adaptability in such scenarios?\n\n4. The evaluation metrics used in the MuJoCo experiments are primarily focused on prediction accuracy. Have you considered additional metrics that might better capture your model's practical performance?  For example,  task-specific performance such as scores or cumulative rewards.\n\n5. It seems to lack a sufficient variety of baselines for comparison. Have you considered additional zero-shot / few-shot or multitask / meta-learning studies?", "limitations": "Refer to Weaknesses and Questions.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper demonstrates interesting ideas by introducing a novel framework for zero-shot transfer of neural ODEs using function encoders. This approach enables adaptation to unseen scenarios without retraining using the neural ODEs and function encoders. This research shows its potential to enhance the adaptability and safety of autonomous systems, bridging the gap between training and testing data.", "weaknesses": "The paper presents a promising framework for zero-shot transfer of neural ODEs, but there are several areas for improvement. Firstly, the reliance on a large and diverse dataset for training is a significant limitation. The approach requires extensive data that spans the entire function space of possible dynamics, which may not always be feasible. This dependency on comprehensive datasets should be addressed by exploring data-efficient learning methods or leveraging transfer learning techniques.\n\nSecondly, the computational overhead involved in training multiple neural ODEs is substantial. This might hinder the scalability and real-time applicability of the proposed method. The authors could investigate more efficient training algorithms or consider approximations that reduce computational costs without compromising accuracy.\n\nThirdly, the paper does not explicitly enforce the orthogonality of basis functions, relying instead on implicit regularization through the loss function. This might lead to suboptimal representation of the function space, affecting the model's performance. \n\nMoreover, the experiments lack diversity in evaluating the model's robustness to completely unseen environments or significantly different conditions. Including more varied test scenarios or stress tests could provide a deeper understanding of the model's adaptability and limitations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**The paper presents a promising framework for zero-shot transfer of neural ODEs, but there are several areas for improvement. Firstly, the reliance on a large and diverse dataset for training is a significant limitation. The approach requires extensive data that spans the entire function space of possible dynamics, which may not always be feasible. This dependency on comprehensive datasets should be addressed by exploring data-efficient learning methods or leveraging transfer learning techniques.**\n\nAny approach that aims to model a large set of dynamics from online data inherently must learn from a large dataset. This is because the space of functions is infinite dimensional, and this large, diverse dataset is effectively teaching the model which subspace is most important. Therefore, the ability to infer behavior from data necessitates the presence of a large training dataset. \n\nOur approach learns basis functions to span the functions present in the training set. However, it’s important to note that any function in the span of the basis can be perfectly represented, and thus our approach is an efficient transfer learning technique for this domain. However, examining strategies for further improving data efficiency is an interesting future direction. For example, it may be possible to augment the training dataset with additional common functions, such as linear transformations between inputs and outputs or trigonometric functions, as this would increase the diversity of the training dataset without requiring more data. \n\n**Secondly, the computational overhead involved in training multiple neural ODEs is substantial. This might hinder the scalability and real-time applicability of the proposed method. The authors could investigate more efficient training algorithms or consider approximations that reduce computational costs without compromising accuracy.**\n\nOverhead is indeed an important consideration, and we have two strategies to address this. During the offline training phase, the key consideration is memory usage. This is because neural networks converge faster with larger batch sizes, and so leveraging the largest batch size possible is key. To address this, we run the basis functions sequentially during training. Since only one basis function is being called at a time (both during the forward pass and back-propagation), the memory overhead is the same as a single neural ODE. This does increase the training time by a factor of $k$. However, as neural ODEs alone cannot model multiple dynamics functions, we view this cost as acceptable for the benefits our approach provides. \n\nDuring the online phase, memory usage is inherently reduced as all memory usage results from online data only. So, memory is not a concern. Instead, the compute speed is the most important factor for real-time control. To address this, we run the basis functions in parallel. As there is no dependence between them, each basis function can run at the same time. Thus, by running the basis functions in parallel, there is no additional compute time overhead. Therefore, this method is equivalent to a vanilla neural ODE with respect to real-time compute speeds. \n\nWe have improved the discussion in the appendix, section C, to include this additional information. \n\n**Thirdly, the paper does not explicitly enforce the orthogonality of basis functions, relying instead on implicit regularization through the loss function. This might lead to suboptimal representation of the function space, affecting the model's performance.**\n\nPlease see the Response to all Reviewers, section 3. \n\n**Moreover, the experiments lack diversity in evaluating the model's robustness to completely unseen environments or significantly different conditions. Including more varied test scenarios or stress tests could provide a deeper understanding of the model's adaptability and limitations.**\n\nVerifying OOD generalization capabilities is an interesting future direction. Please see the response to all reviewers, section 2. \n\n**The experiments lack diversity in evaluating the model's robustness to completely unseen environments or significantly different conditions. How confident are you in your model’s adaptability in such scenarios?**\n\nPlease see the response to all reviewers, section 2. It’s worth noting that the test environments are sampled from the same distribution of environments, but are unseen by the model during training. \n\n**The evaluation metrics used in the MuJoCo experiments are primarily focused on prediction accuracy. Have you considered additional metrics that might better capture your model's practical performance? For example, task-specific performance such as scores or cumulative rewards.**\n\nThe MuJoCo experiments are designed specifically to highlight prediction accuracy. We agree that other metrics might be useful as well, and so we leverage a task-specific efficiency metric in the quadrotor experiments. For more information see section 4.3, especially the last paragraph. \n\n**It seems to lack a sufficient variety of baselines for comparison. Have you considered additional zero-shot / few-shot or multitask / meta-learning studies?**\n\nWe are unaware of any other zero-shot techniques for neural ODEs. It is possible to use a transformer for the same type of data, however transformers suffer notable drawbacks. First, their memory usage is quadratic, and so they are unable to make use of large, online datasets. Second, their forward pass time is quite slow, and so they are not suitable for real time model-predictive control as many forward passes must be made every timestep, e.g. 100s of forward passes in 30 milliseconds. Meta-learning techniques are similar in that they adapt models given instance-specific data. However, these techniques perform additional finetuning or retraining based on this data, which makes them ill-suited for real-time control.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bACZs3peGC", "reviewer": "Reviewer_w5ay", "summary": "The work explores the use of neural ODEs as basis functions for function encoders. This requires dealing with the additional integration step, with the weighted combination of obtained ODEs representing the behaviour of the function to approximate via its integral. With an inner product and its tractable Monte Carlo estimation scheme, the authors derive an algorithm to train basis functions which span the space of dynamical systems, where each basis function is a neural ODE. The FE + ODE approach is put to the test against ODE and FE schemes for fitting a space of Van der Pol systems, long term prediction for RL, as well as quadrotor MPC control with improvements in zero-shot performance against the baselines used.", "questions": "- The term zero-shot does indeed refer to the ability of networks to perform in novel circumstances without retraining, and does apply here. However, it can be slightly misleading as adaptability here requires analysis of new data from the new setting. Can the authors elucidate the scales involved in the tradeoff? In other words, when does it become interesting to train 100 base models (on the 100 datasets) to gain a deployment advantage that still requires new data to function, versus fine-tuning one model or retraining?\n- Do the authors have any insights on changes in levels of performance whether the new ODE is in the convex hull of the available basis or outside it? (for example in the Van der Pol example, if the basis datasets contain trajectories for values of $\\mu$ between 0.1 and 3, can the system perform for $\\mu = 5$ ?", "limitations": "- A major numerical challenge working with continuous time neural networks is that of integration, with a plethora of solvers and schemes available. The authors touch upon this topic in their limitations sections and the impact of the integration horizon selection on behaviour predictability as well as the compute overhead involved (also in appendix C). It would have been interesting to give readers a better sense about the tradeoffs with numerical comparisons that go beyond verbal description.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The authors deal with a very important problem in machine learning and all the more in robotics: that of zero-shot generalisation to changes in dynamics/environment (here modelled via hidden parameters)\n\n- Interesting extension of the FE paradigm with a clear derivation of the required additions to adapt the basis function algorithm to neural ODEs\n- Good readability and presentation\n- Relatively significant combination of neural ODEs and Function Encoders with interesting performance improvements", "weaknesses": "- Some clumsiness in the math notation: in equation (3) $N$ is used at the denominator when it should be $m$ instead.\n- The assumption of orthogonality for the validity of the coefficients is vital. However, both the authors in [13] and in this paper do not share any analysis on the validity of this assumption. It would be interesting to see how well this holds in practice (computing inner product of obtained basis functions/ODEs should be easy) and how many iterations are required to reach orthogonality in the examples provided.\n- The introductory paragraph of section 3 where the link between a new unknown function $f$, its observed trajectory $\\mathcal{D}$ and ODE basis functions is made, could benefit from more motivation and explanation. It is not immediately clear why reasoning at the derivative level while dealing with integral trajectory data is more advantageous. Perhaps a figure would help as the manuscript is text heavy.\n- Explanations of the MuJoCo results are not very clear to me, they seem more like observations than interpretations. Why is the oracle unstable (does it require more data and training given the conditioning on hidden params)? What explains the differences between both experiments: FE + RES does well on half cheetah (better than FE + NODE) and terrible on ant? Where does the lack of inductive bias intervene in this case ?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**The assumption of orthogonality for the validity of the coefficients is vital. However, both the authors in [13] and in this paper do not share any analysis on the validity of this assumption. It would be interesting to see how well this holds in practice (computing inner product of obtained basis functions/ODEs should be easy) and how many iterations are required to reach orthogonality in the examples provided.**\n\nPlease see the response to all reviewers, section 3. Also section A.5 in [13] may be of interest. \n\n**The introductory paragraph of section 3 where the link between a new unknown function $f$, its observed trajectory $D$, and ODE basis functions is made, could benefit from more motivation and explanation. It is not immediately clear why reasoning at the derivative level while dealing with integral trajectory data is more advantageous. Perhaps a figure would help as the manuscript is text heavy.**\n\nTo clarify, in continuous time dynamical systems, the dynamics is inherently a differential equation that is being integrated through time. However, it is typically the case that system observations occur at discrete time intervals, and do not include derivative measurements. Hence, neural ODEs are advantageous because they allow you to learn a model of the true system derivatives while using integral trajectory data. \n\n**Explanations of the MuJoCo results are not very clear to me, they seem more like observations than interpretations. Why is the oracle unstable (does it require more data and training given the conditioning on hidden params)? What explains the differences between both experiments: FE + RES does well on half cheetah (better than FE + NODE) and terrible on ant? Where does the lack of inductive bias intervene in this case ?**\n\nThe apples-to-apples comparison is between FE + Res. and FE + NODE + Res. These two approaches differ only in basis function architecture, and FE + NODE + Res. consistency outperforms. The other relevant comparison is FE + NODE + Res. and the Oracle. These two approaches both benefit from the inductive bias of neural ODEs, and both have information on the hidden parameters. However, the format of the information is different. Oracle gets this information explicitly as an additional input, whereas FE + NODE +Res. implicitly gets this information via the coefficients of the basis functions. We expect that Oracle is performing badly because it is generalizing to an entire space of dynamics with one NODE, which is a very complex function to learn. We believe that Ant is much more difficult and varied than Cheetah, and the relative ease of Cheetah is likely why both Res. methods perform so well on Cheetah. We have improved the discussion in section 4.2 to include these details. \n\n**The term zero-shot does indeed refer to the ability of networks to perform in novel circumstances without retraining, and does apply here. However, it can be slightly misleading as adaptability here requires analysis of new data from the new setting. Can the authors elucidate the scales involved in the tradeoff? In other words, when does it become interesting to train 100 base models (on the 100 datasets) to gain a deployment advantage that still requires new data to function, versus fine-tuning one model or retraining?**\n\nTwo things to note. Rather than training a function for every dataset, we train $k$ basis functions to span all datasets. So, if we have 100 datasets but only want to train 10 basis functions, this is possible without issue. Secondly, it does require a small amount of online data. This data is necessary for any method because you need some information to identify the current dynamics. For example, finetuning a neural ODE  based on online data is another approach, but this also requires online data. Indeed, finetuning a model likely requires even more data than our approach, and incurs a significant computational overhead that is going to take significantly longer than our approach. In contrast, we can compute the coefficients of the basis functions in milliseconds, as the inner product calculation is effectively a sample mean, and then we can instantly adapt our predictions based on the data. \n\n**Do the authors have any insights on changes in levels of performance whether the new ODE is in the convex hull of the available basis or outside it? (for example in the Van der Pol example, if the basis datasets contain trajectories for values of $\\\\mu$ between 0.1 and 3, can the system perform for $\\\\mu=5$?**\n\nExamining OOD generalization is an interesting future direction. Please see the response to all reviewers, section 2, for the experiment you described. \n\n**A major numerical challenge working with continuous time neural networks is that of integration, with a plethora of solvers and schemes available. The authors touch upon this topic in their limitations sections and the impact of the integration horizon selection on behaviour predictability as well as the compute overhead involved (also in appendix C). It would have been interesting to give readers a better sense about the tradeoffs with numerical comparisons that go beyond verbal description.**\n\nWe leverage RK4, and there are indeed more accurate integrators available. However, there is inherently a trade off with respect to both training time and execution time. Integrators such as adaptive step size solvers can potentially make 20 or more calls to the neural ODE during a forward pass, while RK4 makes only 4. The increased number of neural ODE forward passes greatly increases memory usage and compute time. We experimented with better integrators, but ultimately found this tradeoff to be unfavorable. We find RK4 to be the right balance of speed and accuracy for this work. We have added an additional section of the appendix discussing the choice of integrator.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
# Response to all Reviewers:

We thank the reviewers for their comments and keen insights. We have made the following major changes to the paper in response to their feedback. 

## **1. Hyper-Parameters**:  

We have added an ablation on both the number of basis functions and the number of example data points.  
    
### **1a. Number of Basis Functions**: 

We ablate different numbers of basis functions at predicting dynamics in the Half-Cheetah environment. We find that the algorithm is generally insensitive to the number of basis functions, and performance eventually decays as the number of basis functions approaches 0. See the attached PDF for the corresponding figure. The results indicate that 100 basis functions are sufficiently expressive for the dynamics of the ablated environment.  

### **1b. Number of Example Data Points**: 

We ablate the sensitivity to the number of data points on predicting dynamics in the Half-Cheetah environment. We find that increasing data only slightly improves performance and that there are diminishing returns. See the attached PDF for the corresponding figure. Note that if the function is within the span of the basis, then its prediction accuracy only depends on the error between the true basis function coefficients and the Monte Carlo estimate of the coefficients. As the estimate is computed via a sample mean, the error in this prediction decreases to 0 as more data is collected due to the law of large numbers.  


### **2. Generalization**: 

For dynamics functions outside of the training set, it can be shown that any function within the span of the basis can be perfectly computed. Functions that lie outside the span of the basis may still have low error if they are sufficiently close to the learned subspace. To illustrate this point, we train the Van Der Pol example on $\mu \in [0.1, 3.0]$, and evaluate it on $\mu=4.0$.  We observe that the model is still able to reasonably approximate this out-of-distribution $\mu$. See the figure in the attached PDF. Exploring the exact OOD generalization capabilities of this algorithm, along with possible error bounds, is an active direction of future work, but is outside the scope of this paper. 

### **3. Orthonormality**: 

This discussion relates to the function encoder algorithm of the paper “Zero-Shot Reinforcement Learning via Function Encoders”. Nonetheless, we present the following discussion for clarity as it is relevant to this work. 

Consider a set of basis functions $g_1, …, g_k$. Suppose that $g_1, …, g_k$ is not orthonormal. Consider a function $f$, and suppose $f$ happens to be in the span of $g_1, …, g_k$. Then $f$ can be expressed as $f=b^\top g$, where $b$ is a set of coefficients and $g$ is the concatenation of $g_1, …, g_k$. Consider the coefficients calculated via the inner product, 

$c^\\top =\\begin{bmatrix} \\langle f, g_1 \\rangle \\\\ \\vdots \\\\ \\langle f, g_k \\rangle \\end{bmatrix} =\\begin{bmatrix} \\langle b^\\top g, g_1 \\rangle \\\\ \\vdots \\\\ \\langle  b^\\top g, g_k \\rangle \\end{bmatrix} =b^\\top \\begin{bmatrix} \\langle g_1, g_1 \\rangle & … &  \\langle g_1, g_k  \\rangle \\\\ \\vdots & \\ddots & \\vdots \\\\ \\langle g_k, g_1 \\rangle & … & \\langle  g_k, g_k \\rangle \\end{bmatrix}$

Consider the loss function $l=\\vert f - \\hat{f} \\vert ^2=\\vert f - c^\\top g \\vert ^2$. If $c=b$, then the loss will be 0. Observe that $c=b$ if and only if the Gram matrix is identity, and the Gram matrix is identity only for an orthonormal basis. In other words, the minimizer of the loss function is an orthonormal basis. Thus, in order for gradient descent to decrease loss, the basis functions converge towards orthonormality.
This intuition is empirically validated in “Zero-Shot Reinforcement Learning Via Function Encoders”, Appendix section A.5. Please see that paper for more information. 

As a final note, as mentioned in section 2.2, line 106, the coefficients can be computed via least squares after training. Least squares does not require an orthonormal basis as it uses the Gram matrix to account for the inner products between basis functions.

In summary, it has been shown in prior works that the basis functions converge towards orthonormality. As stated in section 2.2, we may also use least squares at execution time, which sidesteps the issue. We have added this discussion to the appendix.

Meta-review: This paper presents a neural ODEs-based method for modeling differential equations by learning basis functions, focusing on capturing system dynamics of autonomous systems within a structured function space. 

The paper was reviewed by four reviewers who all are inclined to suggest for acceptance. The most reviewers agree that (1) the proposed method, the use of neural ODEs as function encoders, is intuitive and sound, (2) dealing with important problems, zero-shot generalization to dynamic environments, in machine learning and autonomous systems. The reviewers also commented that (3) the paper is well written with good readability and presentation. 

Considering these strengths, this paper is recommended for acceptance. The authors are encouraged to utilize their constructive and detailed discussion with reviewers towards further raising the quality and impact of their work.
Decision: Accept (poster)