Neural Dynamical Systems: Balancing Structure and Flexibility in Physical Prediction
Authors: Viraj Mehta
Ian Char
Willie Neiswanger
Youngseog Chung
Andrew Oakleigh Nelson
Mark D Boyer
Egemen Kolemen
Jeff Schneider
Venue:  (2021)
Keywords: nuclear fusion
physics
differential equations
dynamical systems
control
dynamics

Abstract: We introduce Neural Dynamical Systems (NDS), a method of learning dynamical models in various gray-box settings which incorporates prior knowledge in the form of systems of ordinary differential equations. NDS uses neural networks to estimate free parameters of the system,  predicts residual terms,  and numerically integrates over time to predict future states.  A key insight is that many real dynamical systems of interest are hard to model because the dynamics may vary across rollouts.  We mitigate this problem by taking a trajectory of prior states as the input to NDS and train it to dynamically estimate system parameters using the preceding trajectory. We find that NDS learns dynamics with higher accuracy and fewer samples than a variety of deep learning methods that do not incorporate the prior knowledge and methods from the system identification literature which do.  We demonstrate these advantages first on synthetic dynamical systems and then on real data captured from deuterium shots from a nuclear fusion reactor. Finally, we demonstrate that these benefits can be utilized for control in small-scale experiments.

Under review as a conference paper at ICLR 2021
NEURAL DYNAMICAL SYSTEMS: BALANCING STRUC- TURE AND FLEXIBILITY IN PHYSICAL PREDICTION
Anonymous authors Paper under double-blind review
ABSTRACT
We introduce Neural Dynamical Systems (NDS), method of learning dynamical models in various gray-box settings which incorporates prior knowledge in the form of systems of ordinary differential equations. NDS uses neural networks to estimate free parameters of the system, predicts residual terms, and numerically integrates over time to predict future states. A key insight is that many real dy- namical systems of interest are hard to model because the dynamics may vary across rollouts. We mitigate this problem by taking trajectory of prior states as the input to NDS and train it to dynamically estimate system parameters using the preceding trajectory. We find that NDS learns dynamics with higher accuracy and fewer samples than variety of deep learning methods that do not incorporate the prior knowledge and methods from the system identification literature which do. We demonstrate these advantages first on synthetic dynamical systems and then on real data captured from deuterium shots from a nuclear fusion reactor. Fi- nally, we demonstrate that these benefits can be utilized for control in small-scale experiments.
INTRODUCTION
The use of function approximators for dynamical system modeling has become increasingly com- mon. This has proven quite effective when substantial amount of real data is available relative to the complexity of the model being learned (Chua et al., 2018 Janner et al. 2019 Chen et al.)
1990). These learned models are used for downstream application such as model- reinforce- ment learning (Nagabandi et al. 2017 Ross & Bagnell 2012) or model-predictive control (MPC)
(Wang & Ba 2019).
Model-based control techniques are exciting as we may be able to solve new classes of problems with improved controllers. Problems like dextrous robotic manipulation (Nagabandi et al., 2019). game-playing (Schrittwieser et al. 2219)), and nuclear fusion are increasingly being approached us- ing model-based reinforcement learning techniques. However, learning a dynamics model using, for example, deep neural network require large amounts of data. This is especially problematic when trying to optimize real physical systems, where data collection can be expensive. As an alter- native data-hungry machine learning methods, there is also long history of fitting models to system using techniques from system identification, some of which include prior knowledge about the system drawn from human understanding (Nelles) 2013 Liung et al. 2009: [Shllbeg & Jacob- sen 2008). These models, especially in the gray-box setting, are typically data-efficient and often contain interpretable model parameters. However, they the given prior knowledge is approximate or incomplete in nature. They also do not generally adapt the situation where trajectories are drawn from a variety of parameter settings at test time. This an especially crucial point many systems of interest exhibit path-dependent dynamics, which we aim to recover on the fly.
well suited for the situation where
In total, system identification methods are sample efficient but inflexible given changing parameter settings and incomplete or approximate knowledge. Conversely, deep learning methods are more flexible at the cost of many more samples. In this paper, we aim to solve both of these problems by biasing the model class towards our physical model of dynamics. Physical models of dynamics often given in the form of systems of ordinary differential equations (ODEs), which are ubiqui- tious and may have free parameters that specialize them to given physical system. We develop a model that neural networks to predict the free parameters of an ODE system from the previous

---

Under review as a conference paper at ICLR 2021
timesteps as well residual terms added to each component of the system. To train this model, we integrate over the ODE and backpropagate gradients from the prediction error. This particular combination of prior knowledge and deep learning components is effective in quickly learning the dynamics and allows us to adjust system behavior in response to a wide variety of dynamic parame- ter settings. Even when the dynamical system is partially understood and only a subset of the ODEs are known, we find that our method still enjoys these benefits. We apply our algorithm to learning models in three synthetic settings: generic model of ballistics, the Lorenz system (Lorenz 1963), and a generalized cartpole problem, which we use for control as well. We also learn a high-level model of plasma dynamics for a fusion tokamak from real data.
The contributions of this paper are
We introduce Neural Dynamical Systems (NDS), new class of model for learning dynamics that can incorporate prior knowledge about the system.
We show that these models naturally handle the issue of partial or approximate prior knowledge, irregularly spaced data, and system dynamics that change across instantiations, which gener- alizes the typical system identification setting. We also show that these advantages extend to control settings.
We demonstrate this model's effectiveness on real dynamics problem relevant to nuclear fusion and on synthetic problems where we can compare against a ground truth model.
2 RELATED WORK
System Identification and Deep Learning with Structure There is long tradition of forecasting physical dynamics with either machine learning techniques based domain knowledge of the dynamics, especially in the field of system identification, where Ljung (2010), Schoukens & Ljung (219)) and[Cressie & Wikle 2015) are good summaries. Often, this space is discussed spectrum from a purely soooo--noweedeebaar with several shades of gray in between.
system (white-box) to a purely data-driven system (black-box)
White-box models use prior knowledge to precisely give the relationship between quantities of in- terest over time and there is extensive literature on solving them (Brenan et al. 1995). 'Shades of gray" may distinguish between levels of prior knowledge or how equations cover subsets of the state space [Ljung] 2010). Other prior work focuses on online parameter estimation (Vahidi et al.) (005). but this relies on an ongoing trajectory through the system and is difficult to use in our setting.
In nonlinear black-box settings, there are a variety of techniques used to solve system identification models. Volterra series, a generalization of Taylor series which respects dependency on the past, have been used for system identification (Rugh 198I). Block models such as the [ammersttei (930) and Weiner (Billings 1980) models and their combination can also identify systems. Feed- forward and recurrent neural networks have been widely used to model dynamical systems (Chua) et al. 2018 Nagabandi et al.
2017: Hochreiter & Schmidhuber 1997), with additional constraints on stability (Mankk& & Kolter 2020) or the Hamiltonian (Chen et al. and many others added.
Nonlinear autoregressive moving average models with exogenous variables (NARMAX) have also been used widely to model dynamical systems and this class is a superset of nearly everything else discussed (Brunton al.) 2015 Rahrooh & Shepard 2009). Broadly, none of these algorithms are well-suited to setting where the dynamic parameters of the system change across rollouts.
There have also been several approaches for including physical structure in deep models. Raissi et al. (2019) use automatic partial derivative computation to force a neural network to fit a given ODE or PDE solution. de Avila Belbute-Peree et al. (2018) uses a linear complementarity problem to differentiate through 2d physics simulations however their method is not general to more dimensions or other types of problems besides mechanics. Cranmer et al. (2019) uses graph networks to discover physical laws. Chen et al. (2019). Sanchez- et al. (2019) and (ndanne et al. (2020) force the network to respect Hamiltonian and Lagrangian constraints but without specific problem data on the system. PPsihooiios & Ungar 1992) predicts physical parameters for given ODE model and Rico-Martinez et al. (1994) predict residuals. Thompson & Kramer (1994) similarly builds a hybrid parameter prediction function into a dynamical model. These last three works are especially similar to ours, though they use tiny networks, are groblem-specifee in their setup, and don't 't take advantage of backpropagation through numerical ODE solver.
2

---

Under review as a conference paper at ICLR 2021
Neural Ordinary Differential Equations As most numerical ODE solvers are algorithms involv- ing differentiable operations, it has always been possible in principle to backpropagate through the steps of these solvers dating back to at least Runge (1895). However, since each step of the solver involves calling the derivative function, na've backpropagation incurs an (n) memory cost, where is the number of derivative calls made by the solver. Historically Pontryagin (2018) and recently Chen et al. (2018) demonstrated that by computing gradients through the adjoint sensitivity method, the memory complexity of backpropagating through family of ODE solvers can be reduced to ((1) for a fixed network, as opposed to the naive (nn). However, this work only used generic neu- ral networks as the derivative function and did not consider dynamics modeling. They also provide a PyTorch package which we have built off of in our work.
There has been some work using neural ordinary differential equations to solve physical problems. Portwood et al. (2019) used a fully-connected neural ODE with an RNN encoder and decoder to model Navier-Stokes problems. |Rudy al. (2019) used neural network integrated with a Runge- Kutta method for noise reduction and irregularly sampled data. There has also been work learning the structure of dynamical systems, first with a convolutional warping scheme inspired by advection- diffusion PDEs (de Bezenac et al., 2018), then with Neural ODE which was forced to respect boundary conditions and a partial observation mechanism (Ayed et al. 2019).
Machine Learning for Nuclear Fusion As far back as 1995, van Milligen et al. (1955) showed that by approximating the differential operator with a (single-layer, in their case) neural network, one could fit simple cases of the Shaa-Shharraov equation for magnetolydrodynamic equilibria. Recent work has shown that plasma dynamics are amenable to neural network prediction. In particular, Kates-Harbeck et al. (2019) used a convolutional and LSTM-based architecture to predict possible plasma disruptions (when a plasma instability grows large and causes a loss of plasma containment and pressure).
There has also been work in the field of plasma control: a neural network model of the neutral beam injection for the DIII-D tokamak has been deployed in order to diagnose the effect of controls on shots conducted at the reactor (Boyer et al., 2019b). Additionally, (Boyer et al. 2019a) used classic control techniques and a simpler model of the dynamics to develop controller that allows characteristics of the tokamak plasma to be held at desired levels. Others have used contextual Bayesian optimization to choose single-state controls which direct the plasma to desirable states (Char et al. 2019 Chung et al. 2020).
PROBLEM SETTING
Typically, a dynamical system & (a, u, () with some parameters Ø is the conventional model for system identification problems. Here, state is X, control is E U, and time is t E IR. The objective is to predict future states given past states, past and future controls, and prior knowledge of the form of f. We denote ((,, I, no) = xo + fo fo( t)dl as the state obtained by integrating our dynamical system around f to time
We consider in this work more general setting and address the problem of prediction and control over class of dynamical systems, which we define as the set (a = fo u, Φ b) where & is the space of parameters for the dynamical system (e.g. spring constant or terminal velocity). We can generate a trajectory from a class by sampling a 6 P(Φ) for some distribution P and choosing initial conditions and controls. In real data, we can view nature as choosing, but not disclosing, ø. For a particular example j. we sample @ ~ P(D) and xi ~ P(Xo) and are given controls u indexed as u(() and input data {n(Ø) li, u, 2o during training. At test time, we give shorter, prefix time series {a(ω, t;, u. (o)) T but assume access future controls. Then the prediction objective for class of systems for N examples for timesteps [t:]F is
T
f = arg min Po0)
|| t,, u, xo) - 11,
(1)
This objective differs from the traditional one in that implicitly, identifying & for each trajectory needs to be done from the problem data in order be able to predict the data generated by fa-
3

---

Under review as a conference paper at ICLR 2021
x²
Rosidual Prediction
X
System Parameter
ODE Solver
Prior Know Knowledgg ODKs
Estimation
x
u²
fly x]
g(x)
= x(p - 2)
xy 32
x u
An example Neural Dynamical System. Here, blue boxes are fully connected neural networks, gray boxes are problem data and output, the green box the prior knowledge dynamical system, the purple box is data output by ODE solver to query derivatives, and the black box is an ODE solver. The ODEs and system parameters are problem dependent, but here we consider the Lorenz system (defined in Example 1) as example. Our notation for is unfortunately overloaded by our method and the Lorenz system-the the x from our method is bolded in the figure.
Figure 1:
Similarly, the control problem is
V.," c(u(t), s.t. x(t) = Io =o [( (al,
[.
u = min P(O))
(2)
for some cost functional e. We will primarily explore the prediction problem in this setting, but as secondary considerations, we explore robustness to noise, the ability to handle irregularly spaced input data, and the ability to recover the parameters ø which generated the original trajectories. We will also consider the control problem in simple setting.
METHODS
We build up the description of our proposed method by first describing the two methods that inspire it: gray box system identification through optimization (Ljung et al., 2009), and using a Neural ODE (Chen et al. 2018) predict future states in a dynamical system.
To apply grey box optimization [Ljung et al. 2009) to dynamical system i fo( u, f) for
problem data (I,, Y would use nonlinear least squares (Coleman Li 1996) to find
= arg min Σ 111 fa u(t), dt - 1/" fz u(t),
(3)
This makes good use of the prior knowledge component of our system but prone to compounding errors through integration and does not leverage data that may have come from alternate system parameters.
A data driven approach would be to minimize the same objective with fully connected neural ODE (Chen et al. 2018) he in place of f. However, we find that this procedure requires large amounts of training data and doesn't 't leverage any prior knowledge might have, though it is flexible to classes dynamical systems.
We define Neural Dynamical System by taking the advantages of both these methods in the setting where we know the full and correct ODEs and then show how to generalize it to situations where only some ODEs known they are approximate. Specifically, Neural Dynamical System (NDS) is a class of dynamical systems where neural network predicts some part of fo(2, u, usually parameters
term which is added to f.
NDS with Full System Dynamics Consider class of dynamical systems defined in Section where x E IR", u E RTh & E Rd dr, de N and let 6, 0, T be trainable neural network weights. Let he( (t,er) be neural net mapping state history and control sequence to the dp parameters of the system ò and an embedding òy E dd Also let co (2,, ue) be a similar network taking a single state and control that outputs an embedding be E Rde Finally, let d, (bb, be) be network which takes the two output embeddings from the previous network and outputs residual terms f. Intuitively, would like the observed history estimate our system parameters, and some combination

---

Under review as a conference paper at ICLR 2021
of the observed history and current observation to estimate residuals, which influences the design of our model, the neural dynamical system (a visualization of which is shown in Figure D. written
④ = (T Me: t) +i T, bn.
ha 1 Uterr)
be
Cal Ue)
f
(b ba)
(4)
Prior knowlodge
History encoder
Context encoder
Residial prediction
where 9 are domain-specifer ODEs which are the input 'domain knowledge' about the system being modeled. Note that if the prior knowledge is identically zero, this method reduces to the Neural ODE predictions we discussed at the beginning of this section. We also study an ablated model, NDS0, which lacks the residual component r and context encoder network dr. We note here that the context encoder intended to potentially correct for model misspecification and noise but in the noiseless case with model which is perfect, it may not be necessary. We explore this throughout Section 5
Example 1: Lorenz system. To illustrate the full construction, we operate on the example of the the Lorenz system: a chaotic dynamical system originally defined to model atmospheric processes (oorenz 1963). The system has 3-dimensional state (which we'll denote by z, y, 2). 3 parameters, P, G. and 3, and no control input. The system is given by
i oy - x) " = x) y zy Bz.
(5)
For given instantiation of the Lorenz system, have values of [8, o, p] that are constant across the trajectory. So, we can instantiate he which outputs 3- |3. à p). We use the DOPRIS method (Dormand & Prince| 1980) to integrate the full neural dynamical system in Equation with g given by the system in Equation 5 using the adjoint method of Chen et al. 2018)) We use the state aT as the initial condition for this integration. This gives sequence { T which we evaluate
and supervise with a loss of the form
T Σ (=
((0,) /T ( =
10 - -1,,11
(6)
Because of the way we generate our input data, this is equivalent to Equation [ We assume in our setting with full dynamics that the true dynamies lie the function class established Equation By the method in Chen et al. (2018) we can backpropagate gradients through this loss into the parameters of our NDS. Then algorithms the SGD family will converge to a local minimum of our loss function.
4
NDS with Partial System Dynamies Suppose we only had prior knowledge about some of the components of our system and none about others. We can easily accomodate this incomplete infor- mation by simply 'zeroing out' the function. This looks like
(2) (o) (a, () if g is known, g [80
(0(2, u. 1)
else.
substituted into equation 4 this setup, the residual term essentially makes the unknown dimen- sions unstructured Neural ODEs, which still can model time series well (Portwood et al. 2019).
NDS with Approximate System Dynamics For Neural Dynamical Systems to be useful, they must handle situations where the known model approximate. This is transparently handled by our formulation of Neural Dynamical Systems: the parameters of the approximate model I are predicted by ho (2I,,, u1.T") and the residuals f are predicted by d- (b,,, be). This is the same as in the case where we have the correct dynamics, but we remove the assumption of a perfect model.
Example 2: Nuclear Fusion System. In this paper, we apply this technique to plasma dynamics in tokamak, two quantities of interest are the stored energy of the plasma, which we denote E and its rotational frequency, W. The neutral beams and microwave heating allow us to add power (P) and torque (T) to the plasma. The plasma also dissipates energy and rotational mo- mentum via transport across the boundary of the plasma, radiative cooling, and other mechanisms. While the detailed evolution of these quantities is described by turbulent transport equations, for the purposes of control and design studies, physicists often reduced, volume- aoveme-averaged models. The
a tokamak.
5

---

Under review as a conference paper at ICLR 2021
simple linear model (up to variable parameters) used for control development in Boyer et al. (2019a) is used in this work.
E
T
==P
(8)
Fm
Here, n, is ion density, m, is ion mass, and Ro is the tokamak major radius (values are in A.4 We use the constant known values for these. Te and Tm the confinement times of the plasma energy and angular momentum, which we treat as variable parameters (because they are!). These are predicted by the neural network in our model. We again use the model in Equation 4 to give us a neural dynamical system which can learn the real dynamics starting from this approximation in Section
5 EXPERIMENTS
In the following experiments, we aim to show that our methods improve predictions of physical systems by including prior dynamical knowledge. These improvements hold even as we vary the configurations between structured and fluid settings. We show that our models learn from less data and are more accurate, that they handle irregularly spaced data well, and that they learn the appropriate parameters of the prior knowledge systems even when they only ever trajectories.
We use L2 error as our evaluation measure for predictive accuracy as given by Equation 6 We also evaluate model's ability to predict the system parameters by computing the L2 error, i.c. Ex 1... In the settings where we use percent change in L2 error of the trajectories relative to noise- or jitter-free baseline for the same experiment. We believe this is the appropriate metric as it abstracts away the original differences in accuracy between the methods and focuses on the effects of the noise or jitter.
are adding either noise or, will be defined, 'jitter',
For synthetic examples, we consider the Lorenz system in equation 5 and the Ballistic system (A..II. We learn over trajectories {(F(, u,, t)), where the 2t, are generated by numerically integrating ah (x. u, t) using scipy's odeint function (Virtanen et al. 2019), with To and & uniformly sampled from X' and T. and Me, given. Note that @ remains tixed throughout a single trajectory. Details on the ranges of initial conditions and parameters sampled are in the appendix. We evaluate the synthetic experiments generated in the same way as the training data. We use a timestep of 0.5 seconds for the synthetic trajectories. On the Ballistic system this allows to trajectories that do not reach their peak and those that start to fall. Since the Lyapunov exponent of the Lorenz system is less than 3, in 16 predicted timesteps We believe it is important to look at the progress of the system across this threshold to understand whether the NDS model is robust to chaotic dynamics since the Lorenz system used for structure is itself chaotic, we want to make sure that the system does not blow up over longer timescales.
test set of 20,000 trajectories that is fixed for particular random seed
get both predictable and unpredictable data (Frøyland & Alfsen 1984)).
We compare our models with other choices along the spectrum of structured to flexible models from both machine learning and system identification. The models we looked at for experiments are the Full NDS, Partial NDS. a NDSO, a fully connected neural network (FC), a fully connected neural ODE (FC NODE), an LSTM, MATLAB's Gray Box Optimization (GBO), and sparse regression algorithm (SR) due to Brunton et al. (2015). Details on each algorithm are given in Appendix A.2
We can view the Partial NDS and NODE ablations of the Full NDS model which remove some and all of the prior knowledge, respectively. Each model takes 32 timesteps of state and control information as input and are trained on predictions for the following 16 timesteps. The ODE-based models are integrated from the initial conditions of the last given state. All neural networks are all trained with learning rate of 3 x 10-3, which was seen to work well across models. We generated a training set of 100,000 trajectories, test set 20,000 trajectories, and validation set of 10,000 trajectories. Training was halted if validation error did not improve for 3 consecutive epochs.
5.1 SYNTHETIC EXPERIMENTS
We first present results pair of synthetic physical systems where the data is generated in noiseless and regularly spaced setting. We then add noise (in A.9) and irregular spacing to our data to highlight the performance of these methods as conditions become more challenging.
Sample Complexity and Overall Accuracy In order to test sample complexity in learning or fitting, we generated data for a full training dataset of 100,000 trajectories. We then fit our models on different fractions of the training dataset: 1, 0.25, 0.05, 0.01. We repeated this process with 5

---

Under review as a conference paper at ICLR 2021
Sample Complexity Ballitti System
Sample Complexity Lorenz System
Nos
lamt NSS
Ray Connected Me ooe
30
30
Test MSE
Test MSE
MCd0
Spams Reg oation
Groy box Oetivisnte
Number of Training Samples
Number Training Samples
Figure 2: 1.2 loss between predicted and real trajectory as we train on more samples. The NDS models learn with much fewer samples and converge to much lower errors on both systems. Shaded regions are standard errors.
different random seeds and plotted the L2 error of the model over the dataset fraction seen by the model in Figure The error regions are the standard error of the errors over the various seeds.
As seen in Figure 4 the learning of Neural Dynamical Systems looks very different to that of the comparison models. We also see that with small amounts of data, the NDS models greatly outper- form the Neural ODE, but with the full dataset, their performances get closer. This makes sense as the Neural ODE is likely able to infer the structure of the system with large amounts of data. Also, the Fully Connected Neural ODE outperforms the other baselines, which we posit is due to the fact that it implicitly represents that this system as continuous time dynamical process and should change in a continuous fashion. From a camplecooplexxtty perspective makese sense that the better initialization of NDS should matter most when data is limited. A table of the full results of experiments can be seen in A.II
We notice that the NDSO slightly outperforms the NDS with higher variance on these systems. Since has a perfect model of the system, the residual components aren't 't necessary for the model to perform well, however, there is no way the network can 'correct' for a bad estimate.
Curiously, we see on the ballistic system that the partial NDS outperforms the full NDS in the small data setting, but they converge to similar performance with slightly more data. A potential explanation for this is that errors propagate through the dynamical model when the parameters are wrong, while the partial systems naturally dampen errors since, for example, å only depends on the other components through a neural network. Concretely this might look like a full NDS predicting the wrong Rayleigh number a which might give errors y which would then propagate x and Conversely, this wouldn't happen as easily in partial NDS because there are neural networks intermediating the components of the system.
Noise, Irregular Sampling, and Parameter Identification We include further experiments on irregular sampling in Figure [6] where we see that the NDS models are the only ones which do not suffer from varied spacing between timesteps. The Neural ODE model does modestly worse, while the feedforward and recurrent machine learning models struggle. We hypothesize that the initialization of the NDS models helps it generalize to this setting as the prior knowledge models are necessarily invariant to the timestep.
We also include experiments with the performance of our methods under uniform noise in Figure 7 NDS models may be less robust to noise than the machine learning and system identification com- petitors since they experience a greater relative change in performance (however they still exhibit small absolute changes in performence). They still outperform the baselines on an absolute basis. We hypothesize that here the relatively inflexible prior knowledge causes errors to compound in a way that is only somewhat mitigated by the residual terms.
Additionally, we tested the subset of models which identify model parameters to see how close they the true values in Figure A.6 Here, than NDSO. We also see that the NDS models benefit from the generalization of the neural network parameter estimation over the GBO which does a new optimization per trajectory.
that the NDS models do well but slightly worse

---

Under review as a conference paper at ICLR 2021
100
To Cetecotee
Figure 3: L.2 loss between predicted and real trajectories on the fusion problem as we train on more samples. The Fusion NDS outperforms other models, including fixed version of the prior knowledge model and a fully connected Neural ODE.
Figure 4: typical learning curve for NDS. We believe the quick drop in error seen in these models can be thought of as a consequence of a much better initialization in function space.
5.2 FUSION EXPERIMENTS
We explored the concept of approximate system dynamics in a simplified fusion system. We predict the state of the tokamak summarized by its stored energy and rotational frequency given the time series of control input in the form of injected power and torque. As mentioned in Section we have simplified physical model given by Equation g that approximately gives the dynamics of these quantities and how they relate to one another through time. Though there is a lot of remaining work to apply this model real experiment, approaches merging theoretical models with data to make useful predictions can be embedded into useful controller designs and improve the state of fusion.
Our full dataset consisted of 17,686 trajectories, which we randomly partitioned into 1000 as a test set and 16,686 training set [ The data are measured from the D-IIID tokamak via magnetic reconstruction (Ferron et al. 1998) and charge-exchange recombination spectroscopy (Haskey al.] 2018). Similar to our synthetic experiments, sections and train on 32 timesteps to predict 16 timesteps. We compare with the same models as in the previous section, but using
each trajectory into overlapping 48 timestep
Fusion Neural Dynamical System described in Equation 4 with g given by Equation [ As we discussed above, the dynamics this equation are approximate. To illustrate this, we have included the accuracy of the naive dynamics with no learning
our data with fixed confinement times тę = Tm 0.1s as the Nominal Fusion Model in Figure 3 We use a larger fully connected network with 6 layers with 512 hidden nodes to attempt capture the added complexity of the problem.
Sample Complexity and Overall Accuracy When comparing our NDS models, the machine learning baselines, the system ID baselines, and a nominal model from Boyer et al. (2019b). we that the Fusion NDS model performs best by large margin. Although the fully connected neural ODE performs competitively, it fails to reach the same performance. We speculate that the dynamical model helps with generalization whereas the fully connected network may overfit the training data and fail to reach good performance on the test set. Here the NDSO unable to perform well compared to the NDS, as the approximate dynamics mean that the model error is somewhat catastrophic for predictions. We see however that the NDSO outperforms the Nominal Physics Model as it is able to estimate the parameters for each example rather than fixing values of the parameters for the whole dataset.
We these results as highly encouraging and will continue exploring uses of NDS in fusion appli- cations.
Data is loaded and partially processed within the OMFIT framework (Meneghini et al. 2015). We used the on the "ROCESSING" module which has recently been developed for this task and is publicly available branch of the OMFIT source code. Details of the preprocessing
in the Appendix.
8

---

Under review as a conference paper at ICLR 2021
MSE of Model
Performance MPC
# of Training Examples
5K
IK
5K
1K
FC
0.031 + 0.009
0.058 1 0.018
52 3
41 +
FC NODE
0.028 + 0.011
0.049 0.013
55 =
46 +
LSTM
0,081 = 0.023
2
25 +
Full NDS
0.020 0.006
0.029 0.007
72 +
+
Partial NDS
0.022 + 0.009
0.033 0.011
69 - 8
55 + 6
NDSO
0.023 + 0.013
0.028 0.014
71 11
57 =
SR
0.037 = 0.023
0.041 + 0.015
65 -
56 +
GBO
0.046 + 0.019
n/a
49 5
n/a
Table Modeling and Control on the EvilCartpole system.
5.3 CONTROL EXPERIMENT
We also explored the use of these models for control purposes using model-predictive control CCa macho & Alba 2013). For this purpose, modified the Cartpole problem from [Brockman al. (2016) so that there are a variety of parameter values for the weight of the cart and pole as well as pole length as specified in Section A.1.2 Typically, 'solved' cartpole environment would imply consistent performance of 200 from a control algorithm. However, there are three factors that make this problem more difficult. First, in order allow each algorithm identify the system in order to make appropriate control decisions, we begin each rollout with 8 random actions. The control never fails at this point but would certainly fail soon after if continued. Second, the randomly sampled parameters per rollout make the actual control problem more difficult as the environment responds less consistently to control. For example, MPC using the typical Cartpole environment as a model results in rewards of approximately 37. Third, all training data for these algorithms uses random actions with no exploration, which has been seen to degrade the performance most model-based RL or control algorithms (Mozer & Bachrach] 1990).
We then trained dynamics models on this 'EvilCartpole' enviroment for each our comparison algorithms datasets of trajectories the environment with random actions. At that point, rolled out trajectories on our EvilCartpole environment using MPC with control sequences and ran- dom shooting with 1,000 samples and horizon of 10 timesteps. The uncertainties standard errors over 5 separately trained models.
As shown in Table [ the NDS algorithms outperform all baselines on the cartpole task for both the modeling and control objectives. We see that all algorithms degrade in performance as the amount of data is limited. We see Table/othat with larger amounts of data the Fully Connected and Neural ODE models perform as well as the NDS models. We hypothesize that this is due to the fact that the cartpole dynamies are ultimately not that complicated and with sufficient data unstructured machine learning algorithms can learn the appropriate dynamics to reach modestly performing controller as well as NDS.
CONCLUSION
In conclusion, we give framework that merges theoretical dynamical system models with deep learning by backpropagating through numerical ODE solver. This framework succeeds when there is a partial or approximate model of the system. We show there is an empirical reduction in sample complexity and increase in accuracy on two synthetic systems and dataset. In the future, we wish to expand upon our work to make more sophisticated models in the nuclear fusion setting as we move toward practical use. We also hope explore applications of this framework in other area which have ODE-based models of systems.
real nuclear fusion
REFERENCES
Ibrahim Ayed, Emmanuel de Bézenac, Arthur Pajot, Julien Brajard, and Patrick Gallinari. Learning Dynamical Systems from Partial Observations. arXiv: 136 [physics], February 2019. URL
11136
arXiv: 1902. 1136.
S.A. Billings. Identification of nonlinear systems-a a survey, IEE Proceedings Control Theory and Applications, 127(6):272, 1980. ISSN 01437054.
doi: 19.10499/p-..1980.

---

Under review as a conference paper at ICLR 2021
0047.
0tts//p-d.1980.000
M. D. Boyer, K. G. Erickson, B. A. Grierson, D. C. Pace, J. T. Scoville, J. Rauch, B. J. Crowley, R. Ferron, R. Haskey, D. A. Humphreys, R. Johnson, R. Nazikian, and C. Pawley. Feedback control of stored energy and rotation with variable beam energy and perveance on DIII-D. Nuclear doi: 00.1088/14414326642 https://doi.co org/ 11w.0111111010011111222-0
Fusion, 59(7):076044, May 2019a. ISSN 0029-5515.
URL
M. D. Boyer, S. Kaye, and K. Erickson. Real-time capable modeling of neutral beam injection on NSTX-U using neural networks. Nuclear Fusion, 59(5)055008, March 2019b. ISSN 0029-5515. doi: 2Fab0762
URL /ttt::///oa...cc//0..00181111111
K. E. Brenan, S. L. Campbell, and L. R. Petzold. Numerical Solution of Initial- Value Problems in Enferentiil-Algebree Equations. Classics in Applied Mathematics, Society for Industrial and Applied Mathematics, Jamuary 1995. ISBN 978-0- 978-0-89877-333--11 doi: 0111133719711111999 97716/11771111992222
URL https:
/ //epbbs..pbp//// s iam. wttp..oo//../oooo/////////
Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojeiech Zaremba. Openai gym, 2016.
Steven L. Brunton, Joshua L. Proctor, and J. Nathan Kutz. Discovering governing equations from data: Sparse identification of nonlinear dynamical systems. September 2015. doi: 10. 1073/pass. 1517384113. URL https: petttss//aariiv org/ abtp//ass.11./.// 03580v1
Eduardo Camacho and Carlos Bordons Alba. Model predictive control. Springer Science & Business Media, 2013.
Ian Char, Youngseog Chung. Willie Neiswanger, Kirthevasan Kandasamy, Andrew Oakleigh Nelson, Mark Boyer, Egemen Kolemen, and Jeff Schneider. Offline Contextual Bayesian Optimization. pp. 4627 4627-4638. 2019. URL http: http::://ppprr... http:/pppprr...urss.s...oo 8711-offline- oow--fffineecoottxxxuaaaa-aaaee- ializiiaiottiiiiiiiiii www..ps.cccppppp..........
Ricky T. Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud, Neural Ordinary Differ- ential Equations. arXiv: 0066673666600 [cs, stat], June 2018. URL htttppp//rrrdd..rr///3 1806. 07366
arXiv: 1806.07366.
S. Chen, S. A. Billings, and P. M. Grant. Non-linear system identification using neural networks. International Journal of Control, 51(6):1191-1214, Jamuary 1990. ISSN 0020-7179. doi: 10. 1080/002071790000 17900993416 URL Publisher: Taylor & Francis eeprint: httss///bbaogg/o//////0001111119
1080 /
Zhengdao Chen, Jianyu Zhang, Martin Arjovsky, and Léon Bottou. Symplectic Recurrent Neural Networks. arXiv: (9tp::999.111000 13334 [cs, stat), September 2019. URL 1909. 13334
arviv 1333.13334.
Kurtland Chua, Roberto Calandra, Rowan McAllister, and Sergey Levine. Deep Reinforcement Learning Handful of Trials using Probabilistic Dynamics Models. a805 /2/ 14 [cs, stat], May 2018. UR/.appp///xxv...c/r///////// 12114 arXiv: 1805.12114.
Youngseog Chung, Ian Char. Willie Neiswanger, Kirthevasan Kandasamy, Andrew Oakleigh Nel- son, Mark D. Boyer, Egemen Kolemen, and Jeff Schneider. Offline Contextual Bayesian Op- timization for Nuclear Fusion. arXiv:2001.011 fes, stat), January 2020. URL http:
7/ wwcxii..rrg/bs//00111. 01793
arXiv: 2001.01793.
Thomas F. Coleman and Yuying Li. An Interior Trust Region Approach for Nonlinear Minimiza- tion Subject Bounds. SIAM Journal on Optimization, 6(2):418 445,, May 1996. ISSN 1052- 6234. doi: 10. 10.1133/0806023, URL. httt::::::/// //epubs i am. org/ doi /abo//b//// 10. 1377 08C6023
httg/di/// Publisher: Society for Industrial and Applied Mathematics.
Miles Cranmer, Sam Greydanus, Stephan Hoyer, Peter Battaglia, David Spergel, and Shirley Ho. Lagrangian neural networks, 2020.
10

---

Under review as a conference paper at ICLR 2021
Miles D. Cranmer, Rui Xu, Peter Battaglia, and Shirley Ho. Learning Symbolic Physics with Graph Networks. arXiv:1909.0588. /astro-ph, physics:physics, stat], November 2019. URL http:
05862
arXiv: 1909.05862.
Noel Cressic and Christopher K. Wikle. Statistics for Opatio-Temporal Data. John Wiley & Sons, November 2015, ISBN 978-1-119-243044 Google- Bealle-Boos--lle ID: ALLos--DDDLAAAAAAAAEEE 4L.dCgAAQBAJ.
Filipe de Avila Belbute-Peres, Kevin Smith, Kelsey Allen, Josh Tenenbaum, and J. Zico Kolter. End-to-end differentiable physics for learning and control. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Gar- nett (eds.), Advances in Neural Information Processing Systems 31, pp. 7178- 7189. Curran Associates, Inc., 2018. URL htttp:/ppaers wwt.eexxsiip...ccppppaaoop
pdf
Emmanuel de Bezenac, Arthur Pajot, and Patrick Gallinari. Deep Learning for Physical Processes: Incorporating Prior Scientific Knowledge. arXiv:1711.07970 [cs. stat], January 2018. URL
http://arxiv/ org/ abs/1711.07970
arXiv: 1711.07970.
R. Dormand and J. Prince. A family of embedded Runge-Kutta formulae. Journal of Computational and Applied Mathematics, 6(1):19-26, March 1980. ISSN 0377-0427. doi:
10.00160071-6600x889900003333 URL htttp://wer www.wwwwwww...et.tttttmmmmm com seinm//ciene////// att.cieep.1//011106X6600000
J. R. Ferron, M. L. Walker, L.. L. Lao, H. E. St John, D. A. Humphreys, and J. A. Leuer. Real time equilibrium reconstruction for tokamak discharge control. Nuclear Fusion, 3857):0055-1066 July 1998. ISSN 0029-5515. doi: 10.
URL https:::///o..oo
10. 10.108882F0022 5515#2F 38S222223228FF88
Jan Frøyland and Knut H. Alfsen. Lyapunov-exponent spectra for the Lorenz model. Physical Review A, 29(5):998--2911 May 1984. doi: 10. wtw.3//hssRe..2.29992.... URL https://
opr..ap...rg//oi//// 110. 11t.:o1yee//////e...22.2.. 29. 2928
Alex Graves. Generating Sequences With Recurrent Neural Networks. August 2013. URL https: //arxivv.rr//bb/1130000 0850v5
A. Hammerstein. Nichtlineare Integralgleichungen nebst Anwendungen. Acta Mathematica, 54: doi: 10.1007/BF02547519 URL hhtpps:// projecteucl id. org.orgg/eeo worg/euclleeooooooo..oo.oo.. id..d........... awt.//1/aaee....
117-176, 1930. ISSN 0001-5962, 1871-2509.
Publisher: Institut Mittag-Leffler.
S. R. Haskey, B. A. Grierson, L. Stagner, C. Chrystal, A. Ashourvan, A. Bortolon, M. D. Boyer, K. H. Burrell, C. Collins, R. J. Groebner, D. H. Kaplan, and N. A. Pablant. Active spectroscopy measurements of the deuterium temperature, rotation, and density from the core to scrape off layer the DIII-D tokamak (invited). Review of Scientific Instruments, 89(10):00D110, August doi: 0021066/150088444 URL
2018. ISSN 0034-6748.
swt.aaaiiocittaiinnnno.ooo.. org/
doi/ful1/10 1063 5038349
Sepp Hochreiter and Jürgen Schmidhuber. Long Term Terorror Memory. Neural Computation, Novem- ber 1997. URI. https: // acm../ccc..c......omm.mmm/...... www.aooooo..eo...... /abs/////a/b///////////////// 1162 202//eco.. 1997. .9.77.9.5 1735
Michael Janner, Justin Fu, Marvin Zhang, and Sergey Levine. When to Trust Your Model: Model- Based Policy Optimization. arXiv:1906.0822. [cs, stat], June 2019. URL http:://axxiz.
org/ bogg/bbs/ie abs/ 08253
arXiv: 1906.08253.
Julian Kates-Harbeck, Alexey Svyatkovskiy, and William Tang. Predicting disruptive instabilities in controlled fusion plasmas through deep learning. Nature, 568(7753):526-331 April 2019.
ISSN 1476-4687. doi: 10.1033//41188--019-11111
URL https:////// wwwp///wwww.////////////////ttww www.anatueee.. com/
aiticcees/s1188-111-111111
Lennart I.jung. Perspectives System Identification. pp. 13, 2010.

---

Under review as a conference paper at ICLR 2021
Lennart Ljung, Rajiv Singh, Qinghua Zhang, Peter Lindskog. and Anatoli Iouditski. Develop- ments in The MathWorks System Identification Toolbox. IFAC Proceedings Volumes, 42(10): 522-527, 2009. ISSN 14746670. doi: 10.3182200090006-333322 0000200090770003-PP2220000000000 URL https:
//1inking iighb. el wtt.e/eeeeer..coeeeeeeeeeeeaaaaa
638 7006
Edward N. Lorenz. Deterministic Nonperiodic Flow. Journal of the Armospheric Sciences, 20 (2):330-14,, March 1963. ISSN 0022-4928. doi: 10.17759155000006600066600001110 0.CO:2. 281 281963290209300133320 3 0.
Gaurav Manek and J. Zico Kolter. Learning Stable Deep Dynamics Models. January 2020. URL
https: //t::///axxii org. wwtgg//b///e 20w..//0..000.0o00 001.60106666611
O. Meneghini, S. P. Smith, L. L. Lao. O. Izacard, Q. Ren, J. M. Park, J. Candy, Z. Wang, C. Luna, V. A. Izzo, B. A. Grierson, P. B. Snyder, C. Holland, Penna, G. Lu, P. Raum, A. McCubbin, D. M. Orlov, E. A. Belli, N. M. Ferraro, R. Prater, T. Н. Osborne, A. D. Turnbull, and G. M. Staebler. Integrated modeling applications for tokamak experiments with OMFIT. Nuclear Fu- sion, 55(8):083088, July 2015. ISSN 0029-5515. doi: 10. 1008:/0029-5555555/888/// https://dde coboo.orrg .rgg/ / 10. 108882000290
URL
Michael Mozer and Jonathan Bachrach. Discovering the structure of a reactive environment by exploration. In D. S. Touretzky (ed.), Advances in Neural Information Processing Sys- tems 2. pp. 439-446. Morgan-Kauffann 1990. UtLpttp::/ www.. enve www.rrmeeeteeieiiiieeeeeeeeeeeee by exp oatiooniio........
292 dldviiccoeringg the structureeeereee a
pdf
Anusha Nagabandi, Gregory Kahn, Ronald S. Fearing, and Sergey Levine. Neural Net- work Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Funinn. arXiv:1708. 02596 [cs], December 2017. URL httt:::/arriv..rge http://bs//111....
arXiv: 1708.02596.
Anusha Nagabandi, Kurt Konoglie, Sergey Levine, and Vikash Kumar. Deep Dynamics Models for Learning Dexterous Manipulation. arXiv:1909. 1652 [cs], September 2019. URL http: 11652
//axxiv.org/aa/////
arXiv: 1909.11652.
Oliver Nelles. Nonlinear System Identification: From Classical Approaches to Neural Networks and Fuezy Models. Springer Science & Business Media, March 2013. ISBN 978-3-662-04323-3. Google-Books-ID tyjrCAAAQBAJ.
Lev Semenovich Pontryagin. Mathematical theory of optimal processes. Routledge, 2018.
Gavin D. Portwood, Peetak P. Mitra, Mateus Dias Ribeiro, Tan Minh Nguyen, Balasubra- manya T. Nadiga, Juan A. Saenz, Michael Chertkov, Animesh Garg, Anima Anandkumar, Andreas Dengel, Richard Baraniuk, and David P. Schmidt. Turbulence forecasting via Neu-
ral ODE arXiv:/911 .5180 [physics], November 2019. URL arXiv: 1911.05180.
abs/
1911. 05180
Dimitris C. Psichogios and Lyle H. Ungar. A hybrid neural network-first princi- ples approach to process modeling. AIChE Journal, 38110)1499-1511, 1992. doi: 10.0000ai..6003811000
ISSN 1547-5905.
URL https://aiccee.
onl mollnnel wnneibbrrrryyyy wi /ttpe//.cco//////////11000000000
69038 1003
eprint:
wwwp:eeewww.nniiiir.a.eeeeeeeooe http:///////////o.oo00000ooo1.00 690338003..
Alireza Rahrooh and Scott Shepard. Identification nonlinear systems using NARMAX model. Nonlinear Analysis: Theory, Methods & Applicutions, 71(12)=e198- December 2009. doi: 10.
ISSN 0362-546X.
150. URL http: http:://www.scienceee..... receeneedirect....
com/
M. Raissi, P. Perdikaris, and G. E. Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial dif- ferential equations. Journal of Computational Physics, 378:686-707, February 2019. ISSN 0021-9991. doi: 10. 201 9JCoPh 378 686R
URL ht http:: : adsabs sarvard.edu/aas..
12

---

Under review as a conference paper at ICLR 2021
R. Rico-Martinez, J. S. Anderson, and Yannis Kevrekidis. Continuous-time nonlinear sig- nal processing: A neural network based approach for gray box identification. Decem-
ber 1994. URL org
cont nuouss - t mememe-meee non
Stephane Ross and Andrew Bagnell. Agnostic System Identification for Model Reinforce- ment Learning. arXiv: /203. 1007 [cs, stat], July 2012. URL http:://rzxvv.org/////
1203.1007 arXiv: 1203.1007.
Samuel H. Rudy, J. Nathan Kutz, and Steven L.. Brunton. Deep learning of dynamics and signal- noise decomposition with timesstpppig constraints. Journal of Computational Physics, 396: 483-506, November 2019. ISSN 0021-9991,
doi: 10.10166f..cpp2211.06660 URL http:
//www /ttpa//eeaeaaai.......aa.aaaaaaa
Wilson John Rugh. Nonlinear system theory. Johns Hopkins University Press Baltimore, MD, 1981.
C. Runge. Ueber die numerische Auflösung von Differeentaallleihunngnn Mathematische Annalen,
46(2):167-178, June 1895. ISSN 1432-1807. doi: 10.1007/BF01144600. URLhttps:://aa
0xc/10.1007/B201465
Alvaro Sanchez-Gonzalez, Victor Bapst, Kyle Cranmer, and Peter Battaglia. Hamiltonian Graph
Networks with ODE Integrators. ptt.:1999919999 [physics], September 2019. URL http:
//aziiv.zgg/as//19903 12790
arXiv: 1909.12790.
Johan Schoukens and Lennart Ljung. Nonlinear System Identification: A Oriented Roadmap.
arXiv: 0009000000003.000 [cs], February 2019. URL
arXiv: 1902.00683.
Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Si- mon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, Timothy Lill- icrap, and David Silver. Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model. arXiv: 1911..82655 [cs, staf], November 2019. URL http:://erxlv.orr//abb//// org/abs/ 1911.08265 arXiv: 1911.08265.
B. Sohlberg and E. W. Jacobsen. GREY BOX MODELLING - BRANCHES AND EXPERI- ENCES. IFAC Proceedings Volumes, 41(2):11415- 11420, January 2008. ISSN 1474-6670.
doi: 10.3182/2000700655-K2 KR --0-000-------11-1119009090 0094444
URL http: www.wwscenceeirecttto com/
science/arriclleepl//111112
Michael L. Thompson and Mark A. Kramer. ing prior knowledge and neural networks.
Modeling chemical processes
AIChE Journal, (0(8):132--1440
1994.
ISSN 1547-5905. doi: 10. 10.1002//ic.6900400866
URL http://aiche. .eprint:
seerrary.wiey..oom doi abs/ 1ttp:/101.00///11.../// 1002/aic. 690400806 http:://nnniiiiiiaayyyywleeoooop
A. Vahidi, A. Stefanopoulou, and H. Peng. Recursive least squares with forgetting for online es- timation of vehicle mass and road grade: theory and experiments. Vehicle System Dynamics, doi:
43(1):31-55, January 2005. ISSN 0042-3114.
URL Publisher: Taylor & Francis
https://doi. org /10 1t1/:1111100000000 1080 eprint: http:://oo..org/ 10. 00000000000211111111331150000000
B. Ph. van Milligen, V. Tribaldos, and A. Jiménez. Neural Network Differential Equa- tion and Plasma Equilibrium Solver. Physical Review Letters, 75(20):3594 3597, Novem- ber 1995. doi: 10. 10w./h
URL https:///inkk apw.app..... ops.orr//oi/1c.
11C3/hysReviete 75.3599
Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Courna- peau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nel- son, Eric Jones, Robert Kern, Eric Larson, C. Carey, ilhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, Ian Henriksen, E. A. Quintero, Charles R. Harris, Anne M. Archibald, Antônio H. Ribeiro, Fabian Pedregosa, Paul van Mul- bregt, and SciPy 0 Contributors. SciPy 1.0 J.O-Fundamental Algorithms for Scientific Computing
13

---

Under review as a conference paper at ICLR 2021
in Python. arXiv: 10121 [physics], July 2019. URL (ttLtppp//aaar..oo///////11111 10121 arXiv: 1907.10121.
Tingwu Wang and Jimmy Ba. Exploring Model-based Planning with Policy Networks.
arXiv: arXiv: 08649 [cs, stat], June 2019. URL
08649
arXiv: 1906.08649.
14

---

Under review as a conference paper at ICLR 2021
A APPENDIX
A.1 ADDITIONAL DYNAMICAL SYSTEMS USED
A.1.1 BALLISTIC SYSTEM
We also predict trajectories for ballistics: an object is shot out of cannon the presence of air resistance. It has mass and drag coefficient and follows nearly parabolic trajectory. This system has two-dimensional state space (altitude y and horizontal range r) and parameters (mass and drag coefficient), which reduce down to one: terminal velocity et- It is a second order system of differential equations which we reduce to first order using the standard refactoring.
The system is given by
g
Vt
(9)
1/
1
Ut
(here, is the constant of gravitational acceleration).
A.1.2 CARTPOLE SYSTEM
In section 5.3
we discuss experiments on a modified Cartpole system with randomly sampled pa- rameters. Here give full delineation of that system defined in Brockman et al. (2016) with our modifications.
This system has three parameters, one control, and four state variables. The parameters are I, the length of the pole, me. the mass the cart, and mps the mass of the pole. The control is F, the horizontal force applied to the cart. In the current setup,
be one of +10. The state variables the lateral position x, the angle of the pole from vertical 8, and their first derivatives, & and A.
The system is given by the following equations
9 sin + cos f - F-m 162 sin my +mp
Ô=0 i
mp co02
m. +m
(10)
4 b F mp, sin 0 cos0 0)
m. + mp
We give a reward of +1 for every timestep that |0| < 15 and |x] I 2.4. Initial condi- tions are uniformly sampled from [-0.05, 0.05] at test time and training U[-11,1], 1]), ~ ([[00.2, 0.2]) (which slightly wider than the To test), and Å U([-0.1,0.11 0.1). The parameters also uniformly sampled at train and test with I I ([0.66, 1.2)). m. ~ U(0.5, 2|) and m, ~ U([0.03, 0.2]).
U 2)),
I threshold used at
For the partial NDS for Cartpole, remove the equations corresponding to ă and ø.
A.2 COMPARISON METHODS
In our paper, we compared the following methods in our experiments:
Full NDS: A Neural Dynamical System with the full system dynamics for the problem being analyzed. The full construction of this model is given by Equation For the functions hag co. d,,, we use fully connected networks with 2 layers, Softplus activations, 64 hidden nodes in each layer, and batch normalization.
A Neural Dynamical System with partial system dynamics for the problem being analyzed. These follow Equation 7 as applied to Equation 4 For the Ballistic system, we only provide equations for è and I, excluding the information about vertical motion from our network. For the Lorenz system, we only provide equations for & and excluding information about motion in the direction. For the Cartpole system, we only provide information about ó and
Partial NDS:
15

---

Under review as a conference paper at ICLR 2021
9. These equations were chosen somewhat arbitrarily to illustrate the partial NDS effectiveness. We use similar neural networks here as for the Full NDS.
NDSO: Full NDS with residual terms removed. This serves as an ablation which shows the use of the residual terms.
Fully Connected (FC): A Fully-Connected Neural Network with 4 hidden layers containing 128 nodes with ReLU activations and batch normalization.
Fully Connected Neural ODE (FC NODE): A larger version of the Neural ODE given in Chen et al. (2088) we use hidden layers with 128 nodes, batch norm, and Softplus activations for ž. This can be interpreted as version of our NDS with prior knowledge, i.c. g(x) 0.
LSTM: A stacked LSTM with 8 layers as in Graves (2013). The data is fed in sequentially and we regress the outputs of the LSTM against the true values of the trajectory.
Gray Box Optimization (GBO): We use MATLAB's gray-box system identification toolbox [Ljung et al., 2009) along with the prior knowledge ODEs to fit the parameters à as an alternative using neural networks. This algorithm uses trust-region reflective nonlinear least squares with finite differencing (Coleman & Li 1996) to find the parameter values which minimize the error of the model rollouts over the observed data.
Sparse Identification of Nonlinear Systems (SR): We use the method from Brunton et al. (2055) to identify the dynamical systems of interest. This method uses sparse symbolic regres- sion to learn a linear mapping from basis functions of the state Iy and control dy to the derivatives tt computed by finite differences. Our synthetic systems are in the span of the polynomial basis that we used.
We note that ReLU activations were chosen for all feedforward and recurrent architectures, while in the Neural-ODEE-bsed architectures, we follow the recommendations of Chen et al.) (2018) and the Softplus. The sizes and depths of the baselines were chosen after moderate hyperparameter search.
A.3 HYPERPARAMETER
We trained using Adam with learning rate 3 x 10-3 and an ODE relative and absolute tolerance when applicable of 10-3 This wide tolerance was basically function of training time as with tighter tolerances experiments took a long time to run and we were more concerned with sample complexity than the tightness of the integration. Hyperparameters were predominantly chosen by trial and error.
Over the course of figuring out how this works and then evaluating the models we ran some form this code 347 times. The typical setup was either a 1080Ti GPU and CPU cores CPU cores for roughly day per experiment. We found that most of these trainings were marginally faster on the GPU 1.5x speedup, but weren't 't religious about the GPU as we had many CPU cores and many experiments run parallel.
evaluating,
Lorenz Initial Conditions and Parameters For our Lorenz system, we sampled p 3(15,35)), ~ ( 12)), 8 ~ U(1,3)), To ~ (10,,5) 5)), yo ~ U(0, 5)). 20 ~ U[0, 5)).
Ballistic Initial Conditions and Parameters For our Ballistic System, we sampled masses ~ U(1, 100)), drag coefficients ca U(0.4,3)) 2o U[[1000, 100)), yo U(0, (00)). We then use mg to recover the terminal velocity used in our model.
A.4 FUSION MODEL PARAMETERS
n, is ion density (which approximate constant value of 5 19 deuterium ions per m²), m; is ion mass (which we know since our dataset contains deuterium shots and the mass of a deuterium ion is 3.3436 10 10- kg), and Ro is the tokamak major radius of 1.67
16

---

Under review as a conference paper at ICLR 2021
Ballstic
Number Thankin Searpees
ColasstFraction
Figure 5: L2 distance between & and e. the NDS are trained under the usual L2 supervision, the parameters @ of the system approach the correct values.
A.5 FUSION DATA PREPROCESSING
Data is loaded and partially processed within the OMFIT framework Meneghini et al. (2015). A new module, "SIGNAL PROCESSING", has been developed for this task and publicly available on the "profile prediction data processing" branch of the OMFIT source code. The rest of the processing done on Princeton's 's Traverse computing cluster, and is available in the GitHub sourcecode for this project
http://hwwbb.o//bbabaa/////aaaa/ profile-predictor).
DII[- shots from 2010 through the 2019 campaign are collected from the MDS+ database. Shots with pulse length less than 2s, normalized beta less than 1, non-standard topology are excluded from the start. variety of non-standard data is also excluded, including the following situations:
1. during and after dudtrip trigger (as determined by the pointname "dustripped")
2. during and after ECH (pointname "pech" greater than .01) activation, since ECH is not currently included as an actuator
3. whenever density feedback is off (as determined by the pointname "dsifbonoff")
4. during and after non-normal operation of internal coil, with an algorithm described by Carlos Paz-Soldan
All signals are then put the same 50ms time base by averaging all signal values available between the current time step and 50ms prior. no data is available in the window, the most recent value is floated. Only time steps during the "flattop" of the plasma current are included. The flattop period is determined by DIII-D's "Lip_flat and "ip_flat. wtw.fattdduurtii duration" PT pointnames.
All profile data is from Zipfit Meneghini al. (2015). The profiles linearly interpolated onto 33 equally spaced points normalized toroidal flux coordinates, denoted by p. where = 0 the magnetic axis and p is the last closed flux surface.
A.6 PARAMETER LEARNING WITHOUT EXPLICIT SUPERVISION
For experiments in Figure we stored the parameter estimates phi for the NDS and gray box models and compared them to the true values to see how they perform in identification rather than prediction. None of these models were ever supervised with the true parameters. We see in Figure that the NDS is better able to estimate the parameter values than the gray-box method for both systems tested. We believe this is because our method able to leverage many trajectories to infer the parameters whereas the gray-box method only uses the single trajectory.
A.7 COMPUTATION OF NOISE
In previous experiments, did not add noise the trajectories generated by the synthetic systems. We generate noise added to the trajectories by first sampling a set 100 trajectories and computing for the ith component of the state space the RMS value e; across 100 trajectory
17

---

Under review as a conference paper at ICLR 2021
22
0.3
UPt the Commontion
0.2
1.2
Mar Added
[Mar Adbed
Figure 6: Change in performance when data are irregularly sampled time. The Full NDS performs better than any comparison model under added jitter.
Eilict of ho Bullska System
0.29
Efest Nobe
Stalen
Fatioo
0.22
2.28
Takuthvo
Addidd
Figure 7: Change in performance under added relative noise The NDS seem have more variance performance under high amounts of noise as the comparison models but it still greatly outperforms them on an absolute basis.
samples. Then we sample noise A (0, rc;) from a normal distribution where the variance is c; scaled by 'relative noise' term We vary to control the amount of noise added system in way that generalizes across problems and across components of a problem's state space.
A.8 ROBUSTNESS TO IRREGULAR TIMESTEPS
We also explored how these models respond to data sampled in intervals that are not regularly spaced in fashion discussed in A.10 In Figure
that the ODE-based models are able handle the irregularly spaced data much better than the others. Like with noise, we are focusing on relative performance but even so, the full NDS even does substantially better under large jitter settings. As we have discussed, this makes sense because these models natively handle time. We conjecture that the Full NDS neural networks may learn something slightly more general by this 'domain randomization' given that they are correctly specified models that receive all the information of the differing timesteps. The symbolic regression method fails under jitter, presumably because relies heavily on finite differencing.
A.9 ROBUSTNESS TO ADDED NOISE
We evaluate our experiments on additive noise scaled relative to the system discussed in A.7 As the NDS does substantially better than the other models on the synthetic data, we look at the effect of noise relative to the original performance to focus on the properties of the new model. When large amount of noise is added to the system, NDS's performance degrades faster than that of the other models, though it still outperforms them might be unstable here due to errors propagating through the prior knowledge model. The other models all otherwise perform fairly similarly under added noise.
absolute basis. We think full and partial NDS
18

---

Under review as a conference paper at ICLR 2021
System
Lorenz
Ballisti
Samples
100,000
25,000
5,000
1,000
100,000
25,000
5,000
1,000
PC
35.7 4
46.7 = 0.2
57.0
157.9
864K
912K = 4K
1013K
1477K 5K
NODE
40.5 0.6
41.5 0.6
42.9 0.5
64.5
3K
1:
10K 5.58
19K 1.
Full NDS
33.8
36.6
38.3
47.82
1,3 90
166K1600
16K2160
4,6K435
Partial NDS
34.8
35.8 2.2
37.8 1.1
46.7
1.2K
11K
1.6K 190
1.10
NDS0
33.7 = 2.2
36.2 7.3
8.3
12.5 45.5
1.3K +: 143
1.5K + 347
16K =: 434
5.7K = 743
LSTM
198
200
201 5
201 =
180K
181K = 3K
181K
18IK
SR
3,7K=66
n/a
n/a
2.1M=120K
2.8M4170KK
3.1M=140K
3.1M+180K
GBO
3.2K+200
n/a
a/a
8.2K=600
m/a
I//a
Table 2: Sample Complexity Results as discussed in Section 5.1
and Figure 2
System
Samples
100,000
25,000
5,000 m/a
1,000 n/a
100,000
25,000
5,000
1,000
2.8
38.3
1267.81
Partial NDS
m/a
m/a
1249 1: 324
1194
293
1434.8 1 426
2378 1. 593
32.5
55.4 m/a
GBO
233.8 + 15
IV4
n/a
160990 + 1261
0/a
6/a
0/0
Table 3: Parameter Error Results discussed in Section 5.1 and Figure A.6
A.10 COMPUTATION JITTER
Though, we will discuss, fusion data in section 5.2 has been postprocessed to be regularly spaced, in practice, data on tokamaks and in many other systems come irregularly and are some- times missing. As both the fully connected neural ODE and NDS models are integrated in continu- ous time, they can handle arbitrarily spaced data. For the LSTM and Fully Connected models, we concatenated the times of the datapoints to the associated data and fed this into the model. For each batch training and test data and some value of 'jitter', J. we create a new time series {4; + J V where ji U (l- Since our timestep for synthetic experiments is 0.5s we try values of j of 0.1s and 0.25s. We then generate the batch of trajectories by integrating our systems to the new timesteps.
A.11 TABLES OF RESULTS
A.12 ADDITIONAL CONTROL RESULTS
System
Lorenz
Ballistic
Jitter Level (s)
0.1
0.25
0.1
0.25
FC
0.104
0.109
0.106
0.125
FC NODE
0.008
0.012
0.075
0.077
Full NDS
-0.010
-0.066
-0.012
-0.096
Partial NDS
-0.011
-0.012
0.024
0.016
L.STM
0.032
0.031
0.130
0.132
SR
2.69
2.17
0.239
0.342
GBO
-0.002
0.058
0.002
0.013
Table The effect of jitter on given algorithm for given level of jitter as computed in A.10 The effect is calculated by dividing the performance with jitter by the mean performance without and subtracting 1.
19

---

Under review as a conference paper at ICLR 2021
Model
L2 Error on the Fusion Test Set
FC
1.942 x 10
FC NODE
8.288 x 1020
NDS with Approximate Dynamics
4.837 X 10
NDSO
8.944 x 1020
LSTM
2.527 x 10t1
SR
2.546 x 10
GBO
6,452 x 10
Table 5: The performance of our comparison models on the nuclear fusion problem, as discussed in Section
MSE of Model
Performance of MPC
# of Training Examples
100K
25K
100K
25K
FC
0.016 0.004
0.022 + 0.006
86 +
76 + 4
FC NODE
0.015 0.008
0.019 + 0.007
87 +
78
LSTM
0.081 0.023
0.092
33 6
28 + 5
Full NDS
0.007
0.017+0.002
86 86 3
82 + 5
Partial NDS
0.016
0.017 0.005
79 6
78 5
NDSO
0.013 0.009
0.017 + 0.008
76
74 + 6
SR
0.032 + 0.023
0.034 + 0.017
68 +
66
3
Table 6: Modeling and Control the Cartpole system with large amounts of data.
20


Reviews:
[{"review_id": "ukKiSEAf_7", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "### Summary\nThe paper combines gray box optimization with Neural ODE, for improved predictions of time series data from low-dimensional physics systems. \n\n### Recommendation\nThe jury is still out on how to best combine prior knowledge about a physical system with learnable components, so this is an important research direction; and combining system identification/gray box optimization with Neural ODE is a reasonable approach. However, the results remain ambiguous, and most importantly, I feel the most relevant questions (see below) are not investigated.\n\nThe big questions in system identification methods, and what often limits their usefulness, are a) how to deal with systematic effects not properly accounted for in the model prior and b) how to avoid falling into local optima due to a highly nonlinear optimization landscape, and I feel these are not sufficiently addressed in this paper. First of all, most experiments reported in the main paper (5.1) are synthetic, generated with the exact model prior, on a simple low-dimensional system, without any systematic deviations or even noise. Yet even in this ideal setting, pure NeuralODE is just barely out of the confidence interval of NDS (e.g. fig. 4). The appendix contains some experiments with added uniform noise; the authors state that NDS is affected more strongly affected by noise, but the results are hard to parse, since only relative change is given. It's unclear if NDS is still performing best in this noise setting. These results cast doubts if this approach will be useful in more complicated settings.\n\nSecond, baselines. There should be a baseline for pure GBO (same model and setup as NDS, but turning off the residual, let's call it NDS0). I'd expect this baseline to be comparable or better than NDS at least on the synthetic data in 5.1, since the model is exact in those cases. Instead, there is a GBO baseline using a proprietary MATLAB GBO function, which performs surpringly bad at estimating parameters (fig. 5) for the synthetic data. The cited Ljung et al. doesn't actually describe the MATLAB GBO method, only the interface and GUI. So it's hard to know what the differences in optimization setup are, and whether the Neural-ODE residual somehow helps parameter estimation, or it MATLAB just performs pooly due to a different optimization setup. If it's the former, and NDS actually outperforms NDS0 on parameter estimation, a proper investigation and discussion of this would actually be an interesting finding.\nAlso, the baseline models seem to all have a different NN architecture, with different capacity and different activation. This makes it even harder to extract meaning from the results.\nThe only dataset which is not a toy example is the fusion dataset in 5.2. I'm not a domain expert on this, so I don't know how hard of a task this is, or what's a relevant benchmark method is for this dataset-- however, since this is real data, there must have be an attempt to fit this data to a model by the group operating the tokamak, and such a model fit should be compared to as a baseline. Second, I assume as a real-world dataset this must include measurement noise, and effects not explained by the model prior. Surprisingly however, this is also the experiment where NDS most strongly outperforms pure Neural ODE, and even more surprisingly smaller error bounds than the purely learned methods-- the complete opposite of 5.1., which found optimization through the model prior to cause a lot of instability *even without* noise. This should definitely be investigated more closely, as this could be due to anything from different smoothness, sequence length, luck, type of model ODE, or maybe even regularization effect from the real-world data; and the conclusions to draw for the value of this method would be drastically different depending on the finding.\n\nFinally, another big surprise in this paper to me was that partial NDS strongly outperformed NDS. I'm not sure I follow the authors' explanation--if the model parameter estimation is problematically bad, why would NDS be better than NeuralODE? It is possible that this particular form of omission makes the energy landscape easier to optimize; maybe that's what the paper implies. Or if it's just about training stability due to the coupling of jointly learning residual and model parameters, this should be eliminated by pretraining parameter estimation. But this should be properly investigated, as local minima are a typical problem in system identification, and this result could even hint at a way to alleviate this issue.\n\nIn conclusion, I feel while this is an interesting topic, the actual learnings we can draw on the value of using model priors and GBO are limited. This is not a bad paper, but in its current form I don't think it meets the standards for a top ML conference.\n\n### Further questions\n- How long is the history T'? Do the baselines (NeuralODE, FC, LSTM) have access to the history data? If not, this alone might explain the performance difference\n- Wouldn't you want to feed the output of g_phi into d_tau, so it's easier to compute corrections?\n- what's the performance of NDS w/o the residual, or without g_phi (this should be identical to the NeuralODE baseline; is it?). How much history does it need to estimate parameters well? Does pre-training the parameter estimation module improve performance?\n\n----\n\nUpdate:\nThank you for answering my questions and running the additional NDS0 baseline, this clarifies a vital aspect of the paper. \nI'm still concerned that in its current form, the learnings we can extract from the paper are limited.\nThe new baseline confirms the expected, that for low-dimensional synthetic tasks where the ground truth model is known, pure sysid is a perfectly fine strategy. This is not surprising, and a good sanity check to perform. Unfortunately, this is also the largest part of the experimental results, and I'm not sure we can learn that much more beyond this.\nResults on 5.2 do look promising-- neither NODE nor NDS0 perform as well as NDS, and the new baseline has strengthened this result. But as the only data point demonstrating an advantage of the method, with little analysis to why and when it does well this still feels a bit limited.\nI still think this line of research is very valuable, and I encourage the authors to study the properties of this approach further (e.g. noise, missing systematic terms, investigate why partial NDS performs so well in some tasks), and investigate other non-trivial domains where NDS or variants can shine.\n\nI've raised my score to 5.", "extra_scores": {}, "extra_text": {}}, {"review_id": "s5xsThpPY0", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper is well written and introduces a novel method to learn dynamical models, incorporating prior knowledge in the form of systems of ODE.\n\nThe Neural Dynamic Systems method is described in sufficient details and multiple variations are given for the handling of systems where only partial or approximate knowledge is attainable.\nThe experiments explore three different applications of the NDS method introduced in the paper, to a simple synthetic and noiseless physical system, a simplified fusion system where the system dynamics are approximate, and to a modified Cartpole control problem.\n\nThe experiments show promising results, and it seems likely that the machinery developed in this paper will find impactful applications in the natural sciences and in model-based RL.\nI would suggest exploring alternative RL models than the Cartpole problem, such as a robotics application, where the impact of an NDS approach might lead to more interesting results.\n\nIn the related work section, I recommend adding citations to arxiv:1909.05862 and arxiv:1909.12790, which explore very different graph-based methods to tackle a similar issue of predicting the dynamics of physical systems.\n\nAll in all, it is an interesting contribution to the literature of physical predictions, and I recommend it for acceptance. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "mLHNu7tNs48", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a neural-network architecture for modeling dynamical systems that incorporates prior domain knowledge of the system's dynamics. More specifically, the main contributions are the mechanisms for incorporating such knowledge, in terms of fully or partially known structure (differential equations) of the system, which in turn positively affects the modeling performance. The results from the experimental evaluation (on 3 synthetic and one real-world experiments), in general, show that the proposed Neural Dynamical Systems (NDS), and in particular the ones trained with partial prior knowledge, have better performance than several standard benchmarks (such as NeuralODEs, LSTMs, Sparse Regression etc.).\n\nThe paper has a very strong introduction and the authors provide a good (and quite lengthy) overview of the related work. This, however, given the page-limit, has a severe consequence on the latter parts of the paper, that are poorly structured, inconsistent and hard to follow. In particular, the experimental design as well as the presentation and the discussion of the findings need to be improved. \n\nComments:\n- Some (rather important) details of the method are missing, such as a deeper discussion and motivation related to the architecture design. In particular, what is the role of the context encoder and how does it affects the performance of the model? How does the time-span in the history encoder affects the overall performance (also why x and u in the history encoder are of different lengths)?\n- What is the difference between the Full-NDS and Approx.-NDS, with respect to g(.)?  If there is none, why are these considered different? Can you clarify why x1 and u1 in the historical encoder within Approx.-NDS are of length T',  but in the other u is of length T?\n- It is unclear why the reported MSE losses, in general, are so high? Are these summed (and not averaged) for each task? Even the best performing models, when using all the training examples (Fig2), have errors with magnitudes ~10^3 (Ballistic task) and ~10-10^2 (Lorenz task), which are unusually high. What is the interpretation of this? Maybe a plot of the predicted trajectories vs. the ground truth may help. \n- In Fig2, the performances of the Sparse Regression and GBO are constant w.r.t the number of training samples. After checking the appendices, their performance on the smaller sample size is reported as n/a. Does this mean that they were only trained on the complete data, or something else is happening. Moreover, Sparse Regression in particular, in principle is able to model a Lorenz system quite accurately (this is also reported in the original paper Brunton et. al 2015/2016). However here it seems is orders-of-magnitude worse than the rest- how was it parameterized and did you investigated what is the output?\n- The results show that Partial-NDS seem to perform well overall, therefore the details for their parameterization need to be placed in the main part, not the appendix. Nevertheless, while the authors state that they are a partial ablation of Full-NDS, given their performance there is a trade-of between the amount of partial knowledge and the overall performance. Therefore it would be useful to study this amount of prior knowledge given at input. For instance, how will other parts of the equation structure affect their performance (eg. in the Lorenz tasks if you provide x and z but not y) or what is the least amount that is needed which will still lead to good performance with small amounts of data. Also, can parts of these knowledge be approximations etc. \n- The noise/irregular spacing experiments are mentioned in the beginning of Sec 5.1. Besides the figures given in A9, these findings are never properly discussed nor summarized in the main part (except \"NDS does well\"). \n- The \"Sample Complexity and Overall Accuracy\" segment seems out of place and a bit confusing. The paragraph discusses the Fusion experiment but Figure4 (Lorenz) is referenced. Also can you clarify the meaning of \"three points on the spectrum of added structure\" and what are \"our system identification models\"?\n- Adding a summary of the main contributions w.r.t. the results would be beneficial\n\nOther comments:\n- SINDy (Brunton et. al 2015/2016) doesn't perform Sparse Symbolic Regression - but Sparse Regression.\n- Consider moving the PartialNDS set-up in the main part.\n- How is the Partial-NDS parameterized in the Cartpole experiment?\n- In Eq(9) and (10) why is  x_dot = x_dot (and same for the other 3)?  \n- Last paragraph references Table 5.3. and Table A.12  which don't exist.\n\n-------update-----\nThank you for your response. The discussion and the revised manuscript clarified some of my concerns regarding your work. I appreciate that the authors will focus on the parameterization of PartialNDS and the effect of the amount of employed prior-knowledge. However, my concerns regarding the overall performance, reported as very high errors overall, still remain. This might be due to how the experiments are designed, how the results reported or something else - but nevertheless it needs more attention and further investigation.", "extra_scores": {}, "extra_text": {}}, {"review_id": "4r10zGcQvxd", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This work proposes an hybrid framework where the dynamical system inferred in Neural ODE (NODE) is parameterised by two separated components: a component implementing known dynamics provided by a given ODE, and a free component parameterised by a neural network. The rationale behind this approach is to exploit known properties of the data as opposed to a fully data-driven approach.\nPractically, the framework is simply obtained by modifying the neural dynamics of NODE to account for transformations parameterised by the given ODE, and by letting the network take care of the dimensions for which the dynamics are unknown.\n\nThe experiments are carried out on synthetic data generated from the Lorenz system, and on data representing a simplified fusion system and control experiment. The results show improved predictions whenever the dynamics are introduced in the NODE integrator, either in full or partial form, as compared to the full non-parametric implementation through neural networks.\n\nThe work is interesting and provides a convincing argument for the usability of NODE in settings different from the one proposed in the original paper. The idea of hybrid parameterisation of the dynamics is also interesting and proven to be useful in the experimental setup. However, to my opinion the contribution of this work is quite incremental. The proposed method is still a special case of NODE, obtained by using parameterised dynamics. In this sense, standard applications of NODE (with its standard implementation of ODESolve),  can be used for this purpose, by simply parameterising the latent dynamics with the functional implementing the ODE of interest.\nTo my understanding, the contribution of the work is beyond the methodological framework, and resides in the application on fusion systems. However, I feel that the work should provide a more thorough evaluation of the dynamics, and attempt the implementation of more complex systems.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper presents a framework for modeling dynamical systems by combining prior knowledge available as ODE and implemented via a differentiable solver, with statistical modules. This is a key problem consisting in complementing available partial knowledge on a physical system with information extracted from available data with agnostic statistical methods. In their framework, both the ODE parameters and the residual model parameters are learned. Experiments are performed on synthetic and on a simplified but realistic problem. 

All the reviewers do agree that the topic is important and that the paper has merits and brings an interesting contribution. They highlight some weaknesses in the presentation and more importantly in the experimental assessment. Overall, this is a good paper that should still be somewhat improved for publication. The authors are encouraged to investigate further the analysis of their framework in different settings and to bring more experimental evidence.
Decision: Reject