Paper:
This paper addresses the data-driven identification of latent Ordinary Differential Equation (ODE) representations of partially-observed dynamical systems, i.e. dynamical systems whose some components are never observed, with an emphasis on forecasting applications and long-term asymptotic patterns. Whereas stateof-the-art data-driven approaches rely on an explicit mapping of the observation series onto a latent space, the proposed approach only relies on the definition of an augmented space, higher-dimensional than the manifold spanned by the observed variables, where the dynamics of the observations can be fully described by an ODE. From a numerical point of view, the proposed approach exploits a neural-network ODE representation and the associated variational minimization scheme. Numerical experiments support the relevance of the proposed framework w.r.t. state-of-the-art approaches, including neural ODE schemes, both in terms of short-term forecasting errors and long-term behaviour. We further discuss how the proposed framework relates to Koopman operator theory and Takens' embedding theorem.

Review:
Update: I raised the score from 1 to 3 to acknowledge the authors' consideration for the 2000-2010 literature on learning dynamical systems from partial observations. Unfortunately, the writing is still confusing, some of the claims in the introduction and rebuttal are inexact ([5] does not embed the observations and does work with partially observed environments), and the method lacks originality compared to existing work. Other work that relies on ODE integration and in finding high-dimensional state variables has recently been published and tested on more ambitious datasets, e.g. Ayed et al, "Learning Dynamical Systems from Partial Observations", arXiv 2019.
***
TL;DR: relatively well written (if sometimes confusing) paper that reinvents the inference of latent variables in nonlinear dynamical systems that has been published in the 2000s, and that misses an important chunk of literature (and experiments on dynamical systems such as Lorenz-63) from that time.
This paper proposes an approach for learning dynamical systems from partial observations x, by using an augmented state variable z that follows dynamics that can be described by an ordinary differential equation (ODE) with dynamics f. The authors motivate their work by the problem of dynamical system identification when only partial observations are available. The authors claim that was to date primarily addressed using time-delay embedding, following Takens' theorem. The authors introduce s-dimensional unknown state variables z, dynamical function f (for the ODE on z), flow phi on z, limit cycles on z, observation function H: z -> x that goes from z to n-dimensional observations x, low k-dimensional manifold r (with a map M: x -> r), and state augmentation variable y. The reconstructed state variable u is the concatenation of r and y. One key ingredient of the method is to infer the optimal value of state augmentation variable y during learning (see equations 5 and 6) and inference for forecasting (7); this is not well explained in the abstract and introduction.
I would note that the problem of state space modeling (SSM) and dynamical system identification has been well studied, and the notation and reformulation in this paper is somewhat confusing for those who are used to the notation in SSMs (specifically, expressing the observation approximation as M^{-1}(G(phi(u_{t-1}))). Learning a state-space model involves both learning parameters and inferring the latent states representation (or, in graphical models, the distribution of these latent states) given the parametric models. One approach has been to formulate the state-space model learning by maximum likelihood learning of the model parameters so that the generative model fits observed data x, and this would involve factoring out the distribution of latent states z; the algorithm would rely on Expectation Maximisation, and could involve variational approximations or sampling. While the state space models were hampered by their linearity, several papers in 2000s showed how it is possible to learn nonlinear dynamical models, e.g. [4], [5], [6] and [7] to cite earlier ones. Equations (5) and (6) are similar to the standard equations for a dynamical system expressed in continuous time, with the only difference that the optimisation is with respect to y only, rather than w.r.t. z or u (why not \tilde z or \hat z?).
The paper mentions various initialisation strategies for y (last paragraph of section 3). Why not predict from the past of the observations, like is done in many other similar work?
The literature review mixes older and newer references. For example, on page 1, I would note that the Takens' theorem has been applied in conjuction with Support Vector Regression as early as 1999 [1][2], and with neural networks in 1993 [3].
Most importantly, the ideas of this paper have already been published in [4] (with architecture constraints on the neural network state-space model), in [5] (with any nonlinear neural network state-space model), in [6] (using Restricted Boltzmann Machines) and in [7] (using Gaussian Process latent variable models).
The model is illustrated with experiments on a 2D linear attractor, on the Lorenz-63 attractor. Given the results published in [1] and [2] using SVR on 1D observations of that attractor, and in [5] using a recurrent neural network, I am unconvinced by these results. It seems in particular that the number of training points (around 4000) limits the performance of RNN / LSTM models. The application to Sea Level Anomaly is interesting.
Minor comments:
"Unfortunately, When" (page 1)
There is a missing -1 after M in equation (5) and (10)
In equation (7), should not the sum go from t=0 to T, as x_t is unknown for t>T?
What is prediction and what is extrapolation on Figure 1?
The caption of Fig 1 contains (left)
The figures seem squeezed with the captions / titles un-aesthetically wide. 
Labels on Figure 5 in the appendix seem mixed, and red should be the ground truth
[1] Mattera & Haykin (1999) "Support vector machines for dynamic reconstruction of a chaotic system"
[2] Muller, Smola, Ratsch, Scholkopf, Kohlmorgen & Vapnik (1999) "Using support vector machines for time-series prediction"
[3] Wan (1994) "Time series prediction by using a connectionist network with internal delay lines"
[4] Ghahramani, and Roweis (1999) "Learning nonlinear dynamical systems using an EM algorithm"
[5] Mirowski & LeCun (2009) "Dynamic Factor Graphs for Time Series Modeling"
[6] Taylor, Hinton & Roweis (2006) "Modeling human motion using binary latent variables"
[7] Wang, Fleet & Hertzmann (2006) "Gaussian process dynamical models"