Original: C ONSTRAINED P HYSICAL -S TATISTICS M ODELS FOR D YNAMICAL S YSTEM I DENTIFICATION AND P REDICTION
Revised: C ONSTRAINED P HYSICAL -S TATISTICS M ODELS FOR D YNAMICAL S YSTEM I DENTIFICATION AND P REDICTION

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: BSTRACT

Original: Modeling dynamical systems combining prior physical knowledge and machine learning (ML) is promising in scientiﬁc problems when the underlying processes are not fully understood, e.g. when the dynamics is partially known.
Revised: Modeling dynamical systems combining prior physical knowledge and machine learning (ML) is promising in scientiﬁc problems when the underlying processes are not fully understood, e.g. when the dynamics is partially known.

Original: A common practice to identify the respective parameters of the physical and ML components is to formulate the problem as supervised learning on observed trajectories.
Revised: A common practice to identify the respective parameters of the physical and ML components is to formulate the problem as supervised learning on observed trajectories.

Original: However, this formulation leads to an inﬁnite number of possible decompositions.
Revised: However, this formulation leads to an inﬁnite number of possible decompositions.

Original: To solve this ill-posedness, we reformulate the learning problem by introducing an upper bound on the prediction error of a physical-statistical model.
Revised: To solve this ill-posedness, we reformulate the learning problem by introducing an upper bound on the prediction error of a physical-statistical model.

Original: This allows us to control the contribution of both the physical and statistical components to the overall prediction.
Revised: This allows us to control the contribution of both the physical and statistical components to the overall prediction.

Original: This framework generalizes several existing hybrid schemes proposed in the literature.
Revised: This framework generalizes several existing hybrid schemes proposed in the literature.

Original: We provide theoretical guarantees on the wellposedness of our formulation along with a proof of convergence in a simple afﬁne setting.
Revised: We provide theoretical guarantees on the wellposedness of our formulation along with a proof of convergence in a simple afﬁne setting.

Original: For more complex dynamics, we validate our framework experimentally.
Revised: For more complex dynamics, we validate our framework experimentally.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Dynamical systems prediction and identiﬁcation ﬁnd crucial applications ranging from medicine and the study of tumors (Hanahan & Weinberg, 2011; Lu & Fei, 2014) to oceanic and climate forecasting (Oreskes et al., 1994;
Revised: Dynamical systems prediction and identiﬁcation ﬁnd crucial applications ranging from medicine and the study of tumors (Hanahan & Weinberg, 2011; Lu & Fei, 2014) to oceanic and climate forecasting (Oreskes et al., 1994;

Original: Caers, 2011).
Revised: Caers, 2011).

Original: The modeling of such systems traditionally rely on ordinary or partial differential equations (ODE/PDE) (Madec, 2008; Marti et al., 2010), and their resolution via numerical solvers and data assimilation (Ghil & Malanotte-Rizzoli, 1991).
Revised: The modeling of such systems traditionally rely on ordinary or partial differential equations (ODE/PDE) (Madec, 2008; Marti et al., 2010), and their resolution via numerical solvers and data assimilation (Ghil & Malanotte-Rizzoli, 1991).

Original: In real world applications, two main pitfalls occur: ﬁrst the dynamics may only be partially known and thus do not fully represent the studied phenomena (Rio & Santoleri, 2018); second, the system state may only be partially observed as in ocean models (Gaultier et al., 2013).
Revised: In real world applications, two main pitfalls occur: ﬁrst the dynamics may only be partially known and thus do not fully represent the studied phenomena (Rio & Santoleri, 2018); second, the system state may only be partially observed as in ocean models (Gaultier et al., 2013).

Original: Machine learning (ML) has become a complementary approach to traditional physics based models (denoted MB for model based)
Revised: Machine learning (ML) has become a complementary approach to traditional physics based models (denoted MB for model based)

Original: (Reichstein et al., 2019; Dueben & Bauer, 2018).
Revised: (Reichstein et al., 2019; Dueben & Bauer, 2018).

Original: Both offer advantages: whereas MB approaches generalize and extrapolate better, ML high expressivity approaches beneﬁt from the ongoingexplosive growth of available data such as satellite observations, with reduced costs compared toclassical data assimilation.
Revised: Both offer advantages: whereas MB approaches generalize and extrapolate better, ML high expressivity approaches beneﬁt from the ongoing growth of available data such as satellite observations, with reduced costs compared to data assimilation.

Original: In that perspective, recent lines of work tackle the learning of hybrid models relying on prior physical knowledge and machine learning (Yin et al., 2021; Mehta et al., 2020).
Revised: In that perspective, recent lines of work tackle the learning of hybrid models relying on prior physical knowledge and machine learning (Yin et al., 2021; Mehta et al., 2020).

Original: Efﬁciently learning such decompositions actually means solving two different tasks: system identiﬁcation, i.e. estimating the parameters of the physical model, and prediction, i.e. recovering the trajectories associated to the dynamics.
Revised: Efﬁciently learning such decompositions actually means solving two different tasks: system identiﬁcation, i.e. estimating the parameters of the physical model, and prediction, i.e. recovering the trajectories associated to the dynamics.

Original: Both are essential for hybrid MB/ML models of dynamical systems.
Revised: Both are essential for hybrid MB/ML models of dynamical systems.

Original: Whereas prediction aims at robust extrapolation, identiﬁcation accounts for physical interpretability of the MB/ML model.
Revised: Whereas prediction aims at robust extrapolation, identiﬁcation accounts for physical interpretability of the MB/ML model.

Original: While solving both problems using model-based formulation admits well-known numerical solutions, for example using the adjoint method (Le Dimet & Talagrand, 1986; Courtier et al., 1994), the combination of physical models and deep learning is still an open area of research.
Revised: While solving both problems using model-based formulation admits well-known numerical solutions, for example using the adjoint method (Le Dimet & Talagrand, 1986; Courtier et al., 1994), the combination of physical models and deep learning is still an open area of research.

Original: In this context, ML applications mainly focus on the prediction task, at the expense of the system identiﬁcation: Willard et al.
Revised: In this context, ML applications mainly focus on the prediction task, at the expense of the system identiﬁcation: Willard et al.

Original: (2020) underlines the lack of generalizability of black-box ML models and their inability to produce physically sound results.
Revised: (2020) underlines the lack of generalizability of black-box ML models and their inability to produce physically sound results.

Original: Indeed, Ayed et al.
Revised: Indeed, Ayed et al.

Original: (2020) show that without any prior knowledge, the recovered estimates of a dynamical system states are not physically plausible despite accurate predictions.
Revised: (2020) show that without any prior knowledge, the recovered estimates of a dynamical system states are not physically plausible despite accurate predictions.

Original: Moreover, as noted by Yin et al.
Revised: Moreover, as noted by Yin et al.

Original: (2021), learning a linear MB/ML decompo- sition with the sole supervision on the system trajectories is ill-posed and admits an inﬁnite number of decompositions.
Revised: (2021), learning a linear MB/ML decomposition with the sole supervision on the system trajectories is ill-posed and admits an inﬁnite number of decompositions.

Original: Such observations highlight the need to incorporate physically motivated constraints in the learning of hybrid models, e.g. through regularization penalties, and several works propose additional constraints to guide the model towards physical solutions (Jia et al., 2019;
Revised: Such observations highlight the need to incorporate physically motivated constraints in the learning of hybrid models, e.g. through regularization penalties, and several works propose additional constraints to guide the model towards physical solutions (Jia et al., 2019;

Original: Yin et al., 2021; Linial et al., 2021).
Revised: Yin et al., 2021; Linial et al., 2021).

Original: Finally, to complete prior dynamical knowledge with a data-driven component and ensure interpretability of the decomposition, we work out a principled framework that generalizes previous attempts in the regularization of hybrid models.
Revised: Finally, to complete prior dynamical knowledge with a data-driven component and ensure interpretability of the decomposition, we work out a principled framework that generalizes previous attempts in the regularization of hybrid models.

Original: Our contributions are :
Revised: Our contributions are :

Original: • In section 3.1, we introduce a novel way to recover well-posedness and interpretability in the learning of hybrid MB/ML models through the control of an upper bound.
Revised: • In section 3.1, we introduce a novel way to recover well-posedness and interpretability in the learning of hybrid MB/ML models via the control of an upper bound.

Original: Wefurther extend our framework to incorporate auxiliary data when available to handle complex realworld problems.
Revised: We extend our framework to incorporate auxiliary data when available to handle complex real-world data.

Original: • In section 3.2, we propose a novel alternate-optimization algorithm to learn hybrid models.
Revised: • In section 3.2, we propose a novel alternate-optimization algorithm to learn hybrid models.

Original: • In section 3.3, we provide an analysis of the convergence of the proposed algorithm on a simpliﬁed case and experimentally evidence the soundness of our approach on more complex settings of increasing difﬁculty including challenging real world problems (section 4).
Revised: • In section 3.3, we provide an analysis of the convergence of the proposed algorithm on a simpliﬁed case and experimentally evidence the soundness of our approach on more complex settings of increasing difﬁculty including challenging real world problems (section 4).

Original:  B ACKGROUND AND P ROBLEM S ETUP
Revised:  B ACKGROUND AND P ROBLEM S ETUP

Original: We consider a dynamical system with state at time t denoted Z t =
Revised: We consider a dynamical system with state at time t denoted Z t =

Original: Z ( t ) .
Revised: Z ( t ) .

Original: Z t might be fully or only partially observed: we write Z t =
Revised: Z t might be fully or only partially observed: we write Z t =

Original: ( X t , Y t ) , where X t is the observed component and Y t the unobserved one.
Revised: ( X t , Y t ) , where X t is the observed component and Y t the unobserved one.

Original: The evolution of Z is governed by a differential equation with dynamics :
Revised: The evolution of Z is governed by a differential equation with dynamics :

Original: [Equation]
Revised: [Equation]

Original: The objective is to predict trajectories of X , i.e. to model the evolution of the observable part following dX t dt = f
Revised: The objective is to predict trajectories of X , i.e. to model the evolution of the observable part following dX t dt = f

Original: X ( Z t ) .
Revised: X ( Z t ) .

Original: For simplicity, we omit the index X in f X and write f ( . )
Revised: For simplicity, we omit the index X in f X and write f ( . )

Original: ∆ = f X ( . ) .
Revised: ∆ = f X ( . ) .

Original: Dynamical Hypothesis We assume partial knowledge of the dynamics of the observed X t :
Revised: Dynamical Hypothesis We assume partial knowledge of the dynamics of the observed X t :

Original: [Equation]
Revised: [Equation]

Original: where f k ∈ H k is a known operator with unknown parameters θ ∗ , and f u ∈ H u is the unknown residual dynamics.
Revised: where f k ∈ H k is a known operator with unknown parameters θ ∗ , and f u ∈ H u is the unknown residual dynamics.

Original: H k and H u denote function spaces. Note that the additive hypothesis in eq. (2) is not restrictive (see discussion in appendix B).
Revised: H k and H u denote function spaces, see discussion in appendix B.

Original: Learning Problem
Revised: Learning Problem

Original: Our objective is to approximate f with a function h learned from the observed data.
Revised: Our objective is to approximate f with a function h learned from the observed data.

Original: According to eq.
Revised: According to eq.

Original: (2), we assume h = h k
Revised: (2), we assume h = h k

Original: + h u .
Revised: + h u .

Original: h k ∈ H k  belongs to the same hypothesis space as f k , i.e. it has the same parametric form.
Revised: h k ∈ H k , i.e. belongs to the same hypothesis space as f k : it has the same parametric form.

Original: Its parameters are denoted θ k .
Revised: Its parameters are denoted θ k .

Original: Note that h k ( ., θ ∗ ) =
Revised: Note that h k ( ., θ ∗ ) =

Original: f k .
Revised: f k .

Original: h u ∈ H u is represented by a free form functional with parameters θ u , e.g. a neural network.
Revised: h u ∈ H u is represented by a free form functional with parameters θ u , e.g. a neural network.

Original: The learning problem isthen to estimate from data the parameters of h k so that they match the true physical ones andthose of h uso as to approximate at best the unknown dynamics f .
Revised: The learning problem is to estimate from data the parameters of h k so that they match the true physical ones and h u to approximate at best the unknown dynamics f .

Original: In this regard, an intuitive training objective is to minimize a distance d between h = h k + h u and f :
Revised: In this regard, an intuitive training objective is to minimize a distance d between h = h k + h u and f :

Original: [Equation]
Revised: [Equation]

Original: where p Z is the distribution of the statevalues Z that accounts for varying initial states.
Revised: where p Z is the distribution of the state Z that accounts for varying initial states.

Original: Each Zthen deﬁnes a training sample.
Revised: Each Z deﬁnes a training sample.

Original: Minimizing eq.
Revised: Minimizing eq.

Original: (3) with h = h k
Revised: (3) with h = h k

Original: + h u enables to predict accurate trajectories but may have an inﬁnite number of solutions. For instance, h u may bypass the physical hypothesis h k .
Revised: + h u enables to predict accurate trajectories but may have an inﬁnite number of solutions and h u may bypass the physical hypothesis h k .

Original: Thus, interpretability is not guaranteed.
Revised: Thus, interpretability is not guaranteed.

Original: We  develop in the following section a method to overcome this ill-posedness.
Revised: We now develop our proposition to overcome this ill-posedness.

Original:  M ETHOD
Revised:  M ETHOD

Original: In hybrid modeling, two criteria are essentials: 1.
Revised: In hybrid modeling, two criteria are essentials: 1.

Original: identiﬁability, i.e. the estimated parameters of h k should correspond to the true physical ones; 2.
Revised: identiﬁability, i.e. the estimated parameters of h k should correspond to the true physical ones; 2.

Original: prediction power, i.e. the statistical component h u should complete h k
Revised: prediction power, i.e. the statistical component h u should complete h k

Original: so that h = h k
Revised: so that h = h k

Original: + h u performs accurate prediction over the system states.
Revised: + h u performs accurate prediction over the system states.

Original: To control the contribution of each term h k and h u , we work upper bounds out of eq.
Revised: To control the contribution of each term h k and h u , we work upper bounds out of eq.

Original: (3) (section 3.1).
Revised: (3) (section 3.1).

Original: We then propose to minimize d( h, f ) while constraining the upper bounds, which provide us with a well-posed learning framework (section 3.2).
Revised: We then propose to minimize d( h, f ) while constraining the upper bounds, which provide us with a well-posed learning framework (section 3.2).

Original: Besides, we show that several previous works that introduced constrained optimization to solve related problems are speciﬁc cases of our formulation (Yin et al., 2021;
Revised: Besides, we show that several previous works that introduced constrained optimization to solve related problems are speciﬁc cases of our formulation (Yin et al., 2021;

Original: Jia et al., 2019; Linial et al., 2021).
Revised: Jia et al., 2019; Linial et al., 2021).

Original: Finally, we introduce an alternate optimization algorithm which convergence is shown in section 3.3 for a linear approximation of f .
Revised: Finally, we introduce an alternate optimization algorithm which convergence is shown in section 3.3 for a linear approximation of f .

Original: 3. S TRUCTURAL C ONSTRAINTS FOR D YNAMICAL S YSTEMS
Revised: 3. S TRUCTURAL C ONSTRAINTS FOR D YNAMICAL S YSTEMS

Original: To ensure identiﬁability, we derive regularizations on h k and h u ﬂowing from the control of an upper bound of d( h, f ) .
Revised: To ensure identiﬁability, we derive regularizations on h k and h u ﬂowing from the control of an upper bound of d( h, f ) .

Original: In particular, to minimize d( h k , f k ) would enable us to accurately interpret h k as the true f k , andthus h u as the residual dynamics f u .
Revised: In particular, to minimize d( h k , f k ) would enable us to accurately interpret h k as the true f k , and h u as the residual dynamics f u .

Original: However, since we do not access the parameters of f k , computing d( h k , f k ) is not tractable.
Revised: However, since we do not access the parameters of f k , computing d( h k , f k ) is not tractable.

Original: We then consider two possible situations.
Revised: We then consider two possible situations.

Original: In the ﬁrst one, the only available information on the physical system is the parametric form of f k (or equivalently of h k ), training thus only relies on observed trajectories (section 3.1.1).
Revised: In the ﬁrst one, the only available information on the physical system is the parametric form of f k (or equivalently of h k ), training thus only relies on observed trajectories (section 3.1.1).

Original: In the second one, we consider that auxiliary information about f k is available and will be used to minimize the distance between h k and f k (section 3.1.2).
Revised: In the second one, we consider available auxiliary information about f k that will be used to minimize the distance between h k and f k (section 3.1.2).

Original: While the ﬁrst setting is the more general, the physical prior it relies on is often insufﬁcient to effectively handle real world situations.
Revised: While the ﬁrst setting is the more general, the physical prior it relies on is often insufﬁcient to effectively handle real world situations.

Original: The second setting makes use of more informative priors and better corresponds to real cases aswill be shown in the experimental section (see section 4.2).
Revised: The second setting makes use of more informative priors and better corresponds to real cases as shown in the experiments (section 4.2).

Original: 3.1. C ONTROLLING THE ML C OMPONENT AND THE MB H YPOTHESIS
Revised: 3.1. C ONTROLLING THE ML C OMPONENT AND THE MB H YPOTHESIS

Original: We propose a general approach to constrain the learning of hybrid models when one solely access the functional form of h k .
Revised: We propose a general approach to constrain the learning of hybrid models when one solely access the functional form of h k .

Original: In this case, to make h k accountable in our observed phenomena, a solution is to minimize d( h k , f ) .
Revised: In this case, to make h k accountable in our observed phenomena, a solution is to minimize d( h k , f ) .

Original: Following the triangle inequality we link up both errors d( h, f ) and d( h k , f ) (computations available in appendix C.1):
Revised: Following the triangle inequality we link up both errors d( h, f ) and d( h k , f ) (computations available in appendix C.1):

Original: [Equation]
Revised: [Equation]

Original: We want the physical-statistical model h
Revised: We want the physical-statistical model h

Original: = h k
Revised: = h k

Original: + h u to provide high quality forecasts.
Revised: + h u to provide high quality forecasts.

Original: Minimizing the sole upper bound does not ensure such aim, as h u is only penalized through d( h u , 0) and is not optimized to contribute to predictions.
Revised: Minimizing the sole upper bound does not ensure such aim, as h u is only penalized through d( h u , 0) and is not optimized to contribute to predictions.

Original: We thus propose to minimize d( h, f ) while controlling both d( h u , 0) and d( h k , f ) .
Revised: We thus propose to minimize d( h, f ) while controlling both d( h u , 0) and d( h k , f ) .

Original: Such a control of the upper bound of eq.
Revised: Such a control of the upper bound of eq.

Original: (4) amounts to balancing the contribution of the ML and the MB components.
Revised: (4) amounts to balancing the contribution of the ML and the MB components.

Original: This will be formally introduced in section 3.2.
Revised: This will be formally introduced in section 3.2.

Original: Link to the Literature
Revised: Link to the Literature

Original: The least action principle on the ML component i.e. constraining d( h u , 0) is invoked for a geometric argument in (Yin et al., 2021), and appears as a co-product of the introduction of d( h k , f ) in eq.
Revised: The least action principle on the ML component i.e. constraining d( h u , 0) is invoked for a geometric argument in (Yin et al., 2021), and appears as a co-product of the introduction of d( h k , f ) in eq.

Original: Optimizing d( h k , f ) to match the physical model with observations is investigated in (Forssell & Lindskog, 1997).
Revised: Optimizing d( h k , f ) to match the physical model with observations is investigated in (Forssell & Lindskog, 1997).

Original: The general approach of eq.
Revised: The general approach of eq.

Original: (4) allows us to perform prediction (via h ) and system identiﬁcation (via h k ) on simple problems (see section 4.1).
Revised: (4) allows us to perform prediction (via h ) and system identiﬁcation (via h k ) on simple problems (see section 4.1).

Original: The learning of real-world complex dynamics, via data-driven hybrid models, often fails at yielding a physically sound estimation, as illustrated in section 4.2.
Revised: The learning of real-world complex dynamics, via data-driven hybrid models, often fails at yielding a physically sound estimation, as illustrated in section 4.2.

Original: This suggests that learning complex dynamics requires additional information.
Revised: This suggests that learning complex dynamics requires additional information.

Original: In many real-world cases, auxiliary information is available in the form of measurements providing complementary information on f k .
Revised: In many real-world cases, auxiliary information is available in the form of measurements providing complementary information on f k .

Original: Indeed, a common issue in physics is to infer an unobserved variable of interest (in our case f k parameters θ (cid:63) ) from indirect or noisy measurements that we refer to as proxy data.
Revised: Indeed, a common issue in physics is to infer an unobserved variable of interest (in our case f k parameters θ (cid:63) ) from indirect or noisy measurements that we refer to as proxy data.

Original: For instance, one can access a physical quantity but only at a coarse resolution, as in (Um et al., 2020; Belbute-Peres et al., 2020) and in the real world example detailed in section 4.2.
Revised: For instance, one can access a physical quantity but only at a coarse resolution, as in (Um et al., 2020; Belbute-Peres et al., 2020) and in the real world example detailed in section 4.2.

Original: We show in the next subsection how to incorporate such an information in order to approximate d( h k , f [Equation]
Revised: We show in the next subsection how to incorporate such an information in order to approximate d( h k , f k ) .

Original: 3.1. M ATCHING THE P HYSICAL H YPOTHESES : I NTRODUCING A UXILIARY D ATA
Revised: 3.1. M ATCHING THE P HYSICAL H YPOTHESES : I NTRODUCING A UXILIARY D ATA

Original: We here assume one accesses a proxy of f k , denoted f prk ∈ H k .
Revised: We here assume one accesses a proxy of f k , denoted f prk ∈ H k .

Original: Our goal is to adapt our framework to incorporate such auxiliary information, bringing the regularization induced by f prk within the scope of the control of an upper bound.
Revised: Our goal is to adapt our framework to incorporate such auxiliary information, bringing the regularization induced by f pr k within the scope of the control of an upper bound.

Original: This enables us to extend our proposition towards the solving of real world physical problems, still largely unexplored by the ML community.
Revised: This enables us to extend our proposition towards the solving of real world physical problems, still largely unexplored by the ML community.

Original: We have:
Revised: We have:

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where Γ is a constant of the problem that cannot be optimized (see appendix C.2).
Revised: where Γ is a constant of the problem that cannot be optimized (see appendix C.2).

Original: In that context, we can beneﬁt from auxiliary information providing us with coarse estimates of θ (cid:63) , denoted θ pr ,
Revised: In that context, we can beneﬁt from auxiliary information providing us with coarse estimates of θ (cid:63) , denoted θ pr ,

Original: such that f prk = h k ( ., θ pr ) ≈ f k .
Revised: such that f prk = h k ( ., θ pr ) ≈ f k .

Original: To use the available θ pr to guide our estimation towards the true parameters θ (cid:63) of f k , a simple solution is to directly enforce the minimization of d( h k , f prk ) in the parameter space by minimizing (cid:107) θ k − θ pr (cid:107) 2 , where θ k are the parameters of h k .
Revised: To use the available θ pr to guide our estimation towards the true parameters θ (cid:63) of f k , a simple solution is to directly enforce the minimization of d( h k , f prk ) in the parameter space by minimizing (cid:107) θ k − θ pr (cid:107) 2 , where θ k are the parameters of h k .

Original: Indeed, because f k and f prk have identical parametric forms (as both belong to the same functional space
Revised: Indeed, because f k and f prk have identical parametric forms (as both belong to the same functional space

Original: H k ), minimizing (cid:107) θ k − θ pr (cid:107) 2 will bring h k closer to f prk and thus to f k .
Revised: H k ), minimizing (cid:107) θ k − θ pr (cid:107) 2 will bring h k closer to f prk and thus to f k .

Original: As above, we propose to minimize d( h, f ) while controlling both d( h u , 0) and d( h k , f prk ) , as described in section 3.2.
Revised: As above, we propose to minimize d( h, f ) while controlling both d( h u , 0) and d( h k , f prk ) , as described in section 3.2.

Original: Link to the Literature
Revised: Link to the Literature

Original: In (Linial et al., 2021)
Revised: In (Linial et al., 2021)

Original: f prk stands for true observations used to constrain a learned latent space, minimizing d( h k , f prk ) .
Revised: f prk stands for true observations used to constrain a learned latent space, minimizing d( h k , f prk ) .

Original: Jia et al.
Revised: Jia et al.

Original: (2019) uses synthetic data as f prk to pre-train their model which amounts to the control an upper bound, see appendix C.3.
Revised: (2019) uses synthetic data as f prk to pre-train their model which amounts to the control an upper bound, see appendix C.3.

Original: Finally, this setting ﬁnds an extension, when the model f prk is a learned model, for example trained using eq.
Revised: Finally, this setting ﬁnds an extension, when the model f prk is a learned model, for example trained using eq.

Original: (4), leading to a self-supervision approach described in appendix C.4.
Revised: (4), leading to a self-supervision approach described in appendix C.4.

Original: 3. L EARNING A LGORITHM AND O PTIMIZATION P ROBLEM
Revised: 3. L EARNING A LGORITHM AND O PTIMIZATION P ROBLEM

Original: From the upper bounds, we ﬁrst recover the well-posedness of the optimization and derive a theoretical learning scheme (section 3.2.1).
Revised: From the upper bounds, we ﬁrst recover the well-posedness of the optimization and derive a theoretical learning scheme (section 3.2.1).

Original: We then discuss its practical implementation (section 3.2.2).
Revised: We then discuss its practical implementation (section 3.2.2).

Original: 3.2. W ELL -P OSEDNESS AND A LTERNATE O PTIMIZATION A LGORITHM
Revised: 3.2. W ELL -P OSEDNESS AND A LTERNATE O PTIMIZATION A LGORITHM

Original: Recovering Well-Posedness
Revised: Recovering Well-Posedness

Original: We reformulate the ill-posed learning of min h k ,h u ∈H k ×H u d( h, f ) , by instead optimizing d( h, f ) while constraining the upper bounds.
Revised: We reformulate the ill-posed learning of min h k ,h u ∈H k ×H u d( h, f ) , by instead optimizing d( h, f ) while constraining the upper bounds.

Original: Let us deﬁne S k and S u as
Revised: Let us deﬁne S k and S u as

Original: [Equation]
Revised: [Equation]

Original: where µ k , µ u are two positive scalars and (cid:96) ( h k )
Revised: where µ k , µ u are two positive scalars and (cid:96) ( h k )

Original: =
Revised: =

Original: d( h k , f ) in the case of section 3.1.1 and (cid:96) ( h k ) =
Revised: d( h k , f ) in the case of section 3.1.1 and (cid:96) ( h k ) =

Original: d( h k , f prk ) in the case of section 3.1.2.
Revised: d( h k , f prk ) in the case of section 3.1.2.

Original: Our proposition then amounts to optimizing d( h, f ) over the Minkowski-sum S k + 
Revised: Our proposition then amounts to optimizing d( h, f ) over the Minkowski-sum S k + S u = { h = h k +

Original: [Equation]
Revised: [Equation]

Original: u
Revised: k

Original: [Equation]
Revised: [Equation]

Original:  This constrained optimization setting enables us to recover the well-posedness of the optimization problem under the relative compactness of the family of function H k
Revised: k [Equation] [Equation] This constrained optimization setting enables us to recover the well-posedness of the optimization problem under the relative compactness of the family of function H k

Original: (proof in appendix D.3).
Revised: (proof in appendix D.3).

Original: Proposition 1 (Well-posedness) .
Revised: Proposition 1 (Well-posedness) .

Original: Under the relative compactness of S k , eq.
Revised: Under the relative compactness of S k , eq.

Original: (7) ﬁnds a solution h that writes as h = h k
Revised: (7) ﬁnds a solution h that writes as h = h k

Original: + h u ∈ S k + S u .
Revised: + h u ∈ S k + S u .

Original: Moreover, this solution is unique.
Revised: Moreover, this solution is unique.

Original: Alternate Optimization Algorithm
Revised: Alternate Optimization Algorithm

Original: As the terms in both upper bounds of eqs.
Revised: As the terms in both upper bounds of eqs.

Original: (4) and (5) specifically address either h k or h u , we isolate losses relative to h k and h u and alternate projections of h k on S k and h u on S u , as described in Algorithm 1.
Revised: (4) and (5) specifically address either h k or h u , we isolate losses relative to h k and h u and alternate projections of h k on S k and h u on S u , as described in Algorithm 1.

Original: Said otherwise, we learn h by alternately optimizing h k ( h u being ﬁxed) and h u ( h k being ﬁxed).
Revised: Said otherwise, we learn h by alternately optimizing h k ( h u being ﬁxed) and h u ( h k being ﬁxed).

Original: In practice, we rely on a dual formulation (see section 3.2.2 and the SGD version of Algorithm 1 in Appendix F).
Revised: In practice, we rely on a dual formulation (see section 3.2.2 and the SGD version of Algorithm 1 in Appendix F).

Original: Result:
Revised: Result:

Original: Converged h k and h u
Revised: Converged h k and h u

Original: Set h 0 u = 0 , [Equation] =  min[Equation]
Revised: Set h 0 u = 0 , h 0 k = min h k ∈H k d( h k , f ) , tol ∈ R + while d( h, f ) > tol do h n +1 k = arg min

Original: d

Original: [Equation] h [Equation] , f ) , tol
Revised: h k ∈S k d( h k + h nu , f );

Original: [Equation]

Original: R + while d( h, f ) > tol do

Original: [Equation]
Revised: [Equation]

Original: n ← n + 1 end
Revised: n ← n + 1 end

Original: The convergence of the alternate projections is well studied for the intersection of convex sets or smooth manifolds (von Neumann, 1950; Lewis & Malick, 2008) and has been extended in our setting of Minkowski-sum of convex sets (Lange et al., 2019).
Revised: The convergence of the alternate projections is well studied for the intersection of convex sets or smooth manifolds (von Neumann, 1950; Lewis & Malick, 2008) and has been extended in our setting of Minkowski-sum of convex sets (Lange et al., 2019).

Original: Because d as deﬁned in eq.
Revised: Because d as deﬁned in eq.

Original: (3) is convex, S u and S k are convex sets as soon as H k and H u are convex (Appendix A).
Revised: (3) is convex, S u and S k are convex sets as soon as H k and H u are convex (Appendix A).

Original: Thus, if d( ., f ) is strongly convex, eq.
Revised: Thus, if d( ., f ) is strongly convex, eq.

Original: (8) ﬁnds one and only one solution (Boyd et al., 2004).
Revised: (8) ﬁnds one and only one solution (Boyd et al., 2004).

Original: However, neither the convexity of H u nor of H k is practically ensured.
Revised: However, neither the convexity of H u nor of H k is practically ensured.

Original: Nonetheless, we recover the well-posedness of eq.
Revised: Nonetheless, we recover the well-posedness of eq.

Original: (7) and show the convergence of Algorithm 1 in the simpliﬁed case where h is an afﬁne function of X t (see section 3.3).
Revised: (7) and show the convergence of Algorithm 1 in the simpliﬁed case where h is an afﬁne function of X t (see section 3.3).

Original: For complex PDE where convexity may not hold, we validate our approach experimentally and we evidence in section 4 that this formulation enables us to recover both an interpretable decomposition
Revised: For complex PDE where convexity may not hold, we validate our approach experimentally and we evidence in section 4 that this formulation enables us to recover both an interpretable decomposition

Original: h
Revised: h

Original: = h k + h u and improved prediction and identiﬁcation performances.
Revised: = h k + h u and improved prediction and identiﬁcation performances.

Original: 3.2. P RACTICAL O PTIMIZATION
Revised: 3.2. P RACTICAL O PTIMIZATION

Original: Equation (6) involves the choice of µ k and µ u .
Revised: Equation (6) involves the choice of µ k and µ u .

Original: In practice, we implement the projection algorithm by descending gradients on the parameters of h k and h u , with respect to the following losses:
Revised: In practice, we implement the projection algorithm by descending gradients on the parameters of h k and h u , with respect to the following losses:

Original: [Equation]
Revised: [Equation]

Original: where λ h , λ h k , λ h u are positive real values, dynamically increased/decreased during training.
Revised: where λ h , λ h k , λ h u are positive real values, dynamically increased/decreased during training.

Original: Indeed, d( h u , 0) can be interpreted as a stability loss , preventing the neural networks to trump the physical component.
Revised: Indeed, d( h u , 0) can be interpreted as a stability loss , preventing the neural networks to trump the physical component.

Original: On the other hand, d( h k , f ) can be interpreted has an initialization loss yield a ﬁrst estimate of θ k explaining the dynamics.
Revised: On the other hand, d( h k , f ) can be interpreted has an initialization loss yield a ﬁrst estimate of θ k explaining the dynamics.

Original: Yet, f being unknown: d( h, f ) is not tractable.
Revised: Yet, f being unknown: d( h, f ) is not tractable.

Original: To estimate d( h, f ) , we rely on the trajectories associated to the dynamics.
Revised: To estimate d( h, f ) , we rely on the trajectories associated to the dynamics.

Original: We minimize the distance between the ODE ﬂows φ h and φ f deﬁned by h and f , d φ ( φ h , φ f ) , over all initial conditions X 0 :
Revised: We minimize the distance between the ODE ﬂows φ h and φ f deﬁned by h and f , d φ ( φ h , φ f ) , over all initial conditions X 0 :

Original: [Equation]
Revised: [Equation]

Original: We have: [Equation]
Revised: We have: d φ ( φ h , φ f ) = 0 ⇔ d( h, f ) = 0 .

Original: Deﬁnitions of ﬂows for ODE and in depth consideration on these distances are available in appendix A.
Revised: Deﬁnitions of ﬂows for ODE and in depth consideration on these distances are available in appendix A.

Original: The gradients of d φ ( φ h , φ f ) with respect to the parameters of h k or h u can be either estimated analytically using the adjoint method (Chen et al., 2018) or using explicit solvers, e.g. Rk45, and computing the gradients thanks to the backpropagation, see (Onken & Ruthotto, 2020).
Revised: The gradients of d φ ( φ h , φ f ) with respect to the parameters of h k or h u can be either estimated analytically using the adjoint method (Chen et al., 2018) or using explicit solvers, e.g. Rk45, and computing the gradients thanks to the backpropagation, see (Onken & Ruthotto, 2020).

Original: To compute eq.
Revised: To compute eq.

Original: (10), we rely on a temporal sampling of X : our datasets are composed of n sequences of observations of length N , X i =
Revised: (10), we rely on a temporal sampling of X : our datasets are composed of n sequences of observations of length N , X i =

Original: ( X it 0 , . . .
Revised: ( X it 0 , . . .

Original: , X it 

Revised: , X it

Original: + N ∆ t ) , where each sequence X i follows eq.
Revised: + N ∆ t ) , where each sequence X i follows eq.

Original: (2) and corresponds to one initial condition X it 0 .
Revised: (2) and corresponds to one initial condition X it 0 .

Original: We then sample the space of initial conditions X it 0 to compute a Monte-Carlo approximation to d φ ( φ h , φ f ).
Revised: We then sample the space of initial conditions X it 0 to compute a Monte-Carlo approximation to d φ ( φ h , φ f ).

Original: Let ODESolve be the function integrating any arbitrary initial state x t 0 up to time t with dynamics h , so that x t = ODESolve ( x t 0 , h, t ) .
Revised: Let ODESolve be the function integrating any arbitrary initial state x t 0 up to time t with dynamics h , so that x t = ODESolve ( x t 0 , h, t ) .

Original: The estimate of d φ ( φ h , φ f ) then writes as:
Revised: The estimate of d φ ( φ h , φ f ) then writes as:

Original: [Equation]
Revised: [Equation]

Original: Note that the way to compute ODEsolve differs across the experiments (see section 4).
Revised: Note that the way to compute ODEsolve differs across the experiments (see section 4).

Original: 3. T HEORETICAL A NALYSIS FOR A L INEAR A PPROXIMATION
Revised: 3. T HEORETICAL A NALYSIS FOR A L INEAR A PPROXIMATION

Original: We investigate the validity of our proposition when approximating an unknown derivative with an afﬁne function (interpretable ﬁrst guess approximators).
Revised: We investigate the validity of our proposition when approximating an unknown derivative with an afﬁne function (interpretable ﬁrst guess approximators).

Original: We here consider h k as a linear function.
Revised: We here consider h k as a linear function.

Original: We do not assume any information on f , thus relieving this section from the need of an accurate prior knowledge f k .
Revised: We do not assume any information on f , thus relieving this section from the need of an accurate prior knowledge f k .

Original: In this context, we show the convergence of the learning scheme introduced in Algorithm 1 with (cid:96) =
Revised: In this context, we show the convergence of the learning scheme introduced in Algorithm 1 with (cid:96) =

Original: d( h k , f ) ,hence demonstrating the validity of our framework in this simpliﬁed setting.
Revised: d( h k , f ) , demonstrating the validity of our framework in this simpliﬁed setting.

Original: For more complex cases, for which theoretical analysis cannot be conducted, our framework is validated experimentally in section 4.
Revised: For more complex cases, for which theoretical analysis cannot be conducted, our framework is validated experimentally in section 4.

Original: All proofs of this section are conducted using the distance d φ .
Revised: All proofs of this section are conducted using the distance d φ .

Original: Let X s be the unique solution to the initial value problem:
Revised: Let X s be the unique solution to the initial value problem:

Original: [Equation]
Revised: [Equation]

Original: With h k ( X ) =
Revised: With h k ( X ) =

Original: AX and h u ( X )
Revised: AX and h u ( X )

Original: =
Revised: =

Original: D A , the afﬁne approximation of f writes as:
Revised: D A , the afﬁne approximation of f writes as:

Original: [Equation]
Revised: [Equation]

Original: where A ∈ M p,p ( R ) , D A ∈ R p .
Revised: where A ∈ M p,p ( R ) , D A ∈ R p .

Original: We write X D the solution to eq.
Revised: We write X D the solution to eq.

Original: (12) and X A the solution to eq.
Revised: (12) and X A the solution to eq.

Original: (12) when D A = 0 .
Revised: (12) when D A = 0 .

Original: The alternate projection algorithm with the distance d φ writes as:
Revised: The alternate projection algorithm with the distance d φ writes as:

Original: [Equation]
Revised: [Equation]

Original: where λ D , λ A > 0 .
Revised: where λ D , λ A > 0 .

Original: As the optimization of eq.
Revised: As the optimization of eq.

Original: (13) is not convex on A , the solution existence and uniqueness is not ensured.
Revised: (13) is not convex on A , the solution existence and uniqueness is not ensured.

Original: The well-posedness w.r.t A can be recovered by instead considering a simple discretization scheme, e.g. X t +

Revised: The well-posedness w.r.t A can be recovered by instead considering a simple discretization scheme, e.g. X t +

Original: ≈
Revised: ≈

Original: ( AX t + D A )
Revised: ( AX t + D A )

Original: ∆ t + X t and solving the associated least square regression, which well-posedness is guaranteed, see details in appendix D.2.
Revised: ∆ t + X t and solving the associated least square regression, which well-posedness is guaranteed, see details in appendix D.2.

Original: Such strategy is common practice in system identiﬁcation.
Revised: Such strategy is common practice in system identiﬁcation.

Original: Generic theoretical considerations on existence and uniqueness of solutions to eqs.
Revised: Theoretical considerations on existence and uniqueness of solutions to eqs.

Original: (13) and (14) are hard to retrieve.
Revised: (13) and (14) are hard to retrieve.

Original: Nonetheless, if A is an invertible matrix, we prove in appendix D.4:
Revised: If A is an invertible matrix:

Original: Proposition 2 (Existence and Uniqueness) .
Revised: Proposition 2 (Existence and Uniqueness) .

Original: If ˆ A is invertible, There exists a unique D A , hence a unique X D , solving eq.
Revised: If ˆ A is invertible, There exists a unique D A , hence a unique X D , solving eq.

Original:  Finally, formulating Algorithm 1 as a least square problem in an afﬁne setting (see appendix D.5), we prove the convergence of the alternate projection algorithm (appendix D.6) :
Revised: (proof in appendix D.4) Finally, formulating Algorithm 1 as a least square problem in an afﬁne setting (see appendix D.5), we prove the convergence of the alternate projection algorithm (appendix D.6) :

Original: Proposition 3.
Revised: Proposition 3.

Original: For λ D and λ A sufﬁciently high, the algorithm that alternates between the estimation of A and the estimation of D A following eqs.
Revised: For λ D and λ A sufﬁciently high, the algorithm that alternates between the estimation of A and the estimation of D A following eqs.

Original: (13) and (14) converges.
Revised: (13) and (14) converges.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We validate Algorithm 1 on datasets of increasing difﬁculty (see appendix E), where the system state is either fully or partially observed (resp.
Revised: We validate Algorithm 1 on datasets of increasing difﬁculty (see appendix E), where the system state is either fully or partially observed (resp.

Original: section 4.1 and section 4.2).
Revised: section 4.1 and section 4.2).

Original: We no longer rely on an afﬁne prior and explicit h k and h u for each dataset.
Revised: We no longer rely on an afﬁne prior and explicit h k and h u for each dataset.

Original: Performances are evaluated via standard metrics: MSE (lower is better) and relative Mean Absolute Error (rMAE, lower is better).
Revised: Performances are evaluated via standard metrics: MSE (lower is better) and relative Mean Absolute Error (rMAE, lower is better).

Original: We assess the relevance of our proposition based on eqs.
Revised: We assess the relevance of our proposition based on eqs.

Original: (4) and (5), against NeuralODE (Chen et al., 2018) and state of the art Aphynity Yin et al., 2021) and ablation studies.
Revised: (4) and (5), against NeuralODE (Chen et al., 2018), Aphynity (Yin et al., 2021) and ablation studies.

Original: We denote Ours eq.
Revised: We denote Ours eq.

Original: (4) (resp.
Revised: (4) (resp.

Original: Ours eq.
Revised: Ours eq.

Original: the results when (cid:96) =
Revised: the results when (cid:96) =

Original: d( h k , f ) i.e eq.
Revised: d( h k , f ) i.e eq.

Original: (4), (resp.
Revised: (4), (resp.

Original: (cid:96) =
Revised: (cid:96) =

Original: d( h k , f prk ) i.e. eq.
Revised: d( h k , f prk ) i.e. eq.

Original: When d( h k , f ) (resp.
Revised: When d( h k , f ) (resp.

Original: d( h u , 0) ) is not considered in the optimization, we refer to the results as d( h, f )+d( h u , 0) (resp.
Revised: d( h u , 0) ) is not considered in the optimization, we refer to the results as d( h, f )+d( h u , 0) (resp.

Original: d( h, f )+d( h k , f ) ).
Revised: d( h, f )+d( h k , f ) ).

Original: When h is trained by only minimizing the discrepancy between actual and predicted trajectories the results are denoted «Only d( h, f ) ».
Revised: When h is trained by only minimizing the discrepancy between actual and predicted trajectories the results are denoted «Only d( h, f ) ».

Original: We report between brackets the standard deviation of the metrics over 5 runs and refer to Appendices F and G for training information and additional results.
Revised: We report between brackets the standard deviation of the metrics over 5 runs and refer to Appendices F and G for training information and additional results.

Original: Code and datasets will be provided upon acceptance.

Original: 4. F ULLY O BSERVABLE D YNAMICS
Revised: 4. F ULLY O BSERVABLE D YNAMICS

Original: To illustrate the learning scheme induced by eq.
Revised: To illustrate the learning scheme induced by eq.

Original: (4), we focus on fully observed low dimensional dynamics: a simple example emerging from Newtonian mechanics and a population dynamics model.
Revised: (4), we focus on fully observed low dimensional dynamics: a simple example emerging from Newtonian mechanics and a population dynamics model.

Original: Damped Pendulum (DPL)
Revised: Damped Pendulum (DPL)

Original: Now a standard benchmark for hybrid models, we consider the motion of a pendulum of length L damped due to viscous friction (Greydanus et al., 2019;
Revised: Now a standard benchmark for hybrid models, we consider the motion of a pendulum of length L damped due to viscous friction (Greydanus et al., 2019;

Original: Yin et al., 2021).
Revised: Yin et al., 2021).

Original: Newtonian mechanics provide an ODE describing the evolution of the angle x of the pendulum:
Revised: Newtonian mechanics provide an ODE describing the evolution of the angle x of the pendulum:

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: We suppose access to observations of the system state Z =
Revised: We suppose access to observations of the system state Z =

Original: ( x, ˙ x ) .
Revised: ( x, ˙ x ) .

Original: We consider as physical motion hypothesis h k ( x, θ k )
Revised: We consider as physical motion hypothesis h k ( x, θ k )

Original: =
Revised: =

Original: θ k sin( x ) .
Revised: θ k sin( x ) .

Original: The true pulsation θ ∗ =
Revised: The true pulsation θ ∗ =

Original: g/L of the pendulum has to be estimated with θ k .
Revised: g/L of the pendulum has to be estimated with θ k .

Original: The viscous friction term k ˙ x remains to be estimated by h u .
Revised: The viscous friction term k ˙ x remains to be estimated by h u .

Original: Population Dynamics (LV) Lotka-Volterra ODE system models a prey/predator population dynamics describing the growth of the preys ( x ) without predators ( y ), and the extinction of predators without preys, the non linear terms expressing the encounters between both species:[Equation]
Revised: Population Dynamics (LV) Lotka-Volterra ODE system models a prey/predator population dynamics describing the growth of the preys ( x ) without predators ( y ), and the extinction of predators without preys, the non linear terms expressing the encounters between both species:

Original: [Equation]
Revised: [Equation]

Original: We observe the system state Z =
Revised: We observe the system state Z =

Original: ( x, y ) and set as prior knowledge: h k ( x, y )
Revised: ( x, y ) and set as prior knowledge: h k ( x, y )

Original: = ( θ 1 k x, − θ 2 k y ) .
Revised: = ( θ 1 k x, − θ 2 k y ) .

Original: θ (cid:63) =
Revised: θ (cid:63) =

Original: ( α, γ ) has to be estimated by θ
Revised: ( α, γ ) has to be estimated by θ

Original: k
Revised: k

Original: =
Revised: =

Original: θ 1 k , θ 2 k ) .
Revised: θ 1 k , θ 2 k ) .

Original: h u accounts for the non linear terms ( βxy, δxy ) .
Revised: h u accounts for the non linear terms ( βxy, δxy ) .

Original: Experimental Setting For both DPL and LV experiments, we consider the following setting: we sample the space of initial conditions building 100/50/50 trajectories for the train, validation and test sets.
Revised: Experimental Setting For both DPL and LV experiments, we consider the following setting: we sample the space of initial conditions building 100/50/50 trajectories for the train, validation and test sets.

Original: The sequences share the same parameters; respectively ( gL , k ) , for DPL, and ( α, β, γ, δ ) for
Revised: The sequences share the same parameters; respectively ( gL , k ) , for DPL, and ( α, β, γ, δ ) for

Original: LV.
Revised: LV.

Original: The parameter θ k is set to a neuron (of dimension 1 in the pendulum and 2 for LV) and h u is a 2-layer MLP.
Revised: The parameter θ k is set to a neuron (of dimension 1 in the pendulum and 2 for LV) and h u is a 2-layer MLP.

Original: Further experimental details are available in appendices E.1, E.2 and F.[Table] [Table] [Table]
Revised: Further experimental details are available in appendices E.1, E.2 and F.

Original: Only d( h, f )

Original: [Table]

Original: [Table]

Original: u

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: Identiﬁcation and Prediction Results Tableshows that despite accurate trajectory forecasting, the unconstrained setting «Only d( h, f ) » fails at estimating the models parameters, showing the need for regularization for identiﬁcation.
Revised: shows that despite accurate trajectory forecasting, the unconstrained setting «Only d( h, f ) » fails at estimating the models parameters, showing the need for regularization for identiﬁcation.

Original: Constraining the norm of the
Revised: Constraining the norm of the

Original: ML component can be insufﬁcient: for LV data, both
Revised: ML component can be insufﬁcient: for LV data, both

Original: Aphynity and d( h, f ) + d( h u , 0) do not accurately estimate the model parameters.
Revised: Aphynity and d( h, f ) + d( h u , 0) do not accurately estimate the model parameters.

Original: However, the control of d( h k , f ) , following eq.
Revised: However, the control of d( h k , f ) , following eq.

Original: (4), signiﬁcantly improves the parameter identiﬁcation for both datasets.
Revised: (4), signiﬁcantly improves the parameter identiﬁcation for both datasets.

Original: Indeed, in the PDL case, h k and f are (pseudo)-periodic of the same period, hence the gain in the performances.
Revised: Indeed, in the PDL case, h k and f are (pseudo)-periodic of the same period, hence the gain in the performances.

Original: Finally, our proposition based on eq.
Revised: Finally, our proposition based on eq.

Original: (4) is able to identify the parameters of DPL and LV equation with a precision of respectively 1 . 56% and 7 . 8%
Revised: (4) is able to identify the parameters of DPL and LV equation with a precision of respectively 1 . 56% and 7 . 8%

Original: beating all considered baselines.
Revised: beating all considered baselines.

Original: Regarding prediction performances, in under-constrained settings ( «Only d( h, f ) » in Table 1), h u learns to corrects the inaccurate h k .
Revised: Regarding prediction performances, in under-constrained settings ( «Only d( h, f ) » in Table 1), h u learns to corrects the inaccurate h k .

Original: Table 1 and ﬁgs.
Revised: Table 1 and ﬁgs.

Original: 4 and 5 (appendix G.1) show that our proposition provides more consistent prediction performances.
Revised: 4 and 5 (appendix G.1) show that our proposition provides more consistent prediction performances.

Original: These experiments conﬁrm that the constraints on h k and h u arising from the control of the upper bound of eq.
Revised: These experiments conﬁrm that the constraints on h k and h u arising from the control of the upper bound of eq.

Original: (4) increase interpretability and maintain prediction performances.
Revised: (4) increase interpretability and maintain prediction performances.

Original: Throwback to the Afﬁne Case We verify the convergence proved in section 3.3 using the damped pendulum (eq.
Revised: Throwback to the Afﬁne Case We verify the convergence proved in section 3.3 using the damped pendulum (eq.

Original: (15)) linearized in the small oscillations regime (see appendix E.1).
Revised: (15)) linearized in the small oscillations regime (see appendix E.1).

Original: Making an afﬁne hypothesis following eq.
Revised: Making an afﬁne hypothesis following eq.

Original: (12), we apply our alternate projection algorithm and optimize A and D
Revised: (12), we apply our alternate projection algorithm and optimize A and D

Original: A alternately using SGD.
Revised: A alternately using SGD.

Original: Figure 1 shows that we are able to accurately estimate A and D using our proposition, recovering both the oscillation pulsation and the damping coefﬁcient.
Revised: Figure 1 shows that we are able to accurately estimate A and D using our proposition, recovering both the oscillation pulsation and the damping coefﬁcient.

Original: 4. H IGH D IMENSIONAL D YNAMICS
Revised: 4. H IGH D IMENSIONAL D YNAMICS

Original: We now address the learning of transport equations, describing a wide range of physical phenomena such as chemical concentration, ﬂuid dynamics or material properties.
Revised: We now address the learning of transport equations, describing a wide range of physical phenomena such as chemical concentration, ﬂuid dynamics or material properties.

Original: We evaluate the learning setting induced by eq.
Revised: We evaluate the learning setting induced by eq.

Original: (4) and (5) on two physical datasets depicting the evolution of the temperature
Revised: (4) and (5) on two physical datasets depicting the evolution of the temperature

Original: T advected by a time-dependent velocity ﬁeld U and subject to forcing S , following:
Revised: T advected by a time-dependent velocity ﬁeld U and subject to forcing S , following:

Original: [Equation]
Revised: [Equation]

Original: The system state Z =
Revised: The system state Z =

Original: ( T, U, S ) is partially observed, we only access T .
Revised: ( T, U, S ) is partially observed, we only access T .

Original: Every quantities, observed or to estimate, are regularly sampled on a spatiotemporal grid: at each timestep t , the time varying velocity ﬁeld U t writes as U t =
Revised: Every quantities, observed or to estimate, are regularly sampled on a spatiotemporal grid: at each timestep t , the time varying velocity ﬁeld U t writes as U t =

Original: ( u t , v t ) and u t , v t , T t and the forcing term S t are all of size 64 × 64 .
Revised: ( u t , v t ) and u t , v t , T t and the forcing term S t are all of size 64 × 64 .

Original: Experimental Setting We consider as physical prior the advection i.e h k ( T, θ k ) = −∇ .
Revised: Experimental Setting We consider as physical prior the advection i.e h k ( T, θ k ) = −∇ .

Original: ( Tθ k ) .
Revised: ( Tθ k ) .

Original: Thus, θ k is time-dependent, as we learn it to approximate θ (cid:63)
Revised: Thus, θ k is time-dependent, as we learn it to approximate θ (cid:63)

Original: =
Revised: =

Original: U .
Revised: U .

Original: We identify the velocity ﬁeld θ k from observations of T , learning a mapping between T and U parameterized by a neural network
Revised: We identify the velocity ﬁeld θ k from observations of T , learning a mapping between T and U parameterized by a neural network

Original: G ψ , so that θ k = G ψ ( T t − l , ..., T t ) ≈ U t , which is common practice in oceanography (Béréziat &
Revised: G ψ , so that θ k = G ψ ( T t − l , ..., T t ) ≈ U t , which is common practice in oceanography (Béréziat &

Original: Herlin, 2015).
Revised: Herlin, 2015).

Original: G ψ is optimized following eq.
Revised: G ψ is optimized following eq.

Original: S remains to be learned by h u .
Revised: S remains to be learned by h u .

Original: h k implements  a differentiable semi-Lagrangian scheme (Jaderberg et al., 2015) (see appendix E.3) and h u is a
Revised: h k implements [Table] a differentiable semi-Lagrangian scheme (Jaderberg et al., 2015) (see appendix E.3) and h u is a

Original: ResNet.
Revised: ResNet.

Original: G ψ is a UNet.
Revised: G ψ is a UNet.

Original: Training details and a schema of our model are to be found in appendix F.
Revised: Training details and a schema of our model are to be found in appendix F

Original: [Table]
Revised: [Table]

Original: Synthetic Advection and Source (Adv+S)
Revised: Synthetic Advection and Source (Adv+S)

Original: To test the applicability of the learning setting induced by eq.
Revised: To test the applicability of the learning setting induced by eq.

Original: (4) on partially observed settings, we ﬁrst study a synthetic setting (denoted Adv+S) of eq.
Revised: (4) on partially observed settings, we ﬁrst study a synthetic setting (denoted Adv+S) of eq.

Original: (17) by generating velocity ﬁelds U , simulated following (Boffetta et al., 2001) and adding a source term S inspired by (Frankignoul, 1985).
Revised: (17) by generating velocity ﬁelds U , simulated following (Boffetta et al., 2001) and adding a source term S inspired by (Frankignoul, 1985).

Original: The simulation details are given in appendix E.3.
Revised: The simulation details are given in appendix E.3.

Original: Real Ocean Dynamics (Natl)
Revised: Real Ocean Dynamics (Natl)

Original: We consider acomplex dataset emulating real world observations of the North ATLantic ocean (denoted Natl)
Revised: We consider a dataset emulating real world observations of the North ATLantic ocean (denoted Natl)

Original: (Ajayi et al., 2019).
Revised: (Ajayi et al., 2019).

Original: Modeling the evolution of T in Natl isparticularly challenging as its dynamics is chaotic and highly non-linear.
Revised: Modeling the evolution of T in Natl is challenging as its dynamics is chaotic and highly non-linear.

Original: This simulation is representative of the complexity encountered in real world data.
Revised: This simulation is representative of the complexity encountered in real world data.

Original: The principled approach corresponding to eq.
Revised: The principled approach of eq.

Original: (4) is insufﬁcient here and one must resort to additional physical information.
Revised: (4) is insufﬁcient here and one must resort to additional physical information.

Original: To illustratethe extension developed in section 4.2, we make use ofavailable auxiliary data:measurements from satellite observations provide a coarse estimate ofthe surfaceocean currents velocity ﬁelds (see appendix E).
Revised: We illustrate section 4.2 and make use of auxiliary data: satellite observations provide a coarse estimate of surface velocity ﬁelds (appendix E).

Original: Then, the learning goal is to reﬁne the approximated velocity ﬁelds to ﬁt the ocean dynamics.
Revised: The goal is to reﬁne the approximated velocity ﬁelds to ﬁt the ocean dynamics.

Original: Therefore we proceed as described in eq.
Revised: We proceed as described in eq.

Original: (5) and enforce d( h k , f prk )by supervising G ψ with the proxy data (see appendix E.3).
Revised: (5) and enforce d( h k , f prk ) supervising G ψ with the proxy data (appendix E.3).

Original: Identiﬁcation and Prediction Results Table 2 indicates that for Adv+S dataset, we estimate accurately the unobserved velocity ﬁelds.
Revised: Identiﬁcation and Prediction Results Table 2 indicates that for Adv+S dataset, we estimate accurately the unobserved velocity ﬁelds.

Original: Qualitatively, Figure 2 shows that controlling our proposed upper bound eq.
Revised: Qualitatively, Figure 2 shows that controlling our proposed upper bound eq.

Original: (4) facilitates the recovery of truthful velocity ﬁelds U along with an accurate prediction of T .
Revised: (4) facilitates the recovery of truthful velocity ﬁelds U along with an accurate prediction of T .

Original: For the highly complex Natl, Table 2 shows that the introduction of auxiliary data following the formulation in eq.
Revised: For the highly complex Natl, Table 2 shows that the introduction of auxiliary data following the formulation in eq.

Original: (5) signiﬁcantly helps identiﬁcation, as the dynamics is too complex to be able to recover physically interpretable velocity ﬁelds using the bound of eq.
Revised: (5) signiﬁcantly helps identiﬁcation, as the dynamics is too complex to be able to recover physically interpretable velocity ﬁelds using the bound of eq.

Original: Regarding prediction performances on the Adv+S data, Table 2 shows that thanks to our truthful estimates of U , our model provides more precise prediction than NODE and Aphynity.
Revised: Regarding prediction performances on the Adv+S data, Table 2 shows that thanks to our truthful estimates of U , our model provides more precise prediction than NODE and Aphynity.

Original: For real world data, thanks to the proxy data our model recovers better velocity ﬁelds terms while providing a better estimate for T .
Revised: For real world data, thanks to the proxy data our model recovers better velocity ﬁelds terms while providing a better estimate for T .

Original: Besides, adding prior knowledge in the prediction systems improves prediction performances: appendix G shows that NODE minimizes d( h, f ) by predicting average and blurred frames.
Revised: Besides, adding prior knowledge in the prediction systems improves prediction performances: appendix G shows that NODE minimizes d( h, f ) by predicting average and blurred frames.

Original: This shows the need for regularization when learning on structured physical data.
Revised: This shows the need for regularization when learning on structured physical data.

Original: Ablation Study We present in Table 3 an ablation study on the Adv+S dataset evidencing the inﬂuence of our learning choices on the resolution of both identiﬁcation and prediction tasks (see appendix G for detailed results).
Revised: Ablation Study We present in Table 3 an ablation study on the Adv+S dataset evidencing the inﬂuence of our learning choices on the resolution of both identiﬁcation and prediction tasks (see appendix G for detailed results).

Original: “Joint” rows of Table 3 indicate that the learning of h u and h k is done simultaneously.
Revised: “Joint” rows of Table 3 indicate that the learning of h u and h k is done simultaneously.

Original: As shown in Table 3, the sole optimization of d( h, f ) fails at estimating physically sounded U .
Revised: Table 3 shows that the sole optimization of d( h, f ) fails at estimating physically sounded U .

Original: This evidences the ill-posedness in such unconstrained optimization.
Revised: This evidences the ill-posedness in such unconstrained optimization.

Original: Tableindicates that all introduced regularizations improve the recovery of U w.r.t.
Revised: Tableindicates that all introduced regularizations improve the recovery of U w.r.t.

Original: the «Only d( h, f ) » baseline, while adding d( h u , 0) signiﬁcantly improves both prediction performances and velocity ﬁelds estimation.
Revised: the «Only d( h, f ) » baseline, while adding d( h u , 0) signiﬁcantly improves both prediction performances and velocity ﬁelds estimation.

Original: We highlight that the alternate optimization performs better compared to optimiz- ing jointly all parameters of h k and h u .
Revised: We highlight that the alternate optimization performs better compared to optimizing jointly all parameters of h k and h u .

Original: Notably, our proposition to optimize h k and h u alternately beats all baselines on both T prediction and U identiﬁcation (Table 3, Joint rows).
Revised: Notably, our proposition to optimize h k and h u alternately beats all baselines on both T prediction and U identiﬁcation (Table 3, Joint rows).

Original: Finally, jointly trained models fail at estimating U in Table 3, forcing h u to capture the whole dynamics.
Revised: Finally, jointly trained models fail at estimating U in Table 3, forcing h u to capture the whole dynamics.

Original: [Table]
Revised: [Table]

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Grey-box or hybrid modeling, combining ODE/PDE and data based models, has received an increasing focus in the machine learning community (Rico-Martinez et al., 1994; Thompson & Kramer, 1994; Raissi et al., 2020b).
Revised: Grey-box or hybrid modeling, combining ODE/PDE and data based models, has received an increasing focus in the machine learning community (Rico-Martinez et al., 1994; Thompson & Kramer, 1994; Raissi et al., 2020b).

Original: Hybrid approaches allow for alleviated computational costs for ﬂuid simulation (Tompson et al., 2017; De Avila Belbute-Peres et al., 2020;
Revised: Hybrid approaches allow for alleviated computational costs for ﬂuid simulation (Tompson et al., 2017; De Avila Belbute-Peres et al., 2020;

Original: Wandel et al., 2021), and show better prediction performances through data speciﬁc constraints that preserve physics (Raissi et al., 2020a;
Revised: Wandel et al., 2021), and show better prediction performances through data speciﬁc constraints that preserve physics (Raissi et al., 2020a;

Original: Jia et al., 2019).
Revised: Jia et al., 2019).

Original: They offer increased interpretability via constraints on convolutional ﬁlters (Long et al., 2018; 2019) or on learned residual (Geneva & Zabaras, 2020).
Revised: They offer increased interpretability via constraints on convolutional ﬁlters (Long et al., 2018; 2019) or on learned residual (Geneva & Zabaras, 2020).

Original: Physical knowledge, introduced through ODE/PDE regularization (Psichogios & Ungar, 1992; Bongard & Lipson, 2007; de Bézenac et al., 2018) or Hamiltonian priors (Greydanus et al., 2019; Lee et al., 2021), increases generalization power w.r.t pure ML approaches.
Revised: Physical knowledge, introduced through ODE/PDE regularization (Psichogios & Ungar, 1992; Bongard & Lipson, 2007; de Bézenac et al., 2018) or Hamiltonian priors (Greydanus et al., 2019; Lee et al., 2021), increases generalization power w.r.t pure ML approaches.

Original: Closer to our work, (Mehta et al., 2020;
Revised: Closer to our work, (Mehta et al., 2020;

Original: San & Maulik, 2018;
Revised: San & Maulik, 2018;

Original: Young et al., 2017; Saha et al., 2020) study the learning of a physical model augmented with a statistical component.
Revised: Young et al., 2017; Saha et al., 2020) study the learning of a physical model augmented with a statistical component.

Original: Yin et al.
Revised: Yin et al.

Original: (2021) tackle the same task, ensuring the uniqueness in the decomposition by constraining the norm of the ML component.
Revised: (2021) tackle the same task, ensuring the uniqueness in the decomposition by constraining the norm of the ML component.

Original: We generalize latter approaches and address the well-posedness in the learning of hybrid ML/MB models through additional regularization on the estimated parameters of the physical part.
Revised: We generalize latter approaches and address the well-posedness in the learning of hybrid ML/MB models through additional regularization on the estimated parameters of the physical part.

Original: Indeed, to describe natural phenomena relying on hybrid MB/ML models, one major task lies in the estimation of the MB part parameters.
Revised: Indeed, to describe natural phenomena relying on hybrid MB/ML models, one major task lies in the estimation of the MB part parameters.

Original: This can be done using neural networks (Raissi et al., 2019; Mehta et al., 2020).
Revised: This can be done using neural networks (Raissi et al., 2019; Mehta et al., 2020).

Original: However, identiﬁcation tasks being intrinsically ill-posed (Sabatier, 2000), imposing prior knowledge or regularization is necessary to ensure sound estimations (Stewart & Ermon, 2017).
Revised: However, identiﬁcation tasks being intrinsically ill-posed (Sabatier, 2000), imposing prior knowledge or regularization is necessary to ensure sound estimations (Stewart & Ermon, 2017).

Original: Yet, using only prediction as supervision, the recovered parameters are not physically interpretable (de Bézenac et al., 2018;
Revised: Yet, using only prediction as supervision, the recovered parameters are not physically interpretable (de Bézenac et al., 2018;

Original: Ayed et al., 2020).
Revised: Ayed et al., 2020).

Original: To ensure uniqueness of the estimation solution, Ardizzone et al.
Revised: To ensure uniqueness of the estimation solution, Ardizzone et al.

Original: (2018) use invertible neural networks.
Revised: (2018) use invertible neural networks.

Original: Linial et al.
Revised: Linial et al.

Original: ; Tait & Damoulas (2020); Saemundsson et al.
Revised: ; Tait & Damoulas (2020); Saemundsson et al.

Original: (2020) combine variational encoding (Kingma & Welling, 2013) and a PDE model, sampling the space of initial conditions and parameters to solve both identiﬁcation and prediction.
Revised: (2020) combine variational encoding (Kingma & Welling, 2013) and a PDE model, sampling the space of initial conditions and parameters to solve both identiﬁcation and prediction.

Original: However, such methods only deal with low-dimensional dynamics.
Revised: However, such methods only deal with low-dimensional dynamics.

Original: Besides low dimensional systems, we also show the soundness of our approach on complex high dimensional and partially observed dynamics.
Revised: Besides low dimensional systems, we also show the soundness of our approach on complex high dimensional and partially observed dynamics.

Original:  D ISCUSSION
Revised:  D ISCUSSION

Original: We propose in this work an algorithm to learn hybrid MB/ML models.
Revised: We propose in this work an algorithm to learn hybrid MB/ML models.

Original: For interpretability purposes, we impose constraints ﬂowing from an upper bound of the prediction error and derive a learning algorithm in a general setting.
Revised: For interpretability purposes, we impose constraints ﬂowing from an upper bound of the prediction error and derive a learning algorithm in a general setting.

Original: We prove its well posedness and its convergence in a linear approximation setting.
Revised: We prove its well posedness and its convergence in a linear approximation setting.

Original: Besides theoretical considerations, weempirically evidence the soundness of our approach thanks to ablation studies and comparison with recent baselines on several low and high dimensional datasets.
Revised: Empirically, we evidence the soundness of our approach thanks to ablation studies and comparison with recent baselines on several low and high dimensional datasets.

Original: This work can see several extensions, e.g. considering non uniform  3-D grid for climate models, further considerations on the investigated upper bounds, or less restrictive decomposition hypothesis. 
Revised: This work can see several extensions: considering non uniform 2 or 3-D grid for climate models, further considerations on the investigated upper bounds, or different decomposition hypothesis. A CKNOWLEDGEMENTS

Revised: We would like to thank all members of the MLIA team from the ISIR laboratory of Sorbonne Université for helpful discussions and comments.

Revised: We acknowledge ﬁnancial support from ANR AI

Revised: Chairs program via the DL4CLIM ANR-19-CHIA- 0018-01 project, the LOCUST ANR project (ANR-15-CE23-0027) and the European Union’s Horizon 2020 research and innovation program under grant agreement 825619 (AI4EU).

Revised: The Natl60 data were provided by MEOM research team, from the IGE laboratory from the Université Grenoble Alpes

Original: A D ISTANCE A. D ISTANCE B ETWEEN D YNAMICS
Revised: A D ISTANCE A. D ISTANCE B ETWEEN D YNAMICS

Original: We here give the deﬁnition of the distance d .
Revised: We here give the deﬁnition of the distance d .

Original: Let u and v be two functions of L 2 ( R p , R p ) .
Revised: Let u and v be two functions of L 2 ( R p , R p ) .

Original: We consider the distance:
Revised: We consider the distance:

Original: [Equation]
Revised: [Equation]

Original: Naturally, eq.
Revised: Naturally, eq.

Original: (18) veriﬁes the triangle inequality, the symmetry and the positiveness.
Revised: (18) veriﬁes the triangle inequality, the symmetry and the positiveness.

Original: Moreover, in this case, for all functions f , d( ., f ) is convex.
Revised: Moreover, in this case, for all functions f , d( ., f ) is convex.

Original: Indeed, for u , v two functions, and λ ∈ [0 , 1] :
Revised: Indeed, for u , v two functions, and λ ∈ [0 , 1] :

Original: [Equation]
Revised: [Equation]

Original: Hence the convexity of d( ., f ) .
Revised: Hence the convexity of d( ., f ) .

Original: This consideration sufﬁces to ensure the convexity of S k and S u deﬁned in section 3.
Revised: This consideration sufﬁces to ensure the convexity of S k and S u deﬁned in section 3.

Original: A. D ISTANCE B ETWEEN F LOWS
Revised: A. D ISTANCE B ETWEEN F LOWS

Original: Consider the ODE with X ( t ) , X 0 ∈ R p :
Revised: Consider the ODE with X ( t ) , X 0 ∈ R p :

Original: [Equation]
Revised: [Equation]

Original: Equation (19) admits a unique solution as soon as f is Lipschitz.
Revised: Equation (19) admits a unique solution as soon as f is Lipschitz.

Original: We note X (cid:63) this solution.
Revised: We note X (cid:63) this solution.

Original: Then, we can deﬁned the ﬂow φ f of such ODE as :
Revised: Then, we can deﬁned the ﬂow φ f of such ODE as :

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: With the deﬁnition of eq.
Revised: With the deﬁnition of eq.

Original: [Equation]
Revised: [Equation]

Original: we can deﬁne the distance between two ﬂows of ODE as:
Revised: we can deﬁne the distance between two ﬂows of ODE as:

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: d φ is positive and symmetric.
Revised: d φ is positive and symmetric.

Original: Let φ u , φ v be two ﬂows
Revised: Let φ u , φ v be two ﬂows

Original: [Equation]
Revised: [Equation]

Original: we have the triangle inequality:
Revised: we have the triangle inequality:

Original: [Equation]
Revised: [Equation]

Original: Let φ f be ﬁxed
Revised: Let φ f be ﬁxed

Original: [Equation]
Revised: [Equation]

Original: we also have the convexity of d φ ( ., φ f ) with respect to the ﬁrst argument.
Revised: we also have the convexity of d φ ( ., φ f ) with respect to the ﬁrst argument.

Original: Indeed for λ ∈ [0 , 1] :
Revised: Indeed for λ ∈ [0 , 1] :

Original: [Equation]
Revised: [Equation]

Original: However, in this case the convexity is not ensured with respect to u and v This is the reason why for theoretical investigations, we consider the distance d instead of d φ .
Revised: However, in this case the convexity is not ensured with respect to u and v This is the reason why for theoretical investigations, we consider the distance d instead of d φ .

Original: Nonetheless, d φ ( φ u , φ f ) = 

Revised: Nonetheless, d φ ( φ u , φ f ) =

Original: = ⇒
Revised: = ⇒

Original: φ u = φ f = ⇒ u
Revised: φ u = φ f = ⇒ u

Original: = f .
Revised: = f .

Original: B R EMARK ON A DDITIVE D ECOMPOSITION
Revised: B R EMARK ON A DDITIVE D ECOMPOSITION

Original: First, note that in the case of a metric space the decomposition as deﬁned in eq.
Revised: First, note that in the case of a metric space the decomposition as deﬁned in eq.

Original: (2) always exists.
Revised: (2) always exists.

Original: We now detail an intuition for the well-posedness of such decomposition.
Revised: We now detail an intuition for the well-posedness of such decomposition.

Original: Let H k be a closed convex subset of functions of an Hilbert space ( E, <, > ) , and f the function we want to approximate with partial knowledge (represented by the space of hypothesis H k ).
Revised: Let H k be a closed convex subset of functions of an Hilbert space ( E, <, > ) , and f the function we want to approximate with partial knowledge (represented by the space of hypothesis H k ).

Original: Then, thanks to Hilbert projection lemma, we have the uniqueness of the minimizer of min g ∈H k (cid:107) f − g
Revised: Then, thanks to Hilbert projection lemma, we have the uniqueness of the minimizer of min g ∈H k (cid:107) f − g

Original: (cid:107) ,
Revised: (cid:107) ,

Original: i.e it exists one unique h k ∈
Revised: i.e it exists one unique h k ∈

Original: H
Revised: H

Original: k
Revised: k

Original: such that: ∀ g ∈ H
Revised: such that: ∀ g ∈ H

Original: k , (cid:107) f − h k (cid:107) ≤ (cid:107)
Revised: k , (cid:107) f − h k (cid:107) ≤ (cid:107)

Original: f − g
Revised: f − g

Original: (cid:107) .
Revised: (cid:107) .

Original: Finally, the additive decomposition hypothesis presents a remarkable advantage in the case of a vector space.
Revised: Finally, the additive decomposition hypothesis presents a remarkable advantage in the case of a vector space.

Original: Indeed, if H k is a (closed) vector space, let H ⊥ k be its supplementary in E , then we have the uniqueness in the decomposition: f = f H k + f H ⊥ k , where f H ⊥ k ∈
Revised: Indeed, if H k is a (closed) vector space, let H ⊥ k be its supplementary in E , then we have the uniqueness in the decomposition: f = f H k + f H ⊥ k , where f H ⊥ k ∈

Original: H ⊥ k and f H k ∈ H k .
Revised: H ⊥ k and f H k ∈ H k .

Original: The existence and uniqueness ﬂowing directly from the additive decomposition hypothesis, this can explain why such assumption is common when bridging ML and MB hypothesis.
Revised: The existence and uniqueness ﬂowing directly from the additive decomposition hypothesis, this can explain why such assumption is common when bridging ML and MB hypothesis.

Original: C U PPER B OUNDS C. D ERIVATION OF E QUATION (4)
Revised: C U PPER B OUNDS C. D ERIVATION OF E QUATION (4)

Original: The ﬁrst upper bound is a simple use of the triangle inequality:
Revised: The ﬁrst upper bound is a simple use of the triangle inequality:

Original: [Equation]
Revised: [Equation]

Original: C. D ERIVATION OF E QUATION (5)
Revised: C. D ERIVATION OF E QUATION (5)

Original: To derive the second upper bound, we assume that f prk comes from an overall dynamics f pr obeying the additive decomposition hypothesis of eq.
Revised: To derive the second upper bound, we assume that f prk comes from an overall dynamics f pr obeying the additive decomposition hypothesis of eq.

Original: (2) so that f pr and f prk veriﬁes: f pr = f prk + f pru .
Revised: (2) so that f pr and f prk veriﬁes: f pr = f prk + f pru .

Original: First, with computations similar to eq.
Revised: First, with computations similar to eq.

Original: (4), we have:
Revised: (4), we have:

Original: [Equation]
Revised: [Equation]

Original: Then:
Revised: Then:

Original: [Equation]
Revised: [Equation]

Original: Combining Equations (22) and (23), we retrieve eq.
Revised: Combining Equations (22) and (23), we retrieve eq.

Original: [Equation]
Revised: [Equation]

Original: and we have: Γ =
Revised: and we have: Γ =

Original: d( f pr , f prk ) + d( f pr , f ) .
Revised: d( f pr , f prk ) + d( f pr , f ) .

Original: Γ is a constant of the problem that cannot be optimized.
Revised: Γ is a constant of the problem that cannot be optimized.

Original: C. U PPER -B OUND U SING A UXILIARY D YNAMICS f pr
Revised: C. U PPER -B OUND U SING A UXILIARY D YNAMICS f pr

Original: Let f pr be the dynamics of model data, we can link up the error made by h on true data (following dynamics f ) and the error made by h on model data (with dynamics f pr ) via: 
Revised: Let f pr be the dynamics of model data, we can link up the error made by h on true data (following dynamics f ) and the error made by h on model data (with dynamics f pr ) via: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Thus a pre-training on auxiliary data of dynamics f pr amounts to control the term d( h, f pr ) in the upper-bound of eq.
Revised: Thus a pre-training on auxiliary data of dynamics f pr amounts to control the term d( h, f pr ) in the upper-bound of eq.

Original: C. S ELF -S UPERVISION
Revised: C. S ELF -S UPERVISION

Original: Let h
Revised: Let h

Original: = h k
Revised: = h k

Original: + h u be the function to learn and G ψ the recognition network providing an estimate ˆ θ ik of the parameters from an initial sequence ( X it 0 , . . .
Revised: + h u be the function to learn and G ψ the recognition network providing an estimate ˆ θ ik of the parameters from an initial sequence ( X it 0 , . . .

Original: , X it 0 + k ∆ t ) .
Revised: , X it 0 + k ∆ t ) .

Original: This learning setting corresponds to how velocity ﬁelds are learned from consecutive measurements of the tracer ﬁelds T in section 4.2.
Revised: This learning setting corresponds to how velocity ﬁelds are learned from consecutive measurements of the tracer ﬁelds T in section 4.2.

Original: To compute d( h k , f prk ) in the case where f pr = h (cid:63) , where h (cid:63)
Revised: To compute d( h k , f prk ) in the case where f pr = h (cid:63) , where h (cid:63)

Original: =
Revised: =

Original: h (cid:63)k + h
Revised: h (cid:63)k + h

Original: (cid:63)u is a learned model, we rely on the computed θ k associated to h (cid:63)k (thanks for example to the algorithm of section 3.2 associated to eq.
Revised: (cid:63)u is a learned model, we rely on the computed θ k associated to h (cid:63)k (thanks for example to the algorithm of section 3.2 associated to eq.

Original: (4)) to generate a synthetic dataset with achievable supervision in the space of the parameters θ k .
Revised: (4)) to generate a synthetic dataset with achievable supervision in the space of the parameters θ k .

Original: From a real initial sequence ( X it 0 , . . .
Revised: From a real initial sequence ( X it 0 , . . .

Original: , X it 0 + k ∆ t ) , we can estimate the unknown parameter θ ik associated to sequence i with the recognition network G (cid:63)ψ learned with h (cid:63) , i.e θ ik = G (cid:63)ψ (cid:0) X it 0 , . . .
Revised: , X it 0 + k ∆ t ) , we can estimate the unknown parameter θ ik associated to sequence i with the recognition network G (cid:63)ψ learned with h (cid:63) , i.e θ ik = G (cid:63)ψ (cid:0) X it 0 , . . .

Original: , X it 

Revised: , X it

Original: + k ∆ t (cid:1) .
Revised: + k ∆ t (cid:1) .

Original: Then, integrating from the initial condition X it 0 , we generate a trajectory of known parameters θ ik with dynamics h (cid:63) denoted by: ˜ X i = (cid:0) ˜
Revised: Then, integrating from the initial condition X it 0 , we generate a trajectory of known parameters θ ik with dynamics h (cid:63) denoted by: ˜ X i = (cid:0) ˜

Original: X it 0 , . . . ,
Revised: X it 0 , . . . ,

Original: ˜ X it n (cid:1) .
Revised: ˜ X it n (cid:1) .

Original: Sampling the space of initial conditions, we obtain a synthetic dataset: (cid:0) ( ˜ X 1 , θ 1 k ) , . . .
Revised: Sampling the space of initial conditions, we obtain a synthetic dataset: (cid:0) ( ˜ X 1 , θ 1 k ) , . . .

Original: , ( ˜ X m , θ mk ) (cid:1) enabling us to perform self-supervision for G ψ .
Revised: , ( ˜ X m , θ mk ) (cid:1) enabling us to perform self-supervision for G ψ .

Original: Let ˆ θ ik be the parameters estimated by G ψ from the simulated (cid:0) ˜
Revised: Let ˆ θ ik be the parameters estimated by G ψ from the simulated (cid:0) ˜

Original: X i t , . . . ,
Revised: X i t , . . . ,

Original: ˜ X it + k ∆ t (cid:1) , we make the following approximation:
Revised: ˜ X it + k ∆ t (cid:1) , we make the following approximation:

Original: [Equation]
Revised: [Equation]

Original: D P ROOFS D. N OTE ON THE C ONVEXITY OF S k AND S u Convexity of S k
Revised: D P ROOFS D. N OTE ON THE C ONVEXITY OF S k AND S u Convexity of S k

Original: Proof.
Revised: Proof.

Original: Let u, v ∈ S k :
Revised: Let u, v ∈ S k :

Original: [Equation]
Revised: [Equation]

Original: Hence the convexity of S k .
Revised: Hence the convexity of S k .

Original: Convexity of S u
Revised: Convexity of S u

Original: Proof.
Revised: Proof.

Original: Let t ∈ [0 , 1] and u, v ∈ S u .
Revised: Let t ∈ [0 , 1] and u, v ∈ S u .

Original: [Equation]
Revised: [Equation]

Original: Hence the convexity of S u .
Revised: Hence the convexity of S u .

Original: D. ODE I DENTIFICATION
Revised: D. ODE I DENTIFICATION

Original: Consider the following set: S A
Revised: Consider the following set: S A

Original: = { X ( t ) ∈
Revised: = { X ( t ) ∈

Original: C 1 ([0 , T ] , R p ) such that: ∃ A ∈ M p,p ( R ) , X (cid:48)
Revised: C 1 ([0 , T ] , R p ) such that: ∃ A ∈ M p,p ( R ) , X (cid:48)

Original: = AX } , where T > 0 .
Revised: = AX } , where T > 0 .

Original: S A is not a convex set.
Revised: S A is not a convex set.

Original: Consider u and v in S A , and consider A u and A v so that u (cid:48) ( t ) =
Revised: Consider u and v in S A , and consider A u and A v so that u (cid:48) ( t ) =

Original: A u u ( t ) and v (cid:48) ( t ) =
Revised: A u u ( t ) and v (cid:48) ( t ) =

Original: A v v ( t ) .
Revised: A v v ( t ) .

Original: For λ ∈ ]0 , 1[ : we have:
Revised: For λ ∈ ]0 , 1[ : we have:

Original: [Equation]
Revised: [Equation]

Original: In general the last term is not equal to A λu +(1 − λ ) v ( λu + (1 − λ ) v ) , for some matrix
Revised: In general the last term is not equal to A λu +(1 − λ ) v ( λu + (1 − λ ) v ) , for some matrix

Original: A λu +(1 − λ ) v .
Revised: A λu +(1 − λ ) v .

Original: Thus S A is not a convex set.
Revised: Thus S A is not a convex set.

Original: However, discretizing the trajectories and employing a simple integration scheme leads to considering the following cost function:
Revised: However, discretizing the trajectories and employing a simple integration scheme leads to considering the following cost function:

Original: [Equation]
Revised: [Equation]

Original: As a least square regression problem, L ( A ) is convex with respect to A .
Revised: As a least square regression problem, L ( A ) is convex with respect to A .

Original: A least square regression setting can also be recovered using more complex integration schemes, or several time steps integration.
Revised: A least square regression setting can also be recovered using more complex integration schemes, or several time steps integration.

Original: D. P ROOF FOR W ELL - POSEDNESS OF E QUATION (7)
Revised: D. P ROOF FOR W ELL - POSEDNESS OF E QUATION (7)

Original: We set ourselves in the Hilbert space of squared integrable functions with the canonical scalar product (cid:0) L 2 ( R p , R p ) , <, > (cid:1) .
Revised: We set ourselves in the Hilbert space of squared integrable functions with the canonical scalar product (cid:0) L 2 ( R p , R p ) , <, > (cid:1) .

Original: For further consideration on such functional space we refer to (Droniou, 2001).
Revised: For further consideration on such functional space we refer to (Droniou, 2001).

Original: We assume that H k hence S k is convex and a relatively compact family of functions.
Revised: We assume that H k hence S k is convex and a relatively compact family of functions.

Original: Convexity of S k + S u
Revised: Convexity of S k + S u

Original: [Equation]
Revised: [Equation]

Original: Let f, g ∈ S and λ ∈ ]0 , 1[ :
Revised: Let f, g ∈ S and λ ∈ ]0 , 1[ :

Original: [Equation]
Revised: [Equation]

Original: Hence the convexity of S .
Revised: Hence the convexity of S .

Original: C LOSENESS OF S u
Revised: C LOSENESS OF S u

Original: We show that S u is a closed set.
Revised: We show that S u is a closed set.

Original: Indeed, S u = g − 1 ([0 , µ u ]) , where g ( u ) = (cid:107) u (cid:107) , Because g is 1-Lipschitz (using the triangle inequality), g is continuous.
Revised: Indeed, S u = g − 1 ([0 , µ u ]) , where g ( u ) = (cid:107) u (cid:107) , Because g is 1-Lipschitz (using the triangle inequality), g is continuous.

Original: Therefore
Revised: Therefore

Original: S u is closed set as the inverse image of a closed set by continuous function.
Revised: S u is closed set as the inverse image of a closed set by continuous function.

Original: Sequential Limit We now show that S is a closed set thanks to the sequential characterisation: let f n a converging sequence of elements of S and denote f its limit.
Revised: Sequential Limit We now show that S is a closed set thanks to the sequential characterisation: let f n a converging sequence of elements of S and denote f its limit.

Original: We prove that f n converges in S .
Revised: We prove that f n converges in S .

Original: Because ∀ n, f n ∈ S , we have: f n = f nk + f nu , where f nu ∈ S u and f nk ∈ S k .
Revised: Because ∀ n, f n ∈ S , we have: f n = f nk + f nu , where f nu ∈ S u and f nk ∈ S k .

Original: Thanks to the relative compactness of S k , we can extract a converging sub-sequence, of indexes n j , from f nk so that f n j k → f k ∈ S k .
Revised: Thanks to the relative compactness of S k , we can extract a converging sub-sequence, of indexes n j , from f nk so that f n j k → f k ∈ S k .

Original: Because f n → f , the sub-sequence f n j converges: f n j → f .
Revised: Because f n → f , the sub-sequence f n j converges: f n j → f .

Original: By deﬁnition, f n j u is a sequence of S u and we also have that: f n j u = f n j − f n j k .
Revised: By deﬁnition, f n j u is a sequence of S u and we also have that: f n j u = f n j − f n j k .

Original: Because the right member of the equation converges (as a sum of converging functions), the left member of the equation converges i.e. f n j u converges.
Revised: Because the right member of the equation converges (as a sum of converging functions), the left member of the equation converges i.e. f n j u converges.

Original: Since S u is a closed set f n j u converges in S u .
Revised: Since S u is a closed set f n j u converges in S u .

Original: We write f u its limit.
Revised: We write f u its limit.

Original: Therefore, f n j u = f n j − f n j k → f − f k = f u ∈ S u .
Revised: Therefore, f n j u = f n j − f n j k → f − f k = f u ∈ S u .

Original: Hence, f = f u + f k with f u ∈ S u and f k ∈ S k .
Revised: Hence, f = f u + f k with f u ∈ S u and f k ∈ S k .

Original: Therefore S is a closed set.
Revised: Therefore S is a closed set.

Original: Finally, we can apply Hilbert projection lemma on the closed convex set S and retrieve the uniqueness of the minimizer of eq.
Revised: Finally, we can apply Hilbert projection lemma on the closed convex set S and retrieve the uniqueness of the minimizer of eq.

Original: Remark The relative compactness of a family of functions is a common assumption in functional analysis.
Revised: Remark The relative compactness of a family of functions is a common assumption in functional analysis.

Original: For example, in the study of differential equation Cauchy-Peano theorem provides the existence to the solution of an ODE under the assumption of relative compactness.
Revised: For example, in the study of differential equation Cauchy-Peano theorem provides the existence to the solution of an ODE under the assumption of relative compactness.

Original: Also, Ascoli theorem provides the relative compactness of a family of function F under the hypothesis of the equi-continuity of F and the relative compactness of the image space
Revised: Also, Ascoli theorem provides the relative compactness of a family of function F under the hypothesis of the equi-continuity of F and the relative compactness of the image space

Original: A ( x ) =
Revised: A ( x ) =

Original: { f ( x ) | f ∈ F} .
Revised: { f ( x ) | f ∈ F} .

Original: D. P ROOF OF P ROPOSITION
Revised: D. P ROOF OF P ROPOSITION

Original: We now set ourselves in the Hilbert space (cid:0) L 2 ([0 , T ] , R p ) , <, > (cid:1) of squared integrable functions, where <, > is the canonical scalar product of L 2 ([0 , T ] , R p ) .
Revised: We now set ourselves in the Hilbert space (cid:0) L 2 ([0 , T ] , R p ) , <, > (cid:1) of squared integrable functions, where <, > is the canonical scalar product of L 2 ([0 , T ] , R p ) .

Original: Proof.
Revised: Proof.

Original: Let A be a given invertible matrix.
Revised: Let A be a given invertible matrix.

Original: We consider the following space S D
Revised: We consider the following space S D

Original: = { X ∈
Revised: = { X ∈

Original: C 1 ([0 , T ] , R p ) such that: ∃ D ∈ R p , X (cid:48) =
Revised: C 1 ([0 , T ] , R p ) such that: ∃ D ∈ R p , X (cid:48) =

Original: AX + D and X ( t = 0)
Revised: AX + D and X ( t = 0)

Original: =
Revised: =

Original: X 0 } , where T > 0 .
Revised: X 0 } , where T > 0 .

Original: We show that S D is a closed convex set.
Revised: We show that S D is a closed convex set.

Original: Convexity Indeed, let λ ∈ ]0 , 1[ and u, v ∈ S D .
Revised: Convexity Indeed, let λ ∈ ]0 , 1[ and u, v ∈ S D .

Original: λu
Revised: λu

Original: + (1 − λ ) v is differentiable and:
Revised: + (1 − λ ) v is differentiable and:

Original: [ λu + (1 − λ ) v )] (cid:48)
Revised: [Equation]

Original: =
Revised: +

Original: λu (cid:48) + (1 − λ ) v (cid:48) = A ( λu + (1 − λ ) v ) + D,
Revised: [Equation]

Original: Where D = λD u + (1 − λ ) D v .
Revised: Where D = λD u + (1 − λ ) D v .

Original: Hence λu + (1 − λ ) v ∈ S D .
Revised: Hence λu + (1 − λ ) v ∈ S D .

Original: Closeness via Afﬁne-Space To prove the closeness of S D , we prove that it is an afﬁne space of ﬁnite dimension.
Revised: Closeness via Afﬁne-Space To prove the closeness of S D , we prove that it is an afﬁne space of ﬁnite dimension.

Original: Let g the application that to any vector D ∈ R
Revised: Let g the application that to any vector D ∈ R

Original: d
Revised: d

Original: associate the solution X D .
Revised: associate the solution X D .

Original: Let D 0 ∈ R D , we show that g D 0 : D → g ( D 0 + D ) − g
Revised: Let D 0 ∈ R D , we show that g D 0 : D → g ( D 0 + D ) − g

Original: ( D 0 ) is a linear application.
Revised: ( D 0 ) is a linear application.

Original: Naturally, for g D 0 (0 R p ) = 

Revised: Naturally, for g D 0 (0 R p ) =

Original: L 2 .
Revised: L 2 .

Original: Then for D (cid:54) = 

Revised: Then for D (cid:54) =

Original: R p we have:
Revised: R p we have:

Original: [Equation]
Revised: [Equation]

Original: Therefore g D 0 is a linear function and g is an afﬁne function.
Revised: Therefore g D 0 is a linear function and g is an afﬁne function.

Original: Moreover, g is an injection.
Revised: Moreover, g is an injection.

Original: Indeed, if two functions are equals, then they have at most one inverse image by g thanks to Cauchy-Lipschitz theorem.
Revised: Indeed, if two functions are equals, then they have at most one inverse image by g thanks to Cauchy-Lipschitz theorem.

Original: Therefore it deﬁnes a bijection of R d in g ( R d ) .
Revised: Therefore it deﬁnes a bijection of R d in g ( R d ) .

Original: Since, S D = g ( R d ) , S D is an afﬁne space of dimension p and g is continuous in particular for the canonical norm induced on L 2 ([0 , T ] , R p ) .
Revised: Since, S D = g ( R d ) , S D is an afﬁne space of dimension p and g is continuous in particular for the canonical norm induced on L 2 ([0 , T ] , R p ) .

Original: Therefore S D is an afﬁne space of ﬁnite dimension and is a closed set.
Revised: Therefore S D is an afﬁne space of ﬁnite dimension and is a closed set.

Original: Finding a Unique Minimizer We conclude by applying Hilbert projection lemma: our problem of minimizing (cid:82) T
Revised: Finding a Unique Minimizer We conclude by applying Hilbert projection lemma: our problem of minimizing (cid:82) T

Original: [Equation]
Revised: [Equation]

Original: X
Revised: X

Original: [Equation]
Revised: [Equation]

Revised: τ

Revised: [Equation]

Original: − X
Revised: − X

Original: [Equation]
Revised: [Equation]

Original: , amounts to an orthogonal projection problem.
Revised: , amounts to an orthogonal projection problem.

Original: Because S D is a closed convex set, we have existence and uniqueness of such projection.
Revised: Because S D is a closed convex set, we have existence and uniqueness of such projection.

Original: Therefore, it exists a unique function X D ∈ S D and a unique vector D minimizing its distance to the function X s .
Revised: Therefore, it exists a unique function X D ∈ S D and a unique vector D minimizing its distance to the function X s .

Original: D. A LGORITHM IN L INEAR S ETTING
Revised: D. A LGORITHM IN L INEAR S ETTING

Original: We detail in Algorithm 2 the alternate projection algorithm in a linear setting.
Revised: We detail in Algorithm 2 the alternate projection algorithm in a linear setting.

Original: We denote
Revised: We denote

Original: Y =
Revised: Y =

Original: ( X it 0 +∆ t , X it 0 + n ∆ t ) and X =
Revised: ( X it 0 +∆ t , X it 0 + n ∆ t ) and X =

Original: ( X it 0 , X it 

Revised: ( X it 0 , X it

Original: +( n − 1) .
Revised: +( n − 1) .

Original: ∆ t ) .
Revised: ∆ t ) .

Original: For readability purposes we set ∆ t = 1 .
Revised: For readability purposes we set ∆ t = 1 .

Original: Result:
Revised: Result:

Original: A ∈ M p,p ( R ) , D ∈ R p k
Revised: A ∈ M p,p ( R ) , D ∈ R p k

Original: = 0 , D 

Revised: = 0 , D

Original: = 0 , A − 1 0 = 

Revised: = 0 , A − 1 0 =

Original: A 00 =
Revised: A 00 =

Original: min
Revised: min

Original: A (cid:107)Y − XA (cid:107) while (cid:107) D k − D k − 1 (cid:107) > (cid:15) and (cid:107)
Revised: A (cid:107)Y − XA (cid:107) while (cid:107) D k − D k − 1 (cid:107) > (cid:15) and (cid:107)

Original: A k − A k − 1 (cid:107) > (cid:15) do
Revised: A k − A k − 1 (cid:107) > (cid:15) do

Original: D k
Revised: D k

Original: +

Revised: +

Original: =
Revised: =

Original: min D (cid:107)Y − XA k − D (cid:107) 

Revised: min D (cid:107)Y − XA k − D (cid:107)

Original: + λ (cid:107) D (cid:107)
Revised: + λ (cid:107) D (cid:107)

Original: A k +1 =
Revised: A k +1 =

Original: min
Revised: min

Original: A (cid:107)Y − XA − D k +1 (cid:107) 22 + γ (cid:107) Y − XA (cid:107)k ← k + 1 end
Revised: A (cid:107)Y − XA − D k +1 (cid:107) 22 + γ (cid:107) Y − XA (cid:107)k ← k + 1 end

Original: D. P ROOF TO P ROPOSITION
Revised: D. P ROOF TO P ROPOSITION

Original: Naturally, one could estimate jointly D and A using least square regression.
Revised: Naturally, one could estimate jointly D and A using least square regression.

Original: However, the idea is to verify the convergence of such alternate algorithm in a simple case.
Revised: However, the idea is to verify the convergence of such alternate algorithm in a simple case.

Original: We conduct the proof for the ﬁrst dimension of Y to lighten notations, meaning that we are regressing the ﬁrst dimension of Y against the X .
Revised: We conduct the proof for the ﬁrst dimension of Y to lighten notations, meaning that we are regressing the ﬁrst dimension of Y against the X .

Original: A similar reasoning for the other dimension completes the proof.
Revised: A similar reasoning for the other dimension completes the proof.

Original: Proof.
Revised: Proof.

Original: We ﬁrst give the analytical solution for D .
Revised: We ﬁrst give the analytical solution for D .

Original: Let A n be ﬁxed.
Revised: Let A n be ﬁxed.

Original: Estimation of D
Revised: Estimation of D

Original: Consider:
Revised: Consider:

Original: [Equation]
Revised: [Equation]

Original: where D =
Revised: where D =

Original: ( d, . . .
Revised: ( d, . . .

Original: , d ) ∈ R Q .
Revised: , d ) ∈ R Q .

Original: For Q samples, we ﬁnd d so that ∂L ∂d = 0 :[Equation] λ
Revised: For Q samples, we ﬁnd d so that ∂L ∂d = 0 :

Original: [Equation]
Revised: [Equation]

Original: where Y − XA = 1 Q (cid:80) Qi =

Revised: where Y − XA = 1 Q (cid:80) Qi =

Original: ( y i − X i A n ) .
Revised: ( y i − X i A n ) .

Original: Estimation of A Let D be ﬁxed and consider:
Revised: Estimation of A Let D be ﬁxed and consider:

Original: [Equation]
Revised: [Equation]

Original: Similarly, we aim to cancel the ﬁrst derivative of L A with respect to all parameters of A =
Revised: Similarly, we aim to cancel the ﬁrst derivative of L A with respect to all parameters of A =

Original: ( a 1 , .., a p ) :
Revised: ( a 1 , .., a p ) :

Original: ∂ L A
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where B = X t X .
Revised: where B = X t X .

Original: Equation (30) indicates that as soon a D converges, A n converges.
Revised: Equation (30) indicates that as soon a D converges, A n converges.

Original: Thus, we now prove the convergence of ( D n ) .
Revised: Thus, we now prove the convergence of ( D n ) .

Original: Then, for n > 1 consider:
Revised: Then, for n > 1 consider:

Original: [Equation]
Revised: [Equation]

Original: where K = (cid:107) XB − 1 X t (cid:107) .
Revised: where K = (cid:107) XB − 1 X t (cid:107) .

Original: Therefore, for λ, γ , sufﬁciently large,
Revised: Therefore, for λ, γ , sufﬁciently large,

Original: [Equation]
Revised: [Equation]

Original: n

Original: [Equation]

Original: n −
Revised: n −

Original: [Equation]
Revised: [Equation]

Original: converges as a positive decreasing sequence.
Revised: converges as a positive decreasing sequence.

Original: Finally, the sequence of ( D n ) converge and so the sequence of ( A n ) .
Revised: Finally, the sequence of ( D n ) converge and so the sequence of ( A n ) .

Original: In conclusion, the proposed algorithm converges.
Revised: In conclusion, the proposed algorithm converges.

Original: E D ATASETS
Revised: E D ATASETS

Original: In this section, we provide exhaustive simulation details for the damped pendulum, Lotka-Volterra, and both geophysical datasets.
Revised: In this section, we provide exhaustive simulation details for the damped pendulum, Lotka-Volterra, and both geophysical datasets.

Original: E. D AMPED -P ENDULUM
Revised: E. D AMPED -P ENDULUM

Original: For the damped pendulum data, eq.
Revised: For the damped pendulum data, eq.

Original: (15) is integrated with ∆ t = 0 .
Revised: (15) is integrated with ∆ t = 0 .

Original: 2 s using a Runge-Kutta 45 scheme from t = 0 up to t = 10 s .
Revised: 2 s using a Runge-Kutta 45 scheme from t = 0 up to t = 10 s .

Original: Both the pulsation ω 0 and the damping coefﬁcient k are ﬁxed across the dataset.
Revised: Both the pulsation ω 0 and the damping coefﬁcient k are ﬁxed across the dataset.

Original: We generate 100/50/50 sequences respectively for train, validation and test sampling over the initial conditions so that ( θ, ˙ θ ) ∼ U (cid:0)
Revised: We generate 100/50/50 sequences respectively for train, validation and test sampling over the initial conditions so that ( θ, ˙ θ ) ∼ U (cid:0)

Original: [ − π/ 2 , π/ 2] × [ − 0 .
Revised: [ − π/ 2 , π/ 2] × [ − 0 .

Original: 1] (cid:1) .
Revised: 1] (cid:1) .

Original: Small Oscillations To linearize the pendulum, we consider the small oscillations regime and take the initial conditions so that : ( θ, ˙ θ ) ∼ U (cid:0) [ − π/ 6 , π/ 6] × [ − 0 .
Revised: Small Oscillations To linearize the pendulum, we consider the small oscillations regime and take the initial conditions so that : ( θ, ˙ θ ) ∼ U (cid:0) [ − π/ 6 , π/ 6] × [ − 0 .

Original: 1] (cid:1) .
Revised: 1] (cid:1) .

Original: In that case eq.
Revised: In that case eq.

Original: (15) writes as:
Revised: (15) writes as:

Original: [Equation]
Revised: [Equation]

Original: (31) and following notations of section 3.3, we have: D A = 0 and A =
Revised: 31) and following notations of section 3.3, we have: D A = 0 and A =

Original: [Equation]
Revised: [Equation]

Original: E. L OTKA -V OLTERRA
Revised: E. L OTKA -V OLTERRA

Original: For Lotka-Volterra data, eq.
Revised: For Lotka-Volterra data, eq.

Original: (16) is integrated with ∆ t = 0 .
Revised: (16) is integrated with ∆ t = 0 .

Original: 05 using a Runge-Kutta 4-5 scheme from t = 0 up to t = 20 .
Revised: 05 using a Runge-Kutta 4-5 scheme from t = 0 up to t = 20 .

Original: All parameters α, β, γ, δ are set to 1 across the dataset.
Revised: All parameters α, β, γ, δ are set to 1 across the dataset.

Original: We generate 100/50/50 sequences respectively for train, validation and test sampling over the initial prey and predators populations so that ( x, y ) ∼ U (cid:0) [0 , 2] 2 (cid:1) .
Revised: We generate 100/50/50 sequences respectively for train, validation and test sampling over the initial prey and predators populations so that ( x, y ) ∼ U (cid:0) [0 , 2] 2 (cid:1) .

Original: Practical Issues and Adaptation Assuming that α and γ have positive values makes the following problem arises: the trajectories deﬁned by h k for the prey are unbounded, whereas the trajectories deﬁned by eq.
Revised: Practical Issues and Adaptation Assuming that α and γ have positive values makes the following problem arises: the trajectories deﬁned by h k for the prey are unbounded, whereas the trajectories deﬁned by eq.

Original: (16) are.
Revised: (16) are.

Original: Minimizing d( h k , f ) over long term horizon will lead in an underestimation of α to match the bounded behaviour of true data.
Revised: Minimizing d( h k , f ) over long term horizon will lead in an underestimation of α to match the bounded behaviour of true data.

Original: Therefore, we enforce d( h k , f ) on the prey component as soon as the number of predator is small.
Revised: Therefore, we enforce d( h k , f ) on the prey component as soon as the number of predator is small.

Original: In practice, we set this threshold to 0 .
Revised: In practice, we set this threshold to 0 .

Original: E. G EOPHYSICAL D ATASETS
Revised: E. G EOPHYSICAL D ATASETS

Original: We present in this section introductory tools for the understanding of the ﬂuid dynamics data presented in section 4.2.
Revised: We present in this section introductory tools for the understanding of the ﬂuid dynamics data presented in section 4.2.

Original: We ﬁrst introduce the physical modeling of ocean dynamics.
Revised: We ﬁrst introduce the physical modeling of ocean dynamics.

Original: Then, we outline the Adv+S dataset simulation which draws from ocean modeling.
Revised: Then, we outline the Adv+S dataset simulation which draws from ocean modeling.

Original: Finally, we introduce the Natl dataset and the proxy data used in the experiments.
Revised: Finally, we introduce the Natl dataset and the proxy data used in the experiments.

Original: Introduction
Revised: Introduction

Original: To Ocean Modeling The increase in ocean observations thanks to satellites and ﬂoats enabled a great development in Earth modeling over the last decades.
Revised: To Ocean Modeling The increase in ocean observations thanks to satellites and ﬂoats enabled a great development in Earth modeling over the last decades.

Original: The ocean circulation, that is the current velocity ﬁelds dynamics, are now realistically modeled in tri-dimensional structured models such as NEMO (Madec, 2008).
Revised: The ocean circulation, that is the current velocity ﬁelds dynamics, are now realistically modeled in tri-dimensional structured models such as NEMO (Madec, 2008).

Original: Such models rely on in-depth physical knowledge of the studied system and its representation through partial differential equations.
Revised: Such models rely on in-depth physical knowledge of the studied system and its representation through partial differential equations.

Original: Integrated over depth, the equations associated to the transport of the Sea Surface Temperature T by a time-varying horizontal velocity ﬁeld U can be written as:
Revised: Integrated over depth, the equations associated to the transport of the Sea Surface Temperature T by a time-varying horizontal velocity ﬁeld U can be written as:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: (33) where f is the Coriolis parameter,
Revised: (33) where f is the Coriolis parameter,

Original: h the depth of the surface layer obtained from sea surface height ( SSH ) observations, g (cid:48) the reduced gravity which takes the stratiﬁcation in density of the ocean into account such that g (cid:48)
Revised: h the depth of the surface layer obtained from sea surface height ( SSH ) observations, g (cid:48) the reduced gravity which takes the stratiﬁcation in density of the ocean into account such that g (cid:48)

Original: ≈ g. 10 − 3 .
Revised: ≈ g. 10 − 3 .

Original: In a two-dimensional setting, ∇ ( TU ) refers to the advection of a scalar quantity T by a velocity ﬁeld U =
Revised: In a two-dimensional setting, ∇ ( TU ) refers to the advection of a scalar quantity T by a velocity ﬁeld U =

Original: ( u, v ) and writes as : ∇ ( TU )
Revised: ( u, v ) and writes as : ∇ ( TU )

Original: =
Revised: =

Original: ∂T∂x u
Revised: ∂T∂x u

Original: + ∂T∂y v .
Revised: + ∂T∂y v .

Original: The mixing terms, referred to as D T/U and the forcings F T/U , are not known.
Revised: The mixing terms, referred to as D T/U and the forcings F T/U , are not known.

Original: In the context of the presented work, the physical state Z t =
Revised: In the context of the presented work, the physical state Z t =

Original: ( T t , U t ) , f X and f Y from eq.
Revised: ( T t , U t ) , f X and f Y from eq.

Original: (1) can be interpreted as follows:
Revised: (1) can be interpreted as follows:

Original: f X represents the dynamics of the observed T , i.e. f X ( T )
Revised: f X represents the dynamics of the observed T , i.e. f X ( T )

Original: = −∇ .
Revised: = −∇ .

Original: ( TU )+
Revised: ( TU )+

Original: D T + F T in eq.
Revised: D T + F T in eq.

Original: f Y represents the dynamics of U in eq.
Revised: f Y represents the dynamics of U in eq.

Original: (33), i.e. f Y ( U, h ) =
Revised: (33), i.e. f Y ( U, h ) =

Original: − ( U. ∇ ) U + f ∧
Revised: − ( U. ∇ ) U + f ∧

Original: U − g (cid:48) ∇ h + D U
Revised: U − g (cid:48) ∇ h + D U

Original: + F U .
Revised: + F U .

Original: Whereas T is observed by satellites, U is not known.
Revised: Whereas T is observed by satellites, U is not known.

Original: However, the Sea Surface Height (SSH) could be used to compute coarse estimates of U .
Revised: However, the Sea Surface Height (SSH) could be used to compute coarse estimates of U .

Original: Indeed, under hypothesis such as stationarity ( ∂U∂t = 0 ), incompressibility ( ( U. ∇ ) U = 0) ), forcings can be omited.
Revised: Indeed, under hypothesis such as stationarity ( ∂U∂t = 0 ), incompressibility ( ( U. ∇ ) U = 0) ), forcings can be omited.

Original: In this case, eq.
Revised: In this case, eq.

Original: (33) can be rewritten into
Revised: (33) can be rewritten into

Original: [Equation]
Revised: [Equation]

Original: When projected onto x and y axis, eq.
Revised: When projected onto x and y axis, eq.

Original: (34) becomes
Revised: (34) becomes

Original: [Equation]
Revised: [Equation]

Original: Note that eq.
Revised: Note that eq.

Original: (34) and eq.
Revised: (34) and eq.

Original: (35) do not hold at ﬁne scales as the stationarity and incompressibility assumptions only hold at large scale.
Revised: (35) do not hold at ﬁne scales as the stationarity and incompressibility assumptions only hold at large scale.

Original: In this case, the SSH h can be regarded as a stream function.
Revised: In this case, the SSH h can be regarded as a stream function.

Original: Both datasets considered in the paper follow the same equations approximating the tracer equation (eq.
Revised: Both datasets considered in the paper follow the same equations approximating the tracer equation (eq.

Original: (17)) inspired by eq.
Revised: (17)) inspired by eq.

Original: [Equation]
Revised: [Equation]

Original: We study the equations 32 and 33 in an incremental approach.
Revised: We study the equations 32 and 33 in an incremental approach.

Original: In the following parts, we describe how T , U and S are computed in both datasets Adv+S and Natl.
Revised: In the following parts, we describe how T , U and S are computed in both datasets Adv+S and Natl.

Original: E.3. A DV +S
Revised: E.3. A DV +S

Original: We ﬁrst investigate a dataset generated following simplifying assumptions (Adv+S).
Revised: We ﬁrst investigate a dataset generated following simplifying assumptions (Adv+S).

Original: We don’t rely on true U and S , we instead build them so that they correspond to our hypothesis.
Revised: We don’t rely on true U and S , we instead build them so that they correspond to our hypothesis.

Original: Building a Velocity Field U
Revised: Building a Velocity Field U

Original: Under stationarity and incompressibility hypothesis, U can be approximated from a stream function H .
Revised: Under stationarity and incompressibility hypothesis, U can be approximated from a stream function H .

Original: Note that, in this dataset, H is not equal to the SSH h , it is simulated following (Boffetta et al., 2001):
Revised: Note that, in this dataset, H is not equal to the SSH h , it is simulated following (Boffetta et al., 2001):

Original: [Equation]
Revised: [Equation]

Original: As introduced precedently (see eq.
Revised: As introduced precedently (see eq.

Original: (34)), eq.
Revised: (34)), eq.

Original: (33) can be simpliﬁed
Revised: (33) can be simpliﬁed

Original: and we compute U
Revised: and we compute U

Original: = ( u, v ) so that it follows:
Revised: = ( u, v ) so that it follows:

Original: [Equation]
Revised: [Equation]

Original: Note that B varies periodically with time according to B = B 0 + (cid:15) cos( ωt + φ ) .
Revised: Note that B varies periodically with time according to B = B 0 + (cid:15) cos( ωt + φ ) .

Original: We computedifferent velocity ﬁelds sampling random parameters B 0 , k, c, ω, (cid:15), φ .
Revised: We computedifferent velocity ﬁelds sampling random parameters B 0 , k, c, ω, (cid:15), φ .

Original: Building a Source Term S
Revised: Building a Source Term S

Original: In eq.
Revised: In eq.

Original: (32), the diffusion term D T is omitted.
Revised: (32), the diffusion term D T is omitted.

Original: We generate the source term S so that it represents the forcing term F T in eq.
Revised: We generate the source term S so that it represents the forcing term F T in eq.

Original: To illustrate heat exchanges, we draw from Frankignoul (1985).
Revised: To illustrate heat exchanges, we draw from Frankignoul (1985).

Original: This source term is a non linear transformation of U =
Revised: This source term is a non linear transformation of U =

Original: ( u, v ) multiplied by the difference between the ocean temperature and a reference temperature:
Revised: ( u, v ) multiplied by the difference between the ocean temperature and a reference temperature:

Original: S ( U, T ) =
Revised: S ( U, T ) =

Original: w e × ( T − T e ) where w e = (cid:26)[-
-]
Revised: w e × ( T − T e ) where w e = (cid:26)[-
-]

Revised: [Equation]

Original: if ∂ H ∂t < 10 −1 otherwise.
Revised: if ∂ H ∂t < 10 −1 otherwise.

Original: where T e is the sequence mean image (computed without source).
Revised: where T e is the sequence mean image (computed without source).

Original: Dataset Generation Using computed U and S , we integrate eq.
Revised: Dataset Generation Using computed U and S , we integrate eq.

Original: (36) with ∆ t = 

Revised: (36) with ∆ t =

Original: s over 30 days, using a Semi-Lagrangian scheme (see explanations below).
Revised: s over 30 days, using a Semi-Lagrangian scheme (see explanations below).

Original: We generate 800/100/ sequences respectively for train, validation and test sampling over the initial conditions, which are images of size 64 × 64 sampled from Natl dataset.
Revised: We generate 800/100/ sequences respectively for train, validation and test sampling over the initial conditions, which are images of size 64 × 64 sampled from Natl dataset.

Original: Finally, for integration, we impose East-West periodic conditions, implying that what comes out the left part re-enters at the right, and reciprocally.
Revised: Finally, for integration, we impose East-West periodic conditions, implying that what comes out the left part re-enters at the right, and reciprocally.

Original: We also impose velocity to be null on both top and bottom parts of the image.
Revised: We also impose velocity to be null on both top and bottom parts of the image.

Original: Semi-Lagrangian Integration Unlike Eulerian scheme, relying on time discretization of the derivative, the semi Lagrangian scheme relies on the constancy of the solution of a PDE along a characteristic curve .
Revised: Semi-Lagrangian Integration Unlike Eulerian scheme, relying on time discretization of the derivative, the semi Lagrangian scheme relies on the constancy of the solution of a PDE along a characteristic curve .

Original: Consider a solution to the advection equation, i.e. eq.
Revised: Consider a solution to the advection equation, i.e. eq.

Original: (36) with S = 0 .
Revised: (36) with S = 0 .

Original: The method of characteristics consists in exhibiting curves ( x ( s ) , t ( s )) along which the derivative of the solution T is simple, i.e ∂T∂s ( x ( s ) , t ( s ))
Revised: The method of characteristics consists in exhibiting curves ( x ( s ) , t ( s )) along which the derivative of the solution T is simple, i.e ∂T∂s ( x ( s ) , t ( s ))

Original: = 0 .
Revised: = 0 .

Original: For a 1D constant advection scheme, computations lead to:
Revised: For a 1D constant advection scheme, computations lead to:

Original: [Equation]
Revised: [Equation]

Original: giving therefore, T ( x, t )
Revised: giving therefore, T ( x, t )

Original: =
Revised: =

Original: T 0 ( x − Ut ) , linking the value of the solution at all time to its initial condition.
Revised: T 0 ( x − Ut ) , linking the value of the solution at all time to its initial condition.

Original: Therefore from a single observation at t 0 , it sufﬁces to estimate the original departure points x 0 − Ut to infer the prediction at t .
Revised: Therefore from a single observation at t 0 , it sufﬁces to estimate the original departure points x 0 − Ut to infer the prediction at t .

Original: However, when U is not constant in time, the method remains doable, not along characteristic lines deﬁned by : ( x 0 + Ut ), but along characteristic curves which are given by:
Revised: However, when U is not constant in time, the method remains doable, not along characteristic lines deﬁned by : ( x 0 + Ut ), but along characteristic curves which are given by:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: A great deal in the semi-Lagrangian literature involves solving correctly eq.
Revised: A great deal in the semi-Lagrangian literature involves solving correctly eq.

Original: We use the conventional mid-point integration rule and the semi-Lagrangian is implemented using Pytorch function gridsample , following in (Jaderberg et al., 2015).
Revised: We use the conventional mid-point integration rule and the semi-Lagrangian is implemented using Pytorch function gridsample , following in (Jaderberg et al., 2015).

Original: Further developments can be found for example in (Diamantakis, 2014).
Revised: Further developments can be found for example in (Diamantakis, 2014).

Original: E.3. N ATL
Revised: E.3. N ATL

Original: This second dataset depicts the actual ocean circulation, i.e. we consider both eq.
Revised: This second dataset depicts the actual ocean circulation, i.e. we consider both eq.

Original: (32) and eq.
Revised: (32) and eq.

Original: In this case, no assumptions are made on U and S represents both diffusion terms D T and forcing terms F T .
Revised: In this case, no assumptions are made on U and S represents both diffusion terms D T and forcing terms F T .

Original: We access daily data over a year of ocean surface temperature of the North Atlantic observations model resulting from (Ajayi et al., 2019) 1 .
Revised: We access daily data over a year of ocean surface temperature of the North Atlantic observations model resulting from (Ajayi et al., 2019) 1 .

Original: The dataset covers a 2300 km × 2560 km zone at 1.5km resolution, in the North Atlantic Ocean.
Revised: The dataset covers a 2300 km × 2560 km zone at 1.5km resolution, in the North Atlantic Ocean.

Original: In this real-life dataset, sea surface height (SSH) partial derivative with respect to x and y serves as proxies to the (unobserved) velocity ﬁelds U .
Revised: In this real-life dataset, sea surface height (SSH) partial derivative with respect to x and y serves as proxies to the (unobserved) velocity ﬁelds U .

Original: Indeed, recall that simplifying hypotheses led us to eq.
Revised: Indeed, recall that simplifying hypotheses led us to eq.

Original: We divide the Natl zone into 270 patches of size 64 × 64 .
Revised: We divide the Natl zone into 270 patches of size 64 × 64 .

Original: For each region, we extract sea surface temperatures, velocity ﬁelds, source terms and height variables.
Revised: For each region, we extract sea surface temperatures, velocity ﬁelds, source terms and height variables.

Original: We sample 200/20/50 sequences of 1 year, for respectively train, validation and test.
Revised: We sample 200/20/50 sequences of 1 year, for respectively train, validation and test.

Original: In this case, ∆ t = 86400 s (1 day).
Revised: In this case, ∆ t = 86400 s (1 day).

Original: F T RAINING D ETAILS
Revised: F T RAINING D ETAILS

Original: All experiments were conducted on NVIDIA TITAN
Revised: All experiments were conducted on NVIDIA TITAN

Original: X GPU using Pytorch (Paszke et al., 2019).
Revised: X GPU using Pytorch (Paszke et al., 2019).

Original: Hyper-Parameters Interpretation From eq.
Revised: Hyper-Parameters Interpretation From eq.

Original: (4), two independent terms appear justifying an alternate projections approach.
Revised: (4), two independent terms appear justifying an alternate projections approach.

Original: First, we highlight that strictly minimizing d( h k , f ) biases our estimation of h k .
Revised: First, we highlight that strictly minimizing d( h k , f ) biases our estimation of h k .

Original: However, it may yield a good estimation of h k provided that f k contributes signiﬁcantly to the prediction of f .
Revised: However, it may yield a good estimation of h k provided that f k contributes signiﬁcantly to the prediction of f .

Original: Hence, we interpret this loss as an initialization loss.
Revised: Hence, we interpret this loss as an initialization loss.

Original: Thus, in most applications, we progressively decrease its magnitude along training as detailed in appendices F.1 to F.3.
Revised: Thus, in most applications, we progressively decrease its magnitude along training as detailed in appendices F.1 to F.3.

Original: On the other hand, d( h u , 0) aims at constraining the free form function h u to make its action as small as possible.
Revised: On the other hand, d( h u , 0) aims at constraining the free form function h u to make its action as small as possible.

Original: We interpret this loss as a stability penalty.
Revised: We interpret this loss as a stability penalty.

Original: Finally, aiming to recover exact trajectories of observations, we proceed as suggested in (Yin et al., 2021) progressively increasing the hyper-parameters associated to d( h, f ) .
Revised: Finally, aiming to recover exact trajectories of observations, we proceed as suggested in (Yin et al., 2021) progressively increasing the hyper-parameters associated to d( h, f ) .

Original: The practical implementation is summarized in the following algorithm:
Revised: The practical implementation is summarized in the following algorithm:

Original: Algorithm
Revised: Algorithm

Original: Initialization: θ 0 u = 0 , θ 0 k = min h k ∈H k d( h k , f ) , λ h , λ h k , λ h u for epoch = 1 : N epochs do for batch = 1 : B k do θ n
Revised: Initialization: θ 0 u = 0 , θ 0 k = min h k ∈H k d( h k , f ) , λ h , λ h k , λ h u for epoch = 1 : N epochs do for batch = 1 : B k do θ n

Original: +1 k
Revised: +1 k

Original: = θ nk − τ 1 ∇ θ k [ λ h d( h, f ) + λ h k (cid:96) ( h k )] endfor batch
Revised: = θ nk − τ 1 ∇ θ k [ λ h d( h, f ) + λ h k (cid:96) ( h k )] endfor batch

Original: =
Revised: =

Original: B k : B u do θ n +1 u
Revised: B k : B u do θ n +1 u

Original: = θ nu − τ 1 ∇ θ u
Revised: = θ nu − τ 1 ∇ θ u

Original: [ λ h d( h, f ) + λ h u d( h u , 0)] end λ
Revised: [ λ h d( h, f ) + λ h u d( h u , 0)] end λ

Original: [Equation]
Revised: [Equation]

Original: end
Revised: end

Original: F. D AMPED P ENDULUM
Revised: F. D AMPED P ENDULUM

Original: Architecture Details
Revised: Architecture Details

Original: The physical parameters to be learned is a scalar of dimension 1 , and h u is a 1-hidden layer MLP with 200 -hidden neurons with leaky-relu activation.
Revised: The physical parameters to be learned is a scalar of dimension 1 , and h u is a 1-hidden layer MLP with 200 -hidden neurons with leaky-relu activation.

Original: Optimization For this dataset we use RMSProp optimizer with learning rate 0 .
Revised: Optimization For this dataset we use RMSProp optimizer with learning rate 0 .

Original: 0004 forepochs with batch size 128.
Revised: 0004 forepochs with batch size 128.

Original: We supervise the trajectories up to t =
Revised: We supervise the trajectories up to t =

Original: ∆ t × 50 , i.e we enforce d φ over ( t 0 + ∆ t, .., t 0 + 50∆ t ) .
Revised: ∆ t × 50 , i.e we enforce d φ over ( t 0 + ∆ t, .., t 0 + 50∆ t ) .

Original: Overall the number of optimization subsequences for training is 17000.
Revised: Overall the number of optimization subsequences for training is 17000.

Original: We alternate projection on S k and S u by descending the gradient 10 -batches on h k then 10 -batches on h u .
Revised: We alternate projection on S k and S u by descending the gradient 10 -batches on h k then 10 -batches on h u .

Original: Hyperparameters We initialize λ
Revised: Hyperparameters We initialize λ

Original: h k = 0 .
Revised: h k = 0 .

Original: 1 and decrease it geometrically down to λ h k = 0 .
Revised: 1 and decrease it geometrically down to λ h k = 0 .

Original: We initialize λ h = 0 .
Revised: We initialize λ h = 0 .

Original: 1 and increase it geometrically up to λ h = 100 .
Revised: 1 and increase it geometrically up to λ h = 100 .

Original: λ h u is ﬁxed through training at 0 .
Revised: λ h u is ﬁxed through training at 0 .

Original: The hyper-parameters were chosen by randomly exploring the hyper-parameters space by sampling them so that λ ∼ U (1 , 0 . 1 , . . .
Revised: The hyper-parameters were chosen by randomly exploring the hyper-parameters space by sampling them so that λ ∼ U (1 , 0 . 1 , . . .

Original: , 10 − 4 ) .
Revised: , 10 − 4 ) .

Original: We select the ones with the lowest prediction errors, i.e with lowest d φ ( h, f ) .
Revised: We select the ones with the lowest prediction errors, i.e with lowest d φ ( h, f ) .

Original: For the ablation study of Table 1, we set to 0 the hyper-parameters associated to the non-considered loss.
Revised: For the ablation study of Table 1, we set to 0 the hyper-parameters associated to the non-considered loss.

Original: The training time for this dataset is 1 hour.
Revised: The training time for this dataset is 1 hour.

Original: F. L OTKA -V OLTERRA
Revised: F. L OTKA -V OLTERRA

Original: Architecture Details
Revised: Architecture Details

Original: The physical parameters to be learned is a vector of dimension 2 accounting for ( α, β ) in eq.
Revised: The physical parameters to be learned is a vector of dimension 2 accounting for ( α, β ) in eq.

Original: (16), and h u is a 1-hidden layer MLP with 200 -hidden neurons with leaky-relu activation.
Revised: (16), and h u is a 1-hidden layer MLP with 200 -hidden neurons with leaky-relu activation.

Original: Optimization We use Adam optimizer with learning rate 0 .
Revised: Optimization We use Adam optimizer with learning rate 0 .

Original: 0005 for 200 epochs with batch size 128.
Revised: 0005 for 200 epochs with batch size 128.

Original: Overall the number of sequences for training is 15000.
Revised: Overall the number of sequences for training is 15000.

Original: We supervise the trajectories up to t =
Revised: We supervise the trajectories up to t =

Original: ∆ t × 25 , i.e we enforce d φ over ( t 0 + ∆ t, .., t 0 + 25∆ t ) .
Revised: ∆ t × 25 , i.e we enforce d φ over ( t 0 + ∆ t, .., t 0 + 25∆ t ) .

Original: We alternate projection on S k and S u by descending the gradient 10 -batches on h k then 10 -batches on h u .
Revised: We alternate projection on S k and S u by descending the gradient 10 -batches on h k then 10 -batches on h u .

Original: Hyperparameters We initialize λ
Revised: Hyperparameters We initialize λ

Original: h k = 0 .
Revised: h k = 0 .

Original: 1 and decrease it geometrically down to λ h k = 0 .
Revised: 1 and decrease it geometrically down to λ h k = 0 .

Original: We initialize λ h = 0 .
Revised: We initialize λ h = 0 .

Original: 001 and increase it geometrically up to λ h = 1 .
Revised: 001 and increase it geometrically up to λ h = 1 .

Original: λ h u is ﬁxed through training at 0 .
Revised: λ h u is ﬁxed through training at 0 .

Original: The hyper-parameters were chosen by randomly exploring the hyper-parameters space by sampling them so that λ ∼ U (1 , 0 . 1 , . . .
Revised: The hyper-parameters were chosen by randomly exploring the hyper-parameters space by sampling them so that λ ∼ U (1 , 0 . 1 , . . .

Original: , 10 − 4 ) .
Revised: , 10 − 4 ) .

Original: We select the ones with the lowest prediction errors (i.e lowest d( h, f ) ).
Revised: We select the ones with the lowest prediction errors (i.e lowest d( h, f ) ).

Original: For the ablation study of Table 1, we set to 0 the hyper-parameters associated to the non-considered loss.
Revised: For the ablation study of Table 1, we set to 0 the hyper-parameters associated to the non-considered loss.

Original: The training time for this dataset is 2 hours.
Revised: The training time for this dataset is 2 hours.

Original: F. A DV +S
Revised: F. A DV +S

Original: Architectures Details
Revised: Architectures Details

Original: The physical parameters to be estimated are the velocity ﬁelds U , of dimension (2 , 64 , 64) .
Revised: The physical parameters to be estimated are the velocity ﬁelds U , of dimension (2 , 64 , 64) .

Original: As U varies over time, we follow data assimilation principles to map a sequence of 4 consecutive measurements of the tracer ﬁeld T to the associated velocity ﬁeld (Gaultier et al., 2013).
Revised: As U varies over time, we follow data assimilation principles to map a sequence of 4 consecutive measurements of the tracer ﬁeld T to the associated velocity ﬁeld (Gaultier et al., 2013).

Original: To do so, we parameterize a recognition network G ψ by U-net with at most 512 latent channels also following the implementation of (Isola et al., 2017), taking as input a sequence of 4 time steps of T : ( T t 0 , .., T t 

Revised: To do so, we parameterize a recognition network G ψ by U-net with at most 512 latent channels also following the implementation of (Isola et al., 2017), taking as input a sequence of 4 time steps of T : ( T t 0 , .., T t

Original: +3∆ t ) .
Revised: +3∆ t ) .

Original: The residual dynamics h u is learned by a convolutional ResNet, with 1 residual block taking as entry the same sequence of T .
Revised: The residual dynamics h u is learned by a convolutional ResNet, with 1 residual block taking as entry the same sequence of T .

Original: We implement h k via a semi-lagrangian scheme, taking as input T t and the estimated U t to predict T t +1 .
Revised: We implement h k via a semi-lagrangian scheme, taking as input T t and the estimated U t to predict T t +1 .

Original: Optimization We use Adam optimizer with learning rate 0 .
Revised: Optimization We use Adam optimizer with learning rate 0 .

Original: 0001 for 30 epochs with batch size 32.
Revised: 0001 for 30 epochs with batch size 32.

Original: We supervise the trajectories up to t =
Revised: We supervise the trajectories up to t =

Original: ∆ t × 6 , i.e we enforce d φ on ( T t 

Revised: ∆ t × 6 , i.e we enforce d φ on ( T t

Original: +∆ t , ..., T t 0 +6∆ t ) .
Revised: +∆ t , ..., T t 0 +6∆ t ) .

Original: Overall the number of sequences for training is 36800 .
Revised: Overall the number of sequences for training is 36800 .

Original: We alternate projection on S k and S u by descending the gradient 4 -batches on h k then 6 -batches on h u .
Revised: We alternate projection on S k and S u by descending the gradient 4 -batches on h k then 6 -batches on h u .

Original: Hyperparameters, setting of eq.
Revised: Hyperparameters, setting of eq.

Original: (4) We initialize λ h k = 0 .
Revised: (4) We initialize λ h k = 0 .

Original: 1 and decrease it geometrically down to λ h k = 0 .
Revised: 1 and decrease it geometrically down to λ h k = 0 .

Original: We initialize λ
Revised: We initialize λ

Original: h
Revised: h

Original: = 0 .
Revised: = 0 .

Original: and increase it geometrically every epoch up to λ h,f = 1000 .
Revised: and increase it geometrically every epoch up to λ h,f = 1000 .

Original: λ h u is ﬁxed through training at 0 .
Revised: λ h u is ﬁxed through training at 0 .

Original: We select the hyperparameters with the lowest prediction errors (i.e lowest d( h, f ) ).
Revised: We select the hyperparameters with the lowest prediction errors (i.e lowest d( h, f ) ).

Original: For the ablation study of Table 1, we set to 0 the hyper-parameters associated to the non-considered loss.
Revised: For the ablation study of Table 1, we set to 0 the hyper-parameters associated to the non-considered loss.

Original: The training time for this dataset is 8 hours.
Revised: The training time for this dataset is 8 hours.

Original: F. N ATL
Revised: F. N ATL

Original: Architecture Details
Revised: Architecture Details

Original: The architectures in this setting are identical to the ones described in appendix F.3.
Revised: The architectures in this setting are identical to the ones described in appendix F.3.

Original: Optimization We use Adam optimizer with learning rate 0 .
Revised: Optimization We use Adam optimizer with learning rate 0 .

Original: 00001 for 50 epochs with batch size 32.
Revised: 00001 for 50 epochs with batch size 32.

Original: Overall the number of sequences for training is 67000 .
Revised: Overall the number of sequences for training is 67000 .

Original: We enforce d φ over 6 time-steps, i.e we supervise the predictions on timesteps: ( t 0 + ∆ t, .., t 0 + 6∆ t ) .
Revised: We enforce d φ over 6 time-steps, i.e we supervise the predictions on timesteps: ( t 0 + ∆ t, .., t 0 + 6∆ t ) .

Original: We use dropout in both G ψ and h u .
Revised: We use dropout in both G ψ and h u .

Original: Hyperparameters, setting of eq.
Revised: Hyperparameters, setting of eq.

Original: For this setting, λ h geometrically increases from 0.01 up to 100.
Revised: For this setting, λ h geometrically increases from 0.01 up to 100.

Original: We initialize λ
Revised: We initialize λ

Original: h k = 0 .
Revised: h k = 0 .

Original: 1 and decrease it geometrically down to λ h k = 0 .
Revised: 1 and decrease it geometrically down to λ h k = 0 .

Original: λ h u is ﬁxed through training at 0 .
Revised: λ h u is ﬁxed through training at 0 .

Original: We alternate projection on S k and S u by descending the gradient 10 -batches on both h k and h u .
Revised: We alternate projection on S k and S u by descending the gradient 10 -batches on both h k and h u .

Original: The selected model is the one with lowest prediction errors on validation set (i.e lowest d( h, f ) ), sampling uniformly the hyperparameters: λ ∼ U (1 , 0 . 1 , . . .
Revised: The selected model is the one with lowest prediction errors on validation set (i.e lowest d( h, f ) ), sampling uniformly the hyperparameters: λ ∼ U (1 , 0 . 1 , . . .

Original: , 10 − 4 ) .
Revised: , 10 − 4 ) .

Original: Hyperparameters, setting of eq.
Revised: Hyperparameters, setting of eq.

Original: (5) Because the dynamics of Natl is highly non linear and chaotic, we follow (Jia et al., 2019) and ﬁrst warm-up the parameters recognition network G ψ on the velocity ﬁelds proxies for 10 epochs.
Revised: (5) Because the dynamics of Natl is highly non linear and chaotic, we follow (Jia et al., 2019) and ﬁrst warm-up the parameters recognition network G ψ on the velocity ﬁelds proxies for 10 epochs.

Original: For this setting, λ h geometrically increase from 0 .
Revised: For this setting, λ h geometrically increase from 0 .

Original: 01 up to 1 .
Revised: 01 up to 1 .

Original: λ h k is set equal to λ h .
Revised: λ h k is set equal to λ h .

Original: λ h u is ﬁxed through training at 0 .
Revised: λ h u is ﬁxed through training at 0 .

Original: After warm-up, we alternate projection on S k and S u by descending the gradient 100 -batches on h k and 300 on h u .
Revised: After warm-up, we alternate projection on S k and S u by descending the gradient 100 -batches on h k and 300 on h u .

Original: In this setting of eq.
Revised: In this setting of eq.

Original: (5), the selected model is the one with lowest d( h, f ) + d( h k , f prk ) error, sampling uniformly the hyperparameters: λ ∼ U (1 , 0 . 1 , . . .
Revised: (5), the selected model is the one with lowest d( h, f ) + d( h k , f prk ) error, sampling uniformly the hyperparameters: λ ∼ U (1 , 0 . 1 , . . .

Original: , 10 − 4 ) .
Revised: , 10 − 4 ) .

Original: The training time for this dataset is 12 hours.
Revised: The training time for this dataset is 12 hours.

Original: Baselines We train NODE (Chen et al., 2018) and Aphynity (Yin et al., 2021) on both the Adv+S and Natl dataset.
Revised: Baselines We train NODE (Chen et al., 2018) and Aphynity (Yin et al., 2021) on both the Adv+S and Natl dataset.

Original: For the training of Aphinity, we set the learning rate at 0 . 0001 and train onepochs.
Revised: For the training of Aphinity, we set the learning rate at 0 . 0001 and train onepochs.

Original: We initialize λ
Revised: We initialize λ

Original: h
Revised: h

Original: = 0 .
Revised: = 0 .

Original: and increase it geometrically every epoch up to λ h = 100 .
Revised: and increase it geometrically every epoch up to λ h = 100 .

Original: λ h u is ﬁxed through training at 0 .
Revised: λ h u is ﬁxed through training at 0 .

Original: For the training of NODE, we set the learning rate at 0 . 00004 and train on 50 epochs.
Revised: For the training of NODE, we set the learning rate at 0 . 00004 and train on 50 epochs.

Original: To perform prediction, we ﬁrst encode the 4 -consecutive measurements of T (as a 3 × 64 × 64 state) then learn to integrate this state in time thanks to a network h .
Revised: To perform prediction, we ﬁrst encode the 4 -consecutive measurements of T (as a 3 × 64 × 64 state) then learn to integrate this state in time thanks to a network h .

Original: h is a 3-layer convolutional networks, with 64 hidden channels.
Revised: h is a 3-layer convolutional networks, with 64 hidden channels.

Original: It is integrated using RK4 scheme available from https://github.com/rtqichen/torchdiffeq .
Revised: It is integrated using RK4 scheme available from https://github.com/rtqichen/torchdiffeq .

Original: G A DDITIONAL R ESULTS AND S AMPLES G. R ESULTS FOR P ENDULUM AND L OTKA -V OLTERRA D ATASETS
Revised: G A DDITIONAL R ESULTS AND S AMPLES G. R ESULTS FOR P ENDULUM AND L OTKA -V OLTERRA D ATASETS

Original: We provide respectively in ﬁgs.
Revised: We provide respectively in ﬁgs.

Original: 4 and 5 phase diagrams for the damped pendulum and Lotka-Volterra experiments.
Revised: 4 and 5 phase diagrams for the damped pendulum and Lotka-Volterra experiments.

Original: Both graphs in the phase space indicate that the trajectories and their nature are well handled by the learned decomposition, providing a periodic phase space for Lotka-Volterra (ﬁg. 5), and a converging spiral for the damped pendulum (ﬁg. 4).
Revised: Both graphs in the phase space indicate that the trajectories and their nature are well handled by the learned decomposition, providing a periodic phase space for Lotka-Volterra (ﬁg. 5), and a converging spiral for the damped pendulum (ﬁg. 4).

Original: G. R ESULTS FOR A DV +S AND N ATL
Revised: G. R ESULTS FOR A DV +S AND N ATL

Original: In this section, we provide additionial results on both Adv+S and Natl datasets.
Revised: In this section, we provide additionial results on both Adv+S and Natl datasets.

Original: A thorough ablation study (table 4) gives results with constant hyperparameters λ h and λ h k (row Vanilla Optim), which validates our hyper-parameters interpretation.
Revised: A thorough ablation study (table 4) gives results with constant hyperparameters λ h and λ h k (row Vanilla Optim), which validates our hyper-parameters interpretation.

Original: Indeed, the results are better when respectively increasing and decreasing λ h and λ h k .
Revised: Indeed, the results are better when respectively increasing and decreasing λ h and λ h k .

Original: Besides, the row Ours eq.
Revised: Besides, the row Ours eq.

Original: (5) refers to a training performed as introduced in appendix C.4 with f pr = h (cid:63) trained on eq.
Revised: (5) refers to a training performed as introduced in appendix C.4 with f pr = h (cid:63) trained on eq.

Original: Figure 7 shows predictions up todays on the Adv+S data.
Revised: Figure 7 shows predictions up todays on the Adv+S data.

Original: Finally, ﬁgs. 9 and 10 provide results on Natl dataset associated to training relying on both eq.
Revised: Finally, ﬁgs. 9 and 11 provide results on Natl dataset associated to training relying on both eq.

Original: (4) and eq.
Revised: (4) and eq.

Original: (5) and with NODE (Chen et al., 2018).
Revised: (5) and with NODE (Chen et al., 2018).

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: Training
Revised: Training

Original: [Table]
Revised: [Table]

Revised: t

Revised: [Figure]

Revised: t+3 t+ t+ t+ t+

Original: [Figure]
Revised: [Figure]