Original: L EARNING TO IDENTIFY PHYSICAL LAWS OF H AMILTONIAN SYSTEMS VIA META - LEARNING
Revised: I DENTIFYING PHYSICAL LAW OF H AMILTONIAN SYSTEMS VIA META - LEARNING

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Hamiltonian mechanics is an effective tool to represent many physical processes with concise yet well-generalized mathematical expressions.
Revised: Hamiltonian mechanics is an effective tool to represent many physical processes with concise yet well-generalized mathematical expressions.

Original: A well-modeled
Revised: A well-modeled

Original: Hamiltonian makes it easy for researchers to analyze and forecast many related phenomena that are governed by the same physical law.
Revised: Hamiltonian makes it easy for researchers to analyze and forecast many related phenomena that are governed by the same physical law.

Original: However, in general, identifying a functional or shared expression of the Hamiltonian is very difﬁcult.
Revised: However, in general, identifying a functional or shared expression of the Hamiltonian is very difﬁcult.

Original: It requires carefully designed experiments and the researcher’s insight that comes from years of experience.
Revised: It requires carefully designed experiments and the researcher’s insight that comes from years of experience.

Original: We propose that meta-learning algorithms can be potentially powerful data-driven tools for identifying the physical law governing Hamiltonian systems without any mathematical assumptions on the representation, but with observations from a set of systems governed by the same physical law.
Revised: We propose that meta-learning algorithms can be potentially powerful data-driven tools for identifying the physical law governing Hamiltonian systems without any mathematical assumptions on the representation, but with observations from a set of systems governed by the same physical law.

Original: We show that a well meta-trained learner can identify the shared representation of the
Revised: We show that a well meta-trained learner can identify the shared representation of the

Original: Hamiltonian by evaluating our method on several  physical systems with various experimental settings.
Revised: Hamiltonian by evaluating our method on several types of physical systems with various experimental settings.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Hamiltonian mechanics, a reformulation of Newtonian mechanics, can be used to describe classical systems with focusing on modeling continuous-time evolution of system dynamics with a conservative quantity called Hamiltonian (Goldstein et al., 2002).
Revised: Hamiltonian mechanics, a reformulation of Newtonian mechanics, can be used to describe classical systems by focusing on modeling continuous-time evolution of system dynamics with a conservative quantity called Hamiltonian (Goldstein et al., 2002).

Original: Interestingly, the formalism of the Hamiltonian provides both geometrically meaningful interpretation (Arnol’d et al., 2001) and efﬁcient numerical schemes (Feng & Qin, 2010) representing the state of complex systems in phase space with symplectic structure.
Revised: Interestingly, the formalism of the Hamiltonian provides both geometrically meaningful interpretation (Arnol’d et al., 2001) and efﬁcient numerical schemes (Feng & Qin, 2010) representing the state of complex systems in phase space with symplectic structure.

Original: Although formalism was originally developed for classical mechanics, it has been applied to various ﬁelds of physics, such as ﬂuid mechanics (Salmon, 1988), statistical mechanics (Reichl, 1999), and quantum mechanics (Sakurai & Commins, 1995).
Revised: Although formalism was originally developed for classical mechanics, it has been applied to various ﬁelds of physics, such as ﬂuid mechanics (Salmon, 1988), statistical mechanics (Reichl, 1999), and quantum mechanics (Sakurai & Commins, 1995).

Original: While it has many useful mathematical properties, establishing an appropriate Hamiltonian of the unknown phenomena is a challenging problem.
Revised: While it has many useful mathematical properties, establishing an appropriate Hamiltonian of the unknown phenomena is a challenging problem.

Original: A Hamiltonian for a system can be modeled by a shared expression of the Hamiltonian and physical parameters.
Revised: A Hamiltonian for a system can be modeled by a shared expression of the Hamiltonian and physical parameters.

Original: For instance, the Hamiltonian of an ideal pendulum is described as H = p [Equation] ml [-
-] + mgl (1 − cos q ) (shared expression), with mass m , pendulum length l , and gravity constant g (physical parameters), whereas q and p are the angle of the pendulum and the corresponding conjugate momentum (state of the system), respectively.
Revised: For instance, the Hamiltonian of an ideal pendulum is described as H = p 2 2 ml [-
-]2 + mgl (1 − cos q ) (shared expression), with mass m , pendulum length l , and gravity constant g (physical parameters), whereas q and p are the angle of the pendulum and the corresponding conjugate momentum (state of the system), respectively.

Original: Once an appropriate functional of the Hamiltonian is established from observing several pendulums, a new pendulum-like system can be readily recognized by adapting new physical parameters on the expression.
Revised: Once an appropriate functional of the Hamiltonian is established from observing several pendulums, a new pendulum-like system can be readily recognized by adapting new physical parameters on the expression.

Original: Therefore, identifying an appropriate expression of the Hamiltonian is an important yet extremely difﬁcult problem in most science and engineering areas where there still remain numerous unknown processes where it is even uncertain whether a closed-form solution or mathematically clear expression exists.
Revised: Therefore, identifying an appropriate expression of the Hamiltonian is an important yet extremely difﬁcult problem in most science and engineering areas where there still remain numerous unknown processes where it is even uncertain whether a closed-form solution or mathematically clear expression exists.

Original: In the recent era of deep learning, we can consider the use of learning-based algorithms with sufﬁcient data to discover Hamiltonian.
Revised: In the recent era of deep learning, we can consider the use of learning-based algorithms with sufﬁcient data to discover Hamiltonian.

Original: To determine the Hamiltonian underlying the unknown physical process, the Hamiltonian should satisfy two fundamental conditions: (1) it should ﬁt well on previously observed data or motions, (2) it should generalize well on newly observed data from new systems if the systems share the same physical law with previous ones.
Revised: To determine the Hamiltonian underlying the unknown physical process, the Hamiltonian should satisfy two fundamental conditions: (1) it should ﬁt well on previously observed data or motions, (2) it should generalize well on newly observed data from new systems if the systems share the same physical law with previous ones.

Original: The ﬁrst condition has been mitigated by explicitly incorporating symplectic structure or conservation laws on neural networks, called Hamiltonian neural networks (HNN)
Revised: The ﬁrst condition has been mitigated by explicitly incorporating symplectic structure or conservation laws on neural networks, called Hamiltonian neural networks (HNN)

Original: (Greydanus et al., 2019) for learning Hamiltonian dynamics.
Revised: (Greydanus et al., 2019) for learning Hamiltonian dynamics.

Original: HNN and its variants have been shown to be effective in learning many useful properties of the Hamiltonian
Revised: HNN and its variants have been shown to be effective in learning many useful properties of the Hamiltonian

Original: (Toth et al., 2020;
Revised: (Toth et al., 2020;

Original: Chen et al., 2020; Zhong et al., 2020; Sanchez-Gonzalezet al.,
Revised: Chen et al., 2020; Zhong et al., 2020a; Sanchez-Gonzalez

Original: In their experiments, it has been shown that HNN and its variants work well on learning conservation laws or continuous-time translational symmetry, enable the learning of complex systems stably by incorporating numerical integrators and generalize on multiple initial conditions or controls for the given system.

Original: However, there is limited work regarding a trained model that works well on totally new systems governed by the same physical law with novel physical parameters.

Original: To consider the second condition, we propose that meta-learning, which aims to train a model well generalized on novel data from observing a few examples, can be potentially key to learning a functional of Hamiltonian as a data-driven method.
Revised: To consider the second condition, we propose that meta-learning, which aims to train a model well generalized on novel data from observing a few examples, can be a potential key to learning a functional of Hamiltonian as a data-driven method.

Original: There have been several representative categories of meta-learning algorithms, such as the metric-based method (Snell et al., 2017; Sung et al., 2018), black-box method (Santoro et al., 2016; Bertinetto et al., 2019), and gradient-based method
Revised: There have been several representative categories of meta-learning algorithms, such as the metric-based method (Snell et al., 2017; Sung et al., 2018), black-box method (Santoro et al., 2016; Bertinetto et al., 2019), and gradient-based method

Original: (Rusu et al., 2019; Flennerhag et al., 2020).
Revised: (Rusu et al., 2019; Flennerhag et al., 2020).

Original: Among these methods, we especially focus on the gradientbased method, which is readily compatible with any differentiable model and ﬂexibly applicable to a wide variety of learning problems (Finn et al., 2017; Xu et al., 2018; Hospedales et al., 2020).
Revised: Among these methods, we especially focus on the gradientbased method, which is readily compatible with any differentiable model and ﬂexibly applicable to a wide variety of learning problems (Finn et al., 2017; Xu et al., 2018; Hospedales et al., 2020).

Original: One of the most successful algorithms of the gradient-based method is Model-Agnostic MetaLearning (MAML)
Revised: One of the most successful algorithms of the gradient-based method is Model-Agnostic MetaLearning (MAML)

Original: (Finn et al., 2017), which consists of a task-speciﬁc adaptation process and a meta-optimization process.
Revised: (Finn et al., 2017), which consists of a task-speciﬁc adaptation process and a meta-optimization process.

Original: The key observations supporting its potential are the resemblance between these processes and the identiﬁcation of the physical laws of the Hamiltonian.
Revised: The key observations supporting its potential are the resemblance between these processes and the identiﬁcation of the physical laws of the Hamiltonian.

Original: The schematic is shown in Figure 1.
Revised: The schematic is shown in Figure 1.

Original: The task-adaptation process, which adapts the initial model parameters to a task-speciﬁc train set, resembles the process of adapting hypothesized governing equations to observations of several physical systems.
Revised: The task-adaptation process, which adapts the initial model parameters to a task-speciﬁc train set, resembles the process of adapting hypothesized governing equations to observations of several physical systems.

Original: The meta-optimization process, which updates the initial model parameters by validating each task-speciﬁc adapted parameters to a task-speciﬁc test set, is similar to correcting the hypothesized governing equations by validating each system-speciﬁc Hamiltonian on new data from the corresponding physical systems.
Revised: The meta-optimization process, which updates the initial model parameters by validating each task-speciﬁc adapted parameters to a task-speciﬁc test set, is similar to correcting the hypothesized governing equations by validating each system-speciﬁc Hamiltonian on new data from the corresponding physical systems.

Original: In addition, (Raghu et al., 2020) proposed that the recent success behind these meta-learning algorithms was due to providing qualitative shared representation across tasks rather than learning initial model parameters that encourage rapid adaptation (Finn et al., 2017).
Revised: In addition, (Raghu et al., 2020) proposed that the recent success behind these meta-learning algorithms was due to providing qualitative shared representation across tasks rather than learning initial model parameters that encourage rapid adaptation (Finn et al., 2017).

Original: This hypothesis may support our suggestion that a meta-learner can be efﬁcient in identifying the shared representation of a Hamiltonian.
Revised: This hypothesis may support our suggestion that a meta-learner can be efﬁcient in identifying the shared representation of a Hamiltonian.

Original: From this point of view, we experiment on several  physical systems to verify whether these meta-learning algorithms are beneﬁcial to our desired learning problems.
Revised: From this point of view, we experiment on several types of physical systems to verify whether these meta-learning algorithms are beneﬁcial to our desired learning problems.

Original: Our contributions are summarized as follows:
Revised: Our contributions are summarized as follows:

Original: • We formulate the problem of identifying the shared representation of unknown Hamiltonian as a meta-learning problem.
Revised: • We formulate the problem of identifying the shared representation of unknown Hamiltonian as a meta-learning problem.

Original: • For learning to identify the Hamiltonian representations, we incorporate the HNN architecture on meta-learning algorithms.
Revised: • For learning to identify the Hamiltonian representations, we incorporate the HNN architecture on meta-learning algorithms.

Original: • After meta-training the meta-learner, we adapt the model on new systems by learning the data of partial observations and predict the dynamics of the systems as a vector ﬁeld in phase space.
Revised: • After meta-training the meta-learner, we adapt the model on new systems by learning the data of partial observations and predict the dynamics of the systems as a vector ﬁeld in phase space.

Original: • We evaluate our method on several  physical systems to explore the efﬁciency of our methods with various experimental settings.
Revised: • We evaluate our method on several types of physical systems to explore the efﬁciency of our methods with various experimental settings.

Original:  P RELIMINARIES 2. H AMILTONIAN N EURAL N ETWORKS
Revised:  P RELIMINARIES 2. H AMILTONIAN N EURAL N ETWORKS

Original: In Hamiltonian mechanics, the state of a system can be described by the vector of canonical coordinates, x = ( q , p ) , which consist of position, q = ( q 1 , q 2 , ..., q n ) and its conjugate momentum, p = ( p 1 , p 2 , ..., p n ) in phase space, where n is the degree of freedom of the system.
Revised: In Hamiltonian mechanics, the state of a system can be described by the vector of canonical coordinates, x = ( q , p ) , which consist of position, q = ( q 1 , q 2 , ..., q n ) and its conjugate momentum, p = ( p 1 , p 2 , ..., p n ) in phase space, where n is the degree of freedom of the system.

Original: Then, the time evolution of the system is governed by Hamilton’s equations ˙ x = (cid:16) ∂H∂ p , − ∂H∂ q (cid:17) =
Revised: Then, the time evolution of the system is governed by Hamilton’s equations ˙ x = (cid:16) ∂H∂ p , − ∂H∂ q (cid:17) =

Original: Ω ∇ x H ( x ) , where
Revised: Ω ∇ x H ( x ) , where

Original: H ( x ) : R 2 n → R is the Hamiltonian that is conservative during the process and Ω = (cid:20)
Revised: H ( x ) : R 2 n → R is the Hamiltonian that is conservative during the process and Ω = (cid:20)

Original: I n − I n (cid:21) is a 2 n × 2 n skew-symmetric matrix.
Revised: I n − I n (cid:21) is a 2 n × 2 n skew-symmetric matrix.

Original: From the Hamiltonian equations, the Hamiltonian vector ﬁeld in phase space, which is interpreted as the time evolution of the system ˙ x , is the symplectic gradient of the Hamiltonian Ω ∇ x H ( x ) , which is determined by the Hamiltonian function and the state of the system itself.
Revised: From the Hamiltonian equations, the Hamiltonian vector ﬁeld in phase space, which is interpreted as the time evolution of the system ˙ x , is the symplectic gradient of the Hamiltonian Ω ∇ x H ( x ) , which is determined by the Hamiltonian function and the state of the system itself.

Original: Then, the trajectory of the state can be computed by integrating the symplectic gradient of the Hamiltonian.
Revised: Then, the trajectory of the state can be computed by integrating the symplectic gradient of the Hamiltonian.

Original: If the Hamiltonian does not depend on the time variable, the Hamiltonian remains constant during the time evolution, because moving along the direction of the symplectic gradient keeps the Hamiltonian constant (Arnol’d, 2013).
Revised: If the Hamiltonian does not depend on the time variable, the Hamiltonian remains constant during the time evolution, because moving along the direction of the symplectic gradient keeps the Hamiltonian constant (Arnol’d, 2013).

Original: In (Greydanus et al., 2019), the Hamiltonian function can be approximated by neural networks, H θ , called HNN.
Revised: In (Greydanus et al., 2019), the Hamiltonian function can be approximated by neural networks, H θ , called HNN.

Original: To make the Hamiltonian function constant in motion, the loss of HNN is deﬁned by the distance between the true vector ﬁeld and the symplectic gradient of H θ ,
Revised: To make the Hamiltonian function constant in motion, the loss of HNN is deﬁned by the distance between the true vector ﬁeld and the symplectic gradient of H θ ,

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: 2. M ODEL -A GNOSTIC M ETA -L EARNING AND F EATURE R EUSE H YPOTHESES
Revised: 2. M ODEL -A GNOSTIC M ETA -L EARNING AND F EATURE R EUSE H YPOTHESES

Original: A key assumption behind MAML is that separately trained models for each task share meta-initial parameters θ that could be improved rapidly for any task (Finn et al., 2017).
Revised: A key assumption behind MAML is that separately trained models for each task share meta-initial parameters θ that could be improved rapidly for any task (Finn et al., 2017).

Original: Suppose that each given task, T i , composed of D i
Revised: Suppose that each given task, T i , composed of D i

Original: = {D tri , D tei } , is drawn from a task distribution, T i ∼ p ( T ) .
Revised: = {D tri , D tei } , is drawn from a task distribution, T i ∼ p ( T ) .

Original: The learning algorithms usually consist of bi-level optimization processes; (inner-loop) the task-speciﬁc adaptation to each train set,
Revised: The learning algorithms usually consist of bi-level optimization processes; (inner-loop) the task-speciﬁc adaptation to each train set,

Original: [Equation]
Revised: [Equation]

Original: and (outer-loop) the meta-optimization on each test set,
Revised: and (outer-loop) the meta-optimization on each test set,

Original: [Equation]
Revised: [Equation]

Original: where θ could be any differentiable model’s parameters that are expected to learn the shared representations of various tasks, and α and β are the learning rates of the inner and outer loops, respectively.
Revised: where θ could be any differentiable model’s parameters that are expected to learn the shared representations of various tasks, and α and β are the step sizes of the inner and outer loops, respectively.

Original: Meanwhile, (Raghu et al., 2020) observed that during the inner-loop process, the task-speciﬁc distinction of the model parameters θ is mostly from the last layer of the networks, whereas the entire body of the model hardly changed.
Revised: Meanwhile, (Raghu et al., 2020) observed that during the inner-loop process, the task-speciﬁc distinction of the model parameters θ is mostly from the last layer of the networks, whereas the entire body of the model hardly changed.

Original: Therefore, they hypothesized that the body of the model behaves as a shared representation across the different tasks, whereas the head of the model behaves as a task-speciﬁc parameter, which is called the feature reuse hypothesis.
Revised: Therefore, they hypothesized that the body of the model behaves as a shared representation across the different tasks, whereas the head of the model behaves as a task-speciﬁc parameter, which is called the feature reuse hypothesis.

Original: From this hypothesis, they proposed a gradient-based meta-learning algorithm called Almost No Inner Loop (ANIL) by slightly modifying MAML by freezing all but updating the last layer of the networks during the inner-loop process.
Revised: From this hypothesis, they proposed a gradient-based meta-learning algorithm called Almost No Inner Loop (ANIL) by slightly modifying MAML by freezing all but updating the last layer of the networks during the inner-loop process.

Original: They showed that ANIL performs on par or better than MAML on several benchmarks, and has a computational beneﬁt compared to its counterpart.
Revised: They showed that ANIL performs on par or better than MAML on several benchmarks, and has a computational beneﬁt compared to its counterpart.

Original: For the algorithm, when the meta-learner consists of l layers θ = ( θ (1) , ..., θ ( l − 1) , θ ( l ) ) , the inner-loop update is modiﬁed as
Revised: For the algorithm, when the meta-learner consists of l layers θ = ( θ (1) , ..., θ ( l − 1) , θ ( l ) ) , the inner-loop update is modiﬁed as

Original: [Equation]
Revised: [Equation]

Revised: As many physical processes could be expressed as an invariant shared expression of Hamiltonian and variable physical parameters, such meta-learning scheme, which encourages to separate the invariant part and varying part, can be expected to be more efﬁcient to learn new systems by the relatively small number of parameter update.

Original:  M ETHOD 3. I DENTIFYING S HARED R EPRESENTATION OF H AMILTONIAN VIA M ETA -L EARNER
Revised:  M ETHOD 3. I DENTIFYING S HARED R EPRESENTATION OF H AMILTONIAN VIA M ETA -L EARNER

Original: The main goal of our study is to train a model to identify the shared representation of the Hamiltonian using observations of dynamics from several systems that are assumed to be governed by the same physical law with different physical parameters.
Revised: The main goal of our study is to train a model to identify the shared representation of the Hamiltonian using observations of dynamics from several systems that are assumed to be governed by the same physical law with different physical parameters.

Original: From a meta-learning point of view, each system is regarded as a task T i , where the physical parameters of the system are drawn from the distribution of p ( T ) .
Revised: From a meta-learning point of view, each system is regarded as a task T i , where the physical parameters of the system are drawn from the distribution of p ( T ) .

Original: The observations of the system T i can be split into D i
Revised: The observations of the system T i can be split into D i

Original: =
Revised: =

Original: {D tri , D tei } , where D tri and D tei denote the task-speciﬁc train and test sets, respectively.
Revised: {D tri , D tei } , where D tri and D tei denote the task-speciﬁc train and test sets, respectively.

Original: The observations of both D tri and D tei are given by a set of tuples of canonical coordinates x
Revised: The observations of both D tri and D tei are given by a set of tuples of canonical coordinates x

Original: = ( q , p ) and their time derivatives ˙ x = ( ˙ q , ˙ p ) as the ground truth.
Revised: = ( q , p ) and their time-derivatives ˙ x = ( ˙ q , ˙ p ) as the ground truth.

Original: For each system, the task-speciﬁc model parameters are obtained from Equation 2 or Equation 4 by computing the task-speciﬁc loss using Equation 1 on each train set D tri , 
Revised: For each system, the task-speciﬁc model parameters are obtained from Equation 2 or Equation 4 by computing the task-speciﬁc loss using Equation 1 on each train set D tri , [Equation]

Original: [Equation]
Revised: [Equation]

Original: and the meta-optimization can be operated on the batch of systems as Equation 3 by minimizing the loss over the batch of physical parameters sampled from p ( T ) .
Revised: and the meta-optimization can be operated on the batch of systems as Equation 3 by minimizing the loss over the batch of physical parameters sampled from p ( T ) .

Original: Each loss is computed by evaluating each task-speciﬁc adapted model parameters θ (cid:48)
Revised: Each loss is computed by evaluating each task-speciﬁc adapted model parameters θ (cid:48)

Original: i to each test set D tri , 
Revised: i to each test set D tri , (cid:88)

Original: [Equation]
Revised: [Equation]

Original: Depending on the inner-loop methods, we call the algorithms Hamiltonian Model-Agnostic MetaLearning (HAMAML) when using Equation 2, and Hamiltonian Almost No Inner-Loop (HANIL) when using Equation 4.
Revised: Depending on the inner-loop methods, we call the algorithms Hamiltonian Model-Agnostic MetaLearning (HAMAML) when using Equation 2, and Hamiltonian Almost No Inner-Loop (HANIL) when using Equation 4.

Original: 3. L EARNING N EW S YSTEM D YNAMICS FROM P ARTIAL O BSERVATIONS
Revised: 3. L EARNING N EW S YSTEM D YNAMICS FROM P ARTIAL O BSERVATIONS

Original: It can be expected that if the learner is efﬁcient in identifying the underlying physical nature of an unknown process, the model can appropriately predict the dynamics of novel systems from their partial observations.
Revised: It can be expected that if the learner is efﬁcient in identifying the underlying physical nature of an unknown process, the model can appropriately predict the dynamics of novel systems from their partial observations.

Original: To evaluate whether a learner is efﬁcient in identifying the representation of an unknown Hamiltonian, we set up the following validating scheme on several  physical systems.
Revised: To evaluate whether a learner is efﬁcient in identifying the representation of an unknown Hamiltonian, we set up the following validating scheme on several types of physical systems.

Original: After meta-training, novel systems are given as meta-test sets D new =
Revised: After meta-training, novel systems are given as meta-test sets D new =

Original: {D trnew , D tenew } generated from the system’s Hamiltonian with novel physical parameters T new ∼ p ( T ) .
Revised: {D trnew , D tenew } generated from the system’s Hamiltonian with novel physical parameters T new ∼ p ( T ) .

Original: The train set D trnew is used to adapt the learner on the new system dynamics ( θ → θ (cid:48) new ) .
Revised: The train set D trnew is used to adapt the learner on the new system dynamics ( θ → θ (cid:48) new ) .

Original: The new observed systems are given as K trajectories, which are reminiscent settings of the K -shot learning problem (Snell et al., 2017).
Revised: The new observed systems are given as K trajectories, which are reminiscent settings of the K -shot learning problem (Snell et al., 2017).

Original: Starting from the K initial states, each trajectory is obtained by integrating the time derivatives of the systems from 0 s to T s, with sampling rate LT which yields L rolled out sequence of states.
Revised: Starting from the K initial states, each trajectory is obtained by integrating the time-derivatives of the systems from 0 s to T s, with sampling rate LT which yields L rolled out sequence of states.

Original: The test set D tenew is for validating the performance of the adapted model θ (cid:48) new .
Revised: The test set D tenew is for validating the performance of the adapted model θ (cid:48) new .

Original:  R ELATED W ORKS 4. L EARNING D YNAMICS WITH N EURAL N ETWORKS
Revised:  R ELATED W ORKS 4. L EARNING D YNAMICS WITH N EURAL N ETWORKS

Original: Several works on learning dynamical systems usually use various forms of graph neural networks (Battaglia et al., 2016;
Revised: Several works on learning dynamical systems usually use various forms of graph neural networks (Battaglia et al., 2016;

Original: Santoro et al., 2017; Sanchez-Gonzalez et al., 2018;
Revised: Santoro et al., 2017; Sanchez-Gonzalez et al., 2018;

Original: Kipf et al., 2018;
Revised: Kipf et al., 2018;

Original: Battaglia et al., 2018; Sanchez-Gonzalez et al., 2020), where each node usually represents the state of individual objects and each edge between two arbitrary nodes usually represents the relation of the nodes.
Revised: Battaglia et al., 2018; Sanchez-Gonzalez et al., 2020), where each node usually represents the state of individual objects and each edge between two arbitrary nodes usually represents the relation of the nodes.

Original: (Chen et al., 2018) introduced a new type of deep architecture, which is considered as the differentiable ordinary differential equations (ODEs) solver parametrized by neural networks, called neural ODEs.
Revised: (Chen et al., 2018) introduced a new type of deep architecture, which is considered as the differentiable ordinary differential equations (ODE) solver parametrized by neural networks, called neural ODEs.

Original:  Neural ODEs have been developed (Dupont et al., 2019; Finlay et al., 2020) and applied for normalizing ﬂows (Grathwohl et al., 2019; Durkan et al., 2019).
Revised: The most related to our work, (Greydanus et al., 2019) introduced Hamiltonian Neural Networks (HNNs) to learn the dynamics of Hamiltonian systems by parameterizing the Hamiltonian with neural networks. The HNNs have been developed through combining graph networks for learning interacting systems (Sanchez-Gonzalez et al., 2019) or generative networks for learning Hamiltonian from high-dimensional data (Toth et al., 2020).

Original: The most related to our work, HNN (Greydanus et al., 2019) is learning dynamics that incorporate the Hamiltonian with neural networks. 
Revised: (Chen et al., 2020) proposed Symplectic recurrent neural networks to handle observed trajectories of complex Hamiltonian systems by incorporating leapfrog integrator to recurrent neural networks. (Zhong et al., 2020a) considered additional control terms to learn the conservative Hamiltonian dynamics and the dissipative

Original: Similarly, there are several works on explicitly learning the Lagrangian (Lutter et al., 2019; Cranmer et al., 2020a).
Revised: Hamiltonian dynamics (Zhong et al., 2020b). For learning Hamiltonian in an intrinsic way, (Jin et al., 2020) proposed Symplectic networks where the architecture consists of the compositional modules for preserving the symplectic structure. In most existing studies for learning Hamiltonian, one model is trained per one system, and the evaluations are restricted to the same physical system during training with new initial conditions. Our focus is learning the shared representation of

Revised: Hamiltonian which can be reused to predict new related systems governed by the same physical law that appear throughout the previously observed systems.

Original: 4. I DENTIFYING P HYSICAL L AWS OR G OVERNING E QUATIONS FROM D ATA
Revised: 4. I DENTIFYING P HYSICAL L AWS OR G OVERNING E QUATIONS FROM D ATA

Original: In earlier works, symbolic regression was utilized to search the mathematical expressions of the governing equations automatically from observed data (Bongard & Lipson, 2007; Schmidt & Lipson, 2009).
Revised: In earlier works, symbolic regression was utilized to search the mathematical expressions of the governing equations automatically from observed data (Bongard & Lipson, 2007; Schmidt & Lipson, 2009).

Original: These methods have been developed by constructing a dictionary of candidate nonlinear functions (Brunton et al., 2016;
Revised: These methods have been developed by constructing a dictionary of candidate nonlinear functions (Brunton et al., 2016;

Original: Mangan et al., 2017) or partial differentiations(Rudy et al., 2017;
Revised: Mangan et al., 2017) or partial differentiations (Rudy et al., 2017;

Original: Schaeffer, 2017) by incorporating a sparsity-promoting algorithm to extract the governing equations.
Revised: Schaeffer, 2017) by incorporating a sparsity-promoting algorithm to extract the governing equations.

Original: Recently, they have been reﬁned through combining with neural networks (Udrescu & Tegmark, 2020; Both et al., 2019; Atkinson et al., 2019) or graph structures (Cranmer et al., 2019; 2020b).
Revised: Recently, they have been reﬁned through combining with neural networks (Udrescu & Tegmark, 2020; Both et al., 2019; Atkinson et al., 2019; Sahoo et al., 2018) or graph structures (Cranmer et al., 2019; 2020).

Original: In contrast to our work, the existing methods of identifying the governing representations used the symbolic representation underlying the assumptions that the unknown physical laws are expressed by combinations ofmathematically known expressions.
Revised: In contrast to our work, the existing methods of identifying the governing representations used the symbolic representation underlying the assumptions that the unknown physical laws are expressed by combinations of known mathematical terms. 4. G RADIENT - BASED M ETA - LEARNING

Revised: The core learning method of our work is related to gradient-based meta-learning algorithms (Finn et al., 2017).

Original: E XPERIMENTS 5. P HYSICAL S YSTEMS
Revised: The learning method has been developed by training additional learning rate (Li et al., 2017), removing the second-order derivatives (Nichol et al., 2018) and stabilizing the training procedure (Antoniou et al., 2019). Meanwhile, in another direction of the research, the model parameters are separated by the shared representation part which is mainly updated in the outer-loop and taskspeciﬁc varying part which is mainly updated in the inner-loop (Lee & Choi, 2018; Alet et al., 2018; Flennerhag et al., 2020; Raghu et al., 2020;

Revised: Zhou et al., 2020).

Revised: Our study mainly focuses on whether a meta-learning could identify the shared expression of the Hamiltonian from several observed systems.

Revised: MAML is chosen as the representative of the meta-learning algorithm.

Revised: Also, for verifying whether the separative learning scheme improves the model to identify the shared expression of the

Revised: Hamiltonian, we choose ANIL as the representative of the meta-learning with the separative learning scheme.

Revised: Since, without any structural difference, it is sufﬁcient to verify that the existence of the separative learning scheme would be beneﬁcial.

Original: Spring-Mass. Hamiltonian of the system is described by H = p 2 2 m + k ( q − q 0 ) 2, where the physical parameters m , k , and q 0 are the mass, spring constant, and equilibrium position, respectively.
Revised: Spring-Mass. Hamiltonian of the system is described by H = p 2 2 m + k ( q − q 0 ) 2, where the physical parameters m , k , and q 0 are the mass, spring constant, and equilibrium position, respectively.

Original: q and p are the position and conjugate momentum of the system, respectively.
Revised: q and p are the position and conjugate momentum of the system, respectively.

Original: Pendulum.
Revised: Pendulum.

Original: Hamiltonian of the system is described by H = p 2 2 ml 2 + mgl (1 − cos( q − q 0 )) , where the physical parameters m , l , and q 0 are the mass, pendulum length, and equilibrium angle from the vertical, respectively.
Revised: Hamiltonian of the system is described by H = p 2 2 ml 2 + mgl (1 − cos( q − q 0 )) , where the physical parameters m , l , and q 0 are the mass, pendulum length, and equilibrium angle from the vertical, respectively.

Original: q and p are the pendulum angle from the vertical and conjugate momentum of the system, respectively.
Revised: q and p are the pendulum angle from the vertical and conjugate momentum of the system, respectively.

Original: g denotes the gravitational acceleration.
Revised: g denotes the gravitational acceleration.

Original: Kepler problem.
Revised: Kepler problem.

Original: The system consists of two objects that are attracted to each other by gravitational force.
Revised: The system consists of two objects that are attracted to each other by gravitational force.

Original: Hamiltonian of the system is described by H =
Revised: Hamiltonian of the system is described by H =

Original: | p | 2 2 m − GMm | q − q 0 | , where the physical parameters
Revised: | p | 2 2 m − GMm | q − q 0 | , where the physical parameters

Original: M and m are the mass of the two bodies and we set the object of M to be stationary at q 

Revised: M and m are the mass of the two bodies and we set the object of M to be stationary at q

Original: = ( q x , q y ) of the coordinate system.
Revised: = ( q x , q y ) of the coordinate system.

Original: The state is represented by q = ( q 1 , q 2 ) and p = ( p 1 , p 2 ) which denote the position and conjugate momentum of the object m in two-dimensional space, respectively.
Revised: The state is represented by q = ( q 1 , q 2 ) and p = ( p 1 , p 2 ) which denote the position and conjugate momentum of the object m in two-dimensional space, respectively.

Original: G denotes the gravitational constant
Revised: G denotes the gravitational constant.

Original: . 5. E XPERIMENTAL SETTINGS
Revised: [Table] 5. E XPERIMENTAL SETTINGS

Original: Datasets.
Revised: Datasets.

Original: During the meta-training, we generate 10,000 tasks for meta-train sets of all systems.
Revised: During the meta-training, we generate 10,000 tasks for meta-train sets of all systems.

Original: Each meta-train set consists of task-speciﬁc train set and test set given by 50 randomly sampled point states x and their time derivatives ˙ x in phase space with task-speciﬁc physical parameters.
Revised: Each meta-train set consists of task-speciﬁc train set and test set given by 50 randomly sampled point states x and their time-derivatives ˙ x in phase space with task-speciﬁc physical parameters.

Original: The distributions of the sampled states and physical parameters for each physical process are described in Appendix A.1.
Revised: The distributions of the sampled states and physical parameters for each physical process are described in Appendix A.1.

Original: During the meta-testing, the distributions of sampled states and physical parameters are the same as in the meta-training stage.
Revised: During the meta-testing, the distributions of sampled states and physical parameters are the same as in the meta-training stage.

Original: To evaluate learners on the efﬁcacy of identifying Hamiltonian of the new systems, meta-test sets are constructed as the following ways.
Revised: To evaluate learners on the efﬁcacy of identifying Hamiltonian of the new systems, meta-test sets are constructed as the following ways.

Original: (1) Observing the new systems as point dynamics in phase space: D trnew consists of randomly sampled points in phase space with K = { 25 , 50 } and L = 1 , and D tenew consists of equally ﬁne-spaced points in phase space.
Revised: (1) Observing the new systems as point dynamics in phase space: D trnew consists of randomly sampled points in phase space with K = { 25 , 50 } and L = 1 , and D tenew consists of equally ﬁne-spaced points in phase space.

Original: (2) Observing the new systems as trajectories in phase space: D trnew consists of randomly sampled trajectories and D tenew consists of equally ﬁne-spaced points in phase space.
Revised: (2) Observing the new systems as trajectories in phase space: D trnew consists of randomly sampled trajectories and D tenew consists of equally ﬁne-spaced points in phase space.

Original: The number of sampled trajectories with K = { 5 , 10 } and L = 5 sequences during T = 1 s .
Revised: The number of sampled trajectories with K = { 5 , 10 } and L = 5 sequences during T = 1 s .

Original: The trajectories are obtained by integrating the symplectic gradient of the Hamiltonian from initial states using the adaptive step-size Runge-Kutta method (Hairer et al., 1993).
Revised: The trajectories are obtained by integrating the symplectic gradient of the Hamiltonian from initial states using the adaptive step-size Runge-Kutta method (Hairer et al., 1993).

Original: Fine-spaced test sets consist of equally spaced grids for each coordinate in the region of the phase space where we sampled the point states.
Revised: Fine-spaced test sets consist of equally spaced grids for each coordinate in the region of the phase space where we sampled the point states.

Original: More details about data sets are represented in the Appendix A.1.
Revised: More details about data sets are represented in the Appendix A.1.

Original: Baselines.
Revised: Baselines.

Original: We took several learners as baselines to assess the efﬁcacy of our proposed methods, such as (1) training HNN on D trnew from scratch (random initialization), (2) pretrained HNN using standard supervised learning on the meta-train set, (3) meta-trained naive fully connected neural networks (Naive NN), which are given the inputs x and the outputs ˙ x with MAML, and (4) with ANIL.
Revised: We took several learners as baselines to assess the efﬁcacy of our proposed methods, such as (1) training HNN on D trnew from scratch (random initialization), (2) pretrained HNN across all of the meta-train set, (3) meta-trained naive fully connected neural networks (Naive NN), which are given the inputs x and the outputs ˙ x with MAML, and (4) with ANIL.

Original: The architectures and training details of the baselines and our methods are represented in the Appendix A.2.
Revised: The architectures and training details of the baselines and our methods are represented in the Appendix A.2.

Original: Evaluations.
Revised: Evaluations.

Original: The evaluation metricfor the learners is the average of the mean squared errors (MSEs) between  the symplectic gradients of predicted Hamiltonian Ω ∇ x H θ ( x )and true vector [Table] ﬁelds ˙ x across the test sets of new systems D tenew from adapting the learners to the corresponding train set of the new systems D trnew .
Revised: The evaluation metric is the average of the mean squared errors (MSE) between true vector ﬁelds ˙ x and the symplectic gradients of predicted Hamiltonian Ω ∇ x H θ ( x ) across the test sets of new systems D tenew by adapting the learners to the corresponding train set of the new systems D tr new .

Original: In all tasks, the averaged MSEs of randomly sampled 10 new systems are averaged for evaluating the learners after 10 gradient steps adapting to each D trnew .
Revised: For all types of systems, the averaged MSE of randomly sampled 10 new systems are averaged for evaluating the learners after 10 gradient steps adapting to each D trnew .

Original: We also predict the state trajectories and  energies by integrating the output vector ﬁelds of the learnersthat are adapted to a new system by observing samples of state dynamics.
Revised: We also predict the state trajectories and the corresponding energies by integrating the output vector ﬁelds of the learners adapted to a new system by observing samples of point dynamics.

Original: Following (Greydanus et al., 2019), we evaluate the MSEs of the predicted trajectories and energies from their corresponding ground truth at each time step.
Revised: Following (Greydanus et al., 2019), we evaluate the MSEs of the predicted trajectories and energies from their corresponding ground truth at each time step.

Revised: The predicted values are computed by the learners adapted torandomly sampled point dynamics of new systems in phase space after 50 gradient steps.

Revised: [Figure]

Original:  R ESULTS
Revised: 5. R ESULTS

Original:  The quantitative results of meta-testing performance are shown in Table 1.
Revised: Quantitative results. The quantitative results of meta-testing performance are shown in Table 1.

Original: It is shown that HANIL outperforms the others for all experimental settings in all  physical systems.
Revised: It is shown that HANIL outperforms the others for all experimental settings in all types of physical systems.

Original: When observing 25-shot point dynamics and 5-shot trajectories, the number of given samples in the phase space is the same, and the same is true for observing 50-shot point dynamics and 10-shot trajectories.
Revised: When observing 25-shot point dynamics and 5-shot trajectories, the number of given samples in the phase space is the same, and the same is true for observing 50-shot point dynamics and 10-shot trajectories ( L = 5 ).

Original: Comparing the point dynamics and trajectories with the same number of given samples, learning from observing point dynamics is slightly more accurate than that from observing trajectories.
Revised: Comparing the point dynamics and trajectories with the same number of given samples, learning from observing point dynamics is slightly more accurate than that from observing trajectories.

Original:  In Figure 2, predicted pendulum dynamics by adapting the learners from observing partial observations are represented as phase portraits by the corresponding gradient steps.
Revised: Predicted vector ﬁelds. In Figure 2, predicted pendulum dynamics by adapting the learners from observing partial observations are represented as phase portraits by the corresponding gradient steps.

Original: Note that those of spring-mass systems are shown in Figure 4 in Appendix A.3.
Revised: Note that those of spring-mass systems are shown in Figure 5 in Appendix A.3.

Original: In Figure 2 (a), the initial outputs of vector ﬁelds from the learners are represented.
Revised: In Figure 2 (a), the initial outputs of vector ﬁelds from the learners are represented.

Original: During the adaptation to the given observations, the output vectors of each learner are evolved to ﬁt on the observations based on their own prior belief or representation learned from the meta-train set.
Revised: During the adaptation to the given observations, the output vectors of each learner are evolved to ﬁt on the observations based on their own prior belief or representation learned from the meta-train set.

Original: In detail, HNN from scratch fails to predict the dynamics of new systems from partial observations.
Revised: In detail, HNN from scratch fails to predict the dynamics of new systems from partial observations.

Original: At least hundreds of states should be given as train set, and thousands of gradient steps are required for training HNN for learning a system (Greydanus et al., 2019).
Revised: At least hundreds of states should be given as train set, and thousands of gradient steps are required for training HNN for learning a system (Greydanus et al., 2019).

Original: However, in our meta-testing, up to 50 states are givenas train set to adapt to the new system with few gradient steps.
Revised: However, in our meta-testing, up to 50 states are given to adapt to the new system with few gradient steps.

Original: Thus, the number of samples and gradient steps is too small to train HNN without any inductive bias.
Revised: Thus, the number of samples and gradient steps is too small to train HNN without any inductive bias.

Original: Pretrained HNN also fails, even though it is trained using the meta-train sets.
Revised: Pretrained HNN also fails, even though it is trained using the meta-train sets.

Revised: A model simply pretrained across all tasks may output the averaged values of time-derivative at each state point.

Revised: As the time-derivatives of each state would varying sensitive to the physical parameters of the systems, the simple averaged values are likely to have very different patterns from the actual vector ﬁelds.

Original: That is because the standard supervised learning scheme is not  efﬁcientfor the model to learn appropriate shared representation across the systems.
Revised: Therefore, such pretrained model would not be efﬁcient to learn appropriate shared representation across the systems.

Original: Naive NNs, with MAML and ANIL also fail to predict the dynamics because naive NNs are hard to grasp the continuous and conservative structure of the vector ﬁelds.
Revised: Naive NNs, with MAML and ANIL also fail to predict the dynamics because naive NNs are hard to grasp the continuous and conservative structure of the vector ﬁelds where the number of given data are not sufﬁcient to adapt to new systems.

Original: Thus, the phase portraits of them are discontinuous or dissipative-like shape.
Revised: Thus, the phase portraits of them are discontinuous or dissipative-like shape.

Original: HANIL canaccurately predict the new systems dynamics from partial ob- [Figure]
Revised: HANIL can

Original: [Figure]
Revised: [Figure]

Original: servations with few gradient steps, while HAMAML is slower than HANIL to adapt the true vector ﬁelds because of the larger number of parameters to update in  adaptation process.
Revised: accurately predict the dynamics of new systems from partial observations with few gradient steps, while HAMAML is slower than HANIL to adapt the true vector ﬁelds because of the larger number of parameters to update in the adaptation process.

Original:  In Figure 3, we also evaluate the learners adapted to the new systems through their predictions of state and energy trajectories from the initial states of the new systems.
Revised: Trajectory predictions. In Figure 3, we also evaluate the learners adapted to the new systems through their predictions of state and the corresponding energies starting from the initial states during 20 s .

Original: Adapted HANIL (blue lines)  to new systems predicts the states and energy trajectories almost the same as the ground truth ( dashed lines) with relatively small errors, whereas the others fail to predict the right trajectories and energies of the system at each time step. 
Revised: HANIL (blue lines) adapted to new systems predicts the states and energy trajectories with relatively small errors from the ground truth (black dashed lines), whereas the others fail to predict the right trajectories and energies of the system at each time step. Ablation study.

Original: C ONCLUSIONS
Revised: We conduct ablation studies to verify the efﬁcacy of the separative learning schemes by comparing the learning curves during task-adaptation, and the effects of the size of the meta-train sets. We evaluate the same averaged MSEs as described in Section 5.2 but varying gradient steps from 0 to 50 to see the learning curves and varying the number of tasks from 10 to 10,000 which are observed during meta-training to verify the effects of the size of the meta-train set. In addition, in order to see how the number of parameters updated during the inner-loop affects the performance, we add another comparison meta-learned learner called HANIL-Inverse (HANIL-INV). The method updates all but except the ﬁrst layer of the network during the inner-loop, while HANIL only updates the last layer of the network.

Revised: Therefore, during the inner-loop, the number of updated parameters of the HANIL-INV is between the HAMAML and HANIL.

Revised: As shown in both ablation studies in

Revised: Figure 4, HANIL-Inv converges to the error between HAMAML and HANIL, while the HNN from scratch and the pretrained HNN are hardly improved in both studies.

Revised: As comparing the meta-trained learners with the others, meta-learning improves the performance to predict new systems through their ability to learn the shared representation.

Revised: As comparing HAMAML and meta-trained learners with the separative learning scheme such as HANIL-INV and HANIL, it seems beneﬁcial to separately learn the shared representation and physical parameters through the separative learning scheme.

Revised: In addition, since the updated parameters are limited to the last layer during the inner-loop,

Revised: HANIL converges with the lowest errors, which would be most efﬁcient to generalize the related systems with the same physical law, as fewer updated parameters are required for the new system.

Revised: Meanwhile, in Figure 4, the point where the effect of the size of tasks to the meta-trained learners begins to be noticeable is between 200 and 500 and slowly converges at around 10,000.

Revised:  C ONCLUSIONS

Original: By observing the resemblance between seemingly unrelated problems, identifying the Hamiltonian and meta-learning, we formulate the problem of identifying the Hamiltonian as a meta-learning problem.
Revised: By observing the resemblance between seemingly unrelated problems, identifying the Hamiltonian and meta-learning, we formulate the problem of identifying the Hamiltonian as a meta-learning problem.

Original: We incorporate HNN, which is an efﬁcient architecture for learning Hamiltonian, with meta-learning algorithms in order to discover the shared representation of unknown Hamiltonian across observed physical systems.
Revised: We incorporate HNN, which is an efﬁcient architecture for learning Hamiltonian, with meta-learning algorithms in order to discover the shared representation of unknown Hamiltonian across observed physical systems.

Original: Comparing the baseline models with various experiments, we show that our proposed methods, especially HANIL, is efﬁcient to learn totally new systems dynamics governed by the same underlying physical laws.
Revised: Comparing the baseline models with various experiments, we show that our proposed methods, especially HANIL, is efﬁcient to learn totally new systems dynamics governed by the same underlying physical laws.

Original: The results state that our proposed methods have the ability to extract the meta-transferable knowledge, which can be considered as physical nature across the observed physical systems during meta-training.
Revised: The results state that our proposed methods have the ability to extract the meta-transferable knowledge, which can be considered as physical nature across the observed physical systems during meta-training.

Original: A A PPENDIX A. S YSTEM D ETAILS
Revised: A A PPENDIX A. S YSTEM D ETAILS

Original: Spring-Mass.
Revised: Spring-Mass.

Original: The physical parameters are randomly sampled from ( m, k, q 0 ) ∈
Revised: The physical parameters are randomly sampled from ( m, k, q 0 ) ∈

Original: ([0 . 5 , 5] , [0 . 5 , 5] , [ − 5 , 5]) .
Revised: ([0 . 5 , 5] , [0 . 5 , 5] , [ − 5 , 5]) .

Original: The initial states are randomly sampled from ( q, p ) ∈ ([ − 10 , 10] 2 ) .
Revised: The initial states are randomly sampled from ( q, p ) ∈ ([ − 10 , 10] 2 ) .

Original: Finespaced test sets of new systems consist of 50 equally spaced grids for each coordinate in the region of the phase space where we sampled the point states.
Revised: Finespaced test sets of new systems consist of 50 equally spaced grids for each coordinate in the region of the phase space where we sampled the point states.

Original: Therefore, there are 2,500 grids points in the test sets.
Revised: Therefore, there are 2,500 grids points in the test sets.

Original: Pendulum.
Revised: Pendulum.

Original: The physical parameters are randomly sampled from ( m, l, q 0 ) ∈
Revised: The physical parameters are randomly sampled from ( m, l, q 0 ) ∈

Original: ([0 . 5 , 5] , [0 . 5 , 5] , [ − π, π ]) .
Revised: ([0 . 5 , 5] , [0 . 5 , 5] , [ − π, π ]) .

Original: We ﬁx the gravitational acceleration as g = 1 .
Revised: We ﬁx the gravitational acceleration as g = 1 .

Original: The initial states are randomly sampled from ( q, p ) ∈ ([ − 2 π, 2 π ] , [ − 20 , 20]) .
Revised: The initial states are randomly sampled from ( q, p ) ∈ ([ − 2 π, 2 π ] , [ − 20 , 20]) .

Original: Fine-spaced test sets of new systems consist of 50 equally spaced grids for each coordinate in the region of the phase space where we sampled the point states.
Revised: Fine-spaced test sets of new systems consist of 50 equally spaced grids for each coordinate in the region of the phase space where we sampled the point states.

Original: Therefore, there are 2,500 grids points in the test sets.
Revised: Therefore, there are 2,500 grids points in the test sets.

Original: Kepler Problem.
Revised: Kepler Problem.

Original: The physical parameters are randomly sampled from ( M, m, q x , q y ) ∈ ([0 . 5 , 2 . 5] 2 , [ − 2 .
Revised: The physical parameters are randomly sampled from ( M, m, q x , q y ) ∈ ([0 . 5 , 2 . 5] 2 , [ − 2 .

Original: We ﬁx the gravitational constant as G = 1 .
Revised: We ﬁx the gravitational constant as G = 1 .

Original: The initial states are randomly sampled from ( q , p ) ∈ ([ − 5 , 5] 4 ) .
Revised: The initial states are randomly sampled from ( q , p ) ∈ ([ − 5 , 5] 4 ) .

Original: Fine-spaced test sets of new systems consist of 10 equally spaced grids for each coordinate in the region of the phase space where we sampled the point states.
Revised: Fine-spaced test sets of new systems consist of 10 equally spaced grids for each coordinate in the region of the phase space where we sampled the point states.

Original: Therefore, there are 10,000 grids points in the test sets.
Revised: Therefore, there are 10,000 grids points in the test sets.

Original: A. I MPLEMENTATION D ETAILS
Revised: A. I MPLEMENTATION D ETAILS

Original: For all tasks, we took the baseline model as fully connected neural networks with the size of state dimensions - 64 Softplus - 64 Softplus - 64 Softplus - state dimensions and the HNN model as fully connected neural networks with the size of state dimensions - 64 Softplus - 64 SoftplusSoftplus - 1 dimension.
Revised: For all tasks, we took the baseline model as fully connected neural networks with the size of state dimensions - 64 Softplus - 64 Softplus - 64 Softplus - state dimensions and the HNN model as fully connected neural networks with the size of state dimensions - 64 Softplus - 64 SoftplusSoftplus - 1 dimension.

Original: We searched the hyperparameters, exploring the size of hidden dimensions from { 32 , 64 , 128 , 256 } , the number of layers from { 1 , 2 , 3 , 4 } , and the activation function from { Sigmoid, Tanh, Relu, Softplus } .
Revised: We searched the hyperparameters, exploring the size of hidden dimensions from { 32 , 64 , 128 , 256 } , the number of layers from { 1 , 2 , 3 , 4 } , and the activation function from { Sigmoid, Tanh, Relu, Softplus } .

Original: During meta-training or pretraining, we use the Adam optimizer (Kingma & Ba, 2015) on outer-loop with learning rate of 0.001 and use gradient descent on innerloop with learning rate of 0.002.
Revised: During meta-training or pretraining, we use the Adam optimizer (Kingma & Ba, 2015) on outer-loop with learning rate of 0.001 and use gradient descent on innerloop with learning rate of 0.002.

Original: For all systems, we set the number of task batches of 10, inner gradient updates of 5, and episodes of outer loop of 100 for meta-optimization.
Revised: For all systems, we set the number of task batches of 10, inner gradient updates of 5, and episodes of outer loop of 100 for meta-optimization.

Original: During the metatesting, we also use the Adam optimizer with a learning rate 0.002.
Revised: During the metatesting, we also use the Adam optimizer with a learning rate 0.002.

Original: There is no weight decay for all.
Revised: There is no weight decay for all.

Original: A. A DDITIONAL R ESULTS
Revised: A. A DDITIONAL R ESULTS

Original: [Figure]
Revised: [Figure]