Machine learning structure preserving brackets for forecasting irreversible processes

Abstract: Forecasting of time-series data requires imposition of inductive biases to obtain predictive extrapolation, and recent works have imposed Hamiltonian/Lagrangian form to preserve structure for systems with \emph{reversible} dynamics. In this work we present a novel parameterization of dissipative brackets from metriplectic dynamical systems appropriate for learning \emph{irreversible} dynamics with unknown a priori model form. The process learns generalized Casimirs for energy and entropy guaranteed to be conserved and nondecreasing, respectively. Furthermore, for the case of added thermal noise, we guarantee exact preservation of a fluctuation-dissipation theorem, ensuring thermodynamic consistency. We provide benchmarks for dissipative systems demonstrating learned dynamics are more robust and generalize better than either "black-box" or penalty-based approaches.


Reviews and author responses:

The paper introduces a new framework for learning neural ODEs obeying metriplectic dynamics. These differ from previous work on Hamiltonian dynamics in that it includes both conservative and dissipative terms guaranteeing that system entropy is non-decreasing. Dissapative dynamics are extremely common in real-world systems and so this is an important inductive bias to be able to account for. The authors compare their proposed method with black box and penalty based approaches and find that their proposed method outperforms both.  The writing and motivation in the paper are very strong, though there are a few terms missing, namely the $s_i$ in the experiments. They seem to be specific entropy, but I didn't catch a definition in the paper. Overall on this front, the motivation is clear and the approach taken seems to be well founded. To my knowledge, the approach taken is unique though it seems well-grounded in previous work and a natural extension from previous work on learning Hamiltonian dynamics. 

The main weakness of the paper is in the experiments and presentation of the results. The method shows promise and it's positive to see that H and S do seem to enjoy the proposed theoretical benefits in practice. There are a few aspects of the experiments that seem under-explained and perhaps overfit to the experiments:
- D is defined explicitly to be rank 1. This is not a requirement in the general framework and the matrix is too small for this to seem motivated by storage costs, but it seems like it would reduce the influence of the disappative term. It's hard to tell if this is theoretically or experimentally motivated. 
- The configurations of the networks seem to vary significantly and this isn't explained. All tests seem to be done on small systems with correspondingly small networks. While the additional parameters of the penalty and GNODE methods may motivate smaller networks for fair comparison, the significant differences make it hard to tell if the better fit is due to the method or if it's just that neural ODEs are hard to train and the smaller networks led to better fits on relatively simple dynamics. 
- The penalty comparison does not feel very robust. One value of the multipliers is reported without indication as to how they were chosen. It seems odd that the penalty method would completely underperform the black box approach as is seen in figure one and without the error bars, I'd guess it was a bad optimization run. Due to the small number of runs, it still seems possible that they were influence by an outlier.
- These seem like one particle systems, so if the target of interest is mapping the trajectory, then I'd like to see results reported on the discrepency between the true and targeted trajectories and not just H as the black box method is not trained to optimize performance on H. 

Finally, there were also a few discrepancies between what was stated in the checklist and what appeared in the paper. First, I do not think it is reasonable to take credit for releasing code when the code is not included in the supplementary material. If the code isn't included in the submission, then the answer to "did we release the code?" is no regardless of future intentions. Error bars are present, but wide and it's unclear if these are the extremes or a number of standard deviations. These are also smaller experiments, so it seems as though more than 5 runs should be feasible particularly when the deviation seems to be so large. 

Overall, the paper is solid and proposed a promising direction, but in it's current version, the experiments are hard to put significant stock into. It would make for a stronger paper if larger systems were explored or more consistent configurations were used. Additional comparisons would also be valuable as vanilla neural ODEs are not a very strong baseline. It would also be interesting to see how the model preserves on non-metriplectic systems (Hamiltonian or unknown systems from collected data) to see if the technique is capable of adapting in those settings (ie, it should be able to represent Hamiltonian dynamics, does it outperform black box ODEs on those tasks?) though obviously it would not be expected to outperform methods designed for those settings.

EDIT: Updated score based on author comments.  Limitations and societal impact are adequately addressed.

 Thanks for all the responses. While I still feel like the comparisons for the experiments are a little weak and improvements to that area could make for a very strong paper, the authors' responses with regard to experiment settings satisfy enough of my concerns that I'd be comfortable increasing my rating based on the assumption that the missing information will be added to the paper or an appendix. I've updated my original score to reflect this. 

 I appreciate the authors’ thorough response to my questions. After reading the authors' response, now I think the paper’s contribution outweighs my initial concerns (especially regarding the originality). I increased my review score accordingly. Followings are my follow-up for your response.
***

1. We'd like to comment that for our unpublished experiments where one assumed measurements of non-observable states were available (e.g. from an analytic solution), the method performed extremely well - however we felt it inappropriate to include these results.
- I agree with it is less relevant to include the fully-observable experiment in the main body of the paper, but it will be nice if I can see this result in the, e.g., supplementary material. I think it can support the superiority of the authors' algorithm under the optimal or ideal condition, as well as how the partially-observable experiment is challenging.
***

2. As discussed in a previous reviewers comment, we have adapted all hyperparameters for NODE, penalty, and GNODE via a grid search, including architecture width/depth, learning rates, and penalty parameters.
- As Reviewer DqnY also mentioned, it will be nice if the authors can include the detailed experimental protocol in the supplementary material.
***

3. To understand why the penalty can't be too large, consider the asymptotic case of arbitrarily large penalty parameter. In this case the data loss ||X-X_data|| is insignificant, and training will return a valid generic model which doesn't match the data. This Pareto front is a critical weakness of physics-informed NNs which impose physics by penalty, as discussed in several excellent recent works. The novel use of Ottingers conjectured brackets eliminates the previously mentioned pathologies associated with the Pareto front in traditional physics-informed ML
- Thank you for your insightful explanation. I think the Pareto front perspective is a very nice discussion on why the penalty approach is not a very attractive approach. I recommend the authors discussing this point in the main body of the paper, with citing the relevant references. The current version of the paper just states that “We will provide results demonstrating that a penalty approach is insufficient to guarantee preservation of metriplectic structure” in line 78, without a deeper explanation (although the experimental results justify this mention).
***

4. Our architecture is indeed modeled off of Ottinger's analysis. We'd like to stress two points. Firstly, the bracket structure proposed by Ottinger is a conjecture and has not been applied either for discovering dynamics or for performing first principles forward simulation, and this work marks (to our knowledge) the first time these brackets have been used for any application. There are many details which go into adapting a mathematical structure/concept into an effective ML architecture which trains well. In the case of this work, our treatment of tensors via cholesky decomposition and anti-symmetrization of high-order stencils has not been used in the context of learning structure preserving dynamics.
- Thank you for your explanation. In fact, I misunderstood the sentence “Our approach is motivated by the work in [42] which we summarize in Sections 4.1–4.3.” in line 103-104, as Sections 4.1–4.3 are just an introduction on Ottinger's work. I compared Ottinger's paper with the authors’ one, and found that the former assume the presence of symmetric and anti-symmetric matrices while the latter explicitly make these conditions from arbitrary learnable parameters by using some linear algebra tricks.
- Minor comment: It is not too important, but in my opinion, numbering (some important) equations in Section 4 would be helpful (like Section 3).
***

5. (From Reviewer DqnY's suggestion) I'd actually be more interested in whether the network will be able to learn to set the friction term to zero to get close enough to Hamiltonian networks to have a significant advantage over a model without the inductive bias (NODE in the current experiments)
- I also think it is a very interesting and important point. I believe the authors can empirically show it by comparing NODE, Penalty, and GNODE (+ Hamiltonian NODE also?) for some simple conservative systems (e.g., undamped harmonic oscillator).

 Thank you for your continued excellent feedback. 

As noted in response to the previous reviewer, we have added an appendix regarding the details of our hyperparameter tuning, and added a comment in the manuscript that values correspond to a comparison of the "best" performance for each method.

As space is limited, we will add the suggested additional studies as appendices. If time permits, we should be able to include both the study demonstrating performance if data on the latent variables is in fact available, along with a study comparing NODE to our scheme for a Hamiltonian system (e.g. harmonic oscillator). We agree that it would be very interesting if strong enforcement of the constraints (compared to a penalty) leads to learning a more parsimonious description where S = 0 more tightly.

The discussion regarding the Pareto front is well-taken, and we appreciate the feedback that we could have made this distinction more explicit in the manuscript - we've elaborated on this important point in our updated draft. We've also numbered the suggested equations.

 Thank you for the continued insightful comments! We will definitely include hyperparameter search details in an appendix, and add more explicit discussion regarding the treatment of D.

Regarding the comparison to a neural ODE for a hamiltonian system - this is an excellent suggestion and one that hadn't occurred to us. If time permits, we will run this comparison and add as an additional appendix (there isn't space to include an additional study in the manuscript).

The authors introduce a hard-constrained, parameterized GENERIC framework for learning irreversible (dissipative) dynamics. To guarantee total energy conservation and entropy production, the authors use reformulation techniques for Poisson (reversible) and irreversible brackets that automatically satisfy the degeneracy conditions for GENERIC. As a result, the authors’ model can learn various dynamical systems more reliably and plausibly, compared to the baselines such as conventional neural ODE or soft-constrained GENERIC (that learns the degeneracy condition via penalty loss.) The authors validate their model for various benchmarks including damped nonlinear oscillators, two gas containers, and stochastic damped oscillators.   Originality: The main contribution of this paper is that the proposed model guarantees the degeneracy conditions (that should be satisfied for obeying the laws of thermodynamics) automatically while the prior work only guides such conditions via regularizers. This can be achieved by reformulating the reversible and irreversible brackets (Section 4.1 and Section 4.2). However, these reformulations are identical to the work by Oettinger ([42] in the paper). The authors pick some appropriate formulas from Oettinger’s paper, and then directly parameterize the formulas to enable them to learn. As a result, this paper can have originality from the perspective of bridging/combination of well-known techniques, but not definitely novel. 

One interesting point is that the authors distinguish the observable (position and momentum) and non-observable (entropy, …) states, and then learn the non-observables unsupervisely. I think this is an original point of this paper that might be important for real-world problems. 

Overall, I think this paper has moderate novelty/originality.

Quality: I think this paper is technically adequate. GENERIC framework and modified reversible/irreversible brackets are principled structures to learn and extrapolate dissipative dynamical systems. Experimental results seem to be solid, except for some minor concerns (see below comments 1 and 2).

Clarity: This paper is extremely well-written and easy to follow. The motivation is clearly presented. Prior works are well summarized (while some works are missing. See below comment 3). The preliminary and specific techniques are appropriately introduced. The experimental session sufficiently provides details on settings and hyper-parameters. Figures effectively show the experimental result.

Significance: Learning irreversible dynamical systems is definitely important for solving real-world problems. This paper provides a solid contribution to such important problems. 

I think this paper is good, especially in terms of clarity. However, the lack of originality and some concerns (see below) prevent me from recommending clear acceptance for this paper. My current evaluation of this paper is borderline.

Some concerns on the paper:

1. The authors used $10^{-4}$ penalty loss weight when they train the penalty-based GENERIC (line 191 and line 233). It seems $10^{-4}$ is too small to learn the degeneracy conditions. Why did you use such small penalty strength? What if one increases the penalty weight, e.g., $10^{-1}$ or 1?

2. For the two gas containers experiment (Section 5.2), the error bar / standard deviation is not reported, unlike the damped oscillator experiment (Section 5.1). Why?

3. There are some uncited prior works that try to model dissipative or chaotic dynamics, such as [1] Y. D. Zhong, B. Dey, and A. Chakraborty. Dissipative SymODEN: Encoding Hamiltonian dynamics with dissipation and control into deep learning. ICLR Workshop on Integration of Deep Neural Models and Differential Equations, 2020, [2] K. Course, T. Evans, and P. Nair. Weak form generalized Hamiltonian learning. NeurIPS, 2020.

***
Post-rebuttal: After reading the authors' response, now I think the paper’s contribution outweighs my initial concerns (especially regarding the originality). I increased my review score accordingly. The authors mention the limitation of this work, the scalable issue, in the conclusion.

 Thanks for the detailed rebuttal. The code release is much appreciated. For future reference, there's also the option of zipping the repository and submitting it as supplementary material. This would also make it significantly easier for reviewers to run your code as Anonymous Github doesn't support downloads. Apart from that, here are a couple items I'd like to follow up on:

>Experimentally, we observed that choosing D to have full rank initially can bias the training toward overly dissipative trajectories that lead to unnecessarily long training. We have observed the rank of a tensor to be something which changes slowly when training with a first order optimizer.

This type of additional detail is very interesting and I'd like to see it in the paper. I don't think the fact that this hyperparameter exists detracts from the method, but it is something a potential user would want to be aware of and it is currently glossed over in the paper. Beyond just noting this detail, I'd recommend including results across rank settings for D so that readers can observe the impact first hand. 

> General additional info on hyperparameter search


Thanks. That helps alleviate some concerns there. In terms of including it in the paper, it's completely understandable that there's not enough page space to handle this, but you can always include an appendix section documenting your hyperparameter search while including a sentence in the main body pointing the reader there. This is especially important for this paper where the experiment settings vary so significantly between tested methods. 

> While we suspect the reviewer is aware, for clarity of communication we point out that Hamiltonian systems are metriplectic; if one chooses S=0 in GENERIC one recovers a Hamiltonian system. If the question is however: does training with the flexibility to include dissipation for a non-dissipative system lead to worse results than a Hamiltonian neural network we would suspect the answer to be yes for sufficiently large extrapolation/forecasting, as any small amount of dissipation will eventually push trajectories onto from one constant energy surface to another. 

Yes, although the question isn't the comparison to Hamiltonian nets specifically (I completely agree with the authors about that comparison). I'd actually be more interested in whether the network will be able to learn to set the friction term to zero to get close enough to Hamiltonian networks to have a significant advantage over a model without the inductive bias (NODE in the current experiments). 

As the authors have noted above, it can often be difficult to train a neural network to consistently produce fixed-rank structure (though if you suspect it may be beneficial, it could help to parameterize it in such a way that zeros are easier to learn - GLU activations for the outputs of neural networks are one such option. I'm less familiar with recent work in inducing sparsity in parameters, but one older method is https://arxiv.org/pdf/1712.01312.pdf though there's probably less costly methods at this point), which makes it particularly interesting as to whether the optimization procedure can overcome the initialization rank. Of course, if we're treating the rank of D as a hyperparameter, then the question becomes moot as you could just set D=0 to remove dissipation.  

> We respectfully ask if the reviewer has suggestions for specific architectures that would be more appropriate beyond neural ODEs? We had considered e.g. resnets or RNNs but decided that they would be strictly worse than NODEs

Neural ODEs are a valid comparison, but they have a number of known issues as the authors point out in the paragraph starting on line 58. Significant work has gone into improving neural ODEs and it would make sense for the comparison to be against NODEs using current methods. In particular, I'd point to reference [25] as a comparison that should be included as the augmented state actually has a loose analogue to your inclusion of unobserved variables. At least one of the methods offering improved numerical stability would also be worth including as they've been around for a few years now and there are PyTorch implementations available. [27] was the one I was specifically thinking of here, but a more recent comparison would be valid too. It's worth noting that most of these techniques can also be used on the proposed method and could lead to stronger results there as well.

 We thank the reviewer for taking time to read the manuscript carefully and provide a thoughtful review. We respond point by point below with reviewer in quotes and response prefaced by (--)

- "One interesting point is that the authors distinguish the observable (position and momentum) and non-observable (entropy, …) states, and then learn the non-observables unsupervisely. I think this is an original point of this paper that might be important for real-world problems."
- -- We thank the reviewer for appreciating this point. While it is commonplace in the ML literature to find works learning dynamical systems from synthetic data, it is often overlooked that for physical systems it is not possible to measure e.g. entropy. We'd like to comment that for our unpublished experiments where one assumed measurements of non-observable states were available (e.g. from an analytic solution), the method performed extremely well - however we felt it inappropriate to include these results.

- " This paper is extremely well-written and easy to follow. The motivation is clearly presented. Prior works are well summarized (while some works are missing. See below comment 3)... There are some uncited prior works that try to model dissipative or chaotic dynamics, such as [1] Y. D. Zhong, B. Dey, and A. Chakraborty. Dissipative SymODEN: Encoding Hamiltonian dynamics with dissipation and control into deep learning. ICLR Workshop on Integration of Deep Neural Models and Differential Equations, 2020, [2] K. Course, T. Evans, and P. Nair. Weak form generalized Hamiltonian learning. NeurIPS, 2020.""
- -- Thank you for the nice comment. We have included the suggested references in our revised manuscript.

- "The authors used  penalty loss weight when they train the penalty-based GENERIC (line 191 and line 233). It seems  is too small to learn the degeneracy conditions. Why did you use such small penalty strength? What if one increases the penalty weight, e.g.,  10^-4 or 1?
- -- As discussed in a previous reviewers comment, we have adapted all hyperparameters for NODE, penalty, and GNODE via a grid search, including architecture width/depth, learning rates, and penalty parameters. The value presented corresponds to the best performing penalty results on test data. To understand why the penalty can't be too large, consider the asymptotic case of arbitrarily large penalty parameter. In this case the data loss ||X-X_data|| is insignificant, and training will return a valid generic model which doesn't match the data. This Pareto front is a critical weakness of physics-informed NNs which impose physics by penalty, as discussed in several excellent recent works (Rohrhofer, F.M., Posch, S. and Geiger, B.C., 2021. On the Pareto Front of Physics-Informed Neural Networks. arXiv preprint arXiv:2105.00862., Wang, S., Teng, Y. and Perdikaris, P., 2020. Understanding and mitigating gradient pathologies in physics-informed neural networks. arXiv preprint arXiv:2001.04536.).

- "For the two gas containers experiment (Section 5.2), the error bar / standard deviation is not reported, unlike the damped oscillator experiment (Section 5.1). Why?"
- -- As mentioned in another response, for the results presented in the manuscript we trained on a single trajectory of the exact solution. For the two gas problem training was sensitive to random initialization, either failing completely or training consistently. Since then, we have slightly changed the problem to train over minibatches corresponding to multiple trajectories with different initial conditions, and are now able to get statistically consistent results. We have updated the manuscript to reflect this and include error bars.

- "Originality: The main contribution of this paper is that the proposed model guarantees the degeneracy conditions (that should be satisfied for obeying the laws of thermodynamics) automatically while the prior work only guides such conditions via regularizers. This can be achieved by reformulating the reversible and irreversible brackets (Section 4.1 and Section 4.2). However, these reformulations are identical to the work by Oettinger ([42] in the paper). The authors pick some appropriate formulas from Oettinger’s paper, and then directly parameterize the formulas to enable them to learn. As a result, this paper can have originality from the perspective of bridging/combination of well-known techniques, but not definitely novel."
- -- Our architecture is indeed modeled off of Ottinger's analysis. We'd like to stress two points. Firstly, the bracket structure proposed by Ottinger is a conjecture and has not been applied either for discovering dynamics or for performing first principles forward simulation, and this work marks (to our knowledge) the first time these brackets have been used for any application. Secondly, it is perhaps a matter of personal taste/opinion what constitutes novelty, so we'd like to respectfully offer our perspective. One could argue by analogy that convolutional networks are just finite difference stencils. However, there are many details which go into adapting a mathematical structure/concept into an effective ML architecture which trains well. In the case of this work, our treatment of tensors via cholesky decomposition and anti-symmetrization of high-order stencils has not been used in the context of learning structure preserving dynamics. The novel use of Ottingers conjectured brackets eliminates the previously mentioned pathologies associated with the Pareto front in traditional physics-informed ML (Rohrhofer, F.M., Posch, S. and Geiger, B.C., 2021. On the Pareto Front of Physics-Informed Neural Networks. arXiv preprint arXiv:2105.00862., Wang, S., Teng, Y. and Perdikaris, P., 2020. Understanding and mitigating gradient pathologies in physics-informed neural networks. arXiv preprint arXiv:2001.04536.). The particulars of how we translate this idea into a machine learnable brackets has been critical to realizing an effective model, and we have not included in this manuscript the dozen or so other strategies which were not effective. Finally we will point out that in the literature, Hamiltonian, Lagrangian, and reversible networks have been published across neurIPS, ICLR, and ICML in the past two years. Looking back at the reviews for these papers on openreview, these have all been identified (correctly in our opinion) as being novel contributions with major impact despite the fact that the authors didn't invent Hamiltonian or Lagrangian dynamics, and feedback from other reviewers suggest that the idea is indeed novel and potentially impactful.
- -- In contrast to the machine learning world, physics historically builds upon centuries of results, and the majority of physics and mathematical works involve bridging ideas from disparate communities. We conjecture that particularly in the context of scientific/physics-informed machine learning there is a recent trend where authors repurpose classical mathematical/physics structures (e.g. group theory, complexes from exterior calculus/differential topology, tensor invariants) to design new ML architectures with better treatment of invariances. (see e.g. Ling, J., Jones, R. and Templeton, J., 2016. Machine learning strategies for systems with invariance properties. Journal of Computational Physics, 318, pp.22-35., Bergomi, M.G., Frosini, P., Giorgi, D. and Quercioli, N., 2019. Towards a topological–geometrical theory of group equivariant non-expansive operators for data analysis and machine learning. Nature Machine Intelligence, 1(9), pp.423-433., Trask, N., Huang, A. and Hu, X., 2020. Enforcing exact physics in scientific machine learning: a data-driven exterior calculus on graphs. arXiv preprint arXiv:2012.11799.).



 We thank the reviewer for their time and effort reading our work and providing thorough feedback. We address point-by-point below with reviewer comments in quotes and response prefaced by (--)

- "D is defined explicitly to be rank 1. This is not a requirement in the general framework and the matrix is too small for this to seem motivated by storage costs, but it seems like it would reduce the influence of the disappative term. It's hard to tell if this is theoretically or experimentally motivated."
--- A very insightful observation. This part of training appears to us to be an important inductive prior. Experimentally, we observed that choosing D to have full rank initially can bias the training toward overly dissipative trajectories that lead to unnecessarily long training. We have observed the rank of a tensor to be something which changes slowly when training with a first order optimizer. When training (both the examples in the paper and in our unpublished work) we begin with a small D and increase it until we get a good fit. We are currently working on a less labor intensive strategy in which we work with an SVD parameterization of D which allows a more direct parameterization of the rank via the singular values. We have added a comment to the paper addressing this aspect of training.

- "The configurations of the networks seem to vary significantly and this isn't explained. All tests seem to be done on small systems with correspondingly small networks. While the additional parameters of the penalty and GNODE methods may motivate smaller networks for fair comparison, the significant differences make it hard to tell if the better fit is due to the method or if it's just that neural ODEs are hard to train and the smaller networks led to better fits on relatively simple dynamics."
--- For all networks considered, we performed a grid search over hyperparameters and the reported architectures and results correspond to the optimal parameters for each method. This includes both large and small neural networks (both in width and depth) for the NODE, penalty, and GNODE architectures, and so we are confident that we are performing a fair comparison in this regard and that results are not overfit, as we performed hyperparameter optimization with a standard training/validation/test split.

- "The penalty comparison does not feel very robust. One value of the multipliers is reported without indication as to how they were chosen. It seems odd that the penalty method would completely underperform the black box approach as is seen in figure one and without the error bars, I'd guess it was a bad optimization run. Due to the small number of runs, it still seems possible that they were influence by an outlier.
These seem like one particle systems, so if the target of interest is mapping the trajectory, then I'd like to see results reported on the discrepency between the true and targeted trajectories and not just H as the black box method is not trained to optimize performance on H."
--- As reported in the previous response, the penalty multipliers were included in the hyper-parameter optimization process and correspond to the best performance we were able to obtain. For the result in figure 1, the penalty method outperforms the black box approach on the test and validation data. We share the reviewers surprise that when extrapolating/forecasting, the method underperforms the NODE. This is precisely the point we are trying to make however - that unless the degeneracy constraints are enforced exactly, in extrapolation there is no guarantee, and you can see that the energy is growing beyond the condition. This is consistent with plot 1b, where it can be seen that while dS/dt \geq 0 approximately in the test/validation set, as soon as time progresses into the test/extrapolation regime dS/dt < 0, with discrepancies as large as the physical dissipation. This suggests that the violation of the constraint in the penalty method is sufficient large to do substantial artificial work on the system. 
--- We appreciate the suggestion to consider there may be an outlier. While our experience running cases not included in this paper has prompted us to view the increase in energy as expected, we appreciate that 5 runs may not be a convincingly large enough statistical sample and was chosen due to time constraints. We are running more experiments now to update the results in the final camera ready submission to give stronger evidence that the presented trends are statistically significant.

- "Finally, there were also a few discrepancies between what was stated in the checklist and what appeared in the paper. First, I do not think it is reasonable to take credit for releasing code when the code is not included in the supplementary material. If the code isn't included in the submission, then the answer to "did we release the code?" is no regardless of future intentions."
--- We thank the reviewer for pointing this out. We had mistakenly interpreted the anonymity requirements as precluding us from including a link to the code in the review phase. After doing some research we have found that we are in fact able to share an anonymized repository, which may be found at the URL (https://anonymous.4open.science/r/generic-nn-F64E). The final submission will include a link to a more polished github page including references to datasets where we are using this code which we were unable to include without revealing our identity.

- "Error bars are present, but wide and it's unclear if these are the extremes or a number of standard deviations. These are also smaller experiments, so it seems as though more than 5 runs should be feasible particularly when the deviation seems to be so large."
--- Thank you for pointing out that we did not define the error bars, which are of a single standard deviation. As mentioned previously, we are currently running more experiments to provide a more statistically significant sample.

- "Overall, the paper is solid and proposed a promising direction, but in it's current version, the experiments are hard to put significant stock into. It would make for a stronger paper if larger systems were explored or more consistent configurations were used. Additional comparisons would also be valuable as vanilla neural ODEs are not a very strong baseline. It would also be interesting to see how the model preserves on non-metriplectic systems (Hamiltonian or unknown systems from collected data) to see if the technique is capable of adapting in those settings (ie, it should be able to represent Hamiltonian dynamics, does it outperform black box ODEs on those tasks?) though obviously it would not be expected to outperform methods designed for those settings."
- -- We respectfully ask if the reviewer has suggestions for specific architectures that would be more appropriate beyond neural ODEs? We had considered e.g. resnets or RNNs but decided that they would be strictly worse than NODEs. We were also looking for an architecture which admits a similar training (i.e. minibatching over short trajectories) to avoid a complicated comparison of methods trained differently. In our opinion, NODEs provided the cleanest comparison and have similar architectures.
- --  These are great suggestions. While we suspect the reviewer is aware, for clarity of communication we point out that Hamiltonian systems are metriplectic; if one chooses S=0 in GENERIC one recovers a Hamiltonian system. If the question is however: does training with the flexibility to include dissipation for a non-dissipative system lead to worse results than a Hamiltonian neural network we would suspect the answer to be yes for sufficiently large extrapolation/forecasting, as any small amount of dissipation will eventually push trajectories onto from one constant energy surface to another. 
- -- The suggestion to consider systems which do not explicitly admit a GENERIC form is another strong suggestion. For the current paper, our pedagogical strategy to handle space constraints was to present the scheme in an abstract form with minimal application jargon. Working with simple systems which admit analytic solutions allows an explicit comparison - we took this position following the example of earlier works accepted to neurIPs in structure preserving discovery of dynamics. For example, ("Hamiltonian Neural Networks" Greydanus et al, NeurIPS 2019) pursued similar toy problems. We are currently applying this framework to problems in material science and fluid mechanics, but felt that the necessary background to explain treatment of data, application-specific jargon and inductive biases, etc would detract from reaching a broader audience. 

 We thank the reviewer for their valuable insight, particularly regarding suggestions for the most state-of-the-art references that have slipped past us. We have added the suggested references to our revision. Our response to individual questions are below prefaced by (--)

- The experiments demonstrate that GNODE outperforms black-box Neural ODE as well as the penalty-based approach. However, in this context the update frequency for the unobserved state estimation appears to be a critical factor. Therefore, the authors should report on how the performance of GNODE and the penalty-based method varies with changes in that update frequency. 

- -- Thank you for the excellent comment. We focused on performing a grid search over traditional parameters (width/depth of networks, learning rate, etc) and set the update parameter via the following argument. The computational cost scales directly with the update frequency, and so to limit the computational cost of training with limited time before the submission deadline, we increased the parameters until both methods worked to keep it at its smallest value. We agree that this is likely an important parameter and are currently performing a follow-up study on its effect.


This work proposes a neural network architecture and training approach that leverage theories of dynamical systems for introducing a novel parametrization which splits the underlying time evolution dynamics into two parts – an energy conserving part, and as irreversible part. This allows one to use this framework for learning dynamics with dissipative states (for example, a damped oscillator). To accommodate systems with dissipative states that may not be directly measured, this work introduces a training scheme which alternates between system identification (assumes knowledge about both measured and non-measured states and tries to learn the underlying dynamics) and state estimation (assumes knowledge about the underlying dynamics and tries to estimate the non-measured states from the measured states). The proposed framework has been demonstrated on 3 example systems. To conclude, this paper addresses a very topical problem and the details have been provided with sufficient clarity.   ### Originality: 

The overall idea is novel and very interesting. 

However, some relevant prior works are missing from the related work section. *Dissipative SymODEN: Encoding Hamiltonian Dynamics with Dissipation and Control into Deep Learning* (DeepDiffEq Worksho at ICLR’2020) and *OnsagerNet: Learning Stable and Interpretable Dynamics using a Generalized Onsager Principle* (arXiv: 2009.02327) have focused on incorporating energy-dissipation into such physics-governed neural networks. The authors should properly refer to these papers and highlight the similarities/differences. *Symplectic ODE-Net: Learning Hamiltonian Dynamics with Control* (ICLR’2020) and *Simplifying Hamiltonian and Lagrangian Neural Networks via Explicit Constraints* (NeurIPS’2020) are also relevant references in this context. The authors may refer to *Benchmarking Energy-Conserving Neural Networks for Learning Dynamics from Data* (arXiv:2012.02334) for comprehensive details about the various existing work that focus on incorporating Hamiltonian/Lagrangian-based inductive bias. 

Also, Lagrangian-based neural networks precede Hamiltonian-based neural networks; *Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning* was published at ICLR’2019, almost 8 months before *Hamiltonian Neural Network* was presented at NeurIPS’2019 – this chronology contradicts the phrase “followed by” on line 68. 

Lastly, references [21] and [36] refer to the same paper *Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning* – the authors should fix this.

### Quality: 
The experiments demonstrate that GNODE outperforms black-box Neural ODE as well as the penalty-based approach. However, in this context the update frequency for the unobserved state estimation appears to be a critical factor. Therefore, the authors should report on how the performance of GNODE and the penalty-based method varies with changes in that update frequency. Also, why $n_{update}$ was chosen to be the same for both of these methods? 

### Clarity: 
This paper is very well-written which helps in conveying the message in an effective way. Also, it provides all the necessary details. 

### Significance: 
In the last few years, a fast growing volume of work has focuses on incorporating Hamiltonian of Lagrangian formulation based inductive bias while learning dynamics of data. However, most of these papers (except *Dissipative-SymODEN* and *OnsagerNet*) encode *energy conservation* in the architecture of neural networks. This paper expands that line of work by incorporating energy-dissipation into a such energy conserving neural networks. As almost all real-world physical systems have some-form of energy dissipation, this reviewer believes that the proposed approach would be relevant for many practical applications. The authors have adequately addressed the limitations of the submitted work.

Meta-review: This is a very good paper and a clear accept. All reviewers agree that there is a novel methodological contribution that extends neural ODE approaches to irreversible dynamical systems. The parametrization of brackets cleverly preserve structures of the dynamical system, leading to exact preservation of thermodynamical consistency and better generalization. The numerical experiments are good, but can be further improved. The paper is written nicely and the main ideas are explained well. 

Decision: accept