Poisson-Dirac Neural Networks for Modeling Coupled Dynamical Systems across Domains

Authors: Razmik Arman Khosrovian, Takaharu Yaguchi, Hiroaki Yoshimura, Takashi Matsubara
Primary area: learning on time series and dynamical systems
Keywords: neural ordinary differential equations, coupled system, Poisson system, Dirac structure

Abstract:
Deep learning has achieved great success in modeling dynamical systems, providing data-driven simulators to predict complex phenomena, even without known governing equations. However, existing models have two major limitations: their narrow focus on mechanical systems and their tendency to treat systems as monolithic. These limitations reduce their applicability to dynamical systems in other domains, such as electrical and hydraulic systems, and to coupled systems. To address these limitations, we propose Poisson-Dirac Neural Networks (PoDiNNs), a novel framework based on the Dirac structure that unifies the port-Hamiltonian and Poisson formulations from geometric mechanics. This framework enables a unified representation of various dynamical systems across multiple domains as well as their interactions and degeneracies arising from couplings. Our experiments demonstrate that PoDiNNs offer improved accuracy and interpretability in modeling unknown coupled dynamical systems from data.

Decision: accept (poster)
Reviewer scores: 5, 8, 6, 8, 6

Reviews and discussion:
<thread>
creation date:
1730458546959

thread_type:
official_review

<thread_object>
creator: reviewer_pxb8

<content>
summary:
This paper introduces a novel neural network architecture, the Poisson-Dirac Neural Network (PDNN), which combines Hamiltonian and Poisson formulations. The PDNN leverages the Dirac structure to model coupled dynamical systems across various domains.

strengths:
1) The paper provides a solid theoretical foundation for the proposed PDNN architecture.

2) The authors conduct extensive experiments to demonstrate the effectiveness of the PDNN in various applications.

weaknesses:
1) While the paper's theoretical contributions are significant, additional background material, such as differential geometry, etc, in the supplementary information would enhance its accessibility to a broader audience.

questions:
1) While the PDNN shows promising results, it would be interesting to compare its performance with state-space model (SSM) based neural operators on the datasets used in this paper.

2) To make the paper more accessible to a broader audience, the authors could consider adding an appendix to provide additional explanations and visualizations of the key concepts and techniques.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730469002237

thread_type:
official_review

<thread_object>
creator: reviewer_f183

<content>
summary:
This paper introduces a new method for stable deep-learning based modelling of dynamical systems. Neural ODEs were originally found to have significant limitations in modelling dynamics long-term due to a lack of physical guarantees (e.g. conservation laws) leading to error accumulation. To address this, Hamiltonian NNs incorporate Hamiltonian mechanics (modelling the hamiltonian through a NN). This paper contributes a generalization of Hamiltonian NNs; Poinsson-Dirac Neural Networks (PoDiNNs) to combine both port-hamiltonian and poisson systems for mechanics, enabling learning of coupled dynamical systems across multiple domains, their interactions and degeneracies. This  constitutes a unification of previous works that address/leverage specific properties of dynamical systems like energy dissipation and external inputs. Authors show that this approach moreover allows for the identification of the specific coupling patterns between interacting dynamical systems. The experimental results show clear and consistent improvements over previous approaches in a range of systems with degenerate dynamics (e.g. constraints), dissipation and external inputs.

strengths:
- The paper is well-written and concise. The modelling choices made by the authors are well-motivated; the incorporation of Dirac structure into a DL-based solving method for mechanics has a lot of possible benefits regarding guarantees and interpretability of the learned functions.
- The experimental results are very convincing; in the chosen experimental setups the proposed framework outperforms all baseline methods consistently.
- The experiments on identifying coupling patterns are fascinating, these results show that the proposed framework is indeed able to recover the underlying coupling in a very interpretable manner.

weaknesses:
- The paper is quite dense, making some of the reasoning and motivation hard to follow. Although in places authors elaborate on their arguments with examples, I think it would be good to provide some more visual guidance. For example, figure 1 contains a very high-level overview of the concept of PoDiNNs, but I think it would be good to expand this diagram and clearly indicate what parts of this system the method proposes to replace with learned functions (NNs) and relate it to e.g. eq 5, def 3. I think this would greatly help with readability.
- The experiments chosen by the authors are examples of systems that fit the modelling criteria set by the authors for their framework. In these settings, the proposed framework outperforms previous approaches. However, I think it would also be good to consider/experiment with settings that do not exhibit e.g. degeneracy just to gauge how well this generalized method works in the settings that the originally proposed HNN and LNN are validated on. Would it for example be possible to show performance of your model vs baselines on eg double pendulum (as in HNN or LNN paper)?
- For completeness I think it is important to compare your method against the baselines also in terms of computational complexity / overhead. Currently, no details on the computational complexity or time complexity on either training or inference of the proposed method are given. Please provide these details.
- An arguable weakness of the method is the reliance on specifying problem-specific constraints, i.e. the number of components is a model hyperparameter, but as shown in the last experiment / Appx E it is important to specify this to at least the number of underlying components in the system, or performance will be significantly impacted. Overspecifying is possible of course, and does not seem to have negative impact, but induces additional computational complexity.

questions:
See above.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730477438768

thread_type:
official_review

<thread_object>
creator: reviewer_rzpg

<content>
summary:
This work studies the application of deep neural networks for data-driven physics simulation. Over the recent years, several neural network simulation models have been proposed among which Hamiltonian Neural Networks (HNNs) and Lagrangian Neural Networks. Although these models are formulated along physics principles and consequently enjoy improved stability of their simulations, two key limitations remain: (1) they focus on mechanical systems, as opposed to for example electric circuits or magnetic fields, and (2) they consider the system as a single monolithic entity. To alleviate these limitations, this work proposes Poisson-Dirac Neural Networks (PoDiNNs), which unifies the port-Hamiltonian and Poisson representations, and explicitly represents the coupling between internal and external components. The empirical evaluation shows that PoDiNNs enjoy stability over longer simulation horizons and in general achieve lower errors than Neural ODE or variants of HNNs.

Unfortunately, I am missing too much of the mathematics and physics background that is required to understand the paper, so I am unable to provide an informative review.

strengths:
**S1:** The empirical results show that PoDiNNs achieve lower errors and provide stable predictions for longer than Neural ODEs and HNN variants. As such, the method seems effective for the tasks for which it was designed.

weaknesses:
**W1:** The content might be difficult to digest for the audience of ICLR, since a specific mathematical and physics background is required to understand the work. In my impression, most ML researchers will lack this background, and also a substantial part of the AI4Science community might find it difficult to understand the paper.

questions:
-
</content>
</thread_object>

</thread>
<thread>
creation date:
1730603117885

thread_type:
official_review

<thread_object>
creator: reviewer_zhg9

<content>
summary:
This work targets the dynamical system identification using observation data, which is a hot topic and essential application. The key differences with respect to existing works are clearly stated: 1) identifying unknown physics rather than predefined symbolics, 2) considering the coupling behaviors in the system or interactions when the problem covers multiple domains rather than single mechanical systems.

strengths:
•	As the reviewer summarizes above, key limitations are correctly identified such that the contributions of this work are clear. To the best of the reviewer’s knowledge, this work is new.

•	The theoretical analysis is solid, where the definitions and theorems clearly show how the Dirac structure encapsulates internal and external component couplings. Moreover, the corresponding examples of different dynamical systems are well-explained to differentiate the proposed method from existing methods like HNN and NODE.

•	The results on multiple systems look promising, where various experiment scenarios and evaluation metrics are comprehensive to validate PoDiNNs’ capabilities.

weaknesses:
Some technical details, as well as the claimed capabilities, are unclear, which might be because the reviewer is unfamiliar with all kinds of multi-domain dynamical systems. The confusions are listed below.

•	The capability to deal with multi-physics problems is claimed several times in the paper. Specifically, Remark 1 explains the representation of inter-coupling using a bivector element (which is also an NN, right?) Remark 4 with Table 2 briefly demonstrates the scenarios to capture coupled physics in multiple domains. The reviewer would like to know how PoDiNNs represent such interactions.  Is it the same way as the traditional simulation tool, e.g., through iterative refinement of two (or more) simulations of a single domain or system? 

•	The proposed work targets unknown physics/dynamics, which is quite challenging as there are no predefined physical symbolics in PINN-alike works. How to ensure the PoDiNNs capture the correct physics without causing overfitting problem or continuous good performance in extrapolation?

•	Moreover, PoDiNNs focus on behaviors that may not be captured by generic models, e.g., NODE, which seem to need intensive resources. Especially, the couplings in multi-physics usually require heavy computation in traditional simulations. The more fine-grained, the heavier. What is PoDiNNs’ capability in this aspect?

•	For the last paragraph of Sec. 3.4, an example of electric circuit is used. Could the reviewer further explain with more details: why ODEs or using NODE alone cannot capture the current flow and balanced voltage level? Subsequently, how does PoDiNN mitigate the issue of limited representation? A toy example with mathematical derivations or diagram will be helpful.

questions:
Please refer to the bullet points in Weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730636763057

thread_type:
official_review

<thread_object>
creator: reviewer_h9ym

<content>
summary:
The authors present a new architecture, specifically they integrate some clever ideas of Dirac structure in order to unify the port-Hamiltonian and Poisson formulations from geometric mechanics.  Both nice innovations for physics-constrained neural networks.

strengths:
The integration of physics-based principles is always welcome in the dynamical systems world.  The authors have a very nice contribution to make here potentially as the integration of physics principles into neural networks is very important.

weaknesses:
The models used seem to be all linear: which begs the question about simple linear model regressions such as dynamic mode decomposition, dynamic mode decomposition with control and time-delay embedded DMD which models missing/coupled physics.  These more baseline (non-NN) methods are simply not talked about or considered and I think they should be.

There are statements that are simply not true:  "two key limitations remain in modeling dynamical systems, especially those described by ordinary differential equations (ODEs). The first limitation is the narrow focus on mechanical systems."  This suggests the authors don't know the field well and it is a concern.  People are modeling all kinds of dynamical systems with ML/AI architectures well beyond mechanical systems.

Further: "The second limitation is that most methods treat the system as a single, monolithic entity."  This is also not true.  Many people are working on coupled systems where time-delay embeddings are often used to gather information for missing, coupled and unmeasured variables.  Again, it is odd that these statements exist in the paper which suggests the authors are not aware of the great body of work on model discovery and dynamical systems methods with ML/AI for systems which are not mechanical and which indeed have coupling.

questions:
The models considered all seem to be linear.  Is that correct?  If so, more standard system ID or linear methods should be considered instead of all this sophisticated ML/AI architectures.  

Can this generalize to nonlinear models?  Or simply be applied to nonlinear models with success?  Although you did Fitzhugh-Nagumo and  Chua's model, these are only "slightly" nonlinear and system ID models can work pretty well with those.

What have you not considered non-neural network methods like DMD... or system ID methods like mamba/S4 not been compared?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732567087567

thread_type:
official_comment

<thread_object>
creator: area_chair_fcti

<content>
title:
Last day for reviewers to ask questions to the authors!

comment:
Dear reviewers,

Tomorrow (Nov 26) is the last day for asking questions to the authors. With this in mind, please read the rebuttal provided by the authors, as well as the other reviews. If you have not already done so, please explicitly acknowledge that you have read the rebuttal and reviews, provide your updated view _accompanied by a motivation_, and raise any outstanding questions for the authors.

**Timeline**: As a reminder, the review timeline is as follows:
- November 26: Last day for reviewers to ask questions to authors.
- November 27: Last day for authors to respond to reviewers.
- November 28 - December 10: Reviewer and area chair discussion phase.

Thank you again for your hard work, 

Your AC
</content>
</thread_object>

</thread>


Metareview:
metareview:
The reviews for this submission were leaning towards acceptance of the paper. Among other things, reviewers highlighted as strengths the use of well-explained examples of different dynamical systems that allow to differentiate the proposed method from existing methods, and which demonstrated convincing results in terms of lower errors and more stable predictions over longer time horizons. The presentation of the paper received both praise and requests for improvement, with the accessibility of the paper being the most important improvement area. Several reviewers raised questions about the computational cost of the proposed method compared to baselines, which the authors answered transparently during the rebuttal. The reviewer with the most critical review requested comparisons against other methods such as system ID methods, but has unfortunately not engaged with the authors anymore after their rebuttal, despite requests to do so. Despite one other reviewer agreeing that the paper would be strengthened by a comparison to these types of methods, this reviewer maintained their recommendation that the paper is marginally above the acceptance threshold. Taking all of this into account, the recommendation is to accept this paper. The authors are strongly encouraged to include the additional clarifications that were helpful during the rebuttal in their camera ready version of the manuscript.