Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper proposes to model the evolution of coupled homogeneous dynamical systems by combining neural ODE and message passing in graph neural networks. A NODE is used to model a single subsystem and message passing is used to describe the interactions of the various subsystems. Simulation results are reported on a number of bechmarks.
strengths_and_weaknesses: Strengths:
- The paper is well-written and easy to follow
- The idea is intriguing and with wide applicability

Weaknesses:
- Experimental results are not compared against baselines. There is a some comparison on 2 systems with the L2S system, but I am not sure why they did not report results on all systems. This makes it difficult to assess the quality of the results.
- Some more technical details on the dynamical systems setting are needed.
- The methodological novelty is clearly limited (from a technical point of view), as the contribution consists of combining two existing methodologies.
questions: - I checked the appendix but I did not find the setting for the parameters of the Lorenz system (and the others). Maybe I missed this detail, but this is a very relevant detail. In fact, the Lorenz system can behave in very different ways (e.g. show chaotic behavior or not) depending on the setting of its parameters. This will have a significant effect on the time horizon of the predictions and it is something that must be discussed.

- Are 2.5 seconds enough to fully explore the Lorenz attractor? Can you report some more details/plots showing the quality of the predictions?
limitations: The main and obvious limitation is the focused applicability to coupled homogeneous systems. This aspect has been clearly discussed in the paper.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes a message-passing neural ODE method to learn the simulations of homogeneous systems. In this method, smaller sub-systems are individually modeled by a neural ODE. The same weights are shared among each sub-system. A message-passing mechanism is introduced to model the interactions between sub-systems.

Experiments were made, including:
- Test and evaluate the method on five different systems. The method shows good results in these systems.
- Experiments about finetuning (and zero-shot generalization) the model to a system with more nodes, higher complexity, and changed graph structures. And the method shows good generalization abilities. 
- Experiments about the effects of message passing and message size.
- Comparison with the L2S baseline (proposed by Sanchez-Gonzalez et al. [31])

strengths_and_weaknesses: Strengths
The proposed method shows novel points compared to the regular neural ode methods and the regular graph neural network methods.
The authors made a fair amount of experiments to show the accuracy of this method, the effects of message passing, and the generalizability of the proposed model.

Weaknesses
The systems that this paper makes experiments on are a little simple and small - although the underlying ode can be complex, each sub-system usually contains only several degrees of freedom (e,g, only several particles), and the number of sub-systems in a whole system is also relatively small. On the contrary, as mentioned in the paper's "related work" section, there are other learning-based simulation methods that have made successful results on more complex systems, such as clothes, fluids, etc. The authors may at least have some discussions about whether (and how) this paper can be applied to more complex systems.
questions: Did you specifically choose "Tsitouras 5/4 Runge-Kutta (Tsit5)" as the integration method for solving ODEs. Do you think using other methods would significantly affect the results or not?

Different timesteps (0.05s, 0.1s, 0.2s) were used in different experiments. Were there specific reasons to choose these values? Would the method stay robust if different (especially larger) timesteps are used in the training data? Would larger timesteps cause significant delays in message passing?

In the finetune results (figure 3), for some cases (e.g., figure 3a, d, e), finetune has much better results than training from scratch. Is the model much harder to converge when the system contains more sub-systems? If a smaller system is not given, but the model needs to directly learn on a relatively large system, can it still learn well?

In "Table 2", is "Gene Dynamics" 4*4 or 8*8, is "Quadrotors" 3 or 6-quadrotor system?

In line 270, the number of epochs taken to reach minimum test error is used in the evaluation metrics. For the proposed method and L2S baseline, do they take a similar or very different amount of times for one single epoch?



limitations: The authors discussed the limitations of this paper:
- The method can only be applied to homogeneous systems;
- The underlying graph structure of the system needs to be given;
- Potential issues such as delays in communication between nodes are not considered in this paper.
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: In this paper, the authors tackle the problem of generating data from a modular system, where each submodule is assumed to be identical, e.g. they follow the same dynamics. The authors introduce a novel extension of neural ODES, which they call message-passing neural ODES (MP-NODE),  where the state is augmented with messages, allowing for the individual submodules to interact with one another.
strengths_and_weaknesses: # Strengths

MP-NODE is a simple and elegant solution, which I am a huge fan of. The inductive bias of the proposed approach is fantastic as assuming that each submodule follows the same ODE allows for weight sharing, substantially reducing the number of parameters **and** easily allows for generalization. In fact, this is the coolest thing to me; that you can easily deploy the method on a system with more nodes than it was trained on *without* increasing the number of parameters!

I also thought the large variety of experiments was very compelling and really showed the possibilities of this approach over a wide range of domains.

# Weaknesses

The one disappointing thing is the lack of error bars! Especially given how close some of these training curves are, errors bars are essential!

questions: I have a condition concerning the initial conditions for the messages. How are they set? And how does the initial condition affect the performance of the method?
limitations: The limitation section was very honest which I very much appreciated!
review_title: null

Meta-review: metareview: The manuscript describes a novel combination of message passing / graph neural network ideas with NODE. Reviewers agree that this is an innovative contribution. Authors have provided various non-trivial experiments to demonstrate its performance.
confidence: Certain
Decision: Accept