Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This manuscript proposes a method to infer the dynamics of interacting objects while incorporating uncertainty estimates. They use a gaussian process based ODE method, while modeling the dynamics for a set of interacting variables as a combination of independent and interacting terms. The authors also consider the case where they have partial observations of the system. They show the utility of their method on the bouncing balls and charges datasets.
strengths_and_weaknesses: The paper is quite clearly written. The authors detail out the method and the relationship to other methods succinctly. The method they develop is an extension to the GP-ODE method, with additional constraints to take into account the interactions and independent variables. As such, it has limited originality. This also limits the significance of the paper.

While the authors show the advantages of using their method as opposed to the GP-ODE (lower MSE and better ELL) on both datasets, it is unclear why the I-GPODE would have a better ELL than the GPODE, since the main change that the authors are proposing is an additional structure. Moreover, their method does not seem to be as accurate as the GDE (I-NODE) method in the MSE (Tables 3 and 5), which we also see in Figure 2. This dampens the enthusiasm for the paper.

The test performed in Section 5.2.3 is a good one - they first learn the dynamics of three interacting objects, then simulate just one object, and see that competing methods do not learn the individual dynamics as well. This may be because the authors are explicitly learning the individual as opposed to the interactive dynamics.
questions: Please include figures such as in Figure 2 for the comparison between GPODE and I-GPODE. It is unclear why the ELL is much worse using GPODEs.

While the test in Section 5.2.3 is well performed, it would be good to list how I-GPODE does on three interacting objects as well as on the individual objects.

It would have been good to show the results of this method in a real-world scenario.
limitations: The authors have a limited discussion of the limitations and potential negative societal impact of their work.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors introduce a GP interaction model (I-GPODE) which is capable of modelling the continuous time dynamics of a set of interacting objects, whilst providing a robust quantification of uncertainty using latent GP-ODEs. The kinematics of each individual object and the interactions between all objects present are modelled separately; this decomposition allows the complex continuous-time dynamics to be handled by the model. The work leverages recent advances in neural ODEs and efficient sampling from GPs using pathwise updates. Results presented are better or comparable than competing methods in terms of MSE, but considerably better than all competing methods in terms of ELL.
strengths_and_weaknesses:  ### Strengths
The I-GPODE introduced in this work is a novel contribution, addressing the problem of *uncertainty-aware* modelling of interacting object dynamics. The authors do benchmark against competing models which can also be applied to this problem setting, and some of these approaches do outperform the I-GPODE in terms of MSE in some cases, however the ELLs presented throughout the results section make a very compelling case that the I-GPODE significantly improves upon existing work in terms of robust uncertainty quantification in this setting.

The introduction, background and related work sections give a very good overview of where this work sits in terms of the wider research area; Table 1 also helped lend a lot of context. The generative model itself and the variational inference scheme employed are both laid out in a clear and concise manner; Figure 1 does a good job of quickly outlining the key ideas before you even get to the main contributions in Section 3.

The submission is well written and presented, easy to follow and technically sound. Overall, this is a great piece of work which addresses an open problem, therefore I recommend acceptance.

### Weaknesses
Whilst there is discussion of how the model complexity scales with the number of evaluation time points, I think some discussion of how the number of objects considered affects both the performance metrics and the computational feasibility of this approach (e.g. computational complexity and/or runtimes) would have been interesting to see. 

questions: As mentioned in the Weaknesses section above, how are the performance metrics and computational performance affected by an increasing number of objects? What is the upper limit in terms of what is feasible to model? I think a relatively brief set of results considering a scenario where $N_a \gg 5$ would make a nice addition to the results section. 
limitations: There is some discussion of the limitations of the work (e.g. overfitting occurring if the dimensionality of the ODE state is larger than necessary), however I think this could have been expanded a little. There is no discussion of negative societal impact, although I don’t feel that this is an issue for the subject matter in question.
review_title: null

final_score: rating: 9: Very Strong Accept: Technically flawless paper with groundbreaking impact on at least one area of AI/ML and excellent impact on multiple areas of AI/ML, with flawless evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors propose a novel uncertainty-aware modeling of continuous-time dynamics of interacting objects. They model the trajectory of interacting systems using latent GP-ODEs by incorporating the interaction term explicitly into the differential function, and use inducing points to scale their variational inference. They show on several datasets that their method outperform other state-of-the-art approaches, that either don't model interactions between the systems explicitly or aren't uncertainty-aware.
strengths_and_weaknesses: This is a great paper, as it is the first proposed method for uncertainty-aware modeling of continuous-time dynamics of interacting objects. This is an important contribution to the field of latent variable modeling. The paper is very clear, well-written, and results support the claims made in the paper.
questions: I really enjoyed the paper and the proposed model, and have only a few minor points.

In tables 4 and 5, why is I-GPODE-L performing so poorly (I-GPODE-L-S > I-GPODE >> I-GPODE-L)? 
The improvement of I-GPODE-L-S over I-GPODE is discussed but I would like to have some explanation on why I-GPODE-L is performing worse than I-GPODE. 

In Figure 2 and Appendix Figure 5, why do the shaded lines not follow the inferred trajectory? 
The links to citation and figures are not clickable, I am not sure if this was intentional or not. On line 123, you also forget to include the section number in "Finally, please see Section for an investigation"

Equation 6 with generalization to multiple sequences in the appendix
In the Appendix table, I would recommend highlighting the best result in bold, as is done in the main paper. The tables in the appendix are quite hard to read.

I would recommend citing [Learning interpretable continuous-time models of latent stochastic dynamical systems, Duncker et al.., ICML 2019], that is the first paper proposing GP-SDE. This work was prior to [Scalable Inference in SDEs by Direct Matching of the Fokker–Planck–Kolmogorov Equation, Solin et al., NeurIPS 2021] and differed in their Inference algorithm. 
limitations: I don't see any potential negative societal impact of their work.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The paper takes a Bayesian approach to model dynamical systems with Gaussian processes and combines it with a message passing modeling of the dynamics.
strengths_and_weaknesses: Strengths:
- The presentation is pretty clear.
- The method makes sense as it combines two well established approaches.

Weaknesses:
- Limited novelty. It is a relatively small modification of GPODE.
- Limited significance. I-NODE achieves better results so main gain is calibration which is a small gain.
- Results are presented in a wasteful, table 2 and 3 should be joint together for better clarity.
- The claim in the first line in the abstract that "for the first time uncertainty-aware modeling of continuous-time dynamics of interacting objects" is false. E.g. Xu et al "A Bayesian-Symbolic Approach to Reasoning and Learning in Intuitive Physics".

Small remark: you have $p(Y_{1:N}|\cdot)$ in eq. 6 which isn't clear
questions: Can you show that the functions you learn $f_s,f_b$ learn the write functions in the sense that if you disable the interactions then $f_b$ alone would return the correct dynamics?
limitations: NA
review_title: null

Meta-review: metareview: The paper proposes a Gaussian-process approach to modelling nonlinear dynamical systems with multiple interacting objects. 

The main strength of the paper is the empirical performance in terms of uncertainty quantification, backed up by clear writing and logical experimentation. Some reviewers were concerned about novelty, but I consider that additional structure proposed here to be a significant and interesting improvement on the GPODE. 

One reviewer raised a question about computation complexity which was covered in the discussion. Please add this discussion to the manuscript. 

I agree with reviewer gdCd that "The claim in the first line in the abstract that "for the first time uncertainty-aware modelling of continuous-time dynamics of interacting objects" is false" please modify the manuscript appropriately. 

Other than those edits, this paper seems to have excited the reviewers and I'm in agreement that this presents an interesting approach that I expect others to pick up and build on. 


confidence: Certain
Decision: Accept