Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: The paper proposes a novel method for modelling dynamical systems over graphs. The main idea investigated by the authors is to combine Graph Neural Networks together with approximate Koopman embedding. The GNN encodes the input graph to what the authors call "object-centric embedding", whose concatenation over all objects is defacto the approximate Koopman embedding of the system.
One of the key contributions is the reduction in parameters, by assuming that the interactions between different objects in the Koopman space are limited to some fixed number of types, or in other words given the object-centric embedding the Koopman matrix is a block matrix, where each block can only be one of K matrices. In this way the number of parameters is fixed and does not scale with the number of objects, compared to the naive way where it will scale as N^2. In addition to the dynamical modelling the paper adds an extra linear-"control" input in the Koopman embedding space which to affect the dynamics of the system and allow for modelling systems where there is external control being applied. The models are than compared on three small scale tasks, showing better results in mean squared error prediction compared to the three baseline approaches. Additionally, when used for controls on the environments the methods outperforms the one baseline method it is compared to. 


I'm quite borderline on whether the paper should be accepted or rejected, but currently I'm leaning towards a rejection. The main reason for this decision is that in my opinion the experiments presented are somewhat limited with respect to the baselines used and I have some reservations regarding the results presented for IN and PN discussed below.


Detailed comments on paper: 

1. I personally like the main idea of the paper, which is to use previous results from approximating the Koopman operator and combining it with GNNs for more accurate physical modelling of object-object interactions. Additionally, the idea of reducing the parameters is quite important. 

2. Linear control theory - although it is quite natural to add the control as a linear affect in the latent space, and this has been done numerous times before in the literature, I don't recall there to be any theory on Koopman embedding when there is a control signal. Additionally, if the policy that has been used in practice is stochastic, the resulting "induced" dynamical system then also becomes stochastic, and to my knowledge, at least in theory, learning Koopman embedding for such systems has more challenges are requires certain assumptions about the true system, such as co-diagonalization and few others. I think this should be discussed in more detailed by the authors (and please do correct me if I'm wrong on any of these statements) as currently for the readers who are not too familiar I think the text might come across as though that the Koopman theory extends naively to these scenarios as well, which I do not think is the case.

3. It is not very clear how does the "metric" loss affects the solution. I would encourage the authors to provide comparison (only in terms of dynamical modelling, without active control) of whether this metric helps, or have some negative effects on the prediction. I think that for instance if the GNN has some form of weight regularization than this indeed would have some non-trivial effect on the resulting representation. Also, it would useful to have plots of how accurately does the embedding preserve the distance to the true states in order to understand this better.



Comments on the experiments:

1. In the paper there is no discussion about what are the actual observation space of the environments, could these be clarified better.

2. The block diagonal structure approach in general has been presented as working with multiple types of interactions. However, in practice it seems that the authors have only used two types of interaction -> object-same-object and object-other-object interaction. This however, has never been discussed and is maybe false. Could you clarify these details?

3. The results showed in Figure 3 are somewhat in contrast than the results in the original PN paper, specifically the PN paper states that it can achieve MSE of 7.85 for 1000 time steps, and from figure 6 of that paper it shows about 0.05 MSE over 100 steps on a similar rope environment. These results compared to the one presented here in Figure 3 makes me wonder how well did the authors actually managed to reimperilment the IN and PN paper? Could there be any comments on this as this makes many claims of the proposed method being better questionable and hard to understand its significance in relation to previous work. 

4. For the control tasks, it would have been useful to have more than just the single baseline used. There are plenty of algorithms for Reinforcement Learning that could have been used in order to put the method in perspective. E.g. one can apply MPC with ground-truth model (e.g. the simulator) to show the discrepancy with an ideal case. In the RL literature there are plenty of methods for solving smaller problems, parametric and non-parametric: Q-learning, PPO etc... I think this is very important from the reader perspective. 


PS: Please refer to the discussion below with the authors as I have increased my score from 3 to 6.
review_title: Official Blind Review #4

final_score: rating: 6: Weak Accept
review_content: The paper is well written and the proposed idea is novel and builds on a sound theoretical framework of Koopman operator theory. A physical system is a represented as a graph; graph neural network is used to encode the current state to an object-centric embedding where the dynamics are assumed to be linear (Koopman operator theory) and modeled as a transition matrix. The key contribution is to recognize that similar physical interactions can be modeled using same parameters which constraints the transition matrix to be block-wise with shared parameters. Furthermore, the model is extended to add a control matrix to model external control. Experiments are conducted on simulations as well as control for 3 different settings - a a hanging rope/string anchored at the top, soft robot with an anchor, and soft robot in fluids.

Strengths
* Proposed method builds on a sound theoretical basis; although the linear dynamics model appear somewhat limited compared for the complex dynamics, thorough experiments are conducted to demonstrate the effectiveness of the method. The efficiency of the proposed algorithm compared to prior work makes is much practically useful.
* Well written with examples and illustrative figures.
* Quantitative analysis together with ablation studies on the structure are insightful.

Weaknesses
* My primary concern are with the evaluation.
- Experiments are only conducted on synthetic datasets. While experiments on real datasets are understandably difficult especially for quantitative validation, it would help to map the experiments to real problems to gain a more intuitive understanding and thus cater to a broader community.
- This paper and several related works are all evaluated on different problems; it would be useful to evaluate on similar tasks; for instance, strings [Battaglia 2016]. It would make it easier to draw comparisons.
- It's clear that the community would benefit significantly by having a benchmark or a web-based evaluation methodology (similar to OpenAI gym used actively reinforcement learning community). Unfortunately, this paper does not seem to offer a solution to this issue but continue to evaluate in ways similar to previous papers.

review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: This paper proposes to learn compositional Koopman operators using graph neural networks to encode the state into object-centric embeddings and using a block-wise linear transition matrix to regularize the shared structure across objects.  The combination of deep Koopman operator with graph neural nets is very novel and interesting. The experiments are also well designed and the results shown in the paper also indicate the effectiveness and efficiency of the algorithm in both simulation and control. In conclusion, I think this work can inspire more wok into modeling larger and more complex systems by integrating the power of the Koopman theory and the expressiveness of neural networks.
review_title: Official Blind Review #2

final_score: rating: 8: Accept
review_content: This paper introduces an approach to learning compositional koopman operators to efficiently model the dynamics of non-linear systems, consisting of an unspecified number of objects with repetitive dynamics.

The key contribution of this work is the use of a graph neural network that allows the koopman operator to be learned for systems of  multiple objects, and the incorporation of blockwise structure in the koopman gain and control matrices that improves parameter estimation process and is shown to reduce over-fitting.

Results show the proposed approach is effective, although only simple toy problems are examined and controlled. Nevertheless, this is a useful demonstration of learning for soft robot systems, and the idea of using koopman embeddings is likely to be of value to the ICLR community. 

The paper is well written, and I like the idea of incorporating additional structure into the learning process through the expected blockwise structure.

As I understand it, the proposed approach is able to exploit this natural blockwise structure due to the assumption that the same physical dynamics are followed by each block (although objects can also be labelled as rigid/moving). How would this approach extend to objects with vary different properties (eg. 5 objects with connected with springs of different stiffness)?

Could you provide more details on the training process, is the model trained in an end-to-end fashion, or in parts?

For the control experiment, the choice was made to re-evaluate the control after 32 steps. Why was this the case? Is this due to the time taken to simulate/ find controls? MPC would typically re-plan faster than this.

On a related point, the introduction states that the proposed approach is 20 times faster than baselines, but no evidence of this is provided. How was this assessed, and is this at prediction or training time? How long does it take to replan using the proposed model and SQP?
review_title: Official Blind Review #1

Meta-review: This paper proposes using object-centered graph neural network embeddings of a dynamical system as approximate Koopman embeddings, and then learning the linear transition matrix to model the dynamics of the system according to the Koopman operator theory. The authors propose adding an inductive bias (a block diagonal structure of the transition matrix with shared components) to limit the number of parameters necessary to learn, which improves the computational efficiency and generalisation of the proposed approach. The authors also propose adding an additional input component that allows for external control of the dynamics of the system. The reviewers initially had concerns about the experimental section, since the approach was only tested on toy domains. The reviewers also asked for more baselines. The authors were able to answer some of the questions raised during the discussion period, and by the end of it all reviewers agreed that this is a solid and novel piece of work that deserves to be accepted. For this reason I recommend acceptance.
Decision: Accept (Spotlight)