Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 1: Reject
review_content: The paper proposed a new type of graph embedding technique for dynamic graphs based on tensor representation (node x feature x time).  Experiments on edge classification demonstrate improved prediction accuracy. 

+ Clear writing with tensor notations and explanation is well-structured 
+ Improved prediction results on 3/4 real-world dynamic graph datasets

- The theoretical results are a bit artificial. The tensor eigendecomposition used in this paper and [Kilmer and Martin] is for slices of the tensor, similarly for FFT and convolution. The technique is a trivial generalization from matrix results. 
- The paper is missing a large body of baselines, both from the network science community (non-deep learning methods) and from this community (diffusion convolutional RNNs, graph attention networks, etc). 
- The method doesn't scale well, especially for graphs with long-term dynamics. It would be good to show the scaling behavior of the proposed model. 
review_title: Official Blind Review #4

final_score: rating: 3: Weak Reject
review_content: This paper presents a M-product based temporal GCNs to handle dynamic graphs. Experiments on four real datasets are performed to verify the effectiveness of the proposed model.

Overall, I think this paper make a few contributions to advocate tensor M-product. However, there are several big issues as listed below. Given the current status, I could not accept the paper.

Pros:

1, The generalization brought by M-product seems to be general as it includes quite a few graph convolution elements for 3D tensors in a natural way.

2, The experimental setup is reasonable. Datasets are collected from practical problems and of moderately large scale.

3, The paper is clearly written and easy to follow.

Cons & Questions: 

1, My first concern is that M-product formulation does not bring any new insights as people have already used some of the key elements in practice for a long time. For example, the M-transform is just applying 1x1 convolution to multi-channel image. Slice-wise matrix multiplication is also common in practice.

2, Moreover, I think there are several challenges in the M-product formulation which prevent the technique from being practical.

(1) Sharing M such that frontal slices of the transformed signal are the same, i.e., each row of M share the same vector, limits the model capacity significantly. If there is no sharing mechanism, then the model learned on one sequence of graphs could not be applied to another sequence of graphs given two sequences have different lengths. 

(2) If you learn M from data, how could you ensure that M is invertible? In the paragraph before section 4.1, an edge classification formulation is proposed where the inverse M-transform is abandoned. However, if in practice, you do not need the inverse transform, then do those theoretical properties still hold and what is the meaning of introducing such M-product formulation?

3, A few temporal GCN baselines are neither compared or discussed, e.g., [1]. 

4, Could you explain why all the other GCN variants performs significantly worse with a symmetrized adjacency matrix compared to using the asymmetric one? 

[1] Li, Y., Yu, R., Shahabi, C. and Liu, Y., 2017. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. arXiv preprint arXiv:1707.01926.

======================================================================================================

After I read authors' reply and other reviewers' comments, I would like to keep my original rating as the issues have not been properly addressed. I agree with the Reviewer #4 that the theoretical results are a bit artificial and trivial. 
review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: Summary: this work uses tensor methods to improve graph convolution for dynamic graph, where the nodes are fixed and the edges are changing. Specifically, it uses the M-product technique to develop the operations of sequence of matrices that analog to these operations of matrices. In the M-product notations, everything seems to be as neat as matrix operations. The works also shows decent supremacy on edge classification tasks.


Comments: this paper is mathematically interesting. It is well-written in general, but the definitions are dense and hard to follow.

It will be better to give some examples of M-product. For example, what these operations will be if we choose M to be the identity matrix?

M-transfer is a tensor contraction, right?

It seems if you do the operations of the sequence of matrix, there is no need to do iterations like RNN. I am interested in how this will influence the runtime and memory cost.

The M matrix is defined as a lower triangle matrix such as (A \times M)_::t depends on A^(1:t). Is it possible to formulate M such that (A \times M)_::t will depend heavily on A^(t), and less on the farther matices? such that we encode some Markov property?

Does there exist some condition when this method will be equivalent to RNN?


Decision: I feel this work novel and interesting in general. I would like to weakly accept it.

review_title: Official Blind Review #3

Meta-review: The paper proposes a tensor-based extension to graph convolutional networks for prediction over dynamic graphs. 

The proposed model is reasonable and achieves promising empirical results. After discussion, it is agreed that while the problem of handling dynamic graphs is interesting and challenging, the proposed tensor method lacks novelty, the theoretical analysis is artificial, and the empirical study does not cover enough benchmarks. 

The current version of the paper is not ready for publication. Addressing the issues above could lead to a strong publication in the future. 
Decision: Reject