Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper proposes a graph convolutional network that can be in-/equivariant to isometric transformation. The method is applied to a linear toy problem of predicting the result of several diiferential operators and a nonlinear heat diffusion dataset in comparison to finite element analysis (FEA).

A major problem of the paper is its lack of clarity, which is largely due to the unusual definition of established terms, for example:
- The contraction (Eq. 4) is usually an operator defined for a single tensor, not two.
- It is not completely clear to me how Eq. 3 is related to the convolution. Summing over index j look similar to a typical graph convolution, however, here you have an additional index l. Written loosely, a convolution usually has the form $\sum_j f(i, j) g(j) $, but here you have $\sum_j f(i, j) g(j, l)$.
- The notion of a "collection of tensors" does not work well here, in particular when aiming to define a convolution (over a space or group). Given the particular application, the notion of tensor fields might be more appropriate.

The authors mention previous work on equivariant neural networks such as steerable convolutions, tensor field networks, and covariant networks, which they claim are inefficient since they use message-passing, while their approach uses GCNs. This contradicts their statement from above, that GCNs are message-passing neural networks. It should be clarified what exactly makes these approaches less efficient. Moreover, since equivariant neural networks are clearly more suitable for the presented experiments then previous GCNs, some of them should be included in the model comparisons with timings. In particular, going to tensors with large rank (as suggested in Eqs. 8-10) should be rather inefficient compared to equivariant networks using spherical harmonics due to the exponentially increasing number of parameters.

Regarding the timings in Table 4, it is not clear to me, whether the timing include the preprocessing of the adjacency matrix for the inference. Also the caption mentions that a single CPU core was used. Does this refer to both the neural network and the FEA? Otherwise it does not seem like fair comparison.

Pros
====
- incorporating the various differential operators in the network structures in Sec 3.3 is an interesting idea

Cons
====
- lacking clarity in method description and notation
- conceptual differences to steerable convolutions and equivariant networks do not become sufficiently clear
- experiment section not very strong since comparison to equivariant networks is missing and the timings in Table 4 need some clarification

Update: I read the comments of the authors and thank them for the clarifications. The additional baselines improved the paper. I raised my score to reflect that.
review_title: Equivariant GCN with lacking clarity

final_score: rating: 7: Good paper, accept
review_content: Summary:
The paper proposes a network that operates on features of graphs that are embedded in a d-dim Euclidean space. The paper considers equivariance to a group G that is the direct product of permutations of N points and Euclidean transformations. The features they consider are tensor products of the N-dimensional natural representations of permutations and the d-dimensional standard representation of O(d). From the coordinates, an “isometric adjacency matrix” is created, which is such a tensor. This matrix is combined in various G-equivariant ways with the features and then linearly combined with learnable weights to create new features. These operations are interleaved with non-linearities to form the network. The authors compare to several graph network methods and show competitive performance on several tasks.

Strengths:
-	The authors build a bridge between the global permutation equivariant methods on graphs, as they consider matrix-like features and equivariant maps between them, and Euclidean-equivariant methods on point clouds. This combination seems a powerful approach to point-cloud neural networks.

Weaknesses:
-	Important pieces of prior work are missing from the related work section. The paper seems to be strongly related to Tensor Field Networks (TFN) (Thomas et al. 2018), as both define Euclidean and permutation equivariant convolutions on point clouds / graphs. Furthermore, there are several other methods that operate on graph that are embedded in a Euclidean space, such as SchNet (Schütt et al 2017). The graph network methods currently discussed all do not include the point coordinates in their operations. Lastly, the proposed method operates globally linearly on features on a graph, equivariantly to permutations, which is done in prior work, e.g. Maron 2018. 
-	The experimental section only compares to methods that in their convolution are unaware of the point coordinates (except for in the input features). A comparison to coordinate-aware methods, such as TFN or SchNet seems appropriate.
-	The core object, the isometric adjacency matrix G, is ill-defined. In Eq 1 it is defined trough the embedding coordinates and “the transformation invariant rank-2 tensor” T. This object is not defined in the paper, which makes section 3 very confusing to read. In section 3, it appears like that the defined objects D take the role of object G in the above, so what is the role of eq 1? 
-	In section 3, the authors speak of “collections of rank-p tensors”. However, these objects seem to actually be tensors of the shape N^a x d^p, where N is the number of nodes, d is the dimensionality of the embedding, and a and p are natural numbers. These objects transform under both permutations and Euclidean transformations in the obvious way. Why not make this fact explicit? That would make section 3 much easier to read. It seems like that when p=0, then a=1, and when p>0, then a=2.  Except for in sec 3.2.2, in which a p=3 tensor has a=1. 
-	In Sec 3.2, what are f_in and f_out? Are these the dimensionalities of the tensor product representation? Or do they denote the number of copies of the representation? If it’s the former, I don’t see how the network is equivariant. If it’s the latter, I don’t understand the last paragraph of 3.2.2, which says 1H \in R^{N x f_in}, which looks like a 0-tensor.
-	Can the authors clarify “To achieve translation equivariance, a constant tensor can be added to the output collection of tensors.”? The proposed method seems to only lead to translation invariant features. I do not follow how adding a constant tensor leads to translation equivariance that is not invariance.
-	Am I correct in understanding that the method scales cubic with the number of vertices (e.g. eqs 4, 6)? Or is there some sparsity used in the implementation, but not mentioned? Should we expect a method of cubic complexity to scale to 1M vertices? In a naïve implementation, a fast modern GPU with 14.2E12 flops would need 20h for a single 1Mx1M matrix-matrix multiplication (1E18 floating point operations).
-	The authors claim the method scales to 1M vertices, but I cannot find this in the experiments. Table 4 speaks of 155k vertices. How did the authors determine the method scales to 1M vertices? 

Recommendation:
In its current form, I recommend rejection of this paper. Section 3 is insufficiently clear written, the related work lack important references to prior work and the experiments lack a comparison to potentially strong other methods. This is a shame, because I’d like to see this paper succeed, as the core idea is very strong. Significant improvements in the above criticisms can improve my score.

Suggestions for improvement:
-	Be clear about what the G object is and what eq 1 means.
-	Be explicit about types the objects, be more explicit about the indices that refer to the permutation representation, to the indices that refer to the Euclidean representation and the indices that refer to copies of the same representation. I think there is an opportunity to be more clear, more explicit, while reducing notational clutter.
-	Expand the related work section
-	Compare to the strong baselines that use the coordinates.
-	Provide argumentation for the claim to scale to 1M vertices.

Minor points:
-	Eq 7, \times should be \otimes?
-	Eq 14, what is j?
-	The authors write: “A, B and C are X, Y and Z respectively”. Perhaps this could be re-written to the easier to read “A=X, B=Y and C=Z”. This happens each time the word “respectively” is used.
-	Table 3 typo, gluster -> cluster


### Post rebuttal
The authors addressed all my concerns and strongly improved their paper. I think it is now a good candidate for acceptance, as it provides an interesting alternative to / variation on tensor field networks. I raise my rating from 4 to 7.
review_title: Strong core idea, but unclear section 3 and missing related work and baselines

final_score: rating: 5: Marginally below acceptance threshold
review_content: Comments:
Pros:
1.	This paper gives a comprehensive derivation of propositions used in the construction of isoGCN. 
2.	The properties provided are clear
3.	The idea to realize the purpose of being isometric transformation invariant and equivariant is good.

Cons:
1.	More explanations about the superiority of isometric transformation invariance and equivariance need to be addressed since this purpose seems quite important throughout this paper. Besides, how this purpose helps to achieve the superiority needs to be answered.
2.	How to interpret the adjacency matrix up to m hops, what is the construction of the adjacency matrix mentioned in the paper?
3.	In the experiment results, though the proposed isoGCN gave the minimum loss when comparing with other GCN variants, the scale of loss is around 10^(-5/-6) which may not give a significant goodness.
4.	In table 4, though isoGCN gave the least running time, it also gave larger loss compared with the method FrontISTR. How to explain the trade-off between running time and the loss?
5.     The paper is too theoretical and does not fit to ICLR.  Real impact and real application should be emphasized more.

review_title: Isometric Transformation Invariant and Equivariant Graph Convolutional Networks. Summary: This paper proposed a transformation invariant and equivariant GCN, which mainly focused on the construction of the newly defined isometric adjacency matrix. Apply this isometric adjacency matrix to change the formulation of the GCN proposed by Kipf & Welling.

Meta-review: This paper makes an innovative change to the adjacency matrix definition in graph convolutional neural networks (GCNs) (Kipf & Welling, 2017).  The change results in computationally-efficient isometric transformation invariance.   There were a number of concerns raised by reviewers, and the author responses and revisions, and the subsequent discussion, resulted in most of these concerns being satisfactorily addressed.  On reviewer continued to feel the paper was entirely theoretical and therefore not appropriate to ICLR, but that opinion was not shared more broadly and is not held by the area chair.
Decision: Accept (Poster)