Paper: Orbit-Equivariant Graph Neural Networks (ICLR-2024)

Reviewer comment (Presentation):
Would you mind providing some illustrations about the concept of max-orbit?

Author response:
*Question:*

Would you mind providing some illustrations about the concept of max-orbit?

**Response:**

We have adjusted the definition to eliminate potential confusion.

Reviewer comment (Novelty):
I am not sure whether the proposed model has some connection to the paper like DeepSet [1]. Would you mind giving some insights?

Author response:
*Question:*

I am not sure whether the proposed model has some connection to the paper like DeepSet. Would you mind giving some insights?

**Response:**

The aim of DeepSets is to provide a learnable function that can operate on sets, such that the functions are permutation invariant. By contrast, our work defines functions on graphs. A model like DeepSets could be used when implementing a GNN, for f_aggr or f_read (see P.3, Section 2, Graph Neural Networks).

Reviewer comment (Writing):
The motivation of developing orbit-equivariance is still confusing to me. Would you mind providing more examples and situations that such symmetries are required?

Author response:
*Question:*

The motivation of developing orbit-equivariance is still confusing to me. Would you mind providing more examples and situations that such symmetries are required?

**Response:**

The multi-agent application from Morris et al. (2022) and our drug design application are the main motivating examples we have identified for equivariance being too strict of a property. In general, orbit-equivariance could be beneficial in any application that 1) uses node-labeling functions on graphs to optimize some global property and 2) has nodes that are not already individualized.

Reviewer comment (Experiments):
The last concern is about the efficiency of Orbit-1-WL that is applied to determine the orbits, particularly for large-scale graphs in practice. The authors should discuss this point if necessary.

Author response:
In P.5, Section 4, we note that Appendix C.1 has an analysis of the computational complexity of the models. There, it is pointed out that 1-WL is worst-case linear in the number of nodes in the graph. Thus, we anticipate that Orbit-1-WL will scale well to large graphs.

Reviewer comment (Theory):
P.2, Section 2: It is appropriate that the input to an orbit-equivariant function (or, more generally, a node-labeling function) is a graph with an ordered set of nodes. Node sets of graphs in this paper are ordered as \( V=\{1,\ldots, N\} \). Also, examples of node-labeling functions are defined using this ordering. If the node-labeling function takes the graph structure only, we need to show that the function is well-defined regardless of the node ordering. If it is well-defined, the function is automatically equivariant.

Author response:
*Question:*

If the node-labeling function takes the graph structure only, we need to show that the function is well-defined regardless of the node ordering. If it is well-defined, the function is automatically equivariant.

**Response:**

We use V = {1, …, N} in our paper for ease of notation only: it is not required, yet it is a very common notation. Giving a set of nodes is needed to properly define a graph: thus, functions on graphs need to take this into account. The first example function shown on P.4 is a well-defined node-labeling function on a graph that is not equivariant.

Reviewer comment (Presentation):
P.6, Figure 5: Since \( m=o+1(=3) \), it is difficult to understand whether the triples represent \( m \) or \( o+1 \). It would be better to use a different number for both (e.g., \( m=5 \)). In Figure 5, the last element of the partition is 1-index, whereas 0-index is used in the text, which is confusing and should be unified.

Author response:
*Question:*

P.6, Figure 5: Since m = o + 1 (=3), it is difficult to understand whether the triples represent m or o +  1. It would be better to use a different number for both (e.g., m = 5). In Figure 5, the last element of the partition is 1-index, whereas 0-index is used in the text, which is confusing and should be unified.

**Response:**

We have added an extra annotation to show which triple corresponds to o + 1 = 3. Hopefully this eliminates the potential confusion. We have also fixed the inconsistency between the figure and text, in the 0-indexing vs 1-indexing.

Reviewer comment (Theory):
P.6, Theorem 2: Regarding the claim that a function is not orbit-equivariant, does it mean the function is *not necessarily* orbit-equivariant? For example, Unique-ID-GNNs happen to be orbit-equivariant depending on the choice of the underlying GNN (e.g., degenerated GNN that always returns 0).

Author response:
*Question:*

P.6, Theorem 2: Regarding the claim that a function is not orbit-equivariant, does it mean the function is not necessarily orbit-equivariant?

**Response:**

Yes. By the statement “a class of models X is not orbit-equivariant”, we mean that there exists a function f from the class X that is not orbit-equivariant.

Reviewer comment (Theory):
P.7, Section 5: I have a question about whether it is appropriate to apply the cross-entropy loss to the output before transforming in the training of \( m \)-Orbit-Transform-GNN. This operation implies that \( m \)-Orbit-Transform-GNN is trained to output the correct labels before the transformation. If I understand correctly, this is different from what is intended.

Author response:
*Question:*

P.7, Section 5: I have a question about whether it is appropriate to apply the cross-entropy loss to the output before transforming in the training of m-Orbit-Transform-GNN.

**Response:**

Yes, we do intend to apply the loss before the transformation takes place. We have revised P.7, Section 5, Loss to make this clearer.

Reviewer comment (Writing):
P.8, Section 5: GCNs [...] would have achieved an accuracy of 0 on the Alchemy-Max-Orbit, [...]: Does this sentence mean that GCNs have not been tested on the Alchemy-Max-Orbit?

Author response:
*Question:*

P.8, Section 5: GCNs [...] would have achieved an accuracy of 0 on the Alchemy-Max-Orbit, [...]: Does this sentence mean that GCNs have not been tested on the Alchemy-Max-Orbit?

**Response:**

Yes, GCNs are not expressive enough to solve any of the examples in the Alchemy-Max-Orbit datasets.

Reviewer comment (Experiments):
P.8, Table 1: The graph accuracy of RNI-GCN is 0 for all ten trials in Alchemy-Max-Orbit-2. Is this result due to the nature of RNI-GCN? Or does insufficient training cause it, and RNI-GCN could have non-zero accuracy when appropriately trained?

Author response:
*Question:*

P.8, Table 1: The graph accuracy of RNI-GCN is 0 for all ten trials in Alchemy-Max-Orbit-2. Is this result due to the nature of RNI-GCN or rather how it was trained?

**Response:**

The full results in P.37, Section C.5.2, Orbit-Sorting-Cross-Entropy Loss show that the performance of RNI-GCNs plateaus by 8k epochs (under all metrics), so it appears unlikely that further training would improve the model. However, there is nothing inherent to the nature of the model that strictly prevents it from achieving a non-zero graph accuracy.

Reviewer comment (Writing):
P.2, Section 2: aka -> a.k.a. or also known as (I suggest using the full phrase).

Author response:
*Other Minor Comments*

**Response:**

Thank you for paying such close attention to our paper to have made these comments. We have addressed them in our uploaded revision.

Reviewer comment (Writing):
P.4, Section 3: \( \{0\}^{|G|} \) -> \( 0^{|G|} \).

Author response:
*Other Minor Comments*

**Response:**

Thank you for paying such close attention to our paper to have made these comments. We have addressed them in our uploaded revision.

Reviewer comment (Writing):
P.4, Section 3: the number of unique values in \( \{\{ f(G)_v \mid v\in r \}\} \): wouldn't it be easier to write \( | \{ f(G)_v \mid v\in r \} | \) in a mathematical way?

Author response:
*Other Minor Comments*

**Response:**

Thank you for paying such close attention to our paper to have made these comments. We have addressed them in our uploaded revision.

Reviewer comment (Writing):
P.5, Section 4: What does *Indiv* in Orbit-Indiv-GNN stand for? Does it mean individualization?

Author response:
*Question:*

P.5, Section 4: What does Indiv in Orbit-Indiv-GNN stand for? Does it mean individualization?

**Response:**

Yes: it refers to individualization within each orbit.

Reviewer comment (Writing):
P.7, Section 5: \( |T| \) is not used.

Author response:
*Question:*

P.7, Section 5: |T| is not used.

**Response:**

We have rephrased this sentence to eliminate the confusion.

Reviewer comment (Presentation):
In figure 1, please make it explicit that the 3 Fluorine atoms in the molecule belong to the same equivalence class only if the position coordinates of the atoms - are not used as part of the node features

Author response:
We have clarified this in the caption of Figure 1.