Paper: Graph Decoding via Generalized Random Dot Product Graph (ICLR-2024)

Reviewer comment (Experiments):
The experiments only compared the proposed method with the backbone GNN. However, the advances in recent GNN significantly outperform GAE on various tasks. How does the proposed method compare to those methods?

Author response:
Dear Reviewer, we thank you for your contribution to our work. We wanted to address some of the comments you left in your review. 

We purposely choose the vanilla implementations of such architectures since we believe that the proposed methodology only makes sense when compared to the traditional inner product decoder.

Reviewer comment (Evaluation):
Does the generated matrix indeed have negative eigenvalues? If so, does the original matrix that the GNN is trained on also contain negative eigenvalues?

Author response:
Dear Reviewer, we thank you for your contribution to our work. We wanted to address some of the comments you left in your review. 

Yes, both of them have negative eigenvalues. The adjacency matrix normalisation operations in GCNs do not alter the spectra of negative eigenvalues beyond their scale, and the generative model indeed generates negative eigenvalues, as stated in the text.

Reviewer comment (Novelty):
Novelty is limited. Equation (3), the center of the method, has been proposed in [1].

Author response:
We were well aware of the contribution made by [1], we believe that even though our work is not catered around the development of the kernel, its application to a graph decoding process in the context of GAEs provides novelty to the field.

Reviewer comment (Theory):
The meaning of negative eigenvalue in adjacency matrix remains unclear. Does it corresponds to some specific substructure or some graph statistics?

Author response:
We did not delve into the topological effects of the presence of negative eigenvalues, but it is clearly something we want to pursue shortly. Nevertheless we could informally link it with the presence of cycles and triangles.

Reviewer comment (Evaluation):
Uncertainties are not provided in experiment results.

Author response:
We believe that uncertainties are provided in the results tables in the form of measured deviations from the mean in the Supplementary Material. We will include them in the main text in future versions of the work.

Reviewer comment (Novelty):
Lorentz model in hyperbolic GNN [1] also uses a diagonal matrix with $\pm 1$. Could you please compare your Equation 3 with hyperbolic GNNs?

Author response:
Thank you for your question regarding the comparison between our Equation 3 and the Lorentz model in hyperbolic GNNs. Our Equation 3, given by, provides a framework for computing the probability of an adjacency matrix
Unlike the Lorentz model, which is specifically designed for hyperbolic space with its constant negative curvature, our model is more general. The diagonal matrix in our equation does not inherently assume a hyperbolic structure and could potentially be adapted to represent other types of geometric metrics, including Euclidean space.
While the Lorentz model is adept at handling data with hierarchical structures characteristic of hyperbolic space, our model aims to be more versatile, allowing for adjustment of the diagonal matrix to cater to a wider range of graph topologies and data geometries. This adaptability makes our Equation 3 potentially applicable to a broader spectrum of graph-based learning tasks. It is important to note, however, that if it is specified to reflect hyperbolic geometry, our model could operate in a manner similar to the Lorentz model, thus serving as a specific case within our more general framework.

Reviewer comment (Theory):
Can use approximate graph laplacian matrix ($L=D-A$) instead of adjacency matrix to avoid the positive-semi definite problem?

Author response:
About using a Laplacian. Indeed this is a valid approach, however, adopting this paradigm would heavily complicate the generative paradigm of the model. Implementing this model, would duplicate the target matrices needed, as the degree matrix D would also need to be generated for the adjacency matrix reconstruction. We consider this to be out of the scope of our proposed article, however, we will consider this further for future work.

Reviewer comment (Experiments):
Why is node clustering an appropriate experiment to verify the model? Should it be more related to the task of graph structure perception?

Author response:
Dear Reviewer, we thank you for your contribution to our work. We wanted to address some of the comments you left in your review. 

Node clustering is a standard task for unsupervised graph representation learning. The objective of this task is to showcase the increased capability of algorithms in capturing the topology of the graph, trying to reproduce node labels without explicitly optimising for them. For further information the reviewer can check the following paper: Pan, S., Hu, R., Long, G., Jiang, J., Yao, L., & Zhang, C. (2018). Adversarially Regularized Graph Autoencoder for Graph Embedding. arXiv:1802.04407

Reviewer comment (Reproducibility):
The GRPDG method requires knowing in advance the number of positive/negative eigenvalues, so it's very important to know how the authors addressed this. Is it always the same? Is it chosen for each dataset? How? If guessed correctly, the authors try several different values of $q$ (for each dataset maybe?) and keep the result with the best performance. The lack of information on this crucial part is not acceptable.

Author response:
The value of q, the proportion of negative eigenvalues, is searched by greedy search of the parameter per residue. This work does not establish any relationship between q and the data.

Reviewer comment (Writing):
In the preliminary section, the description for the GAE loss functions says the expectation with respect to some q, which is not defined before.

Author response:
The equation defining the GAE is extracted from Kipf, T. N., & Welling, M. (2016). Variational Graph Auto-Encoders. Advances in Neural Information Processing Systems. arXiv:1611.07308 (2016). There, in eq. (1) is stated q as q(Z | X, A) = QN i=1 q(zi | X, A), with q(zi | X, A) = N (zi | µi , diag(σ 2 i )).