Paper: Rethinking Independent Cross-Entropy Loss For Graph-Structured Data (ICLR-2024)

Reviewer comment (Writing):
Some notations or definitions haven't been clearly explained. See the questions.

Author response:
**W1 and W2:** Thanks for your constructional suggestions. We will revise the paper according to the questions and clarify the connection between (5) and (4d) to facilitate readers' understanding of the relationship between maximum likelihood estimation and cross-entropy loss.

Reviewer comment (Theory):
More explanation about the equivalance between (4c) and (4d) should be provided.

Author response:
**Q1:** We will introduce the transition from (4c) to (4d). We aim to learn the parameters $\theta$ to maximize the likelihood of the observed conditional distribution $p\left(y_{1}, \ldots, y_{L} \mid z_{1}, \ldots, z_{L} ; \theta\right)$, known as maximum likelihood estimation. To simplify the computation of the likelihood function, it is common practice to use the logarithm of the likelihood function, which transforms the product of probabilities into a sum (taking the logarithm does not alter the optimal solution of the likelihood).

Reviewer comment (Writing):
In the definitions of $\bar{z}_m$ and $\bar{y}_m$, there are two indices $k$ and $i$ that are confusing.

Author response:
**Q2:** Thanks for your pointing out. We will fix this mistask and use $i$ as the indice uniformly.

Reviewer comment (Theory):
In (5), $y _i\bar{y} _m^\top$ is a matrix, which is not consistent with the shape of output of $g _\phi$.

Author response:
**Q3:** We flatten the matrix $y_i y_m^{\top} \in \mathbb{R}^{C\times C}$ into shape ${1\times C^2}$, and the shape of output of $g_\phi$ is ${n\times C^2}$, where $n$ represents the number of nodes and $c$ represents the number of classes.

Reviewer comment (Experiments):
How did the authors determine the hyperparameters of the compared methods?

Author response:
**Q5:** Following the experimental settings of original papers, for GAT, we choose the model parameters by utilizing an early stopping strategy with a patience of 100 epochs on classification cross-entropy loss. For other models(GCN, SGC, MLP and GrapgSAGE), we utilize the model parameters which perform best on the validation set for testing. The remaining hyper-parameters including learning rate, dropout and weight decay are tuned for different models.

Reviewer comment (Evaluation):
It is not clear why the improvement on balanced data is higher than imbalanced data.

Author response:
**Q6:** In Table 2 of manuscript, we have already demonstrated the advantages on two extremely imbalanced datasets, LastFMAsia and ogbn-arxiv. We attribute this result to the referential ability of joint-cluster distribution modeling, which refers to a cluster of nodes when making decision on target node label during training and inference. The joint distribution weakens the over-confident prediction on the majority classes by assigning prediction confidence on other related minority classes at the cluster, and thus ameliorates the generalization performance. However, in the extremely imbalanced datasets, there are a large number of majority class nodes in the reference cluster of minority class nodes, making the reference information of nodes in the imbalanced datasets not as rich as in the balanced datasets. We think this leads to the improvement on balanced data is higher than imbalanced data.