Posterior Label Smoothing for Node Classification

Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-02 04:01:33

Abstract: Soft labels can improve the generalization of a neural network classifier in many domains, such as image classification. Despite its success, the current literature has overlooked the efficiency of label smoothing in node classification with graph-structured data. In this work, we propose a simple yet effective label smoothing for the transductive node classification task. We design the soft label to encapsulate the local context of the target node through the neighborhood label distribution. We apply the smoothing method for seven baseline models to show its effectiveness. The label smoothing methods improve the classification accuracy in 10 node classification datasets in most cases. In the following analysis, we find that incorporating global label statistics in posterior computation is the key to the success of label smoothing. Further investigation reveals that the soft labels mitigate overfitting during training, leading to better generalization performance.

Main paper: 
 introduction: 
 Adding a uniform noise to the ground truth labels has shown remarkable success in training neural networks for various classification tasks, including image classification and natural language processing [Szegedy et al., 2016a , Vaswani et al., 2017 , Müller et al., 2019 , Zhang et al., 2021] .Despite its simplicity, label smoothing acts as a regularizer for the output distribution and improves generalization performance [Pereyra et al., 2017] . More sophisticated soft labeling approaches have been proposed based on the theoretical analysis of label smoothing [Li et al., 2020, Lienen and Hüllermeier, 2021] . However, the usefulness of smoothing has been under-explored in the graph domain, especially for node classification tasks.In this work, we propose a simple yet effective smoothing method for transductive node classification tasks. Inspired by the previous work suggesting predicting the local context of a node [Hu et al., 2019 , Rong et al., 2020] , such as subgraph prediction, helps to learn better representations, we propose a smoothing method that can potentially reflect the local context of the target node. To encode the neighborhood information into the node label, we propose to relabel the node with a posterior distribution of the label given neighborhood labels.Under the assumption that the neighborhood labels are conditionally independent given the label of the node to be relabeled, we factorize the likelihood into the product of conditional distributions between two adjacent nodes. To compute the posterior, we estimate the conditionals and prior from a graph's global label statistics, making the posterior incorporate the local structure and global label distributions. Since the posterior obtained in this way does not preserve the ground truth label, we finally interpolate the posterior with the ground truth label, resulting in a soft label.The posterior, however, may pose high variance when there are few numbers of neighborhood nodes. To mitigate the issue with the sparse labels, we further propose iterative pseudo labeling to re-estimate the likelihood and prior based on the pseudo labels. Specifically, we use the pseudo labels Submitted to 38th Conference on Neural Information Processing Systems (NeurIPS 2024). Do not distribute.of validation and test sets to update the likelihood and prior, along with the ground truth labels of the training set.We apply our smoothing method to seven different baseline neural network models, including MLP and variants of graph neural networks, and test its performance on 10 benchmark node classification datasets. Our empirical study finds that the soft label with iterative pseudo labeling improves the accuracy in 67 out of 70 cases despite its simplicity. We analyze the cases where the soft label decreases the accuracy and reveals characteristics of label distributions with which the soft labeling may not work. Further analysis shows that using local neighborhood structure and global label statistics is the key to its success. Through the loss curve analysis, we find that the soft label prevents over-fitting, leading to a better generalization performance in classification. 
related work: 
 In this section, we introduce previous studies related to our method. We begin by discussing various node classification methods, followed by an exploration of the application of soft labels in model training.Graph structures are utilized in various ways for node classification tasks. Some studies propose model frameworks based on the assumption of specific graph structures. For example, GCN [Kipf and Welling, 2016] , GraphSAGE [Hamilton et al., 2017] , and GAT [Veličković et al., 2017] aggregate neighbor node representations based on the homophilic assumption. To address the class-imbalance problem, GraphSMOTE [Zhao et al., 2021] , ImGAGN [Qu et al., 2021] , and GraphENS [Park et al., 2022] are proposed for homophilic graphs. H 2 GCN [Zhu et al., 2020] and U-GCN [Jin et al., 2021] aggregate representations of multi-hop neighbor nodes to improve performance on heterophilic graphs. Other studies concentrate on learning graph structure. GPR-GNN [Chien et al., 2020] and CPGNN [Zhu et al., 2021] learn graph structures to determine which nodes to aggregate adaptively.LDS [Franceschi et al., 2019] , IDGL [Chen et al., 2020] and DHGR [Bi et al., 2022 ] take a graph rewiring approach, learning optimized graph structures to refine the given structure. Besides, research such as ChebNet [Defferrard et al., 2016] , APPNP [Gasteiger et al., 2018] , and BernNet [He et al., 2021] focus on learning appropriate filters from the graph signals.Hinton et al. [2015] demonstrate that a small student model trained using soft labels generated by the predictions of a large teacher model shows better performance than a model trained using one-hot labels. This approach, known as knowledge distillation (KD), is widely adopted in computer vision [Liu et al., 2019] , natural language processing (NLP) [Jiao et al., 2020] , and recommendation systems [Tang and Wang, 2018] for compression or performance improvement. In the graph domain, applying KD has been considered an effective method to distill graph structure knowledge to student models. TinyGNN [Yan et al., 2020] highlights that deep GNNs can learn information from further neighbor nodes than shallow GNNs, and it distills local structure knowledge from deep GNNs to shallow GNNs. NOSMOG [Tian et al., 2023] improves the performance of multi-layer perceptrons (MLPs) on graph data by distilling graph structure information from a GNN teacher model.On the other hand, simpler alternatives to generate soft labels are considered. The label smoothing (LS) [Szegedy et al., 2016a] generates soft labels by adding uniform noise to the labels. The benefits of LS have been widely explored. Müller et al. [2019] show that LS improves model calibration. Lukasik et al. [2020] establish a connection between LS and label-correction techniques, revealing LS can address label noise. LS has been widely adopted in computer vision [Zhang et al., 2021] and NLP [Vaswani et al., 2017] studies, but has received little attention in the graph domain.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: We discuss the limitations of the proposed model when the empirical conditional is not distinguishable in Section 4.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.• The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.• The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.• The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.• The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.• If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.• While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
method: 
 In this section, we describe our approach for label smoothing for the node classification problem and provide a new training strategy that iteratively refines the soft labels via pseudo labels obtained from the training procedure.{ } ) = ! "! 𝑃 { } ) = ! !# 𝑃 { } ) = & 𝑃 ) $ × & 𝑃 ) $ = ! "! Posterior Distribution 𝑃( |{ }) ∝ ! "! × ! $ 𝑃( |{ }) ∝ ! "! × ! % 𝑃( |{ }) ∝ ! !# × ! % TPrior Distribution& 𝑃( ) = ! % & 𝑃( ) = ! % & 𝑃( ) = # '( ) # '( *'+,) = ! $ & 𝑃 + ) & 𝑃 + ) & 𝑃 + ){ }Target Node T T Figure 1 : Overall illustration of posterior node relabeling. To relabel the node label, we compute the posterior distribution of the label given neighborhood labels. Note that the node features are not considered in the relabeling process.Consider a transductive node classification with graph G = (V, E, X), where V and E denotes the set of nodes and edges respectively, and X ∈ R |V|×d denotes d-dimensional node feature matrix. For each node i in a training set, we have a labely i ∈ [K],where K is the total number of classes. We use the notation e i ∈ {0, 1} K for one-hot encoding of y i , i.e., e ik = 1 if y i = k and k e ik = 1.In a transductive setting, we observe the connectivity between all nodes, including the test nodes, without having true labels of the test nodes.We propose a simple and effective relabeling method to allocate a new label of a node based on the label distribution of the neighborhood nodes. Specifically, we consider the posterior distribution of node labels given their neighbors. Let N (i) be a set of neighborhood nodes of node i. If we assume the distribution of node labels depends on the graph connectivity, then the posterior probability of node i's label, given its neighborhood labels, isP (Y i = k|{Y j = y j } j∈N (i) ) = P ({Y j = y j } j∈N (i) |Y i = k)P (Y i = k) K ℓ=1 P ({Y j = y j } j∈N (i) |Y i = ℓ)P (Y i = ℓ).(EQUATIONThe likelihood measures the joint probability of the neighborhood labels given the label of node i. To obtain the likelihood, we approximate the likelihood through the product of empirical conditional label distribution between adjacent nodes, i.e., P ({Y j = y j } j∈N (i) |Y i = k) ≈ j∈N (i) P (Y j = y j |Y i = k, (i, j) ∈ E), where P (Y j = y j |Y i = k, (i, j) ∈ E)is the conditional of between adjacent nodes. The conditional between adjacent nodes i and j with label n and m, respectively, is estimated byEQUATION)The prior distribution is also estimated from the empirical observations. We use the empirical proportion of label as a prior, i.e., P (Y i = m) := |{u | y u = m}|/|V|. We also explore alternative designs for the likelihood and compare their performances in Section 4.2.Note that, in implementation, all empirical distributions are computed only with the training nodes and their labels. The empirical distribution might be updated after node relabeling through the posterior computation, but we keep it the same throughout the relabeling process.The posterior distribution can be used as a soft label to train the model, but we add uniform noise ϵ to the posterior to mitigate the risk of the posterior becoming overly confident if there are few or no neighbors. In addition, since the most probable label from the posterior might be different from the ground truth label, we interpolate the posterior with the ground truth label. To this end, we obtain the soft label êi of node i asEQUATIONwhere ẽik ∝ P (Y i = k | {Y j = y j } j∈N (i) ) + βϵ. α and β control the importance of interpolation and uniform noise. By enforcing α > 1/2, we can keep the most probable label of soft label the same as the ground truth label, but we find that this condition is not necessary in empirical experiments.We name our method as PosteL (Posterior Label smoothing). The detailed algorithm of PosteL is shown in Algorithm 1. for i ∈ V train do Approximate likelihood:P ({Y j = y j } j∈Ntrain(i) |Y i = k) ≈ j∈Ntrain(i) P (Y j = y j |Y i = k, (i, j) ∈ E).Compute posterior distribution: P (Y i = k | {Y j = y j } j∈Ntrain(i) ) using Equation (1) . Add uniform noise:ẽik ∝ P (Y i = k | {Y j = y j } j∈Ntrain(i) ) + βϵ.Obtain soft label: êi = (1 -α)ẽ i + αe i . end forPosterior relabeling is a method used to predict the label of a node based on the labels of its neighboring nodes. However, in transductive node classification tasks where train, validation, and test nodes coexist within the same graph, the presence of unlabeled nodes can hinder the accurate prediction of posterior labels. For instance, when a node has no labeled neighbors, the likelihood becomes one, and the posterior only relies on the prior. Moreover, in cases where labeled neighbors are scarce, noisy labels among the neighbors can significantly compromise the posterior distribution.Such challenges are particularly prevalent in sparse graphs. For example, 26.35% of nodes in the Cornell dataset have no neighbors with labels. In such scenarios, the posterior relabeling can be challenging.To address these limitations, we propose to update the likelihoods and priors through the pseudo labels of validation and test nodes. We first train a graph neural network with the soft labels obtained via Equation (3) and predict the labels of validation and test nodes to obtain the pseudo labels. We choose the most probable label as a pseudo label from the prediction. We then update the likelihood and prior with the pseudo labels, leading to the re-calibration of the posterior smoothing and soft labels. By repeating training and re-calibration until the best validation loss of the predictor no longer decreases, we can maximize the performance of node classification. We assume that if posterior label smoothing improves classification performance with a better estimation of likelihood and prior, the pseudo labels obtained from the predictor can benefit the posterior estimation as long as there are not many false pseudo labels.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [Yes] Justification: The computational complexity of the proposed model is proven in Appendix C.Guidelines:• The answer NA means that the paper does not include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.• All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.• Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.• Theorems and Lemmas that the proof relies upon should be properly referenced. 
experiments: 
 The experimental section is composed of two parts. First, we evaluate the performance of our method for node classification through various datasets and models. Second, we provide a comprehensive analysis of our method, investigating the conditions under which it performs well and the importance of each design choice.In this section, we assess the enhancements in node classification performance across a range of datasets and backbone models. Our aim is to validate the consistent efficacy of our method across datasets and backbone models with diverse characteristics. Datasets We assess the performance of our method across 10 node classification datasets. To examine the effect of our method on diverse types of graphs, we conduct experiments on both homophilic and heterophilic graphs. Adjacent nodes in a homophilic graph are likely to have the same label. Adjacent nodes in a heterophilic graph are likely to have different labels. For the homophilic datasets, we use five datasets: the citation graphs Cora, CiteSeer, and PubMed [Sen et al., 2008 , Yang et al., 2016] , and the Amazon co-purchase graphs Computers and Photo [McAuley et al., 2015] . For the heterophilic datasets, we use five datasets: the Wikipedia graphs Chameleon and Squirrel [Rozemberczki et al., 2021] , the Actor co-occurrence graph Actor [Tang et al., 2009] , and the webpage graphs Texas and Cornell [Pei et al., 2020] . Detailed statistics of each dataset are illustrated in Appendix A.We evaluate the performance of PosteL across various backbone models, ranging from MLP, which ignores underlying structure between nodes, to six widely used graph neural networks: GCN [Kipf and Welling, 2016] , GAT [Veličković et al., 2017] ,APPNP [Gasteiger et al., 2018] , ChebNet [Defferrard et al., 2016] , GPR-GNN [Chien et al., 2020] ,and BernNet [He et al., 2021] . We follow the experimental setup and backbone implementations of He et al. [2021] . Specifically, we use fixed 10 train, validation, and test splits with ratios of 60%/20%/20%, respectively, and measure the accuracy at the lowest validation loss. We report the mean performance and 95% confidence interval. The model is trained for 1,000 epochs, and we apply early stopping when validation loss does not decrease during the last 200 epochs. For all models, the learning rate is validated within {0.001, 0.002, 0.01, 0.05}, and weight decay within {0, 0.0005}. The search spaces of the other model-dependent hyperparameters are provided in Appendix B. We validate two hyperparameters for PosteL: posterior label ratio α ∈ {0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0}and uniform noise ratio β ∈ {0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9}.We compare our method with two different soft labeling methods, including label smoothing (LS) [Szegedy et al., 2016b] and knowledge distillation (KD) [Hinton et al., 2015] . For KD, we use an ensemble of average logits from three independently trained GNNs as a teacher model. The temperature parameter for KD is set to four following the previous work [Stanton et al., 2021] .In Table 1 , the classification accuracy and 95% confidence interval for each of the seven models across the 10 datasets are presented. In most cases, PosteL outperforms baseline methods across various settings, demonstrating significant performance enhancements and validating its effectiveness for node classification. Specifically, our method performs better in 67 cases out of 70 settings against the ground truth labels. Furthermore, among these settings, 39 cases show improvements over the 95% confidence interval. Notably, on the Cornell dataset with the GCN backbone, our method achieves a substantial performance enhancement of 14.43%. When compared to the other soft label methods, PosteL performs better in most cases as well. The knowledge distillation method shows comparable performance with the GPR-GNN baseline, but even in this case, there are marginal differences between the two approaches.In this section, we analyze the main experimental result from various perspectives, including design choices, ablations, and computational complexity.Learning curves analysis We investigate the influence of soft labels on the learning dynamics of GNNs by visualizing the loss function of GCNs with and without soft labels. Figure 2 visualizes the differences between training, validation, and test losses with and without the PosteL labels on the Squirrel dataset. From the training loss, we observe that the cross entropy with the PosteL labels converges to a higher loss than that with the ground truth labels. The curve shows that predicting soft labels is more difficult than predicting ground truth labels. On the other hand, the validation and test losses with the soft labels converge to lower losses than those with the ground truth labels. Especially, up to 200 epochs, we observe that no overfitting happens with the soft labels. We conjecture that predicting the correct PosteL label implies the correct prediction of the local neighborhood structure since the PosteL labels contain the local neighborhood information of the target node. Hence, the model trained with PosteL labels could have a better understanding of the graph structure, potentially leading to a better generalization performance. A similar context prediction approach has been proposed as a pertaining method in previous studies [Hu et al., 2019 , Rong et al., 2020] . We provide the same curves for all datasets in Figure 6 and Figure 7 in Appendix D. All curves across all datasets show similar patterns.Influence of neighborhood label distribution Our approach assumes that the distribution of neighborhood labels varies depending on the label of the target node. If there are no significant differences between the neighborhood's label distributions, the posterior relabeling assigns similar soft labels for all nodes, making our method similar to the uniform noise method.Figure 3 shows the neighborhood label distribution for three different datasets. In the PubMed and Texas datasets, we observe a notable difference in the conditionals when w.r.t the different labels of a target node. The PubMed dataset is known to be homophilic, where nodes with the same labels are likely to be connected, and the conditional distributions match the characteristics of the homophilic dataset. The Texas dataset, a heterophilic dataset, shows that some pairs of labels more frequently appear in the graph. For example, when the target node has the label of 1, their neighborhoods will likely have the label of 5. On the other hand, the conditionals of the Actor dataset do not vary much 1 2 3 0.0 0.5 P(Y j |Y i = 1) 1 2 3 0.0 0.5 P(Y j |Y i = 2) 1 2 3 0.0 0.5 P(Y j |Y i = 3) (a) PubMedP(Y j |Y i = 1) 1 2 3 4 5 0.0 0.1 0.2 P(Y j |Y i = 2) 1 2 3 4 5 0.0 0.1 0.2 P(Y j |Y i = 3) 1 2 3 4 5 0.0 0.1 0.2 P(Y j |Y i = 4) 1 2 3 4 5 0.0 0.1 0.2 P(Y j |Y i = 5) (c) ActorFigure 3 : Empirical conditional distributions between two adjacent nodes. We omit the adjacent condition (i, j) ∈ E from the figures for simplicity.PosteL Labels regarding the label of the target node. In such a case, the prior will likely dominate the posterior.Therefore, the posterior may not provide useful information about neighborhood nodes, potentially limiting the effectiveness of our method. This analysis aligns with the results in Table 1 , where the improvement of the Actor dataset is less significant than those of the PubMed and Texas datasets. The neighborhood label distributions for all datasets are provided in Figure 8 and Figure 9 in Appendix E. Effect of iterative pseudo labeling We evaluate the impact of iterative pseudo labeling by analyzing the loss curve at each iteration. Figure 5 illustrates the loss curves for different iterations on the Cornell dataset. As the iteration progresses, the validation and test losses after 1,000 epochs keep decreasing. In this example, the model performs best after four iteration steps. We find that the best validation performance is obtained from 1.13 iterations on average. We provide the average iteration steps in Appendix C used to report the results in Table Test Loss Design choices of likelihood model We explore various valid design choices for likelihood models.We introduce two variants of PosteL: PosteL (normalized) and PosteL (local-H). In Equation ( 2), each edge has an equal contribution to the conditional. The conditional can be influenced by a few numbers of nodes with many connections. To mitigate the importance of high-degree nodes, we alternatively test the following conditional, denoted as PosteL (normalized):P norm. (Y j = m|Y i = n, (i, j) ∈ E) := yu=n v∈N (u) 1 |N (u)| • 1[y v = m] |{y u = n | u ∈ V}| ,where 1 is an indicator function.In PosteL (local-H), we estimate the likelihood and prior distributions of each node from their respective H-hop ego graphs. Specifically, the likelihood of PosteL (local-H) is formulated as follows:P local-H (Y j = m|Y i = n, (i, j) ∈ E) := |{(u, v)|y v = m, y u = n, (u, v) ∈ E, u, v ∈ N (H) (i)}| |{(u, v)|y u = n, (u, v) ∈ E, u, v ∈ N (H) (i)}| ,where N (H) (i) denotes the set of neighborhoods of node i within H hops. Through the local likelihood, we test the importance of global and local statistics in the smoothing process. Ablation studies To highlight the importance of each component in PosteL, we perform ablation studies on three components: posterior smoothing without uniform noise (PS), uniform smoothing (UN), and iterative pseudo labeling (IPL). Table 3 presents the performance results from the ablation studies.The configuration with all components included achieves the highest performance, underscoring the significance of each component. The iterative pseudo labeling proves effective across almost all datasets, with a particularly notable impact on the Cornell dataset. However, even without iterative pseudo labeling, the performance remains competitive, suggesting that its use can be decided based on available resources. Additionally, incorporating uniform noise into the posterior distribution enhances performance on several datasets. Moreover, PosteL consistently outperforms the approach using only uniform noise, a widely used label smoothing method. Complexity analysis The computational complexity of calculating the posterior label is O(|E|K).Since the labeling is performed before the learning stage, the time required to process the posterior label can be considered negligible. The training time increases linearly w.r.t the number of iterations with the pseudo labeling. However, experiments show that an average of 1.13 iterations is needed, making our approach feasible without having too many iterations. The proof of computational complexity is in Appendix C.Our method relies on global statistics estimated from training nodes. However, in scenarios where training data is sparse, the estimation of global statistics can be challenging. To assess the effectiveness of the label smoothing from graphs with sparse labels, we conduct experiments with varying sizes of a training set. We vary the size of the training set from 5% to 40% of an entire dataset and conduct the classification experiments with the same setting used in the previous section. The percentage of validation nodes is set to 20% for all experiments. Table 4 provides the classification performance with sparse labels. Even in scenarios with sparse labels, PosteL consistently outperforms models trained on ground truth labels in most cases. These results show that our method can effectively capture global statistics even when training data is limited.Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?Answer: [Yes] Justification: We provide the source code in the supplemental material, and all the hyperparameters we used are reported in Section 4.Guidelines:• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.• If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.• Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.• While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 
conclusion: 
 In this paper, we proposed a novel posterior label smoothing method, PosteL, designed to enhance node classification performance in graph-structured data. Justification: The abstract and introduction accurately explain our method.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper.• The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.• The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.• It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [Yes] Justification: We provide the source code for the proposed model, along with the environment required to reproduce it and the hyperparameter space we utilized.Guidelines:• The answer NA means that paper does not include experiments requiring code.• Please see the NeurIPS code and data submission guidelines (https://nips.cc/ public/guides/CodeSubmissionPolicy) for more details.• While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).• The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https://nips.cc/public/guides/CodeSubmissionPolicy) for more details.• The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.• The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.• At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).• Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: We provide all details in the appendix and experiments sections.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.• The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes] Justification: We provide the 95% confidence interval for all main experiments.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)• The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean.• It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.• For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).• If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes] Justification: We provide experiments computer resources in Appendix B.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.• The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.• The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.• The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [NA] Justification: This paper proposes a label smoothing method designed to improve the classification performance.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.• The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.• If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: This paper poses no risk for misuse.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.• Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.• We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: We cite the original paper that produced the code package and dataset.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL.• The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.• For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.• If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [Yes] Justification: We provide documentation for the code.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.• The paper should discuss whether and how consent was obtained from people whose asset is used. Justification: This work does not involve crowdsourcing.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.• Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.• For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: We provide detailed statistics about the dataset used for the experiments in In this section, we provide the computer resources and search space for model hyperparameters.Our experiments are executed on AMD EPYC 7513 32-core Processor and a single NVIDIA RTX A6000 GPU with 48GB of memory. We use the same model hyperparameter search space as He et al.[2021]. Specifically, we set the number of layers for all models to two. The dropout ratio for the linear layers is fixed at 0.5. For the GCN [Kipf and Welling, 2016] , the hidden layer dimension is set to 64. The GAT [Veličković et al., 2017] uses eight heads, each with a hidden dimension of eight.For the APPNP [Gasteiger et al., 2018] , a two-layer MLP with a hidden dimension of 64 is used, the power iteration step is set to 10, and the teleport probability is chosen from {0.1, 0.2, 0.5, 0.9}. For the MLP, the hidden dimension is set to 64. For the ChebNet [Defferrard et al., 2016] , the hidden dimension is set to 32, and two propagation steps are used. For the GPR-GNN [Chien et al., 2020] , a two-layer MLP with a hidden dimension of 64 is used as the feature extractor neural network, and the random walk path length is set to 10. The PPR teleport probability is chosen from {0.1, 0.2, 0.5, 0.9}.For BernNet [He et al., 2021] , a two-layer MLP with a hidden dimension of 64 is used as the feature extractor, and the polynomial approximation order is set to 10. The dropout ratio for the propagation layers in both GPR-GNN and BernNet is chosen from {0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9}.In this section, we provide a detailed analysis of the time complexity of Section 3.1. Specifically, we demonstrate the time complexity of obtaining the prior and likelihood distributions separately. Finally, we determine the time complexity of computing the posterior distribution using these distributions.First, the prior distribution P (Y i = m) can be obtained as follows:EQUATIONThe time complexity of calculating Equation ( 4) is O(|V|), so the time complexity of calculating the prior distribution for K classes is O(|V|K).Next, calculating the empirical conditional P (Y j = m|Y i = n, (i, j) ∈ E) from Equation (2) can be performed as follows:P (Y j = m|Y i = n, (i, j) ∈ E) ∝ u:u∈V,yu=n v∈N (u) e vm .(5) The time complexity of calculating Equation ( 5 In Section 3.2, iterative pseudo labeling is proposed, which involves iteratively refining the pseudo labels of validation and test nodes to calculate posterior labels. Since this process requires training the model from scratch for each iteration, the number of iterations can be a significant bottleneck in terms of runtime. Consequently, the iteration counts are evaluated to assess this aspect. The mean iteration counts for each backbone and dataset in Table 1 are summarized in Table 6 . With an overall mean iteration count of 1.13, we argue that this level of additional time investment is justifiable for the sake of performance enhancement.D Learning curves analysis for all datasets  


Decision: The reviewers have raised concerns about the limited novelty of the contributions, particularly due to similarities to Adaptive Label Smoothing (ALS) [1] and Structure-Aware Label Smoothing (SALS) [2]. While SALS is a preprint, it also raises questions about novelty. Although the authors clarified some differences, some concerns remained. The lack of theoretical analysis weakens the contribution further, and the empirical improvements are limited, with some lacking statistical significance.

Given these issues, the paper requires major revisions, and I recommend not accepting it in its current form.

[1] Zhou, Kaixiong, et al. “Adaptive Label Smoothing to Regularize Large-Scale Graph Training.” SDM 2023.

[2] Wang, Yiwei, et al. “Structure-Aware Label Smoothing for Graph Neural Networks.” arXiv preprint (2021).

Reviews: review: paper_summary: paper_summary: This work proposes a preprocessing step to refine labels of nodes in a structured graph that can benefit different graph-related transductive classification tasks. Inspired by the success of label smoothing in other machine learning tasks, the authors propose a label smoothing procedure based on a Bayesian inference that aggregates local and global information to estimate the soft labels. The procedure consists of mixing the soft and hard labels and an iterative regime akin to the Bayes update, which makes the method adaptive to different regularities present in different datasets. Authors conduct experiments applied to various models and datasets to support the efficacy of their methodology. They also provide an ablation study and further analyses of the results that shed light on different aspects of their proposal.
strength_weakness: strength_weakness: 1. **Related work**: Currently, the related works seem to provide references to earlier studies that, for the most part, motivate this work and are not methodologically close to it. For example, there is no reference to closely related works that either adopted label smoothing or conducted a very similar procedure in the context of graph data. Most notable is "[Adaptive Label Smoothing To Regularize Large-Scale Graph Training](https://epubs.siam.org/doi/abs/10.1137/1.9781611977653.ch7)" which appears to have a very similar procedure but a different approach to obtain the soft labels.

Moreover, the current statements imply that the current work is the first to suggest label smoothing for the graph data. To be more concrete, line 81 needs to be expanded, and more closely related works need to be discussed. For example, to compare the current approach and highlight similarities and key distinctions with earlier works that are closely related to it.

Some other related works could be the following: 
- [Structure-Aware Label Smoothing for Graph Neural Networks](https://arxiv.org/abs/2112.00499)
- [Label Efficient Regularization and Propagation for Graph Node Classification](https://ieeexplore.ieee.org/abstract/document/10234505)
- [Node Dependent Local Smoothing for Scalable Graph Learning](https://proceedings.neurips.cc/paper_files/paper/2021/hash/a9eb812238f753132652ae09963a05e9-Abstract.html)


2. **Design decisions and theory**: besides complexity analysis, the study could have been accompanied by convergence analysis and more theoretically founded justification. However, this does not reduce the value of the work as its empirical results provide a strong signal for the effectiveness of the method, which merits future work toward theoretical assessment and explanation of its success.

3. **Background information**: the classification task that uses the preprocessed smooth labels is not defined explicitly, which makes the work less accessible for the readers without prior knowledge.

4. **The IPL step** is proposed to address the presence of "unlabeled nodes"; however, it is hard to follow how the varying training size experiment reported in Table 4 is analogous to the unlabeled node scenario. Perhaps it is due to a lack of background information mentioned in point 3. 

5. **Suggestions to rephrase**:
   line 236: "mitigate the importance", perhaps some rephrasing is needed.
   line 189: "learning curve" -> "loss curve" 
   line 209: "when", some rephrasing might be needed
questions: questions: 1. Can you please elaborate on the similarities and distinctions of your work with [Adaptive Label Smoothing To Regularize Large-Scale Graph Training](https://epubs.siam.org/doi/abs/10.1137/1.9781611977653.ch7)? 

2. line 159, the experimental setup: it is a bit concerning that according to Figures 2, 6, and 7, the baselines appear to have a chance if the right regularization (other than label smoothing) and/or optimal early stopping is applied. What is your thought on that? Also, the weight decay grid search is very limited; is there any reason for that? Moreover, do you have a justification as to why we should not take the early stopping as a hyperparameter as opposed to the current setup that is fixed for each variation (baseline, +LS, +KD, +PosteL)?

3. Table 1: some explanation as to why certain models, such as GPR-GNN or BernNet, appear to not be benefiting from label smoothing.

4. line 117: it is mentioned that the condition is unnecessary based on empirical results. Could you please refer to the experiment in which this is being observed?

5. line 109: is this still true in the case of the IPL? In the IPL, the empirical distributions seem to be updated according to the pseudo-labels on the validation/test, correct?

6. Figures 2, 6, and 7: are they based on one split? Perhaps it would be useful to look at the mean and variance across all splits, similar to the reporting in Table 1.
limitations: limitations: Limitations are addressed in the body of the text. It is perhaps preferable to have the important limitations mentioned in a separate section or in the conclusion as well.
paper_summary: paper_summary: This paper introduces Posterior Label smoothing (PosteL), an innovative approach to enhance node classification on graph-structured data. PosteL integrates local neighborhood information with global label statistics to generate soft labels, aiming to improve model generalization and mitigate overfitting. The authors demonstrate the effectiveness of PosteL through extensive experiments on various datasets and models, showing significant performance improvements over baseline methods.
strength_weakness: strength_weakness: 1.	While the paper mentions the computational complexity, a deeper analysis or comparison with existing methods could provide more insight. For example, maybe you could provide some compared experiments with existing methods on time/resource consumption.
2.	The reliance on global label statistics might introduce bias in cases where the dataset has inherent class imbalance or label noise.
3.	The article "Rethinking the inception architecture for computer vision" appears twice in your reference list; please consolidate these entries. Carefully review your references to maintain standardization.
4.	The author compares two soft label methods that were proposed quite some time ago (from 2015 and 2016, respectively). Are there any experimental results comparing with more recent methods? Otherwise, the persuasiveness of the experiments might not be so strong.
5.	Please maintain consistent terminology throughout the text. The term "over-fitting" in line 46 should be changed to "overfitting" to be consistent with the rest of the context.
6.	Authors could provide more details on the sensitivity analysis of the hyperparameters α and β, which are crucial for the method's performance.
questions: questions: 1.	Could the authors provide more details on the implementation of the iterative pseudo labeling process?
2.	In the iterative pseudo labeling process, the validation and test sets are introduced to training. Whether the model will be overfitting to validation and test sets?
3.	What strategies were used to select the hyperparameters α and β, and how sensitive is the model performance to these choices?
4.	In Figures 6 and 7, it can be observed that not all the validation and test losses with the soft labels converge to lower losses than those with the ground truth labels. Does this indicate that the method has overly mitigated overfitting to the data?
5.	Regarding the overfitting effect mentioned in "Learning curves analysis," I am curious whether other soft label based methods can achieve the same effect, or is this a unique effect of the method presented in this paper?
limitations: limitations: Yes
paper_summary: paper_summary: The paper proposes PosteL, a label smoothing method utilizing posterior distribution for node classification in graph-structured data. It is basically a preprocessing method for GNNs, generating soft labels based on neighborhood context and global label statistics before the training phase.
strength_weakness: strength_weakness: 1. The authors select $\alpha$ and $\beta$ from a wide range but did not explore the parameter sensitivity of PosteL. The sensitivity to hyperparameters could be a potential limitation, necessitating careful tuning, which may reduce the credibility of the experiments.
2. The authors do not seem to clarify the difference between PosteL and other label smoothing methods for node classification (or methods that can be adapted to node classification), which makes the novelty of the method unclear. The paper could explore other smoothing techniques or baselines in more depth for a comprehensive comparison.
questions: questions: 1. Are there any examples to support the assumption that the neighborhood labels are conditionally independent given the label of the node to be relabeled (line 28)? It would be helpful for reader understanding to include this in the paper.
2. Why are the node features not considered in the relabeling process?
3. The authors demonstrate the effectiveness of PosteL with sparse labels, but the reasons behind are lacking explanation. Also, how does the performance compare to other preprocessing methods?
limitations: limitations: The authors discuss the case when the prior likely dominates the posterior, which limits the effectiveness of the proposed PosteL.
paper_summary: paper_summary: This paper proposed label-smoothing to improve the transductive node classification in GNN.
strength_weakness: strength_weakness: 1.	The paper could benefit from discussing related works that combine label-smoothing with Graph Neural Networks (GNNs), such as [1] and [2]. Including these would provide a more comprehensive context for the current research.

2.	The proposed method lacks a theoretical motivation or analysis. Providing this would strengthen the paper's scientific rigor and help readers better understand the underlying principles.

3.	The proposed method bears similarities to the approach in [1]. A direct comparison with this work would clarify the novel contributions of the current study and situate it within the existing research landscape.

4.	Iterative pseudo-labeling is a well-established technique in the field. The paper should address this, explaining how the current application differs from or builds upon previous uses of this method.
Addressing these points could significantly enhance the paper's depth and impact. 

---
[1]: Wang, Y., Cai, Y., Liang, Y., Wang, W., Ding, H., Chen, M., ... & Hooi, B. (2021). Structure-aware label smoothing for graph neural networks. arXiv preprint arXiv:2112.00499.

[2]: Zhang, Wentao, et al. "Node dependent local smoothing for scalable graph learning." Advances in Neural Information Processing Systems 34 (2021): 20321-20332.
questions: questions: See weaknesses
limitations: limitations: The limitation should be clarified in the paper. The current limitations are not clear.

Comments: title: Follow up comments and questions on the response of the authors
Feedback
comment: > Concerning Q2, First, a follow-up question: Loss curves for GT in Figure 2 from the paper and Figure 2 from the attached PDF do not match, e.g., GT labels in 1000 epochs get minimized up to ~0.4 loss, while in the new attached PDF for the same number of epochs, it gets minimized up to ~0.8. Similarly, for the PosteL, e.g., the difference between validation and test is more pronounced based on Figure 2 of the paper than the figure in the attachment. I was expecting them to be the same. Why are they different?
    
The experimental settings for these figures differ. In Figure 2 of the paper, we conducted experiments using the same learning and model hyperparameters to compare the patterns of the loss curves. In contrast, Figure 2 of the attached PDF shows experiments conducted with the best hyperparameters for each baseline. Additionally, the loss curve in Figure 2 of the attached PDF represents the mean loss across 10 different splits, whereas the loss curve in Figure 2 of the paper shows the loss from a single split. Therefore, the loss curves may differ.


> Second, I rephrase my question as I feel the main point is still not fully addressed: According to the original Figure 2 from the paper, early stopping at epoch ~50 provides a similar performance to Postel labels. That holds true for all datasets in Figure 6 and for some datasets in Figure 7. So, the question is, given that we can obtain comparable performance only by performing different early stopping for GT vs PosteL, why not do that?
    
We believe there may have been a miscommunication regarding early stopping in our paper. We evaluate test performance based on the model parameters that achieve the lowest validation loss, so the performance of GT is evaluated around epoch 50. The 200 epochs mentioned refer to the patience set for early stopping, which is configured to prevent excessively long training times.

It is an interesting observation that the loss for GT is similar to or even lower than that for PosteL. We found that even when the validaiton loss with the soft labels is greater than that of the GT labels, the accuracy of the PosteL is better than the model trained on GT labels. Due to the characteristics of the cross entropy loss, two different prediction results with the same label after argmax can have very different level of losses, e.g., for binary prediction, $\arg\max([0,1])=\arg\max([0.4,0.6])=1$. A similar phenomenon is observed in Figure 1 of [3], where some cases with label smoothing show higher loss than the baseline, yet their accuracy is greater. We want to emphasize that, despite this, our method often demonstrates lower test loss compared to GT, and the trend of the loss curve indicates a reduction in overfitting.

> Concerning Q3, my concern is addressed. However, I just want to clarify that my question was meant to be, "Why does GPR-GNN or BernNet appear to not benefit from label smoothing **as much**." It would be useful to add the explanation that the authors provided in their response to the paper as well.

Thank you for further explaining your question. We understand the point and will address it in the revised version.

[1] Wang, Yiwei, et al. "Structure-aware label smoothing for graph neural networks." arXiv preprint (2021).  
[2] Zhou, Kaixiong, et al. "Adaptive label smoothing to regularize large-scale graph training." SDM 2023.  
[3] Xu, Yi, et al. "Towards understanding label smoothing." arXiv preprint (2020).
Thank you for your efforts in making constructive discussions.

> Regarding Q1, the explanation provided is the type of information expected to be included in the paper. For the final revision of the paper, it is highly recommended that similarities be included and that the existence of [1] be acknowledged.

We completely agree. We will revise the current text to clarify that our work is not the first to suggest label smoothing for node classification. In the revised version, we will cite [1] and [2] and explain the similarities and differences between their approaches and ours.

> Likewise, it is important to highlight the key differences between each step in as much detail as possible. ALS's label propagation is similar to posterior label smoothing in this work, with the key difference being that the prior in the former is the hard labels while in the latter is the global statistics, which appears to be one of the elements that accommodate both homophilic and heterophilic graphs. A similar comparison is expected for the "label refinement" and "smooth pacing" steps from ALS. It would be great if the authors also provided such an explanation in their answer to this inquiry, which could further assist reviewers in their final opinion.
    
Thank you for highlighting the key differences between existing work and ours. 
    
Both our method and ALS propose approaches for incorporating label relationships in label smoothing. In our method, the neighborhood label distribution captures these relationships directly from global statistics, whereas ALS uses label refinement, which involves training a separate function during the training process. 

Next, ALS's smooth pacing is a method that gradually increases the proportion of smoothed labels compared to GT labels. This approach could be directly adapted to our method to adjust the parameter $\alpha$.

> Also, in Table 1 of the attached PDF, it would be great to highlight which datasets are homophilic and heterophilic. 
    
> A minor final point regarding Tables 1 and 5 of the paper: it is not self-evident which datasets are homophilic and which are heterophilic. This can be communicated more directly.
    
Thank you for your feedback. The five datasets on the left are homophilic, while the five on the right are heterophilic. We will highlight this in the revised version.
    
> Concerning the response to W3 & W4, while the current experiment conducted under section 4.3 is a valid experiment, it addresses the question of the efficiency of the model in learning. The title of the section, however, sets the expectation for the reader to see the performance of the model against different label sparsity ratios.
Here are some possibilities:  
a) varying node label sparsity across the graph but with a fixed training set size.  
b) varying node label sparsity across train and test sets. (so the sparsity is different per each set)  
c) fixed label sparsity across the graph, but varying training set size while test set size is fixed.  
The authors provide results for "case c", while "case a" seems to assess the robustness of the method against different sparsities because it addresses sparsity more directly, while "case c" includes the confounding effect of both train size and label sparsity.  
My recommendation is to rephrase the current section title, and if authors intend to report robustness against sparsity, "case a" is a better option. If they decide not to include such an experiment in their work, it is then suggested to make necessary clarifications in the title and body of the section corresponding to this experiment, and they may mention this as a future work.

We apologize for any confusion caused. We want to emphasize that in our setting, all of the training data is labeled. We use the term "sparse label" to refer to situations where the proportion of training data is small, resulting in a sparse labeled neighborhood. However, we understand that this terminology might confuse readers, so we will clarify the term and its explanation in the revised version.
I thank the authors for their diligence in addressing the concerns and questions.
Q4, Q5, and Q6 are addressed, and I have no further inquiry.

**Regarding Q1**, the explanation provided is the type of information expected to be included in the paper. For the final revision of the paper, it is highly recommended that similarities be included and that the existence of [1] be acknowledged. Likewise, it is important to highlight the key differences between each step in as much detail as possible. ALS's label propagation is similar to posterior label smoothing in this work, with the key difference being that the prior in the former is the hard labels while in the latter is the global statistics, which appears to be one of the elements that accommodate both homophilic and heterophilic graphs. A similar comparison is expected for the "label refinement" and "smooth pacing" steps from ALS. It would be great if the authors also provided such an explanation in their answer to this inquiry, which could further assist reviewers in their final opinion.

Also, in Table 1 of the attached PDF, it would be great to highlight which datasets are homophilic and heterophilic.


**Concerning the response to W3 & W4**, while the current experiment conducted under section 4.3 is a valid experiment, it addresses the question of the efficiency of the model in learning. The title of the section, however, sets the expectation for the reader to see the performance of the model against different label sparsity ratios.

Here are some possibilities:
- a) varying node label sparsity across the graph but with a fixed training set size.
- b) varying node label sparsity across train and test sets. (so the sparsity is different per each set)
- c) fixed label sparsity across the graph, but varying training set size while test set size is fixed.

The authors provide results for "case c", while "case a" seems to assess the robustness of the method against different sparsities because it addresses sparsity more directly, while "case c" includes the confounding effect of both train size and label sparsity.

My recommendation is to rephrase the current section title, and if authors intend to report robustness against sparsity, "case a" is a better option. If they decide not to include such an experiment in their work, it is then suggested to make necessary clarifications in the title and body of the section corresponding to this experiment, and they may mention this as a future work.


**Concerning Q2**,
First, a follow-up question:
Loss curves for GT in Figure 2 from the paper and Figure 2 from the attached PDF do not match, e.g., GT labels in 1000 epochs get minimized up to ~0.4 loss, while in the new attached PDF for the same number of epochs, it gets minimized up to ~0.8. Similarly, for the PosteL, e.g., the difference between validation and test is more pronounced based on Figure 2 of the paper than the figure in the attachment.

I was expecting them to be the same. Why are they different?

Second, I rephrase my question as I feel the main point is still not fully addressed:
According to the original Figure 2 from the paper, early stopping at epoch ~50 provides a similar performance to Postel labels.

That holds true for all datasets in Figure 6 and for some datasets in Figure 7. So, the question is, given that we can obtain comparable performance only by performing different early stopping for GT vs PosteL, why not do that?

 
**Concerning Q3**, my concern is addressed. However, I just want to clarify that my question was meant to be, "Why does GPR-GNN or BernNet appear to not benefit from label smoothing **as much**." It would be useful to add the explanation that the authors provided in their response to the paper as well.

A minor final point regarding Tables 1 and 5 of the paper: it is not self-evident which datasets are homophilic and which are heterophilic. This can be communicated more directly.

[1] Zhou, Kaixiong, et al. "Adaptive label smoothing to regularize large-scale graph training." SDM 2023.
Thanks for your response, I keep the original score unchanged.
Authors partially addressed some of my concerns. I have raised my score.
We thank the reviewer for a prompt response. While we do not request a change to the reviewer’s assessment, we do ask for the following that can assist other reviewers's fair assessment, as the current response lacks details.

As we have answered in the general response, we believe that our work is clearly different from [1] since [1] is designed under homophilic assumption, whereas our work is based on the assumption that the neighborhood label distribution should differ when the node label differs. The difference is reflected in the results with heterophilic datasets, and we do believe this is a significant difference. If [1] is similar to our work, could the reviewer elaborate on which aspects our approach is similar to [1]?

Just for reviewers' information, we also want to highlight that [1] is still in preprint and has never been published yet as far as we aware. We found some ambiguities in Equation 4 in their paper, where $c$ is used to define $\eta$, there is no information provided on the left side of the equation while implementing their work. We implemented the work based on our educated guesses, but the reproducibility of the work is limited. 

[1] Wang, Yiwei, et al. "Structure-aware label smoothing for graph neural networks." arXiv preprint (2021).
I appreciate the authors' response to our comments. However, upon careful consideration, I find that the proposed method shares significant similarities with the work presented in [1]. As a result, the primary contribution of this study appears to be limited to additional empirical observations. Given these factors, I maintain our original assessment and score.

[1] Wang, Yiwei, et al. "Structure-aware label smoothing for graph neural networks." arXiv preprint (2021).