Title: Gradient Rewiring for Editable Graph Neural Network Training
Venue: NeurIPS 2024
Accepted: yes
Keywords: Gradient

Graph Neural Networks
Abstract: Deep neural networks are ubiquitously adopted in many applications, such as computer vision, natural language processing, and graph analytics. However, well-trained neural networks can make prediction errors after deployment as the world changes. \textit{Model editing} involves updating the base model to correct prediction errors with less accessible training data and computational resources.
Despite recent advances in model editors in computer vision and natural language processing, editable training in graph neural networks (GNNs) is rarely explored. The challenge with editable GNN training lies in the inherent information aggregation across neighbors, which can lead model editors to affect the predictions of other nodes unintentionally. In this paper, we first observe the gradient of cross-entropy loss for the target node and training nodes with significant inconsistency, which indicates that directly fine-tuning the base model using the loss on the target node deteriorates the performance on training nodes. Motivated by the gradient inconsistency observation, we propose a simple yet effective \underline{G}radient \underline{R}ewiring method for \underline{E}ditable graph neural network training, named \textbf{GRE}. Specifically, we first store the anchor gradient of the loss on training nodes to preserve the locality. Subsequently, we rewire the gradient of the loss on the target node to preserve performance on the training node using anchor gradient. Experiments demonstrate the effectiveness of GRE on various model architectures and graph datasets in terms of multiple editing situations. The source code is available at \url{https://github.com/zhimengj0326/Gradient_rewiring_editing}.

Gradient Rewiring for Editable Graph Neural Network Training

Abstract

Deep neural networks are ubiquitously adopted in many applications, such as computer vision, natural language processing, and graph analytics. However, welltrained neural networks can make prediction errors after deployment as the world changes. Model editing involves updating the base model to correct prediction errors with less accessible training data and computational resources. Despite recent advances in model editors in computer vision and natural language processing, editable training in graph neural networks (GNNs) is rarely explored. The challenge with editable GNN training lies in the inherent information aggregation across neighbors, which can lead model editors to affect the predictions of other nodes unintentionally. In this paper, we first observe the gradient of crossentropy loss for the target node and training nodes with significant inconsistency, which indicates that directly fine-tuning the base model using the loss on the target node deteriorates the performance on training nodes. Motivated by the gradient inconsistency observation, we propose a simple yet effective Gradient Rewiring method for Editable graph neural network training, named GRE . Specifically, we first store the anchor gradient of the loss on training nodes to preserve the locality. Subsequently, we rewire the gradient of the loss on the target node to preserve performance on the training node using anchor gradient. Experiments demonstrate the effectiveness of GRE on various model architectures and graph datasets in terms of multiple editing situations. The source code is available at https://github.com/zhimengj0326/Gradient_rewiring_editing.

1 Introduction

Graph Neural Networks (GNNs) have demonstrated exemplary performance for graph learning tasks, such as recommendation, link prediction, molecule property analysis [1, 2, 3, 4, 5, 6, 7]. With message passing, GNNs learn node representations by recursively aggregating the neighboring nodes' representations. Once trained, GNN models are deployed to handle various high-stake tasks, such as credit risk assessment in financial networks [8] and fake news detection in social networks [9]. However, the impact of erroneous decisions in such influential applications can be substantial. For instance, misplaced credit trust in undetected fake news can lead to severe financial loss.

An ideal approach to tackle such errors should possess the following properties: 1) the ability to rectify severe errors in the model's predictions, 2) the capacity to generalize these corrections to other similar instances of misclassified samples, and 3) the ability to preserve the model's prediction accuracy for all other unrelated inputs. To achieve these goals, various model editing frameworks have been developed to rectify errors by dynamically adjusting the model's behavior when errors are detected [10, 11]. The core principle is to implement minimal changes to the model to correct the error while keeping the rest of the model's behavior intact. However, model editing is not a simple

plug-and-play solution. These frameworks often require an additional training phase to prepare for editing before they can be used effectively for editing [10, 11, 12, 13]. Although model editing techniques have shown significant utility in computer vision and language models, there is rare work focused on rectifying critical errors in graph data. The unique challenge arises from the inherent message-passing mechanism in GNNs when edits involve densely interconnected nodes [14, 15]. Specifically, editing the behavior of a single node can unintentionally induce a ripple effect, causing changes that propagate throughout the entire graph. [14] theoretically and empirically demonstrate the complexity of editing GNNs through the lens of the loss landscape of the Kullback-Lieber divergence between the pre-trained node features and edited final node embeddings. Moreover, a simple yet effective model structure, named EGNN, is proposed with stitched peer multi-layer perception (MLP), where only the stitched MLP is trained during model editing.

In this work, we investigate the model editing problem for GNNs from a brand-new gradient perspective , which is compatible with existing work [14]. Specifically, we first found a considerable inconsistency between the gradients of the cross-entropy loss for the target node and the training nodes for GNNs. Such inconsistency implies that direct fine-tuning of the base model using the loss of the target node can lead to a deterioration in the performance on the training nodes. Motivated by the above observation, we propose a simple yet effective Gradient Rewiring method for Editable graph neural network training, named GRE . Specifically, we first calculate and store the anchor gradient of the loss on the training nodes. This anchor gradient represents the original learning direction that we wish to preserve. Then, during the editing process, we adjust the gradient of the loss on the target node based on the stored anchor gradient. This adjustment, or 'rewiring', ensures that the changes made to the target node do not adversely affect the performance on the training nodes. Experiments demonstrate the effectiveness of our proposed method for various model structures and graph datasets. Moreover, the proposed method is compatible with the existing EGNN baseline and further improves the performance.

Flickr

Arxiv

A-photo

2 Preliminary and Related Work

We first introduce the notations used throughout this paper. A graph is given by G = ( V , E ) ,where V = ( v 1 , · · · , v N ) is the set of nodes indexed from 1 to n , and E = ( e 1 , · · · , e m ) ⊆ V × V is the set of edges. n = |V| and m = |E| are the numbers of nodes and edges, respectively. Let X ∈ R n × d be the node feature matrix, where d is the dimension of node features. A ∈ R n × n is the graph adjacency matrix, where A i,j = 1 if ( v i , v j ) ∈ E else A i,j = 0 . ˜ A = ˜ D -1 2 ( A + I ) ˜ D -1 2 is the normalized

adjacency matrix, where ˜ D is the degree matrix of A + I . The node label is defined as y i for node v i . We consider node classification tasks with C classes in this paper.

2.1 Graph Neural Networks

Graph Neural Networks have been successfully applied across various domains/tasks, including knowledge graphs [16, 17], graph condensation [18, 19, 20], event extraction [21], and entity relation tasks [22]. Most graph neural networks follow a neighborhood aggregation procedure to learn node representation via propagating representations of neighbors and then follow up with feature transformation [23]. The l -th layer of graph neural networks is given by:

where x ( l ) i is the representation of node v i at l -th layer and x (0) i is initialized as node feature x i , i..e, the i -th row at node feature matrix X . Many GNNs, such as GCN [24], GraphSAGE [25], and GAT [26], can be defined under this computation paradigm via adopting the different propagation and transformation operations. For example, the l -th layer in GCN can be defined as:

where X ( l ) ∈ R n × d and X ( l -1) ∈ R n × d are the node representation matrix containing the h v for each node v at the layer l and layer l -1 , respectively. W ( l ) ∈ R d × d is a layer-specific trainable weight matrix, and σ ( · ) is a non-linear activation function (e.g., ReLU).

2.2 Model Editing

Model editing aims to modify a base model's responses for a misclassified sample x tg and its analogs. This is typically achieved by fine-tuning the model using only a single pair of input x tg and the desired output y tg , while preserving the model's responses to unrelated inputs [10, 11, 12, 14]. Our contribution lies in the novel application of model editing to graph data, a domain where misclassifications on a few pivotal nodes can trigger substantial financial losses, fairness issues, or even the propagation of adversarial attacks. Consider the scenario of node classification where a well-trained GNN incorrectly predicts a particular node. Model editing can be employed to rectify this erroneous prediction. By leveraging the node's characteristics and the desired label, the model can be updated to correct such behavior. The ideal outcome of model editing is twofold: first, the updated model should correctly predict the specific node and its similar instances; second, the model should maintain its original behavior on unrelated inputs. It is important to note that some model editors require a preparatory training phase before they can be applied effectively [10, 13, 12, 11]. This crucial step ensures that the model editing process is both precise and effective in its application.

3 Methodology

In this section, we first provide the preliminary experimental results as the motivation to rewire gradients for model editing. Subsequently, we propose our gradient rewiring method for editable graph neural networks training (GRE) and an advanced version (GRE+) to improve the effectiveness of model editing, respectively.

3.1 Motivation

In the preliminary experiments, we first pre-train GCN, GraphSAGE, and MLP on the training dataset V train (e.g., Cora, Flickr, ogbn-arxiv, and Amazon Photo datasets) using cross-entropy loss. Subsequently, we find the misclassified samples in the validation dataset and randomly select one sample as the target sample ( x tg , y tg ) . During the model editing, we update the pre-trained model using cross-entropy loss over the target sample using gradient descent, i.e., the models are trained inductively . Following previous work [10, 12, 14], we perform 50 independent edits and report the averaged metrics.

It is well-known that model editing incurs training performance degradation [11, 10, 12] 1 for many model architectures. To deeply delve into the underlying reason, we investigate performance degradation from a model gradient perspective. We further define the training loss as L train = 1 |V train | ∑ i ∈V train CE ( f θ ( x i ) , y i ) , where f θ ( · ) ∈ R C is a prediction model parameterized with θ ∈ R L , C and L are the number of classes and model parameters, CE ( · , · ) is the cross-entropy loss, the target loss is given by L tg = CE ( f θ ( x tg ) , y tg ) . For example, model f θ ( · ) can be instantiated by GNNs with the number of layers defined in Eq. (1) or a simple MLP. For model editing, the gradient for training and target loss is given by g train = ∂ L train ∂θ ∈ R L and g tg = ∂ L tg ∂θ ∈ R L , respectively. To investigate why the model editing leads to training performance degradation, we use gradient RMSE (Root-Mean-Squared-Error), i.e., Grad RMSE = √ ∥ g train -g tg ∥ 2 2 , to measure the model editing discrepancy for training datasets and target sample.

The model editing curves for gradient RMSE 2 , training loss, and target loss across various model architectures (GCN, GraphSAGE, and MLP) are shown in Figure 1. Although the gradient RMSE for training datasets and target sample is close to 0, the model parameters demonstrate significant inconsistent behavior in terms of training loss due to large gradient discrepancy in the initial editing stage. We observe that: 1) Even though the target loss decreases during model editing, the training loss increases significantly. 2) The increasing rates of training loss for GCN and GraphSAGE are significantly higher than that of MLP. The above observations imply that editing training for graph neural networks is more challenging due to higher gradient discrepancy between the training dataset and the target sample.

3.2 Gradient Rewiring Approach

Preliminary results show a high discrepancy in training loss and target loss for GNNs, which implies that the vanilla model editing hampers the performance on the overall training dataset and thus results in a high accuracy drop for node classification tasks. Therefore, we aim to tackle the training dataset performance degradation from the gradient rewiring approach.

GRE We propose a simple yet effective gradient rewiring approach for editable graph neural network training, named GRE. We first formulate a constrained optimization problem to regulate model editing and then solve the constrained optimization problem via gradient rewiring.

Model editing aims to correct the prediction for the target sample while maintaining the prediction accuracy on the training nodes. The objective function focuses on minimizing the loss at the target node. To preserve the predictions on the training nodes, we introduce two constraints: (1) the training loss should not exceed its value prior to model editing (see Eq. (3)); and (2) the differences in model predictions after editing should remain within a predefined range (see Eq. (4)). Define θ 0 and θ ′ as the model parameters before and after model editing. Then we have the following constrained optimization problem:

where θ and θ ′ represent the model parameters before and after model editing, respectively, the hyperparameter δ ′ represents the maximum average prediction difference on training nodes. Notice that the model parameters update adopts gradient descent using target loss without any constraints, i.e., θ ′ = θ 0 -αg tg , where α is step size in model editing. The key idea of our proposed solution is to rewire gradient g tg as g , which is obtained by satisfying the involved constraints. Note that the model editing usually corrects the model prediction on the target sample within a few steps, i.e. there are no significant model parameter differences, thus we adopt Taylor expansion to tackle such constrained

optimization problem. For target loss L tg , we can approximate it as:

To optimize the objective function Eq. (2), it is easy to conclude that the gradient cosine similarity g ⊤ tg g should be maximized. Given the gradient before/after model editing is fixed, the maximization of gradient cosine similarity g ⊤ tg g is equivalent to the minimization of ∥ g tg -g ∥ 2 . To satisfy Eq. (3), we also adopt Taylor expansion on L train and it is easy to obtain that the gradient cosine similarity should be positive, i.e., g ⊤ tg g ≥ 0 . As for the constraint in Eq. (4), similarly, a Taylor expansion is used to express the relationship between the model predictions before and after the model editing, as follows:

Therefore, we can obtain the following approximation on Eq. (4):

where gradient for a model prediction is defined as ˆ g train = ∂ 1 |V train | ∑ i ∈V train f θ 0 ( x i ) ∂θ ∥ ∥ θ = θ 0 ∈ R L × C . Therefore, the model prediction difference constraint can be transformed into ∥ ˆ g ⊤ train g ∥ 2 ≤ ∥ ˆ g train ∥ 2 spect ∥ g ∥ 2 ≤ δ , where ∥ · ∥ spect represents matrix spectrum norm and ∥ ˆ g train ∥ spect is fixed in model editing, and δ = δ ′ α 2 . In a nutshell, our goal is to correct the target sample (i.e., minimize ∥ g tg -g ∥ 2 ) and minimize gradient discrepancy for model prediction among training dataset and target sample (i.e., ∥ g ∥ 2 ), while guaranteeing non-increased training loss (i.e., g ⊤ train g ≥ 0 ). The original constraint optimization problem is simplified as gradient rewiring, i.e.,

where λ ≥ 0 is the hyperparameter to control the balance between target sample correction and gradient discrepancy for model prediction. It is easy to obtain that Eq.(8) is a quadratic program (QP) in L -variables (the number of model parameters is usually high in neural networks). Fortunately, we can effectively solve this problem in the dual space via transforming as a smaller QP problem with only one variable v [27], where the relation between primal and dual variable is g train v -(1+ λ ) g = -g tg . Then we have the following problem:

It is easy to obtain the optimal dual variable v ∗ = -min { g ⊤ train g tg g train ⊤ g train , 0 } and the optimal rewired gradient g ∗ = (1 + λ ) -1 ( g tg -v ∗ g train ) . In other words, the gradient rewiring procedure is quite simple: for the gradient of the target loss g tg , reduce its projection component on g train and then scale it by (1 + λ ) -1 .

Additionally, we highlight that the gradient for training loss g train must be stored before model editing. In this way, gradient rewiring can be conducted to remove the harmful gradient component on target loss that increases training loss. Since shallow GNNs model performs well in practice [28], the model size of GNNs is small and the memory cost O ( L ) for storing anchor gradient is negligible.

GRE+ In GRE, the training loss after model editing is required not to be larger than that before model editing. However, it is still possible that the training loss on specific sub-training sets performs worse after model editing. At the same time, the training loss for the whole training dataset, after model editing, is on par with or even lower than that of before editing. To tackle this issue, we proposed an advanced gradient rewiring approach, named GRE+, via applying loss constraint on multiple disjoint sub-training sets. Specifically, we split training dataset V train into K sub-training sets {V 1 train , V 2 train , · · · , V K train } . Similarly, we define g k train = ∂ L k train ∂θ ∈ R L , where L k train = 1 |V k train | ∑ i ∈V k train CE ( f θ ( x i ) , y i ) .

Following the derivative clue in GRE, we can replace the training loss constraint on the whole training dataset with multiple training loss constraints on training subsets, and obtain the advanced gradient rewiring approach as follows:

Notice that Eq.(10) is a quadratic program (QP) in L -variables (the number of model parameters are usually high in neural networks), and we can effectively solve this problem in the dual space via transforming as a smaller QP problem with only K variables v ∈ R K [27]. Define gradient matrix as G = [ g 1 train , g 2 train , · · · , g K train ] ⊤ ∈ R K × L , then the relation between primal and dual variable is given by G v -(1 + λ ) g = -g tg . The original optimization problem can be transformed into the following dual problem:

The dual problem is a QP with K ≪ L variables, and we usually consider the value of K to be smaller than 5 in practice. Once we tackle dual QP problem (11) for v ∗ , we can recover the rewired gradient as g = (1 + λ ) -1 ( G v + g tg ) . Similarly, the gradient for training loss g k train , where 1 ≤ k ≤ K , is required to be stored before model editing. The corresponding memory cost is given by O ( KL ) .

4 Experiments

In this section, we conduct experiments to evaluate the effectiveness of our proposed GRE and GRE+, with the goal of answering the following three research questions. RQ1: Can the proposed solution correct the wrong model prediction with a lower accuracy drop after model editing in the independent and sequential editing setting? RQ2: What's the tradeoff performance between accuracy drop and success rate in the independent editing setting? RQ3: How sensitive are the proposed GRE and GRE+ methods to the key hyperparameter λ ?

4.1 Experimental Setting

We follow the standard experimental setting for GNNs [14]. Specifically, we first randomly split the train/validation/test dataset. Then, we ensure that each class has 20 samples in the training and 30 samples in the validation sets. The remaining samples are used for the test set. The target node is randomly selected 50 times from the validation set and the well-trained model makes the wrong prediction. The average model editing performance (e.g., success rate, drawdown) is reported for evaluation.

Datasets and Models. In our experiments, we utilize a selection of eight graph datasets from diverse domains, split evenly between small-scale and large-scale datasets. The small-scale datasets include Cora, A-computers [29], A-photo [29], and Coauthor-CS [29]. On the other hand, the large-scale datasets encompass Reddit [25], Flickr [2], ogbn-arxiv [3], and ogbn-products [3]. Note that our approach is based on gradient rewiring, which is orthogonal to model architectures. We adopt two prevalent models GCN [24] and GraphSAGE [25], where both of them are trained with the entire graph at each step. We evaluate our method under the inductive setting , which means the model is trained on a subgraph containing only the training node, and evaluated on the whole graph.

Baselines. Our methods are evaluated against three notable baselines: the traditional gradient descent editor (GD), the Editable Neural Network editor (ENN) [10], and editable training for GNNs[14]. 3 The GD editor is a straightforward application using gradient descent on the target loss with respect to the GNNs model parameters until the desired prediction outcome is achieved. ENN adopts a different approach by initially training the GNN parameters for a few steps to prime the model for subsequent edits. After this preparatory phase, ENN, like GD, applies the gradient descent on the parameters of GNN until the correct prediction is attained. EGNN [14] stitches a peer MLP and only trains MLP during model editing. Note that our method is compatible with EGNN, and different GNN architectures integrated with EGNN (e.g., EGNN-GCN, EGNN-GraphSAGE) are treated as distinct architectures.

Independent, sequential, and batch editing. All independent, sequential, and batch editing processes involve well-trained GNN models using training datasets, with target samples randomly selected multiple times from misclassified instances in the validation dataset. The key differences lie in the base model that needs to be edited. For independent editing, the same well-trained model using the training datasets is edited multiple times. In contrast, for sequential editing, the model is edited iteratively, with each editing step using the previously edited model from the last target sample, incorporating both the training datasets and partial samples from the validation dataset. For batch editing, all batched samples are edited simultaneously in one editing process. 4

Evaluation Metrics. Consistent with preceding studies [10, 12, 11], we assess the effectiveness of the various methods using two primary metrics: (1) Accuracy (Acc) : We use accuracy for the test dataset to evaluate the effectiveness after model editing. (2) DrawDown (DD) : This metric measures the mean absolute difference in test accuracy before and after model editing. A lower drawdown value signifies a superior editor locality. (3) Success Rate (SR) : This metric evaluates the proportion of edits in which the editor successfully amends the model's prediction. Both metrics offer a different perspective on the effectiveness of the editing process.

Table 2: The results on four large-scale datasets after applying one single edit. 'OOM' is the out-of-memory error. The best/second-best results are highlighted in boldface /underlined, respectively. The results for more backbones (e.g., MLP, EGNN-GCN, EGNN-SAGE) are in Appendix D.1.

4.2 Experimental Results in the Independent and Sequential Editing Setting

In many real-world scenarios, well-trained models often produce inaccurate predictions on unseen data. To evaluate the practical effectiveness of editors for independent editing ( RQ1 ), we randomly choose nodes from the validation set that were misclassified during the training. The editor is then

applied to rectify the model's predictions for these misclassified nodes, and we evaluate the drawdown and edit success rate on the test set.

We edit one random single node 50 times and report the mean and standard deviation results in Tables 1 and 2 for small-scale and large-scale graph datasets, respectively. Our observations are made below:

GCN

SAGE

❶ Contrasting model editing on textual data [11, 12, 30], all editors can effectively rectify model predictions in the graph domain. As shown in Table 1, all editors achieve a high success rate (typically from 96% ∼ 100% ) after editing GNNs, which is highly different from transformers with below 50% SR. This finding indicates that GNNs, unlike transformers, can be more easily adjusted to produce correct predictions. However, this improvement comes at the expense of substantial drawdown on other unrelated nodes, underscoring the key challenge of maintaining prediction locality for unrelated nodes before and after editing.

❷ Our proposed GRE and GRE+ notably surpass both GD and ENN in terms of test drawdown. This advantage stems mainly from the rewired gradient based on the pre-stored training loss gradient, which facilitates target sample correction while preserving the training loss. GD and ENN attempt to rectify model predictions by updating the parameters of GNNs without incorporating training loss information. In contrast, GRE and GRE+ maintain much better test accuracy after model editing. For example, for Amazon-photos, the accuracy drop dwindles from roughly 65 . 08% to around 43 . 87% , a 43 . 9% improvement over the baseline. This is due to the gradient rewiring approach that facilitates target sample correction while preserving the training loss. Interestingly, when applied to GNNs, ENN performs markedly worse than the basic editor GD. Moreover, GD performs well in MLP, which is consistent with the low gradient discrepancy of MLP.

❸ Our proposed GRE and GRE+ are compatible with EGNN and further improve the performance. Weobserve that while GRE occasionally underperforms, GRE+ consistently shows better performance than GD in reducing accuracy. For instance, when the A-computers dataset is evaluated with EGNNGCN, GRE, and GRE+ exhibit an average accuracy drop of 4 . 62% and 0 . 51% , respectively, whereas GD shows a decrease of 0 . 73% . Notably, we find that for 7 out of 8 datasets, GRE+ with EGNNSAGE shows a negative drop in accuracy, meaning that the test accuracy actually increases after model editing. This points towards the superior performance of the EGNN-SAGE model architecture.

In the sequential editing setting , we select a sequence of nodes from the validation set that were misclassified during the training phase. The editor is then used to iteratively correct the model's predictions for these sequentially misclassified nodes, and we measure the resulting drawdown and success rate of edits on the test set.

In Figure 2, we report the test accuracy drawdown in the sequential setting, a more challenging scenario that warrants further investigation. In particular, we plot the test accuracy drawdown compared to GD across various GNN architectures and graph datasets. Our observations are as follows: ❹ The proposed GRE and GRE+ consistently outperform GD in the sequential setting. However, the drawdown is significantly higher than in the single edit setting. For instance, GRE+ exhibits a 43 . 87% drawdown for GCN on the A-photo dataset in the single edit setting, which

escalates up to a 65% drawdown in the sequential edit setting. These results also highlight the challenge of maintaining the locality of GNN prediction in sequential editing. ❺ The improvement of GRE+ over GRE is quite limited in the sequential setting. For example, GRE+ exhibits a 24 . 52% drawdown over GRE for GCN on the A-photo dataset in the single edit setting while is on par with GRE in the sequential edit setting. These results further verify the difficulty of sequential editing and indicate more comprehensive training subset selection may be promising.

A-photos

ogbn-arxiv

Flickr

GCN

Graph-

SAGE

4.3 Trade-off Performance Comparison

We further compare the trade-off between the accuracy drawdown and the success rate of our method on various GNN architectures and graph datasets. As shown in Figure 3, we plot Pareto front curves by assigning different hyperparameters for the proposed methods. The upper-left corner point represents the ideal performance, i.e., the highest SR and lowest accuracy drawdown. The results show that GRE+ achieves better trade-off results compared to GRE, and all methods consistently maintain a high success rate on various GNN architectures and graph datasets.

GCN

Graph

-SAGE

ogbn-arxiv

Flickr

4.4 Hyperparameter Study

In this experiment, we investigate the sensitivity of our proposed method w.r.t. λ across a variety of GNN architectures and graph datasets. Specifically, we search for λ from the set of { 0 . 0 , 0 . 1 , 1 . 0 , 10 . 0 , 50 . 0 } . As shown in Figure 4, the test accuracy drop remains relatively stable despite variations in λ , suggesting that meticulous tuning of this parameter may not be crucial. For the ogbn-arxiv dataset, an uptick in accuracy drop corresponds with an increase in λ , reflecting the inherent difficulty of this dataset. Intriguingly, in the case of GRE+5 with 5 training subsets, the test accuracy drop exceeds that of GRE+2 and GRE+3, a pattern that diverges from the trend observed in other datasets.

5 Conclusion

In this paper, we explore the editing of graph neural networks from a new gradient perspective. Through empirical observations, we discover that conventional model editing techniques often underperform due to the gradient discrepancy between the training loss and target loss in GNNs. To address this issue, we propose a gradient rewiring approach. Specifically, we formulate a constrained optimization problem to regulate the model performance during model editing and identify a simple yet effective gradient rewiring approach to explicitly satisfy the constraints. In this way, the proposed approach can correct the target sample while preventing an increase in training loss. Experiments demonstrate the effectiveness of our approach, and our proposed method is also compatible with the existing baseline EGNN and can further improve performance. Future work includes more comprehensive training subset selection in GRE+ and a tailed approach for editable graph neural networks training in the sequential editing setting.

References

Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining , pages 974-983, 2018.

Hanqing Zeng, Hongkuan Zhou, Ajitesh Srivastava, Rajgopal Kannan, and Viktor Prasanna. Graphsaint: Graph sampling based inductive learning method. In International Conference on Learning Representations , 2020.

Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. arXiv preprint arXiv:2005.00687 , 2020.

Zhimeng Jiang, Xiaotian Han, Chao Fan, Zirui Liu, Na Zou, Ali Mostafavi, and Xia Hu. Fmp: Toward fair graph message passing against topology bias. arXiv preprint arXiv:2202.04187 , 2022.

Keyu Duan, Zirui Liu, Peihao Wang, Wenqing Zheng, Kaixiong Zhou, Tianlong Chen, Xia Hu, and Zhangyang Wang. A comprehensive study on large-scale graph training: Benchmarking and rethinking. arXiv preprint arXiv:2210.07494 , 2022.

Xiaotian Han, Zhimeng Jiang, Ninghao Liu, and Xia Hu. G-mixup: Graph data augmentation for graph classification. In International Conference on Machine Learning , pages 8230-8248. PMLR, 2022.

Hongyi Ling, Zhimeng Jiang, Meng Liu, Shuiwang Ji, and Na Zou. Graph mixup with soft alignments. In International Conference on Machine Learning . PMLR, 2023.

Daniele Petrone and Vito Latora. A dynamic approach merging network theory and credit risk techniques to assess systemic risk in financial networks. Scientific Reports , 8(1):5561, 2018.

Kai Shu, Amy Sliva, Suhang Wang, Jiliang Tang, and Huan Liu. Fake news detection on social media: A data mining perspective. ACM SIGKDD explorations newsletter , 19(1):22-36, 2017.

Anton Sinitsin, Vsevolod Plokhotnyuk, Dmitriy Pyrkin, Sergei Popov, and Artem Babenko. Editable neural networks. arXiv preprint arXiv:2004.00345 , 2020.

Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D Manning. Fast model editing at scale. arXiv preprint arXiv:2110.11309 , 2021.

Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D Manning, and Chelsea Finn. Memory-based model editing at scale. In International Conference on Machine Learning , pages 15817-15831. PMLR, 2022.

Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge in language models. arXiv preprint arXiv:2104.08164 , 2021.

Zirui Liu, Zhimeng Jiang, Shaochen Zhong, Kaixiong Zhou, Li Li, Rui Chen, Soo-Hyun Choi, and Xia Hu. Editable graph neural network for node classifications. arXiv preprint arXiv:2305.15529 , 2023.

Shaochen Zhong, Duy Le, Zirui Liu, Zhimeng Jiang, Andrew Ye, Jiamu Zhang, Jiayi Yuan, Kaixiong Zhou, Zhaozhuo Xu, Jing Ma, et al. Gnns also deserve editing, and they need it more than once. In International Conference on Machine Learning , 2024.

Yezi Liu, Qinggang Zhang, Mengnan Du, Xiao Huang, and Xia Hu. Error detection on knowledge graphs with triple embedding. In 2023 31st European Signal Processing Conference (EUSIPCO) , pages 1604-1608. IEEE, 2023.

Junnan Dong, Qinggang Zhang, Xiao Huang, Keyu Duan, Qiaoyu Tan, and Zhimeng Jiang. Hierarchy-aware multi-hop question answering over knowledge graphs. In Proceedings of the ACM Web Conference 2023 , pages 2519-2527, 2023.

Wei Jin, Lingxiao Zhao, Shichang Zhang, Yozen Liu, Jiliang Tang, and Neil Shah. Graph condensation for graph neural networks. arXiv preprint arXiv:2110.07580 , 2021.

Yezi Liu and Yanning Shen. Tinygraph: Joint feature and node condensation for graph neural networks. arXiv preprint arXiv:2407.08064 , 2024.

Qizhang Feng, Zhimeng Stephen Jiang, Ruiquan Li, Yicheng Wang, Na Zou, Jiang Bian, and Xia Hu. Fair graph distillation. Advances in Neural Information Processing Systems , 36:80644-80660, 2023.

Pei Chen, Hang Yang, Kang Liu, Ruihong Huang, Yubo Chen, Taifeng Wang, and Jun Zhao. Reconstructing event regions for event extraction via graph attention networks. In Kam-Fai Wong, Kevin Knight, and Hua Wu, editors, Proceedings of the 1st Conference of the AsiaPacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing , pages 811-820, Suzhou, China, December 2020. Association for Computational Linguistics.

Pei Chen, Haibo Ding, Jun Araki, and Ruihong Huang. Explicitly capturing relations between entity mentions via graph neural networks for domain-specific named entity recognition. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli, editors, Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers) , pages 735-742, Online, August 2021. Association for Computational Linguistics.

Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International conference on machine learning , pages 1263-1272. PMLR, 2017.

Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations , 2017.

William L Hamilton, Rex Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Proceedings of the 31st International Conference on Neural Information Processing Systems , pages 1025-1035, 2017.

Petar Veliˇ ckovi´ c, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In International Conference on Learning Representations , 2018.

Jorge Nocedal and Stephen J Wright. Quadratic programming. Numerical optimization , pages 448-492, 2006.

Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. Simplifying graph convolutional networks. In International conference on machine learning , pages 6861-6871. PMLR, 2019.

Oleksandr Shchur, Maximilian Mumme, Aleksandar Bojchevski, and Stephan Günnemann. Pitfalls of graph neural network evaluation. arXiv preprint arXiv:1811.05868 , 2018.

Zeyu Huang, Yikang Shen, Xiaofeng Zhang, Jie Zhou, Wenge Rong, and Zhang Xiong. Transformer-patcher: One mistake worth one neuron. In The Eleventh International Conference on Learning Representations , 2023.

Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina EliassiRad. Collective classification in network data. AI magazine , 29(3):93-93, 2008.

Mehrdad Farajtabar, Navid Azizan, Alex Mott, and Ang Li. Orthogonal gradient descent for continual learning. In International Conference on Artificial Intelligence and Statistics , pages 3762-3773. PMLR, 2020.

Gobinda Saha, Isha Garg, and Kaushik Roy. Gradient projection memory for continual learning. In International Conference on Learning Representations , 2021.

Cheng Chen, Ji Zhang, Jingkuan Song, and Lianli Gao. Class gradient projection for continual learning. In Proceedings of the 30th ACM International Conference on Multimedia , pages 5575-5583, 2022.

Christian Simon, Piotr Koniusz, Richard Nock, and Mehrtash Harandi. On modulating the gradient for meta-learning. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part VIII 16 , pages 556-572. Springer, 2020.

Aravind Rajeswaran, Chelsea Finn, Sham M Kakade, and Sergey Levine. Meta-learning with implicit gradients. Advances in neural information processing systems , 32, 2019.

Mikhail Khodak, Maria-Florina F Balcan, and Ameet S Talwalkar. Adaptive gradient-based meta-learning methods. Advances in Neural Information Processing Systems , 32, 2019.

Zhekai Du, Jingjing Li, Hongzu Su, Lei Zhu, and Ke Lu. Cross-domain gradient discrepancy minimization for unsupervised domain adaptation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 3937-3946, 2021.

Zhiqiang Gao, Shufei Zhang, Kaizhu Huang, Qiufeng Wang, and Chaoliang Zhong. Gradient distribution alignment certificates better adversarial domain adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 8937-8946, 2021.

A Experimental Setting

A.1 More details on EGNN

We provide more details on editable graph neural networks (EGNN), a method free of neighborhood propagation, designed specifically for correcting misclassified node predictions [14]. EGNN uniquely integrates a peer MLP (matching in the number of hidden units and layers) with GNNs (such as GCN and GraphSAGE), and trains solely the MLP model using the target loss during model editing. This strategy enables EGNN to leverage the propagation-free advantages of MLPs for model editing. However, it's important to note that EGNN is incompatible with EGNN, as the model editing in EGNN refines a stitched MLP, which is not employed during model training. Our proposed methodologies, GRE and GRE+, are founded on gradient rewiring, which is orthogonal to the EGNN approach. In the appendix, we illustrate how our proposed methodologies can further augment the effectiveness of the EGNN approach and MLP models.

A.2 Datasets

The statistical information of all datasets is summarized in Table 3. The details of the datasets utilized for node classification are described as follows:

Cora [31]: This citation network comprises 2,708 publications interconnected by 5,429 links. Each publication is characterized by a 1,433-dimensional binary vector that signifies the presence or absence of specific words from a predetermined vocabulary.

A-computers [29]: This dataset is a segment of the Amazon co-purchase graph. In this network, nodes denote goods, and GREes represent frequent co-purchases of two goods. Node features are encoded as bag-of-words product reviews.

A-photo [29]: Similar to A-computers, this is another segment of the Amazon co-purchase graph. Node features are also bag-of-words encoded product reviews.

Coauthor-CS [29]: Derived from the Microsoft Academic Graph from the KDD Cup 2016 challenge 3, this co-authorship graph has nodes representing authors who are linked if they have co-authored a paper. Node features denote paper keywords for each author's publications, while class labels indicate an author's most active fields of study.

Reddit [25]: This dataset is formulated from Reddit posts, with each node representing a post associated with different communities.

ogbn-arxiv [3]: This dataset represents the citation network among all arXiv papers. Each node denotes a paper, and each GREe signifies a citation between two papers. Node features are generated from the average 128-dimensional word vector of each paper's title and abstract.

ogbn-products [3]: This is an Amazon product co-purchasing network, where nodes represent Amazon products and GREes denote co-purchases of two products. Node features are created from low-dimensional representations of product description text.

A.3 Implementation Details

The hyperparameters for model architecture, learning rate, dropout rate, and training epochs are shown in Table 4. For EGNN, we also adopt GNNs and MLPs with hyperparameters in Table 4. For GRE, we use the hyperparameters γ = { 0 . 0 , 0 . 1 , 1 . 0 , 10 . 0 , 50 . 0 } . For GRE+, we also select hyperparameters γ = { 0 . 0 , 0 . 1 , 1 . 0 , 10 . 0 , 50 . 0 } and K = { 1 , 2 , 3 , 5 } . As for QP problem Eq. (11), we use a standard package qpsolvers with version 3.4.0 to tackle this QP problem with ecos solver.

A.4 Running Environment

For hardware configuration, all experiments are executed on a server with 251GB main memory, 24 AMD EPYC 7282 16-core processor CPUs, and a single NVIDIA GeForce-RTX 3090 (24GB). For software configuration, we use CUDA=11.3.1, python=3.8.0, pytorch=1.12.1, higher=0.2.1, torch-geometric=1.7.2, torch-sparse=0.6.16 in the software environment. Additionally, we use the package of higher in https://github.com/eric-mitchell/mend for ENN implementation.

B Limitations and Discussions

While our proposed GRE and GRE+ methods effectively mitigate the accuracy dropdown compared to conventional gradient descent algorithms, the success of our approaches is largely contingent on the precision of the pre-stored gradient for training loss. Despite the relatively few required model edit steps for single node editing, the accuracy of the pre-stored gradient may not sustain over long-term model editing, as the pre-stored gradient for training loss could exhibit significant discrepancy from the gradient of training loss for the edited model. To address such discrepancy, a straightforward strategy could involve leveraging critical training samples to estimate the true gradient of training loss for the edited model. Another possible direction is to identify critical samples instead of random samples for GRE+ with the aim of further constraining the model's behavior before and after model editing.

Notice that the proposed gradient rewiring method is not inherently specific to graphs, the gradient rewiring method is particularly suitable in the graph domain due to the small model size. Specifically, graph models are typically a few layers and thus are smaller in model size compared to models (e.g., Transformers) used in NLP and CV tasks. This results in lower computational and storage costs for gradients, making our strategy particularly suitable for the graph domain. Additionally, it is more challenging to edit nodes in a graph due to the inherent propagation process within neighborhoods. Such propagation may lead to significant gradient discrepancies within the graph domain.

C Algorithms

We show the algorithms of GRE and GRE+ during model editing in Algorithm 1 and 2, respectively.

D More Experimental Results

In this section, we present experimental results to showcase the improved efficacy of our proposed methods, GRE and GRE+. These techniques enhance the performance of EGNN, a specifically designed editable graph neural network, across both independent and sequential editing settings.

Algorithm 1 Gradient Rewiring Editable (GRE) Graph Neural Networks Training

1: Input: Target samples ( x tg , y tg ) , hyperparameter λ , well-trained GNN model f θ ( · ) , and its corresponding gradient for the training subgraph.

2: Output: Updated GNN model f θ ′ ( · ) .

̸

3: while f θ ( x tg ) = y tg do

4: Compute the model gradient g tg for the target loss L tg .

5: Rewire the target loss gradient g tg by reducing the projection component on g train, then scale with (1 + λ ) -1 :

6: g ∗ = (1 + λ ) -1 ( g tg -v ∗ g train ) .

7: Replace g tg with g ∗ and update the model parameters using the optimizer to obtain θ ′ .

8: end while

Algorithm 2 Gradient Rewiring Editable Plus (GRE+) Graph Neural Networks Training

1: Input: Target samples ( x tg , y tg ) , hyperparameters λ , well-trained GNNs model f θ ( · ) , and its corresponding model gradient for training subgraph.

2: Output: Editable GNNs model f θ ′ ( · ) .

3: while f θ ( x tg )! = y tg do

4: Compute model gradient g tg for target loss L tg .

5: Solve QP problem Eq. (11) via standard QP solver package and obtain the optimal dual variable v ∗ .

6: Calculate the rewired gradient using g ∗ = (1 + λ ) -1 ( G v ∗ + g tg ) .

7: Replace g tg with g ∗ and then adopt optimizer to update model parameters as θ ′ .

8: end while

D.1 More results on Model Architectures for Independent Editing

In this section, we conduct a sequence of 50 single-node edits and present the mean and standard deviation results in Tables 5 for large-scale graph datasets. Similarly, GRE occasionally underperforms, and GRE+ consistently shows better performance than GD with respect to the reduction in accuracy. For instance, when the Reddit dataset is evaluated with EGNN-GCN, GRE and GRE+ exhibit an average accuracy drop of 1 . 48% and -0 . 21% , respectively, whereas GD shows a decrease of 1 . 28% . Moreover, GRE+ with EGNN-SAGE shows a negative drop in accuracy among 6 out of 8 datasets, i.e., the test accuracy actually increases after model editing.

D.2 Experimental Results on other Model Architectures for Sequential Editing Setting

In the sequential editing setting, we take a sequence of 50 misclassified nodes and use the editor to iteratively correct the model's predictions for EGNN across different GNN architectures. The test accuracy drop associated with various model editing methods for different graph datasets is reported in Figure 7. Our observations indicate that the proposed GRE and GRE+ methods consistently outshine GD in this sequential setting. For instance, with the Coauthor-CS dataset and EGNN-GCN, our proposed methods achieve virtually no decrease in accuracy, while GD exhibits a drop of over 7% . Another compelling observation is that the improvement demonstrated by our methods over GD for EGNN is markedly larger than for GNNs. This suggests potential synergies between optimizer selection and model architecture design.

A-photos

ogbn-

arxiv

Flickr

EGNN-

GCN

EGNN-

SAGE

D.3 Trade-off Performance Comparison on other Model Architectures

We extend our evaluation by comparing the trade-off between the accuracy drop and success rate of our method on EGNN across various graph datasets. By adjusting different hyperparameters for the proposed methods, we construct Pareto front curves as shown in Figure 5. The results underscore that both GRE+ and GRE outperform GD in achieving superior trade-off outcomes. Importantly, our proposed methods exhibit robust preservation of the success rate across various GNN architectures and graph datasets.

D.4 More Experimental results on Batch Editing

In this section, we present the experimental results of applying batch editing on four small-scale datasets (Table 6) and four large-scale datasets (Table 7).

For the small-scale datasets, our proposed methods, GRE and GRE+, consistently outperform the baseline methods. For example, on the Cora dataset, GRE+ achieves the highest accuracy with a minimal drawdown and a high success rate. Specifically, GRE+ can reduce 56 . 9% and 30 . 3% drawdown compared with 2nd best baseline in GCN and GraphSAGE architectures, respectively. For the large-scale datasets, GRE+ again demonstrates superior performance. On ogbn-products datasets GRE+ can reduce 2 . 5% and 0 . 5% drawdown compared with 2nd best baseline GRE in GCN and GraphSAGE architectures, respectively, while maintaining a high success rate.

D.5 The Edit Time and Memory Comparison for Editing Methods

In this section, we present the experimental results of the edit time and memory required for editing across four large-scale datasets (Table 8).

We observe that GRE+ takes 1 . 5 2 . 5 × wall-clock editing time than the GD/GRE editor in terms of the wall-clock edit time. This is because GRE+ requires QP solver to obtain the rewired gradient. In terms of memory consumption, the overall memory overhead is insignificant. For example, GRE+ (5) requires 17 . 9% GPU memory than GD editor in obgn-products dataset and GCN architecture. The reason is that the anchor gradient is required to store in memory and QP solver computation in memory.

D.6 More Test Accuracy Results on Sequential Editing

In this subsection, we present the after-editing test accuracy results of applying sequential editing on various datasets for both GCN and GraphSAGE models in Figure 6. The test accuracy is reported as a percentage for each dataset.

Overall, our proposed methods, GRE and GRE+, consistently outperform the baseline methods GD and ENN across all datasets in terms of test accuracy. For example, on the Reddit dataset, the proposed methods can achieve more than 100% improvement over GD and ENN in terms of accuracy. Besides, compared with GRE and GRE+, the improvement is marginal on most occasions except A-computer dataset in GraphSAGE, which indicates the limited effectiveness of the fine-grained gradient rewiring in GRE+.

E More Related Work

Gradient-based method for other tasks. The existing literature on gradient modification mainly incorporates continual learning and meta learning. In continual learning, work [32] proposes gradient projection methods to update the model with gradients in the orthogonal directions of old tasks, without access to old task data. GPM [33] identifies the bases of these subspaces by examining network representations after learning each task using Singular Value Decomposition (SVD) in a single-shot manner and stores them in memory as gradient projection memory. Class gradient projection is proposed in [34] to address the class deviation in gradient projection. In meta-learning, work [35] proposes a meta-learning algorithm to learn to modulate the gradient in the absence of abundant data. The implicit model-agnostic meta-learning (iMAML) algorithm is developed in [36] for optimization-based meta-learning with deep neural networks that remove the need for differentiating through the optimization path. [37] provides a theoretical framework for designing and understanding practical meta-learning methods that integrate sophisticated formalizations of task-similarity.

GCN

SAGE

F More discussion

Comparison with Curriculum Learning. Curriculum learning and model editing are two distinct approaches in the field of machine learning. Curriculum learning is an approach where the network is trained in a structured manner, starting with simpler tasks and gradually introducing more complex ones. This method aims to improve the learning process by mimicking how humans learn. Model editing is a fast and efficient approach to patch the well-trained model prediction for several failed test cases. Although both are multi-stage training stages, there are several key differences: (1) Goals: Curriculum Learning aims to improve the overall learning process by structuring the training data in a way that mimics human learning. In contrast, model editing aims to make targeted adjustments to a pre-trained model to correct undesirable behaviors. (2) Approach: curriculum learning mainly focuses on the sequence and complexity of the training data. Model editing typically modifies the model's parameters or architecture to correct undesirable behavior goals. (3) Additional information in the multi-stage process. Model editing requires failure feedback for well-trained models as the target samples to patch, e.g., test failure cases after production is launched. In other words, such feedback can only be obtained after model pertaining. In curriculum learning, all information is given in multi-stage training. In summary, curriculum learning focuses on structuring the training process to improve overall learning, while model editing focuses on making targeted adjustments to a pre-trained model to correct specific behaviors. Both approaches can be complementary and used together to achieve better model performance.

Comparison with Domain Adaptation. To the best of our knowledge, many existing methods in domain adaptation (DA) [38, 39] integrate source and target gradients in the loss function. For example, [38] aims to minimize the gradient discrepancy for unsupervised DA, and [39] aligns gradient distribution for better adversarial DA. However, these methods can not be applied in graph model editing since (1) gradient discrepancy is required to successfully edit model prediction; (2) model editing collapses (i.e., no gradient discrepancy) at the initial stage; (3) regulating gradient behavior is insufficient for model editing task since the main problem is how to get an edited model instead of cross-domain generalization. To this end, we rewire the gradient before the model parameters update, and derive a closed-form, instead of a learning-based, gradient rewiring method to accelerate model editing.

EGNN-GCN

EGNN-SAGE

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We list our main claims and contributions as the bullet items at the end of the introduction

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: Please check Appendix B

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: In this paper, we don't include theoretical proofs.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We provide a clear algorithm in the method section. Besides, we provide implemental details in the experiment section and appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often

one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We plan to clean up and release the code during the camera-ready stage.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines (https://nips.cc/public/ guides/CodeSubmissionPolicy) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: //nips.cc/public/guides/CodeSubmissionPolicy) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We include all implemented details in the appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: For the experimental results, we include the standard deviation.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We provide compute resources in Appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?

Answer: [Yes]

Justification: We confirm the paper meets the NeurIPS Code of Ethics in every respect.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [No]

Justification: This paper has no positive societal impacts or negative societal impacts.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper poses no safeguard risk

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We follow the license of the existing paper and assets

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: This paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjec Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nModel editing involved fine-tuning a pre-trained model specifically on training examples where it cannot predict the correct output in order to correct these errors. While model training has been explored for CV and NLP models, GNNs pose a unique challenge due to the unordered data type and node-level classification problem. This paper attempts to rectify this error by proposing a Gradient Rewiring+ (GRE+) method for Editable graph neural network training while preserving the original train/test set performance. GRE+ is evaluated on several small and large classification datasets.\n\nSTRENGTHS:\n- The proposed method aims to rectify errors that still persist after training, without compromising the original performance. To quantify this, the authors report three metrics: The accuracy after model editing has been performed, the DropDown metric measuring how this impacts the original performance and the success rate of the method at actually rectifying the error. The three of these provide a clear, multi-faceted view of the effect of the proposed method against the baselines. \n- In most situations, the proposed GRE+ proves to be the best method.\n- The introduction and motivation of the paper is generally well-written and easy to follow.\n\nWEAKNESSES:\n- Biggest difference is the presentation and decision to treat GRE and GRE+ as different methods to compare to baseline approaches. This is because the experimental results in Tables 1 and 2 show that GRE on its own is not really noteworthy as it fails to consistently rank as the 2nd best behind GRE+ and is outperformed by GD and ENN on numerous occassions. \n- This hampers the writing in the method which focuses on what GRE does first, before expanding to talk about GRE+ with more specifics. In contrast to this writing decision, it would be better if GRE and GRE+ were not treated separately, GRE meant GRE+ and instead of having 2 rows for your method in Tabs 1/2, you just had GRE(+), then later did an ablation study (where one option is the GRE config as mentioned in the submitted paper) to show the efficacy/necessity of the whole method.\n- Line 243 This is the first time the word \"Transformers\" is mentioned in the paper and its jarringly brought up out of the blue. The introduction broadly mentioned that model editing exists for CNNs and NLP tasks but not specifically the Transformer architecture. This prose should be deeply revised before the paper is publication ready.\n- There is no analysis/study of the cost/complexity of the model editing methods which is a major weakness. Looking at Fig 3 while it is true that GRE+ improves little/none over GRE, in Tab 1/2 the story is much different. There does not seem to be a situation where GRE is better so I must question why it is presented as a distinct approach itself compared to GRE+. Is it because it is computationally less expensive?\n- Some statements are not properly substantiated with citations:\n\t- L105 \"It is well-known that model editing incurs training performance degredation\"\n\t- L167 \"Since shallow GNNs model performs well in practice\" seems like a handwave. Also, spelling/grammar check.\n- Other nitpicks:\n\t- Tab 1 caption mentions \"OOM\" for some methods but that is only used in entries for Tab - Fig 2 should have larger fonts it is very hard to read.\n\t- L242, should read \", respectively.\" when enumerating like that.\n\nQUESTIONS:\n- Fig 3 where are results on ENN?\n\nLIMITATIONS:\nA limitations section is provided in the supplementary material.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-06-18"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a model editing technique motivated by an observed inconsistency between the gradient of target and training nodes' cross-entropy losses. Their proposed method, Gradient Rewiring for Editable Graph Neural Networks (GRE), stores the anchor of the gradients of the training nodes and uses the anchors during editing to preserve performance. Finally, the authors empirically evaluate their proposed method on a collection of real-world datasets.\n\nSTRENGTHS:\n- I believe the problem to be an important practical problem and that the paper is well-motivated.\n- The main observation of the paper, i.e., that the gradients of the target and the training nodes are inconsistent, is interesting and novel in the context of Graph Neural Networks.\n- The proposed method seems to perform well consistently. The authors have performed a comprehensive empirical evaluation, and I believe the results are solid.\n- The authors are overcoming the quadratic problem in the number of model parameters by solving a dual problem with significantly fewer variables.\n\nWEAKNESSES:\n- The method requires storing the gradients of the training dataset. This would raise the memory consumption. It would be great if the authors could include a table with the memory requirements for different models. \n- There are a lot of writing/grammar issues. In the following, I will give a (non-exhaustive) list of examples of missing articles and text where the writing could generally be improved. This would generally not be a big deal, but in this case, the writing makes the text very hard to follow and comprehend. I would encourage the authors to do a few revision rounds before updating the manuscript. In my personal experience, free tools such as Grammarly also help with identifying weird formulations or grammar mistakes:\n\t- 16-17: the gradient of *the* loss\n\t- Line 21: \"interpreting the features and topology of graph data\" - *interpreting* is a somewhat unusual way to describe learning representations on graphs; please consider rephrasing. \n\t- Line 42: \"(...) intricate editing for GNNs through the lens of landscape.\" - this phrasing is very confusing; what landscape? I suggest the authors rephrase to something more akin to \"through the lens of the loss landscape of the Kullback-Lieber divergence between the initial node features and the final node embeddings\" since this seems to be the technique that [[1]] is proposing.\n\t- Line 46: \"(...) perspective, and is compatible with existing work\" - please consider rephrasing to something similar to \"(...) perspective, which is compatible with existing work\".\n\t- Line 49: \"(...) can lead to a deterioration in the performance *of* the training nodes\" -> \"(...) can lead to a deterioration in the performance *on* the training nodes\" - the performance is of the model on the training nodes\n\t- Line 55: similar, \"performance *of* the training nodes\" -> \"performance *on* the training nodes\"\n\t- The caption of Figure 1 is somewhat confusing; please consider rephrasing.\n\t- Line 93-94: Somewhat confusing: \"motivation to rewire gradients *of* model editing\" or \"motivation to rewire gradients *for* model editing\"?\n\t- Line 95: \"and advanced version (GRE+)\" -> \"and *an* advanced version (GRE+)\"\n\t- Lines 98-99: \"we pre-train (...) on (...) datasets\" -> \"we pre-train (...) on *the* (...) datasets\"\n\t- Lines 101-102: please consider rephrasing to something more akin to \"we fine-tune the well-trained model using the cross-entropy loss of the target sample via gradient descent\"\n\t- Lines 106-108: \"we investigate performance degradation from model gradient perspective\" -> \"we investigate *the* performance degradation from *a* model gradient perspective\"; \"we further define training loss\" -> \"we further define *the* training loss\"; \"where ... is prediction model ... CE is cross-entropy loss\" -> \"where ... is *a* prediction model ... CE is *the* cross-entropy loss\".\n\t- Footnote 3 is very confusing; please consider rephrasing the entire footnote.\n\t- Lines 131-147 are also somewhat confusing and hard to comprehend due to language. Please consider rephrasing all of the paragraphs.\n\t- Line 148: \"where gradient for model prediction is defines as\" -> \"where *the* gradient for *the* model prediction is *defined* as\". Also, the inline *\\frac* is not aesthetically pleasing, but that is a very minor complaint.\n\t- Line 161: Please consider either removing \"*that*\": \"it is easy to obtain *that* the optimal dual variable v=...\" -> \"it is easy to obtain the optimal dual variable $v*$=...\" or rephrasing to something more akin to \"it is easy to see that the optimal dual variable is $v*$=...\"\n\t- Please also consider rephrasing the next lines, from 162 to 164.\n\t- Missing commas on line 172: \"the training loss for the whole training dataset, after model editing, is on par with (...)\"\n\t- Please consider re-phrasing lines 199-200 to something more akin to \"We randomly select a node from the validation set on which the well-trained model makes a wrong prediction.\"\n- Both Tables 1 and 2 have results with different decimal precision for the standard deviation. For instance, GraphSAGE/GRE/CORA/DD has a single decimal ($3.36 \\pm 0.2$) while GraphSAGE/GRE+/CORA/DD contains two decimals ($0.41\\pm 0.07$). Please consider using the same amount of decimals for all of the results. It makes the table more aesthetically pleasing and easier to read.\n- \"OOM\" is mentioned in the caption of Table 1 as being an out-of-memory error, but there are no OOMs in Table 1. However, Table 2 contains OOMs but the caption does not explain what OOM means.\n- Please consider increasing the font of the text for both Figures 1 and 2.\n- Figure 4, first row, first column: the figure covers half the \"N\" of \"GCN\". The plots are also not vertically aligned on the two columns.\n\nOverall, I believe that the proposed method is solid due to the very good empirical results. However, the writing is lacking to a degree that makes the text very hard to follow and comprehend. I would encourage the authors to revise the text significantly, with multiple rounds of proofreading.\n\nDue to concerns about the clarity and writing, I currently recommend rejection. However, I will revise my score if the authors significantly improve the writing during the rebuttal. I think that the overall work could potentially be impactful and useful from a practitioner's perspective.\n\n\n[1]: https://arxiv.org/pdf/2305.15529\n\nQUESTIONS:\nPlease see the weaknesses above.\n\nLIMITATIONS:\nThe authors are discussing limitations in the appendix.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe work introduces a novel method called Gradient Rewiring (GRE) to address the challenge of editable training in graph neural networks. Traditional fine-tuning approaches often struggle with maintaining performance for both target and training nodes. GRE aims to overcome this limitation by rewiring gradients in a way that preserves locality, leading to improved performance for both types of nodes. The method is designed to enhance the training process in graph neural networks by effectively updating node representations while maintaining the network's overall structure.\n\nSTRENGTHS:\n1. The method of the paper is clear, intuitive, and easy to implement.\n\n2. The writing is clear, making it easy to understand and read.\n\nWEAKNESSES:\n1. Although the experiments were conducted on graph datasets, the proposed method is not specifically designed for graphs. This general approach can be tested on various tasks.\n\n2. There are noticeable formatting errors and typos in the paper.\n\nQUESTIONS:\n1. It would be beneficial to provide a more detailed explanation of the experimental process and results in Figure 1. In the first row of Figure 1, the images indicate that the gradients for testing and training become very similar, suggesting that the test loss and train loss should decrease simultaneously. However, in the second and third rows, the test loss decreases while the training loss increases. Please further explain this observation.\n\n2. The training + rewriting pipeline is similar to curriculum learning, where the network is trained in two phases: first on simple samples, then on more difficult ones. I recommend the authors conceptually compare their method with curriculum learning. Additionally, I suggest experimenting with batch data rewriting, which might be more practical compared to rewriting individual samples or sample sequences.\n\nLIMITATIONS:\nThe limitations has been discussed in the Appendix.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles the challenge of editing GNNs. The authors highlight a key issue in GNN editing: the gradient inconsistency between target and training nodes, which can degrade performance when the model is fine-tuned using only the target node’s loss. To address this, they introduce the Gradient Rewiring (GRE) method, which preserves the performance of training nodes by storing an anchor gradient and rewiring the target node’s loss gradient accordingly. The efficacy of GRE is validated through experiments across various GNN architectures and graph datasets, demonstrating its potential to enhance model adaptability without compromising existing accuracies.\n\nSTRENGTHS:\n- This paper is well-motivated and well-organized, addressing a research question that remains to be further investigated. The provided visualization also aids in understanding the motivations behind the study.\n- The authors offer a detailed derivation process, making the methodology easy to follow.\n- The method was tested on large-scale datasets, demonstrating the model's scalability.\n\nWEAKNESSES:\n- The significance of editing graph neural networks remains unclear. Editing seems to lead to significant performance degradation, especially on large-scale datasets. The authors should discuss why such significant efforts are warranted to modify predictions on individual samples.\n- I recognize that considering graph neural network editing from the perspective of gradient rewriting is novel, but the authors have only discussed model editing in related work. The lack of gradient modification literature makes it hard to position this paper.\n- Some experimental details are missing, such as dataset splitting. These details are crucial for evaluating the proposed method.\n- The time complexity is not provided.\n- Some minor issues (e.g., a superscript linking to nothing in the caption of Figure 1).\n\nQUESTIONS:\n- In what scenarios would people be willing to accept a substantial overall performance drop in order to modify predictions for individual samples?\n- Are there any methods developed for editing neural networks on non-graph data based on gradient modification? Or are there any works on gradient modification that are relevant to the method in this paper?\n\nLIMITATIONS:\nAs the authors discussed in Appendix B.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-15"}