Original: GNND ELETE : A G ENERAL U NLEARNING S TRATEGY FOR G RAPH N EURAL N ETWORKS
Revised: GNND ELETE : A G ENERAL U NLEARNING S TRATEGY FOR G RAPH N EURAL N ETWORKS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We consider the problem of graph unlearning, wherein graph neural network (GNN) model is trained to specified accuracy and then deployed while a sequence of requests arrives to delete graph elements (nodes, edges) from the model.
Revised: We consider the problem of graph unlearning, wherein graph neural network (GNN) model is trained to specified accuracy and then deployed while a sequence of requests arrives to delete graph elements (nodes, edges) from the model.

Original: As GNN models are used in real-world implementation, this problem is increasingly vital to address—for example, when a user seeks to hide their connections with others in a social graph or when relationships in a knowledge graph become irrelevant or are not true anymore.
Revised: As GNN models are used in real-world implementation, this problem is increasingly vital to address—for example, when a user seeks to hide their connections with others in a social graph or when relationships in a knowledge graph become irrelevant or are not true anymore.

Original: To unlearn information from a trained GNN, its influence on both
Revised: To unlearn information from a trained GNN, its influence on both

Original: GNN model weights as well as on representations of neighbors in the graph must be deleted from the model.
Revised: GNN model weights as well as on representations of neighbors in the graph must be deleted from the model.

Original: However, existing methods using retraining and weight modification either degrade model weights shared across all nodes or are ineffective because of strong dependency of deleted edges on their local graph neighborhood.
Revised: However, existing methods using retraining and weight modification either degrade model weights shared across all nodes or are ineffective because of strong dependency of deleted edges on their local graph neighborhood.

Original: Realizing these pitfalls, we formalize the required properties for graph unlearning in the form of Deleted Edge Consistency and Neighborhood Influence and develop
Revised: Realizing these pitfalls, we formalize the required properties for graph unlearning in the form of Deleted Edge Consistency and Neighborhood Influence and develop

Original: GNND ELETE , a model-agnostic layer-wise operator that optimize both properties for unlearning tasks.
Revised: GNND ELETE , a model-agnostic layer-wise operator that optimize both properties for unlearning tasks.

Original: GNND ELETE updates latent representations to delete nodes and edges from the model while keeping the rest of the learned knowledge intact.
Revised: GNND ELETE updates latent representations to delete nodes and edges from the model while keeping the rest of the learned knowledge intact.

Original: Experiments on six real-world and two knowledge graphs show that GNND ELETE outperforms existing graph unlearning models by up to 36 .
Revised: Experiments on six real-world and two knowledge graphs show that GNND ELETE outperforms existing graph unlearning models by up to 36 .

Original: 9% in AUC on link prediction tasks and 22 .
Revised: 9% in AUC on link prediction tasks and 22 .

Original: 5% in AUC on distinguishing deleted edges from nondeleted edges.
Revised: 5% in AUC on distinguishing deleted edges from nondeleted edges.

Original: GNND ELETE efficient—e.g., it takes 12.3x less time and 9.3x less space than retraining from scratch on WordNet18.
Revised: GNND ELETE efficient—e.g., it takes 12.3x less time and 9.3x less space than retraining from scratch on WordNet18.

Original: Our code is available here.
Revised: Our code is available here.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Graph neural networks (GNN) are increasingly used in real-world implementations (Xu et al., 2022; 2019; Huang et al., 2021; Morselli Gysi et al., 2021), and the underlying graphs evolve over time in most deployed GNNs.
Revised: Graph neural networks (GNN) are increasingly used in real-world implementations (Xu et al., 2022; 2019; Huang et al., 2021; Morselli Gysi et al., 2021), and the underlying graphs evolve over time in most deployed GNNs.

Original: Traditional machine learning approaches often work offline, wherein a model is trained once using the full training dataset and then locked for inference with few, if any, updates to the model.
Revised: Traditional machine learning approaches often work offline, wherein a model is trained once using the full training dataset and then locked for inference with few, if any, updates to the model.

Original: In contrast, online training can update the model using new training data points as they become available (Orabona, 2019; Nagabandi et al., 2018).
Revised: In contrast, online training can update the model using new training data points as they become available (Orabona, 2019; Nagabandi et al., 2018).

Original: However, neither offline nor online learning can deal with data deletion (Cao & Yang, 2015b; Ginart et al., 2019a)—the task of removing all traces of a data point from a model without sacrificing model performance.
Revised: However, neither offline nor online learning can deal with data deletion (Cao & Yang, 2015b; Ginart et al., 2019a)—the task of removing all traces of a data point from a model without sacrificing model performance.

Original: When data needs to be deleted from a model, the model must be updated accordingly.
Revised: When data needs to be deleted from a model, the model must be updated accordingly.

Original: For example, GNNs must implement privacy provisions for preserving personal privacy (e.g., California Consumer Privacy Act (CCPA) and General Data Protection Regulation (GDPR)), meaning that endowing GNNs with data deletion capability is important yet sparsely studied in the literature (Ginart et al., 2019b; Fu et al., 2022).
Revised: For example, GNNs must implement privacy provisions for preserving personal privacy (e.g., California Consumer Privacy Act (CCPA) and General Data Protection Regulation (GDPR)), meaning that endowing GNNs with data deletion capability is important yet sparsely studied in the literature (Ginart et al., 2019b; Fu et al., 2022).

Original: At the same time, GNNs received significant interest from the learning community due to the ubiquity of graph-structured data (Hu et al., 2020;
Revised: At the same time, GNNs received significant interest from the learning community due to the ubiquity of graph-structured data (Hu et al., 2020;

Original: Ying et al., 2019).
Revised: Ying et al., 2019).

Original: However, despite methods developed for general machine unlearning, none of them can be readily applied to GNNs because graph machine unlearning (as we show in this paper) is fundamentally different because of dependencies between nodes connected by edges.
Revised: However, despite methods developed for general machine unlearning, none of them can be readily applied to GNNs because graph machine unlearning (as we show in this paper) is fundamentally different because of dependencies between nodes connected by edges.

Original: In addition, since the current success of GNN models relies on the idea of "exchanging information in the local neighborhood," an adversarial agent can trivially infer the presence of the data from its neighbors if the impact of data on its local neighborhood is not limited.
Revised: In addition, since the current success of GNN models relies on the idea of "exchanging information in the local neighborhood," an adversarial agent can trivially infer the presence of the data from its neighbors if the impact of data on its local neighborhood is not limited.

Original: Given the broad range of GNN applications and the lack of general graph unlearning methods, there is a need to design algorithms that enable GNN models to unlearn what they have previously learned.
Revised: Given the broad range of GNN applications and the lack of general graph unlearning methods, there is a need to design algorithms that enable GNN models to unlearn what they have previously learned.

Original: This would mean that unavailable, outdated, or privacy-concerned graph elements are not used by the a Deleted Edge Consistency b Neighborhood Influence c Overview of GNND ELETE
Revised: This would mean that unavailable, outdated, or privacy-concerned graph elements are not used by the a Deleted Edge Consistency b Neighborhood Influence c Overview of GNND ELETE

Original: G NN l a y e r
Revised: G NN l a y e r

Original: G NN l a y e r G NN l a y e r ⋯
Revised: G NN l a y e r G NN l a y e r ⋯

Original: Message passing
Revised: Message passing

Original: Aggregation
Revised: Aggregation

Original: Update 𝑊 𝐷 = argmin 𝑊 𝐷 𝜆𝐿 𝐷𝐸𝐶 +(1−𝜆)𝐿 𝑁𝐼
Revised: Update 𝑊 𝐷 = argmin 𝑊 𝐷 𝜆𝐿 𝐷𝐸𝐶 +(1−𝜆)𝐿 𝑁𝐼

Original: Activation
Revised: Activation

Original: Deletion
Revised: Deletion

Original: Freezedmodel parameters 𝑾 𝐷
Revised: Freezedmodel parameters 𝑾 𝐷

Original: Learnable deletion operator 𝜙(𝒉 𝑢′ ,𝒉 𝑣′ )
Revised: Learnable deletion operator 𝜙(𝒉 𝑢′ ,𝒉 𝑣′ )

Original: − 𝜙(𝒉 𝑝 ,𝒉 𝑞 )=
Revised: − 𝜙(𝒉 𝑝 ,𝒉 𝑞 )=

Original: 𝛿 𝑁 𝑣 𝑁 𝑢
Revised: 𝛿 𝑁 𝑣 𝑁 𝑢

Original: Before deletion:
Revised: Before deletion:

Original: Afterdeletion: ≈ 𝜓( ) 𝜓( )
Revised: Afterdeletion: ≈ 𝜓( ) 𝜓( )

Original: Deleted edge 𝑒 𝑢𝑣
Revised: Deleted edge 𝑒 𝑢𝑣

Original: Unconnectednode pairs
Revised: Unconnectednode pairs

Original: G NN
Revised: G NN

Original: G NN 𝑁 𝑢 𝑁 𝑣 𝑆 𝑢𝑣 𝑁 𝑝 𝑁 𝑞
Revised: G NN 𝑁 𝑢 𝑁 𝑣 𝑆 𝑢𝑣 𝑁 𝑝 𝑁 𝑞

Original: Node
Revised: Node

Original: Edge to delete
Revised: Edge to delete

Original: Edge
Revised: Edge

Original: Representation before deletion
Revised: Representation before deletion

Original: Representation after deletion 𝜙,𝜓 Vector operation model anymore, preventing security concerns and performance drops.
Revised: Representation after deletion 𝜙,𝜓 Vector operation model anymore, preventing security concerns and performance drops.

Original: In this paper, we take a step towards building an efficient and accurate machine unlearning method for GNNs.
Revised: In this paper, we take a step towards building an efficient and accurate machine unlearning method for GNNs.

Original: Designing graph unlearning methods is nonetheless challenging.
Revised: Designing graph unlearning methods is nonetheless challenging.

Original: Removing data alone is insufficient to comply with recent demands for increased data privacy because models trained on the original data may still contain information about their patterns and features.
Revised: Removing data alone is insufficient to comply with recent demands for increased data privacy because models trained on the original data may still contain information about their patterns and features.

Original: A naive approach is to delete the data and retrain a model from scratch.
Revised: A naive approach is to delete the data and retrain a model from scratch.

Original: However, this can be prohibitively expensive, especially in large datasets.
Revised: However, this can be prohibitively expensive, especially in large datasets.

Original: Recently, many efforts have been made to achieve efficient unlearning based on the idea of exact learning (Brophy & Lowd, 2021; Sekhari et al., 2021;
Revised: Recently, many efforts have been made to achieve efficient unlearning based on the idea of exact learning (Brophy & Lowd, 2021; Sekhari et al., 2021;

Original: Hase et al., 2021; Ullah et al., 2021).
Revised: Hase et al., 2021; Ullah et al., 2021).

Original: The core idea is to retrain several independent models by dividing datasets into separate shards and then aggregating their predictions during inference.
Revised: The core idea is to retrain several independent models by dividing datasets into separate shards and then aggregating their predictions during inference.

Original: Such methods guarantee the removal of all information associated with the deleted data.
Revised: Such methods guarantee the removal of all information associated with the deleted data.

Original: However, in the context of GNN, dividing graphs destroys the structure of the input graph and thus leads to poor performance on edge-level tasks, such as link prediction.
Revised: However, in the context of GNN, dividing graphs destroys the structure of the input graph and thus leads to poor performance on edge-level tasks, such as link prediction.

Original: To address this issue, Chen et al.
Revised: To address this issue, Chen et al.

Original: (2021c) applies the exact learning to the graph-structured data by proposing a particular graph partition method that can preserve the structural information and weighted prediction aggregation for inference.
Revised: (2021c) applies the exact learning to the graph-structured data by proposing a particular graph partition method that can preserve the structural information and weighted prediction aggregation for inference.

Original: However, this is still less efficient as the cost increase as the number of shards grows.
Revised: However, this is still less efficient as the cost increase as the number of shards grows.

Original: In addition, choosing the optimal number of shards is still unresolved and may require extra hyperparameter tuning.
Revised: In addition, choosing the optimal number of shards is still unresolved and may require extra hyperparameter tuning.

Original: Several approximation-based approaches
Revised: Several approximation-based approaches

Original: Guo et al.
Revised: Guo et al.

Original: ; Ullah et al.
Revised: ; Ullah et al.

Original: ; He et al.
Revised: ; He et al.

Original: ; Shibata et al.
Revised: ; Shibata et al.

Original: (2021) avoid retraining a model from scratch on multiple subsets.
Revised: (2021) avoid retraining a model from scratch on multiple subsets.

Original: While these approaches have shown promise, Mitchell et al.
Revised: While these approaches have shown promise, Mitchell et al.

Original: (2022) show that the methods produce model updates that can harm model performance.
Revised: (2022) show that the methods produce model updates that can harm model performance.

Original: Present Work.
Revised: Present Work.

Original: We introduce GNND ELETE 1 , a model-agnostic approach for graph unlearning.
Revised: We introduce GNND ELETE 1 , a model-agnostic approach for graph unlearning.

Original: We formalize two key GNN deletion properties (Section 4.1): 1)
Revised: We formalize two key GNN deletion properties (Section 4.1): 1)

Original: Deleted Edge Consistency:
Revised: Deleted Edge Consistency:

Original: Predicted probability for deleted edges of the unlearned model should be similar to those for nonexistent edges.
Revised: Predicted probability for deleted edges of the unlearned model should be similar to those for nonexistent edges.

Original: The property enforces GNND ELETE to unlearn information such that deleted edges masquerade as unconnected nodes.
Revised: The property enforces GNND ELETE to unlearn information such that deleted edges masquerade as unconnected nodes.

Original: 2) Neighborhood Influence: We establish a connection between graph unlearning and Granger causality (Granger, 1969) to ensure that local subgraphs after deletion are not affected and thus maintain the original predictive dependencies.
Revised: 2) Neighborhood Influence: We establish a connection between graph unlearning and Granger causality (Granger, 1969) to ensure that local subgraphs after deletion are not affected and thus maintain the original predictive dependencies.

Original: However, existing graph unlearning methods do not consider this essential property, meaning they do not consider local connectivity influence, leading to sub-optimal deletion.
Revised: However, existing graph unlearning methods do not consider this essential property, meaning they do not consider local connectivity influence, leading to sub-optimal deletion.

Original: Using both properties, we introduce GNND ELETE , a layer-wise deletion operator to update node representations.
Revised: Using both properties, we introduce GNND ELETE , a layer-wise deletion operator to update node representations.

Original: When receiving deletion requests, GNND ELETE freezes the model and learns additional small gated weight matrices that are shared across all nodes.
Revised: When receiving deletion requests, GNND ELETE freezes the model and learns additional small gated weight matrices that are shared across all nodes.

Original: Unlike existing methods that attempt to retrain several small models from scratch or directly update model weights which can be inefficient and suboptimal, GNND ELETE uses small learnable matrices for inference without changing GNN model weights, achieving both efficiency and scalability.
Revised: Unlike existing methods that attempt to retrain several small models from scratch or directly update model weights which can be inefficient and suboptimal, GNND ELETE uses small learnable matrices for inference without changing GNN model weights, achieving both efficiency and scalability.

Original: To optimize GNND ELETE , we specify a novel objective function (described in Section 4.3) that satisfies
Revised: To optimize GNND ELETE , we specify a novel objective function (described in Section 4.3) that satisfies

Original: Deleted Edge Consistency and Neighborhood Influence, yielding strong overall deletion.
Revised: Deleted Edge Consistency and Neighborhood Influence, yielding strong overall deletion.

Original: Our Contributions.
Revised: Our Contributions.

Original: We summarize our contributions as follows: 1 ⃝ We formalize the machine unlearning problem and suggest two helpful properties, Deleted Edge Consistency and Neighborhood
Revised: We summarize our contributions as follows: 1 ⃝ We formalize the machine unlearning problem and suggest two helpful properties, Deleted Edge Consistency and Neighborhood

Original: Influence , of machine unlearning in the context of graphs.
Revised: Influence , of machine unlearning in the context of graphs.

Original: 2 ⃝ We propose a GNN data deletion approach GNND ELETE .
Revised: 2 ⃝ We propose a GNN data deletion approach GNND ELETE .

Original: It unlearns the deleted data much more efficiently than existing graph unlearning methods without sacrificing predictive power.
Revised: It unlearns the deleted data much more efficiently than existing graph unlearning methods without sacrificing predictive power.

Original: 3 ⃝ We show that GNND ELETE is of particular interest as it is agnostic to the class of models and training methodology and so is extremely flexible.
Revised: 3 ⃝ We show that GNND ELETE is of particular interest as it is agnostic to the class of models and training methodology and so is extremely flexible.

Original: 4 ⃝ Theoretically, we show the difference between node representations from the baseline and GNND ELETE is bounded, which ensures GNND ELETE ’s strong performance.
Revised: 4 ⃝ Theoretically, we show the difference between node representations from the baseline and GNND ELETE is bounded, which ensures GNND ELETE ’s strong performance.

Original: 5 ⃝ We provide empirical results for both link and node deletion tasks.
Revised: 5 ⃝ We provide empirical results for both link and node deletion tasks.

Original: The results suggest that GNND ELETE can lead to effective unlearning using 12 .
Revised: The results suggest that GNND ELETE can lead to effective unlearning using 12 .

Original: 3 × less time and 9 .
Revised: 3 × less time and 9 .

Original: 3 × computation than retraining from scratch.
Revised: 3 × computation than retraining from scratch.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Machine Unlearning.
Revised: Machine Unlearning.

Original: We organize existing machine unlearning literature into four categories: 1)
Revised: We organize existing machine unlearning literature into four categories: 1)

Original: Re-training: Re-training models from scratch to unlearn is simple yet can be inefficient despite efforts to develop efficient data partitioning and re-training (Bourtoule et al., 2021;
Revised: Re-training: Re-training models from scratch to unlearn is simple yet can be inefficient despite efforts to develop efficient data partitioning and re-training (Bourtoule et al., 2021;

Original: Wu et al., 2020; Liu et al., 2022b; Cao & Yang, 2015a;
Revised: Wu et al., 2020; Liu et al., 2022b; Cao & Yang, 2015a;

Original: Golatkar et al., 2020; Izzo et al., 2021).
Revised: Golatkar et al., 2020; Izzo et al., 2021).

Original: Nevertheless, partitioning graphs is not easy because the graph structure and learned node representations of the shared graphs can be significantly different from the original graph.
Revised: Nevertheless, partitioning graphs is not easy because the graph structure and learned node representations of the shared graphs can be significantly different from the original graph.

Original: Moreover, such methods may not scale to large datasets.
Revised: Moreover, such methods may not scale to large datasets.

Original: Output modification: To reduce computational overhead, methods, such as UNSIR (Tarun et al., 2021) directly modify model outputs.
Revised: Output modification: To reduce computational overhead, methods, such as UNSIR (Tarun et al., 2021) directly modify model outputs.

Original: UNSIR first learns an error matrix that destroys the output and then trains the destroyed model for two epochs with clean data to impair and repair the outputs.
Revised: UNSIR first learns an error matrix that destroys the output and then trains the destroyed model for two epochs with clean data to impair and repair the outputs.

Original: On graphs, the error destroys outputs for all edges, and the training after that falls back to re-training the whole model 3)
Revised: On graphs, the error destroys outputs for all edges, and the training after that falls back to re-training the whole model 3)

Original: Logit manipulation: Other methods achieve unlearning by manipulating the model logits (Izzo et al., 2021; Baumhauer et al., 2020).
Revised: Logit manipulation: Other methods achieve unlearning by manipulating the model logits (Izzo et al., 2021; Baumhauer et al., 2020).

Original: These methods only apply to linear or logit-based models.
Revised: These methods only apply to linear or logit-based models.

Original: Weight modification: Unlearning via weight modification is achieved by running an optimization algorithm.
Revised: Weight modification: Unlearning via weight modification is achieved by running an optimization algorithm.

Original: For example, Ullah et al.
Revised: For example, Ullah et al.

Original: (2021) proposed a new algorithmic machine unlearning based on noisy stochastic gradient descent.
Revised: (2021) proposed a new algorithmic machine unlearning based on noisy stochastic gradient descent.

Original: Guo et al.
Revised: Guo et al.

Original: (2019) achieves certified removal based on Newton update.
Revised: (2019) achieves certified removal based on Newton update.

Original: Other weight modification via optimization methods include Thudi et al.
Revised: Other weight modification via optimization methods include Thudi et al.

Original: (2021a)
Revised: (2021a)

Original: and Neel et al.
Revised: and Neel et al.

Original: Recent unlearning methods perturb gradients (Liu et al., 2020b) or model weights (Chen et al., 2021a).
Revised: Recent unlearning methods perturb gradients (Liu et al., 2020b) or model weights (Chen et al., 2021a).

Original: However, weight modification approaches lack features unique to graphs and incur computation overheads (e.g., calculation of the inverse Hessian).
Revised: However, weight modification approaches lack features unique to graphs and incur computation overheads (e.g., calculation of the inverse Hessian).

Original: In Appendix A, we elaborate further on machine unlearning methods for other tasks and models.
Revised: In Appendix A, we elaborate further on machine unlearning methods for other tasks and models.

Original: Graph Unlearning.
Revised: Graph Unlearning.

Original: GraphEraser
Revised: GraphEraser

Original: (Chen et al., 2021c) tries to solve the graph unlearning problem with graph partitioning and efficient retraining.
Revised: (Chen et al., 2021c) tries to solve the graph unlearning problem with graph partitioning and efficient retraining.

Original: They extend the k -means algorithm to consider both node features and structural information when dividing graphs into shards.
Revised: They extend the k -means algorithm to consider both node features and structural information when dividing graphs into shards.

Original: A learnable aggregator is optimized to aggregate the predictions from each sharded model.
Revised: A learnable aggregator is optimized to aggregate the predictions from each sharded model.

Original: However, GraphEraser (Chen et al., 2021c) is designed for the node classification task, which hinders its applications on other tasks such as link prediction.
Revised: However, GraphEraser (Chen et al., 2021c) is designed for the node classification task, which hinders its applications on other tasks such as link prediction.

Original: GraphEditor (Cong & Mahdavi) relies on the closed-form solution of linear
Revised: GraphEditor (Cong & Mahdavi) relies on the closed-form solution of linear

Original: GNNs for guaranteed information deletion, including node deletion, edge deletion, and node feature update.
Revised: GNNs for guaranteed information deletion, including node deletion, edge deletion, and node feature update.

Original: Performing additional finetuning can boost predictive performance.
Revised: Performing additional finetuning can boost predictive performance.

Original: However, GraphEditor is restricted to linear structure, which is the same case for most machine unlearning algorithms (not only limited to the graph structured data),
Revised: However, GraphEditor is restricted to linear structure, which is the same case for most machine unlearning algorithms (not only limited to the graph structured data),

Original: Thus, it cannot work with existing trained non-linear GNNs or knowledge graphs.
Revised: Thus, it cannot work with existing trained non-linear GNNs or knowledge graphs.

Original: It also has trouble unlearning a large portion of nodes or edges.
Revised: It also has trouble unlearning a large portion of nodes or edges.

Original: Connection with Catastrophic Forgetting and Dynamic Network Embeddings.
Revised: Connection with Catastrophic Forgetting and Dynamic Network Embeddings.

Original: Despite the fact of forgetting knowledge, we argue that catastrophic forgetting (Kemker et al., 2018) is not a good fit for the machine unlearning setting.
Revised: Despite the fact of forgetting knowledge, we argue that catastrophic forgetting (Kemker et al., 2018) is not a good fit for the machine unlearning setting.

Original: Specifically, catastrophic forgetting is 1) not targeted at a particular set of data, 2) not guaranteed to forget, i.e. adversarial agents may still find out the presence of what is forgotten in the training set; 3) not capable of handling an arbitrary amount of deletion requests.
Revised: Specifically, catastrophic forgetting is 1) not targeted at a particular set of data, 2) not guaranteed to forget, i.e. adversarial agents may still find out the presence of what is forgotten in the training set; 3) not capable of handling an arbitrary amount of deletion requests.

Original: Dynamic network embedding (Nguyen et al., 2018) learns network representations by incorporating time information and can deal with data deletion.
Revised: Dynamic network embedding (Nguyen et al., 2018) learns network representations by incorporating time information and can deal with data deletion.

Original: However, these methods are designed to process graphs that are inherently changing over time.
Revised: However, these methods are designed to process graphs that are inherently changing over time.

Original: On the other hand, machine unlearning aims at deleting a specific set of training data, while keeping the majority of the underlying graph fixed.
Revised: On the other hand, machine unlearning aims at deleting a specific set of training data, while keeping the majority of the underlying graph fixed.

Original: It aims at targeted data removal requests and is thus orthogonal to the above two approaches.
Revised: It aims at targeted data removal requests and is thus orthogonal to the above two approaches.

Original: Connection with Adversarial Attacks and Defense on GNNs.
Revised: Connection with Adversarial Attacks and Defense on GNNs.

Original: To determine whether a given data point has been used in the training process of a model, Membership Inference (MI) is a suitable measure for the quality of unlearning (Yeom et al., 2017; Sablayrolles et al., 2019).
Revised: To determine whether a given data point has been used in the training process of a model, Membership Inference (MI) is a suitable measure for the quality of unlearning (Yeom et al., 2017; Sablayrolles et al., 2019).

Original: Defense against
Revised: Defense against

Original: MI attack is also a key problem we care about when building unlearning models.
Revised: MI attack is also a key problem we care about when building unlearning models.

Original: Thudi et al.
Revised: Thudi et al.

Original: (2022) proposed to use a novel privacy amplification scheme based on a new tighter bound and subsampling strategy.
Revised: (2022) proposed to use a novel privacy amplification scheme based on a new tighter bound and subsampling strategy.

Original: Olatunji et al.
Revised: Olatunji et al.

Original: (2021) showed that all GNN models are vulnerable to MI attack and proposed two defense mechanisms based on output perturbation and query neighborhood perturbation.
Revised: (2021) showed that all GNN models are vulnerable to MI attack and proposed two defense mechanisms based on output perturbation and query neighborhood perturbation.

Original: Liu et al.
Revised: Liu et al.

Original: (2022a) treat the data to be unlearned as backdoored data.
Revised: (2022a) treat the data to be unlearned as backdoored data.

Original: Although such works can serve as an interesting unlearning measure, they aim at defending against MI attacks and can not perform the real unlearning procedure, which is a complementary topic to machine unlearning.
Revised: Although such works can serve as an interesting unlearning measure, they aim at defending against MI attacks and can not perform the real unlearning procedure, which is a complementary topic to machine unlearning.

Original: Connection with Techniques for Achieving Privacy in ML Models.
Revised: Connection with Techniques for Achieving Privacy in ML Models.

Original: Privacy-preserving is another related privacy-preserving learning scheme, such as federated learning (FL) (Koneˇcný et al., 2016).
Revised: Privacy-preserving is another related privacy-preserving learning scheme, such as federated learning (FL) (Koneˇcný et al., 2016).

Original: To let FL models unlearn data in a similar privacy-preserving way, Liu et al.
Revised: To let FL models unlearn data in a similar privacy-preserving way, Liu et al.

Original: (2020a) proposed the first federated unlearning framework by leveraging historical parameter updates.
Revised: (2020a) proposed the first federated unlearning framework by leveraging historical parameter updates.

Original: Chen et al.
Revised: Chen et al.

Original: (2021d) proposed a new attack and metrics to improve privacy protection, which provides insights on practical implementations of machine unlearning.
Revised: (2021d) proposed a new attack and metrics to improve privacy protection, which provides insights on practical implementations of machine unlearning.

Original: Golatkar et al.
Revised: Golatkar et al.

Original: (2021) proposed machine unlearning in a mixed-privacy setting by splitting the weights into a set of core and forgettable user weights.
Revised: (2021) proposed machine unlearning in a mixed-privacy setting by splitting the weights into a set of core and forgettable user weights.

Original: Different from privacy-preserving algorithms that aim at protecting data privacy during training and inference, the goal of machine unlearning, most of the time, is to retrieve privacy, which has no conflict.
Revised: Different from privacy-preserving algorithms that aim at protecting data privacy during training and inference, the goal of machine unlearning, most of the time, is to retrieve privacy, which has no conflict.

Original:  P RELIMINARIES
Revised:  P RELIMINARIES

Original: Let G =
Revised: Let G =

Original: ( V , E , X ) denote an attributed graph with n = |V| nodes, set of edges E , and n f -dimensional node features X
Revised: ( V , E , X ) denote an attributed graph with n = |V| nodes, set of edges E , and n f -dimensional node features X

Original: = { x 0 , . .
Revised: = { x 0 , . .

Original: , x n − 1 } where x i ∈ R n f .
Revised: , x n − 1 } where x i ∈ R n f .

Original: We use A to denote the adjacency matrix of G and deg G : V → N to denote degree distribution of graph G .
Revised: We use A to denote the adjacency matrix of G and deg G : V → N to denote degree distribution of graph G .

Original: Further, we use S kuv =
Revised: Further, we use S kuv =

Original: ( V kuv , E kuv , X kuv ) to represent a k -hop enclosing subgraph around nodes u and v .
Revised: ( V kuv , E kuv , X kuv ) to represent a k -hop enclosing subgraph around nodes u and v .

Original: Graph Neural Networks (GNNs).
Revised: Graph Neural Networks (GNNs).

Original: A GNN layer g can be expressed as a series of transformation functions: g ( G )
Revised: A GNN layer g can be expressed as a series of transformation functions: g ( G )

Original: =
Revised: =

Original: ( U PD ◦ A GG ◦ M SG )( G ) that takes G as input and produces n d -dimensional node representations h u for u ∈ V (Figure 1).
Revised: ( U PD ◦ A GG ◦ M SG )( G ) that takes G as input and produces n d -dimensional node representations h u for u ∈ V (Figure 1).

Original: Within layer l , M SG specifies neural messages that are exchanged between nodes u and v following edges in A uv by calculating p luv = M SG ( h l − 1 u , h l − 1 v , A uv ) .
Revised: Within layer l , M SG specifies neural messages that are exchanged between nodes u and v following edges in A uv by calculating p luv = M SG ( h l − 1 u , h l − 1 v , A uv ) .

Original: The
Revised: The

Original: A GG defines how every node u combines neural messages from its neighbors N u and how it computes the aggregated message as P lu =
Revised: A GG defines how every node u combines neural messages from its neighbors N u and how it computes the aggregated message as P lu =

Original: A GG ( { p luv | v ∈ N u } ) .
Revised: A GG ( { p luv | v ∈ N u } ) .

Original: Finally, U PD defines how the aggregated messages P lu and hidden node states from the previous layer are combined to produce h lu , i.e., final outputs of l -th layer as follows h lu = U PD ( P lu , h l − 1 u ) .
Revised: Finally, U PD defines how the aggregated messages P lu and hidden node states from the previous layer are combined to produce h lu , i.e., final outputs of l -th layer as follows h lu = U PD ( P lu , h l − 1 u ) .

Original: The output of the last GNN layer are final node representations, z u =
Revised: The output of the last GNN layer are final node representations, z u =

Original: h Lu , where L is the total number of GNN layers in the model.
Revised: h Lu , where L is the total number of GNN layers in the model.

Original: Unlearning in GNN models.
Revised: Unlearning in GNN models.

Original: Let E d ⊆ E denote edges to be deleted and E r = E\E
Revised: Let E d ⊆ E denote edges to be deleted and E r = E\E

Original: d be remaining edges after removing E d from the graph.
Revised: d be remaining edges after removing E d from the graph.

Original: We use G r =
Revised: We use G r =

Original: ( V r , E r , X r ) to indicate the graph after deleting edge E d , where V r = { u ∈ V
Revised: ( V r , E r , X r ) to indicate the graph after deleting edge E d , where V r = { u ∈ V

Original: | deg G r ( u ) > 0 } are nodes that remain connected to nodes in G r , and X r denotes the corresponding node attributes.
Revised: | deg G r ( u ) > 0 } are nodes that remain connected to nodes in G r , and X r denotes the corresponding node attributes.

Original: While the above mentioned description focuses on edge deletion, GNND ELETE can be used for node editing as well by deleting all edges incident to a node the user wants to delete from the model.
Revised: While the above mentioned description focuses on edge deletion, GNND ELETE can be used for node editing as well by deleting all edges incident to a node the user wants to delete from the model.

Original: Unlearning an edge e uv requires the model to delete all the information and influence associated with e uv as if it has never seen it during the training, but keep the change in downstream performance to a minimum level.
Revised: Unlearning an edge e uv requires the model to delete all the information and influence associated with e uv as if it has never seen it during the training, but keep the change in downstream performance to a minimum level.

Original: Under the current
Revised: Under the current

Original: GNN framework, we need to not only modify the predicted probability of e uv but also need to remove its information from its local neighborhood.
Revised: GNN framework, we need to not only modify the predicted probability of e uv but also need to remove its information from its local neighborhood.

Original: Hence, post-processing or logit manipulation is not effective for a GNN to delete edges, even if such strategies do not affect the rest of the graph at all.
Revised: Hence, post-processing or logit manipulation is not effective for a GNN to delete edges, even if such strategies do not affect the rest of the graph at all.

Original: We also denote a classification layer f which takes as input pairs of node representations h Lu and h Lv and outputs prediction probabilities for the edges e uv .
Revised: We also denote a classification layer f which takes as input pairs of node representations h Lu and h Lv and outputs prediction probabilities for the edges e uv .

Original: Finally, given L GNN layers and f , a GNN model m : G → R |E| can be written as: m ( G ) =
Revised: Finally, given L GNN layers and f , a GNN model m : G → R |E| can be written as: m ( G ) =

Original: ( f ◦ g L · · · ◦ g 1 )( G ) .
Revised: ( f ◦ g L · · · ◦ g 1 )( G ) .

Original: Similarly, the corresponding unlearned model m ′ :
Revised: Similarly, the corresponding unlearned model m ′ :

Original: G → R |E r | can be written as: m ′ ( G r ) =
Revised: G → R |E r | can be written as: m ′ ( G r ) =

Original: ( f ◦ g ′ L · · · ◦ g ′ 1 )( G r ) as a composition of L unlearned GNN layers.
Revised: ( f ◦ g ′ L · · · ◦ g ′ 1 )( G r ) as a composition of L unlearned GNN layers.

Original:  O UR A PPROACH : GNND ELETE
Revised:  O UR A PPROACH : GNND ELETE

Original: Effectively deleting edges from graphs requires that the GNN model ignores the importance of the existence of the edges in set E d .
Revised: Effectively deleting edges from graphs requires that the GNN model ignores the importance of the existence of the edges in set E d .

Original: Given the graph G , the model should be unable to recognize whether a deleted edge e uv is part of the graph and ignore any influence that a deleted edge has in its neighborhood.
Revised: Given the graph G , the model should be unable to recognize whether a deleted edge e uv is part of the graph and ignore any influence that a deleted edge has in its neighborhood.

Original: To this end, we i) present two properties for effective graph unlearning and ii) propose a layer-wise deletion operator that can be applied to any GNN for deleting a given set of edges E d .
Revised: To this end, we i) present two properties for effective graph unlearning and ii) propose a layer-wise deletion operator that can be applied to any GNN for deleting a given set of edges E d .

Original: Problem Formulation (Graph Unlearning).
Revised: Problem Formulation (Graph Unlearning).

Original: Given  a graph G =
Revised: Given is a graph G =

Original: ( V , E , X )  and a set of edgesto delete E d , a graph unlearningalgorithm aims to delete the information of all edges e
Revised: ( V , E , X ) , a fully trained GNN model m ( G ) , and a set of edges E d to delete from m ( G ) . Then, graph unlearning aims to unlearn every edge e

Original: uv ∈ E d from theunderlying GNN model m ( G ) such that theclassification layer f of the unlearned model m ′ ( G ) fails to predict e uv (Deleted Edge Consistency property) and retains the performance of remaining edges e uv / ∈ E d (Neighborhood Influence property).
Revised: uv ∈ E d from the GNN model m ( G ) such that the unlearned model m ′ ( G ) is close to the model that would have been output had deleted edges E d been omitted from training by satisfying Deleted Edge Consistency property for e uv ∈ E d and Neighborhood Influence property for e uv / ∈ E d .

Original: 4. D ELETION P ROPERTIES FOR G RAPH N EURAL N ETWORKS
Revised: 4. D ELETION P ROPERTIES FOR G RAPH N EURAL N ETWORKS

Original: Deleting information from a graph is not a trivial task because the representations of nodes and edges are dependent on the combined neighborhood representations.
Revised: Deleting information from a graph is not a trivial task because the representations of nodes and edges are dependent on the combined neighborhood representations.

Original: The following two properties show intuitive assumptions over the deletion operator for effective unlearning in GNNs: 1) Deleted Edge Consistency.
Revised: The following two properties show intuitive assumptions over the deletion operator for effective unlearning in GNNs: 1) Deleted Edge Consistency The predicted probability from

Revised: the unlearned model

Original: The predicted probability from the unlearned model m ′ for an edge e uv should be random , such that it’s hard to determine whether it is a true edge or not.
Revised: m ′ for an edge e uv should be random , such that it’s hard to determine whether it is a true edge or not.

Original: More specifically, the unlearned GNN layer g ′ l should effectively forget the edge information, and consequently, the combination of the node representations that are present in the deleted edge should be random .
Revised: More specifically, the unlearned GNN layer g ′ l should effectively forget the edge information, and consequently, the combination of the node representations that are present in the deleted edge should be random .

Original: Formally, we have the following property definition:
Revised: Formally, we have the following property definition:

Original: Definition 1 (Deleted Edge Consistency) .
Revised: Definition 1 (Deleted Edge Consistency) .

Original: Let e uv denote an edge to be deleted, g l be the l -th layer in a GNN with output node representation vectors h lu , and the unlearned GNN layer g ′ l with h ′ lu .
Revised: Let e uv denote an edge to be deleted, g l be the l -th layer in a GNN with output node representation vectors h lu , and the unlearned GNN layer g ′ l with h ′ lu .

Original: The unlearned layer g ′ l satisfies the Deleted Edge Consistency property if it minimizes the difference between node-pair representations ϕ ( h ′ u , h ′ v ) , and ϕ ( h p , h q ) of two randomly chosen nodes p, q ∈ V :
Revised: The unlearned layer g ′ l satisfies the Deleted Edge Consistency property if it minimizes the difference between node-pair representations ϕ ( h ′ u , h ′ v ) , and ϕ ( h p , h q ) of two randomly chosen nodes p, q ∈ V :

Original: [Equation]
Revised: [Equation]

Original: where ϕ is a readout function (e.g., the dot product, concatenation, etc.)
Revised: where ϕ is a readout function (e.g., the dot product, concatenation, etc.)

Original: that combines the node representations h ′ lu and h ′ lv , ∈ R denotes the random choice from V , and δ is an infinitesimal constant.
Revised: that combines the node representations h ′ lu and h ′ lv , ∈ R denotes the random choice from V , and δ is an infinitesimal constant.

Original: 2) Neighborhood Influence.
Revised: 2) Neighborhood Influence.

Original: While the notion of causality has been used in explainable machine learning, to the best of our knowledge, we propose the first effort of modifying a knowledge graph using a causal perspective.
Revised: While the notion of causality has been used in explainable machine learning, to the best of our knowledge, we propose the first effort of modifying a knowledge graph using a causal perspective.

Original: Formally, removing edge e uv from the graph requires unlearning the influence of e uv from the subgraphs of both nodes u and v .
Revised: Formally, removing edge e uv from the graph requires unlearning the influence of e uv from the subgraphs of both nodes u and v .

Original: In this work, we propose the
Revised: In this work, we propose the

Original: Neighborhood Influence property which leverages the notion of Granger causality (Granger, 1969;
Revised: Neighborhood Influence property which leverages the notion of Granger causality (Granger, 1969;

Original: Bressler & Seth, 2011) and declares a causal relationship ψ
Revised: Bressler & Seth, 2011) and declares a causal relationship ψ

Original: ( { h u | u ∈ S uv } )
Revised: ( { h u | u ∈ S uv } )

Original: → e uv between variables ψ ( { h u | u ∈ S uv } ) and e uv if we are better able to predict edge
Revised: → e uv between variables ψ ( { h u | u ∈ S uv } ) and e uv if we are better able to predict edge

Original: e uv using all available node representations in S uv than if the information apart from ψ ( { h u | u ∈ S uv } ) had been used.
Revised: e uv using all available node representations in S uv than if the information apart from ψ ( { h u | u ∈ S uv } ) had been used.

Original: Here, ψ ( · ) is an operator that combines the node representations in subgraph S uv .
Revised: Here, ψ ( · ) is an operator that combines the node representations in subgraph S uv .

Original: In the context of graph unlearning, if the absence of node representations decreases the prediction confidence of an edge, then there is a causal relationship between the node representation and the e uv prediction.
Revised: In the context of graph unlearning, if the absence of node representations decreases the prediction confidence of an edge, then there is a causal relationship between the node representation and the e uv prediction.

Original: Here, we characterize the notion of deletion by extending Granger causality to local subgraph causality, i.e., an edge e uv dependent on the subgraphs associated with both nodes u and v .
Revised: Here, we characterize the notion of deletion by extending Granger causality to local subgraph causality, i.e., an edge e uv dependent on the subgraphs associated with both nodes u and v .

Original: In particular, removing e uv should not affect the predictions of S uv yielding the following property:
Revised: In particular, removing e uv should not affect the predictions of S uv yielding the following property:

Original: Definition 2 (Neighborhood Influence) .
Revised: Definition 2 (Neighborhood Influence) .

Original: Let e uv ∈
Revised: Let e uv ∈

Original: E
Revised: E

Original: d denote an edge in G to be deleted, g l be the l -th layer in a GNN with output node representation vectors h lu , and the unlearned GNN layer g ′ l with h ′ lu .
Revised: d denote an edge in G to be deleted, g l be the l -th layer in a GNN with output node representation vectors h lu , and the unlearned GNN layer g ′ l with h ′ lu .

Original: The unlearned layer g ′ l satisfies the Neighborhood Influence property if it minimizes the difference of all node-subset representations ψ ( { h lw | w ∈ S uv } ) comprising e uv with their corresponding node-subset representations ψ ( { h ′ lw | w ∈ S uv/e uv } ) where e uv is deleted, i.e.,
Revised: The unlearned layer g ′ l satisfies the Neighborhood Influence property if it minimizes the difference of all node-subset representations ψ ( { h lw | w ∈ S uv } ) comprising e uv with their corresponding node-subset representations ψ ( { h ′ lw | w ∈ S uv/e uv } ) where e uv is deleted, i.e.,

Original: [Equation]
Revised: [Equation]

Original: where ψ is an operator that combines the elements of S
Revised: where ψ is an operator that combines the elements of S

Original: uv (e.g., concatenation, summation, etc.),
Revised: uv (e.g., concatenation, summation, etc.),

Original: S uv/e uv represent the subgraph excluding the information from e ij , and δ is an infinitesimal constant.
Revised: S uv/e uv represent the subgraph excluding the information from e ij , and δ is an infinitesimal constant.

Original: 4. L AYER -W ISE D ELETION O PERATOR
Revised: 4. L AYER -W ISE D ELETION O PERATOR

Original: Perturbing weights or any other hyperparameter of the GNN model m can affect the decisions for multiple nodes and edges of graph G because we allow information to pass through the network from the local neighborhood of each node.
Revised: Perturbing weights or any other hyperparameter of the GNN model m can affect the decisions for multiple nodes and edges of graph G because we allow information to pass through the network from the local neighborhood of each node.

Original: Hence, to ensure effective deletion of an edge e uv , it is important to eliminate signals with minor contributions for predicting edges and develop knobs that can tune/perturb any source of node/edge information that aid in the prediction of e uv .
Revised: Hence, to ensure effective deletion of an edge e uv , it is important to eliminate signals with minor contributions for predicting edges and develop knobs that can tune/perturb any source of node/edge information that aid in the prediction of e uv .

Original: To provide the flexibility to delete the knowledge of specific nodes/edges, we introduce a model-agnostic deletion operator D EL that can be applied to any type of GNN layer.
Revised: To provide the flexibility to delete the knowledge of specific nodes/edges, we introduce a model-agnostic deletion operator D EL that can be applied to any type of GNN layer.

Original: Deletion Operator.
Revised: Deletion Operator.

Original: Following the notations of Section 3, for the l -th GNN layer g l ( G ) with output dimension d l , we define an extended GNN layer with unlearning capability as ( D EL l ◦ g l )( G ) with the same output dimension d l .
Revised: Following the notations of Section 3, for the l -th GNN layer g l ( G ) with output dimension d l , we define an extended GNN layer with unlearning capability as ( D EL l ◦ g l )( G ) with the same output dimension d l .

Original: Given an edge e uv that is to be removed, D EL is  defined as:
Revised: Given an edge e uv that is to be removed, D EL is applied to the node representations and is defined as:

Original: [Equation]
Revised: [Equation]

Original: where 1 is the identity function, and ϕ : R n × d l → R n × d l can be any differentiable function that takes as input the output node representations of g .
Revised: where 1 is the identity function, and ϕ : R n × d l → R n × d l can be any differentiable function that takes as input the output node representations of g .

Original: In this work, ϕ is considered as an MLP with weight parameters W lD .
Revised: In this work, ϕ is considered as an MLP with weight parameters W lD .

Original: Similarly to other GNN operators, the weights W lD of our D EL operator are shared across all nodes to achieve efficiency and scalability.
Revised: Similarly to other GNN operators, the weights W lD of our D EL operator are shared across all nodes to achieve efficiency and scalability.

Original: Local Update.
Revised: Local Update.

Original: Defining an operator that acts only in the local neighborhood S uv enables targeted unlearning, keeping the previously learned knowledge intact as much as possible.
Revised: Defining an operator that acts only in the local neighborhood S uv enables targeted unlearning, keeping the previously learned knowledge intact as much as possible.

Original: If node u is within the local neighborhood of e uv , D EL is activated.
Revised: If node u is within the local neighborhood of e uv , D EL is activated.

Original: For other nodes, D EL remains deactivated and does not affect the hidden states of the nodes.
Revised: For other nodes, D EL remains deactivated and does not affect the hidden states of the nodes.

Original: This ensures that the model will not forget the knowledge it has gained before during training and the predictive performance on E\ S
Revised: This ensures that the model will not forget the knowledge it has gained before during training and the predictive performance on E\ S

Original: uv will not drop.
Revised: uv will not drop.

Original: By applying the deletion operator D EL to every GNN layer, we expect the final representations to reflect the unlearned information in the downstream task.
Revised: By applying the deletion operator D EL to every GNN layer, we expect the final representations to reflect the unlearned information in the downstream task.

Original: Next, we show a theoretical observation over the unlearned node representations that indicates a stable behavior of the deletion operator:
Revised: Next, we show a theoretical observation over the unlearned node representations that indicates a stable behavior of the deletion operator:

Original: Theorem 1.
Revised: Theorem 1.

Original: (Bounding edge prediction using initial model m and unlearned model m ′ )
Revised: (Bounding edge prediction using initial model m and unlearned model m ′ )

Original: Let e uv be an edge to be removed, W LD be the weight matrix of the deletion operator D EL L , and normalized
Revised: Let e uv be an edge to be removed, W LD be the weight matrix of the deletion operator D EL L , and normalized

Original: Lipschitz activation function σ ( · ) .
Revised: Lipschitz activation function σ ( · ) .

Original: Then, the norm difference between the dot product of the final node representations from the initial model z u , z v and from the unlearned one z
Revised: Then, the norm difference between the dot product of the final node representations from the initial model z u , z v and from the unlearned one z

Original: ′ u , z ′ v is bounded by:
Revised: ′ u , z ′ v is bounded by:

Original: [Equation]
Revised: [Equation]

Original: where W LD denotes the weight matrix of the deletion operator for the l -th GNN layer.
Revised: where W LD denotes the weight matrix of the deletion operator for the l -th GNN layer.

Original: The proof is in Appendix B. By Theorem 1, ⟨ z ′ u , z ′ v ⟩ , and consequently the prediction probability for edge e uv from the unlearned model cannot be dissimilar from the baseline.
Revised: The proof is in Appendix B. By Theorem 1, ⟨ z ′ u , z ′ v ⟩ , and consequently the prediction probability for edge e uv from the unlearned model cannot be dissimilar from the baseline.

Original: Nevertheless, D EL l is a layer-wise operator, which provides stable node embeddings as compared to the initial ones.
Revised: Nevertheless, D EL l is a layer-wise operator, which provides stable node embeddings as compared to the initial ones.

Original: 4. M ODEL U NLEARNING
Revised: 4. M ODEL U NLEARNING

Original: Moving from a layer-wise operator to the whole GNN model, our method GNND ELETE applies
Revised: Moving from a layer-wise operator to the whole GNN model, our method GNND ELETE applies

Original: D EL to every GNN layer, leading to a total number of trainable parameters (cid:80) l ( d l ) 2 .
Revised: D EL to every GNN layer, leading to a total number of trainable parameters (cid:80) l ( d l ) 2 .

Original: As the number of trainable parameters in GNND ELETE is independent of the size of the graph, it is compact and scalable to larger graphs and the number of deletion requests.
Revised: As the number of trainable parameters in GNND ELETE is independent of the size of the graph, it is compact and scalable to larger graphs and the number of deletion requests.

Original: Considering the properties defined in
Revised: Considering the properties defined in

Original: Section 4.1, we design two loss functions and compute them in a layer-wise manner.
Revised: Section 4.1, we design two loss functions and compute them in a layer-wise manner.

Original: Specifically, for the l -th GNN layer we first compute the Deleted Edge Consistency loss:
Revised: Specifically, for the l -th GNN layer we first compute the Deleted Edge Consistency loss:

Original: [Equation]
Revised: [Equation]

Original: and the Neighborhood Influence loss:
Revised: and the Neighborhood Influence loss:

Original: [Equation]
Revised: [Equation]

Original: where [ h ′ l u ; h ′ lv ] denotes the concatenation of two vectors, and ∥ denotes the concatenation of multiple vectors.
Revised: where [ h ′ lu ; h ′ lv ] denotes the concatenation of two vectors, and ∥ denotes the concatenation of multiple vectors.

Original: Note that according to Equations 1 and 2, we choose the functions ϕ, ψ to be the concatenation operators.
Revised: Note that according to Equations 1 and 2, we choose the functions ϕ, ψ to be the concatenation operators.

Original: During the backward pass, the deletion operator at the l -th GNN layer is only optimized based on the weighted total loss at the l -th layer, i.e.
Revised: During the backward pass, the deletion operator at the l -th GNN layer is only optimized based on the weighted total loss at the l -th layer, i.e.

Original: [Equation]
Revised: [Equation]

Original: where λ ∈ [0 , 1] is a regularization coefficient that balances the trade-off between the two properties.
Revised: where λ ∈ [0 , 1] is a regularization coefficient that balances the trade-off between the two properties.

Original: Broad Applicability of GNND ELETE .
Revised: Broad Applicability of GNND ELETE .

Original: GNND ELETE treats node representations in a modelagnostic manner, allowing us to consider graph unlearning in models beyond GNNs.
Revised: GNND ELETE treats node representations in a modelagnostic manner, allowing us to consider graph unlearning in models beyond GNNs.

Original: Graph transformers (Ying et al., 2021; Rampášek et al., 2022) have been proposed recently as an extension of the
Revised: Graph transformers (Ying et al., 2021; Rampášek et al., 2022) have been proposed recently as an extension of the

Original: Transformer architecture (Vaswani et al., 2017) for learning predictions on graphs.
Revised: Transformer architecture (Vaswani et al., 2017) for learning predictions on graphs.

Original: The D EL operator can be, also, applied after the computation of the node representations in such models.
Revised: The D EL operator can be, also, applied after the computation of the node representations in such models.

Original: For example, the MPNN le ( X l , E l , A ) layer in GraphGPS (Rampášek et al., 2022, Equation 2) can be replaced with the unlearned version ( D EL ◦ MPNN le ) .
Revised: For example, the MPNN le ( X l , E l , A ) layer in GraphGPS (Rampášek et al., 2022, Equation 2) can be replaced with the unlearned version ( D EL ◦ MPNN le ) .

Original: Similarly, in the Graphormer layer, D EL operator can be applied after the multi-head attention MHA layer (Rampášek et al., 2022, Equation 8).
Revised: Similarly, in the Graphormer layer, D EL operator can be applied after the multi-head attention MHA layer (Rampášek et al., 2022, Equation 8).

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: Next, we present our empirical results for GNND ELETE .
Revised: Next, we present our empirical results for GNND ELETE .

Original: In particular, we address the following questions: Q1 )
Revised: In particular, we address the following questions: Q1 )

Original: How does GNND ELETE perform as compared to baselines?
Revised: How does GNND ELETE perform as compared to baselines?

Original: Q2 )
Revised: Q2 )

Original: Can GNND ELETE be extended to unlearning nodes and node features?
Revised: Can GNND ELETE be extended to unlearning nodes and node features?

Original: Q3 )
Revised: Q3 )

Original: How does the interplay between Deleted
Revised: How does the interplay between Deleted

Original: Edge Consistency and Neighborhood Influence property affect deletion performance?
Revised: Edge Consistency and Neighborhood Influence property affect deletion performance?

Original: 5. E XPERIMENTAL SETUP
Revised: 5. E XPERIMENTAL SETUP

Original: Datasets.
Revised: Datasets.

Original: We evaluate GNND ELETE on several widely-used graphs at various scales.
Revised: We evaluate GNND ELETE on several widely-used graphs at various scales.

Original: We usehomogeneous graphs: Cora (Bojchevski & Günnemann, 2018), PubMed (Bojchevski & Günnemann, 2018), DBLP (Bojchevski & Günnemann, 2018), CS (Bojchevski & Günnemann, 2018),
Revised: We usehomogeneous graphs: Cora (Bojchevski & Günnemann, 2018), PubMed (Bojchevski & Günnemann, 2018), DBLP (Bojchevski & Günnemann, 2018), CS (Bojchevski & Günnemann, 2018),

Original: OGB-Collab
Revised: OGB-Collab

Original: (Hu et al., 2020), and 2 heterogeneous graphs: OGB-BioKG (Hu et al., 2020), and WordNet18RR (Dettmers et al., 2018).
Revised: (Hu et al., 2020), and 2 heterogeneous graphs: OGB-BioKG (Hu et al., 2020), and WordNet18RR (Dettmers et al., 2018).

Original: Table 5 includes details on graph datasets and deleted edges.
Revised: Table 5 includes details on graph datasets and deleted edges.

Original: GNNs and Baselines.
Revised: GNNs and Baselines.

Original: We test with four GNN architectures and two graph types to show the flexibility of our GNND ELETE operator.
Revised: We test with four GNN architectures and two graph types to show the flexibility of our GNND ELETE operator.

Original: In particular, we test on GCN (Kipf & Welling, 2017), GAT
Revised: In particular, we test on GCN (Kipf & Welling, 2017), GAT

Original: (Veli ˇ ckovi ´ c et al., 2018) and GIN (Xu et al., 2019) for homogeneous graphs, and R-GCN (Schlichtkrull et al., 2018) and R-GAT (Chen et al., 2021b) for heterogeneous graphs.
Revised: (Veli ˇ ckovi ´ c et al., 2018) and GIN (Xu et al., 2019) for homogeneous graphs, and R-GCN (Schlichtkrull et al., 2018) and R-GAT (Chen et al., 2021b) for heterogeneous graphs.

Original: We consider four baseline methods: i) G RAPH E DITOR (Cong & Mahdavi), a method that finetunes on a closed-form solution of linear
Revised: We consider four baseline methods: i) G RAPH E DITOR (Cong & Mahdavi), a method that finetunes on a closed-form solution of linear

Original: GNN models; ii) G RAPH E RASER (Chen et al., 2021c), a re-training-based machine unlearning method for graphs; iii) G RAD A SCENT , which performs gradient ascent on E d with cross-entropy loss and iv) D ESCENT - TO -D ELETE (Neel et al., 2021), a general machine unlearning method.
Revised: GNN models; ii) G RAPH E RASER (Chen et al., 2021c), a re-training-based machine unlearning method for graphs; iii) G RAD A SCENT , which performs gradient ascent on E d with cross-entropy loss and iv) D ESCENT - TO -D ELETE (Neel et al., 2021), a general machine unlearning method.

Original: Implementation Details.
Revised: Implementation Details.

Original: We test GNND ELETE on edge deletion tasks, and show that it can also handle node deletion and node feature update tasks.
Revised: We test GNND ELETE on edge deletion tasks, and show that it can also handle node deletion and node feature update tasks.

Original: In particular, we show the effectiveness of
Revised: In particular, we show the effectiveness of

Original: GNND ELETE on two different settings: i) an easier setting where information far away from the test set is deleted, and ii) a harder setting where information close to testset is deleted.
Revised: GNND ELETE on two different settings: i) an easier setting where information far away from the test set is deleted, and ii) a harder setting where information close to testset is deleted.

Original: For edge deletion tasks, we delete a varying size of E d between [0 .
Revised: For edge deletion tasks, we delete a varying size of E d between [0 .

Original: 5% − 5 .
Revised: 5% − 5 .

Original: 0%] of total edges, with a step size of 0 .
Revised: 0%] of total edges, with a step size of 0 .

Original: For larger datasets like OGB (Hu et al., 2020), we limit the maximum deletion ratio to 2 .
Revised: For larger datasets like OGB (Hu et al., 2020), we limit the maximum deletion ratio to 2 .

Original: We report the average and standard error of the unlearning performance across five independent runs.
Revised: We report the average and standard error of the unlearning performance across five independent runs.

Original: Regarding the evaluating metrics, we use AUROC to evaluate GNND ELETE for the link prediction task, as well as the Membership Inference (MI) (Thudi et al., 2021a) for the node deletion.
Revised: Regarding the evaluating metrics, we use AUROC to evaluate GNND ELETE for the link prediction task, as well as the Membership Inference (MI) (Thudi et al., 2021a) for the node deletion.

Original: An elaborate discussion of the evaluation metrics can be found in Appendix C.1.
Revised: An elaborate discussion of the evaluation metrics can be found in Appendix C.1.

Original: Moreover, in
Revised: Moreover, in

Original: Appendix C.2 we describe two sampling strategies that we apply on E d .
Revised: Appendix C.2 we describe two sampling strategies that we apply on E d .

Original: 5. Q1 : R ESULTS – C OMPARISON TO B ASELINES
Revised: 5. Q1 : R ESULTS – C OMPARISON TO B ASELINES

Original: We compare GNND ELETE to the baseline unlearning techniques and present the results in Tables 12.
Revised: We compare GNND ELETE to the baseline unlearning techniques and present the results in Tables 1-2.

Original: Across three GNN architectures, we find that GNND ELETE achieves the best performance on  E t , outperforming G RAPH E DITOR and G RAPH E RASER by 13.9% and 38.8%.
Revised: Across three GNN architectures, we find that GNND ELETE achieves the best performance on the test edge set E t , outperforming G RAPH E DITOR and G RAPH E RASER by 13.9% and 38.8%.

Original: Further, we observe that GNND ELETE achieves the highest AUROC on E d , outperforming G RAPH E DITOR and
Revised: Further, we observe that GNND ELETE achieves the highest AUROC on E d , outperforming G RAPH E DITOR and

Original: G RAPH E RASER by 32.2% and 25.4%.
Revised: G RAPH E RASER by 32.2% and 25.4%.

Original: GNND ELETE even outperforms R ETRAIN - FROM -S CRATCH by 21.7% under this setting, proving its capability of effectively unlearning the deleted edges.
Revised: GNND ELETE even outperforms R ETRAIN - FROM -S CRATCH by 21.7% under this setting, proving its capability of effectively unlearning the deleted edges.

Original: Interestingly, none of the existing baseline methods have comparable performance to GNND ELETE on these performance metrics, including G RAPH E RASER , which ignores the global connectivity pattern and overfit to specific shards, and G RAPH E DITOR , whose choice of linear architecture strongly limits the power of the GNN unlearning.
Revised: Interestingly, none of the existing baseline methods have comparable performance to GNND ELETE on these performance metrics, including G RAPH E RASER , which ignores the global connectivity pattern and overfit to specific shards, and G RAPH E DITOR , whose choice of linear architecture strongly limits the power of the GNN unlearning.

Original: In contrast to Table 1, results in Table 2 show that
Revised: In contrast to Table 1, results in Table 2 show that

Original: [Table] [Table]
Revised: [Table] [Table]

Original: retraining is ineffective when deleting edges on large graphs.
Revised: retraining is ineffective when deleting edges on large graphs.

Original: Our results demonstrate that baselines like D ESCENT - TO -D ELETE and G RAD A SCENT lose almost their predictive prowess in making meaningful predictions and distinguishing deleted edges because the weight updates are independent of the unlearning task and affect all the nodes, including nodes associated with E t .
Revised: Our results demonstrate that baselines like D ESCENT - TO -D ELETE and G RAD A SCENT lose almost their predictive prowess in making meaningful predictions and distinguishing deleted edges because the weight updates are independent of the unlearning task and affect all the nodes, including nodes associated with E t .

Original: Please refer to the
Revised: Please refer to the

Original: Appendix for results on PubMed (Tables 8-9), DBLP (Tables 10-11), OGB-Collab (Tables 12-13), and WordNet18 (Tables 14-15) using a deletion ratio of 0.5%, 2.5%, and 5%.
Revised: Appendix for results on PubMed (Tables 8-9), DBLP (Tables 10-11), OGB-Collab (Tables 12-13), and WordNet18 (Tables 14-15) using a deletion ratio of 0.5%, 2.5%, and 5%.

Original: 5. Q2 : R ESULTS – O THER UNLEARNING TASKS AND GNND ELETE ’ S EFFICIENCY
Revised: 5. Q2 : R ESULTS – O THER UNLEARNING TASKS AND GNND ELETE ’ S EFFICIENCY

Original: Node Deletion Task.
Revised: Node Deletion Task.

Original: Here, we show the flexibility of GNND ELETE in handling node learning tasks.
Revised: Here, we show the flexibility of GNND ELETE in handling node learning tasks.

Original: We delete 100 nodes and their associated edges from the training data and evaluate the performance of the unlearning method using the Membership Inference attack metric.
Revised: We delete 100 nodes and their associated edges from the training data and evaluate the performance of the unlearning method using the Membership Inference attack metric.

Original: Results in Table 3 show the performance of baselines and GNND ELETE for the DBLP and Wordnet18 using a deletion ratio of 5% .
Revised: Results in Table 3 show the performance of baselines and GNND ELETE for the DBLP and Wordnet18 using a deletion ratio of 5% .

Original: It shows that GNND ELETE outperforms baselines for most GNN models, highlighting its effectiveness in node deletion.
Revised: It shows that GNND ELETE outperforms baselines for most GNN models, highlighting its effectiveness in node deletion.

Original: Across five GNN architectures, we find that GNND ELETE improves on the MI ratio score of all baselines: G RAPH E DITOR (+9.7%), G RAPH E RASER (+22.5%), R ETRAINFROM -S CRATCH (+8.6%),
Revised: Across five GNN architectures, we find that GNND ELETE improves on the MI ratio score of all baselines: G RAPH E DITOR (+9.7%), G RAPH E RASER (+22.5%), R ETRAINFROM -S CRATCH (+8.6%),

Original: G RAD A SCENT (+23.5%), and D ESCENT - TO -D ELETE (+2.4%).
Revised: G RAD A SCENT (+23.5%), and D ESCENT - TO -D ELETE (+2.4%).

Original: See
Revised: See

Original: Appendix Tables 6 for results using different deletion ratios and evaluation metrics.
Revised: Appendix Tables 6 for results using different deletion ratios and evaluation metrics.

Original: Time and Space Efficiency.
Revised: Time and Space Efficiency.

Original: We demonstrate that GNND ELETE is time-efficient as compared to most unlearning baselines.
Revised: We demonstrate that GNND ELETE is time-efficient as compared to most unlearning baselines.

Original: For all methods, we use a 2-layer GCN/R-GCN architecture with a trainable entity and relation embeddings with 128, 64, and 32 hidden dimensions trained on three datasets (PubMed, CS, and OGB-Collab).
Revised: For all methods, we use a 2-layer GCN/R-GCN architecture with a trainable entity and relation embeddings with 128, 64, and 32 hidden dimensions trained on three datasets (PubMed, CS, and OGB-Collab).

Original: We present the results of wall-clock time vs. graph size in Figure 2 and observe that GNND ELETE consistently takes much less time than existing graph unlearning methods.
Revised: We present the results of wall-clock time vs. graph size in Figure 2 and observe that GNND ELETE consistently takes much less time than existing graph unlearning methods.

Original: Specifically, GNND ELETE is 12 .
Revised: Specifically, GNND ELETE is 12 .

Original: 3 × faster than R ETRAIN - FROM -S CRATCH on WordNet.
Revised: 3 × faster than R ETRAIN - FROM -S CRATCH on WordNet.

Original: For smaller graphs like DBLP, GNND ELETE takes 185 seconds less (18.5% faster) than the pre-training stage of G RAPH E DITOR .
Revised: For smaller graphs like DBLP, GNND ELETE takes 185 seconds less (18.5% faster) than the pre-training stage of G RAPH E DITOR .

Original: Despite taking less time, the predictive performance of D ESCENT - TO -D ELETE and G RAD A SCENT is poor compared to GNND ELETE because they are not tailored to incorporate the graph structure for unlearning.
Revised: Despite taking less time, the predictive performance of D ESCENT - TO -D ELETE and G RAD A SCENT is poor compared to GNND ELETE because they are not tailored to incorporate the graph structure for unlearning.

Original: Regarding the space efficiency, we measure the number of training parameters and show that GNND ELETE has the smallest model
Revised: Regarding the space efficiency, we measure the number of training parameters and show that GNND ELETE has the smallest model

Original: [Table]
Revised: [Table]

Original: size.
Revised: size.

Original: In addition, the number of training parameters does not scale with respect to the graph size, proving the efficiency and scalability of GNND ELETE .
Revised: In addition, the number of training parameters does not scale with respect to the graph size, proving the efficiency and scalability of GNND ELETE .

Original: For instance, GNND ELETE takes 9 .
Revised: For instance, GNND ELETE takes 9 .

Original: 3 × less computation than G RAPH E RASER .
Revised: 3 × less computation than G RAPH E RASER .

Original: Due to space limitation, we refer to Appendix Table 7 for additional results and details.
Revised: Due to space limitation, we refer to Appendix Table 7 for additional results and details.

Original: 5. Q3 : R ESULTS – D ELETED E DGE C ONSISTENCY VS . N EIGHBORHOOD I NFLUENCE
Revised: 5. Q3 : R ESULTS – D ELETED E DGE C ONSISTENCY VS . N EIGHBORHOOD I NFLUENCE

Original: We conduct ablations on two key properties of GNND ELETE , namely the Deleted Edge Consistency and Neighborhood Influence, by varying the regularization parameter λ in Equation 7.
Revised: We conduct ablations on two key properties of GNND ELETE , namely the Deleted Edge Consistency and Neighborhood Influence, by varying the regularization parameter λ in Equation 7.

Original: Results in
Revised: Results in

Original: Table 4 show that both properties are necessary to high AUROC on E t and E d .
Revised: Table 4 show that both properties are necessary to high AUROC on E t and E d .

Original: We observe that
Revised: We observe that

Original: GNND ELETE focuses on Neighborhood Influence as λ decreases, explaining why the performance on E t is close to the original while the model cannot distinguish E d from the remaining edges.
Revised: GNND ELETE focuses on Neighborhood Influence as λ decreases, explaining why the performance on E t is close to the original while the model cannot distinguish E d from the remaining edges.

Original: For higher values of λ , GNND ELETE focuses on optimizing the Deleted Edge Consistency property and can better distinguish between E d and E r .
Revised: For higher values of λ , GNND ELETE focuses on optimizing the Deleted Edge Consistency property and can better distinguish between E d and E r .

Original: In summary, we observe an improvement of 5.56% in the average AUROC for λ = 0 .
Revised: In summary, we observe an improvement of 5.56% in the average AUROC for λ = 0 .

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this work
Revised: In this work

Original: [Figure]
Revised: [Figure]

Original: we propose a flexible and easy-to-use deletion operator that can be applied to any type of GNN model.
Revised: we propose a flexible and easy-to-use deletion operator that can be applied to any type of GNN model.

Original: Along with the deletion operator, we introduce Deleted Edge Consistency and
Revised: Along with the deletion operator, we introduce Deleted Edge Consistency and

Original: Neighborhood Influence as two properties that can contribute to more effective graph unlearning.
Revised: Neighborhood Influence as two properties that can contribute to more effective graph unlearning.

Original: Combining the operator with the two properties
Revised: Combining the operator with the two properties

Original: [Figure]
Revised: [Figure]

Original: we introduce a novel loss function for the unlearning task.
Revised: we introduce a novel loss function for the unlearning task.

Original: We evaluate GNND ELETE in the link prediction task and we show that our method outperforms the existing graph unlearning models for the case of edge deletion, showing on parallel competitive results for node deletion.
Revised: We evaluate GNND ELETE in the link prediction task and we show that our method outperforms the existing graph unlearning models for the case of edge deletion, showing on parallel competitive results for node deletion.

Original: Overall, the performance behavior across a variety of problems and the ease of use demonstrate the potential of GNND ELETE to become the graph unlearning workhorse.
Revised: Overall, the performance behavior across a variety of problems and the ease of use demonstrate the potential of GNND ELETE to become the graph unlearning workhorse.

Original: A R ELATED W ORK : C ONT ’ D
Revised: A R ELATED W ORK : C ONT ’ D

Original: Machine Unlearning for Other Tasks.
Revised: Machine Unlearning for Other Tasks.

Original: Given the increasing necessity of machine unlearning, researches have studied machine unlearning algorithms tailored for other tasks.
Revised: Given the increasing necessity of machine unlearning, researches have studied machine unlearning algorithms tailored for other tasks.

Original: For example, Chen et al.
Revised: For example, Chen et al.

Original: (2022) and Li et al.
Revised: (2022) and Li et al.

Original: (2022) propose machine unlearning frameworks for recommendation systems by considering the collaborative information.
Revised: (2022) propose machine unlearning frameworks for recommendation systems by considering the collaborative information.

Original: In addition, recent unlearning methods use Bayesian and latent models (Nguyen et al., 2020; Kassab & Simeone, 2022; Sekhari et al., 2021).
Revised: In addition, recent unlearning methods use Bayesian and latent models (Nguyen et al., 2020; Kassab & Simeone, 2022; Sekhari et al., 2021).

Original: Machine unlearning (MU) has also been applied for other tasks, including verification of MU (Sommer et al., 2020), re-definition of MU (Thudi et al., 2021b), test of MU (Goel et al., 2022), zero-shot MU (Chundawat et al., 2022), tree-based MU (Schelter et al., 2021), coded MU (Aldaghri et al., 2021), pruning based federated MU (Wang et al., 2021), adaptive sequence deletion MU (Gupta et al., 2021),
Revised: Machine unlearning (MU) has also been applied for other tasks, including verification of MU (Sommer et al., 2020), re-definition of MU (Thudi et al., 2021b), test of MU (Goel et al., 2022), zero-shot MU (Chundawat et al., 2022), tree-based MU (Schelter et al., 2021), coded MU (Aldaghri et al., 2021), pruning based federated MU (Wang et al., 2021), adaptive sequence deletion MU (Gupta et al., 2021),

Original: MU for k-means (Ginart et al., 2019a), MU for features and labels (Warnecke et al., 2021), MU for spam email detection (Parne et al., 2021), MU for linear models (Mahadevan & Mathioudakis, 2021).
Revised: MU for k-means (Ginart et al., 2019a), MU for features and labels (Warnecke et al., 2021), MU for spam email detection (Parne et al., 2021), MU for linear models (Mahadevan & Mathioudakis, 2021).

Original: While the above mentioned methods have shown new directions for machine unlearning, they are not comparable to our work as we focus on general graph unlearning instead of a specific ML task.
Revised: While the above mentioned methods have shown new directions for machine unlearning, they are not comparable to our work as we focus on general graph unlearning instead of a specific ML task.

Original: B P ROOF FOR T HEOREM
Revised: B P ROOF FOR T HEOREM

Original: For the L -th GNN layer, we assume for sake of simplicity that the final node representation z u for a node u is computed as:
Revised: For the L -th GNN layer, we assume for sake of simplicity that the final node representation z u for a node u is computed as:

Original: [Equation]
Revised: [Equation]

Original: where σ is the sigmoid function, and, and W 1 , W 2 are the weight parameters for the GNN layer.
Revised: where σ is the sigmoid function, and, and W 1 , W 2 are the weight parameters for the GNN layer.

Original: Similarly, for the unlearned L -th layer we have:
Revised: Similarly, for the unlearned L -th layer we have:

Original: [Equation]
Revised: [Equation]

Original: where W LD ’s are the weight parameters for the DEL operator.
Revised: where W LD ’s are the weight parameters for the DEL operator.

Original: For a given edge e uv ∈ E d that is to be deleted, we expect for the dot product ⟨ z u , z v ⟩ to be maximized, as it is an existent edge in the initial graph, while ⟨ z ′ u , z ′ v ⟩ to be fixed, so that the edge probability is close to 0 .
Revised: For a given edge e uv ∈ E d that is to be deleted, we expect for the dot product ⟨ z u , z v ⟩ to be maximized, as it is an existent edge in the initial graph, while ⟨ z ′ u , z ′ v ⟩ to be fixed, so that the edge probability is close to 0 .

Original: 5 , following the Deleted
Revised: 5 , following the Deleted

Original: Edge Consistency property.
Revised: Edge Consistency property.

Original: Specifically, the difference of the two terms can be bounded as:
Revised: Specifically, the difference of the two terms can be bounded as:

Original: [Equation]
Revised: [Equation]

Original: Then

Original: [Equation] simplifying Equations 8 and [Equation] we have:
Revised: Then, simplifying Equations 8 and 9, we have:

Original: [Equation]
Revised: [Equation]

Original: and applying that to Equation 10:
Revised: and applying that to Equation 10:

Original: [Equation]
Revised: [Equation]

Original: C E XPERIMENTSC. E VALUATION M ETRICS
Revised: C E XPERIMENTS

Revised: In this section, we give further details on the experimental setup.

Original: To measure the effectiveness of the deletion operation, we choose three performance metrics: i) To quantify the link prediction performance on E t ,we compute AUROC andAUPRC on E 
Revised: We firstly present the used evaluation metrics for the edge and node deletion, then we describe the sampling strategies for E t , and E d , dataset statistics, and finally we present the full

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: We choose the following performance metrics to measure the effectiveness of the deletion operation on the set of the deleted edges E d and the test set E t that contains a subset of the remaining edges:. For the edge deletion case, we have: • AUROC and AUPRC on the test set E t : these metrics measure the prediction perfomance of each graph graph learning model over the existent edges of the test set E t .

Original:  of some edges should not affect the unlearned model’s capability on the original test set.
Revised: High values of AUROC and AUPRC on E t show that the unlearned model’s performance on the original test set is not affected by the deletion of an edge subset.

Original: To measure the ability of the unlearned model to differentiate E d  from  E r , we compute AUROC and AUPRC on
Revised: • AUROC and AUPRC on the set of deleted edges E d : these metrics quantify the ability of the unlearned models to distinguish deleted edges (contained in E d ) from the remaining edges (contained in E r ). For the computation of the area under the curve, we take into account the total of the deleted edges in E d and we sample an equal amount of remaining edges from E r . Then, we set the labels of the deleted edges equal to 0 (since they do not exist after deletion), and the labels of the remaining edges to 1 .

Original:  E d by sampling the same amount of present edges  from  E r as the unlearned model f ′ should clearly distinguish between deleted edges and present edges.
Revised: Higher values of the AUC on E d show that the model is more capable of distinguishing edges in E d from edges in E r . For the node deletion case [Table]

Original: Finally, we quantify the  privacy leakage with 
Revised: we capitalize on the evaluation for privacy leakage with Membership

Original: Membership Inference (MI) (Thudi et al., 2021a) attacks.
Revised: Inference ( MI) (Thudi et al., 2021a) attacks.

Original: An unlearned model f ′ effectively forgets E d if an MI attacker returns that E d is not present in the training set, i.e., the probability of predicting the presence of E d decreases.
Revised: An unlearned model f ′ effectively forgets E d if an MI attacker returns that E d is not present in the training set, i.e., the probability of predicting the presence of E d decreases.

Original: We compute the success rate of MI attack by calculating the ratio of presence probability of E d before and after the deletion operator.In the context of GNNs, given the model outputs of e , an MI attacker is a model that can predict if a GNN model is trained on edge e . To the best of our knowledge, there is no existing membership inference attack work on graph edges.
Revised: • MI Ratio: this metric quantifies the success rate of a Membership Inference (MI) attack, by calculating the ratio of presence probability of E d before and after the deletion operator.

Original: Therefore, we adapt the implementation in Olatunji et al.
Revised: We adapt the implementation from Olatunji et al.

Original: (2021) in our experiments.
Revised: (2021) in our experiments.

Revised: If the ratio is higher than 1 , it means that the model contains less information about E d .

Revised: If the ratio is less than 1 , it means the model contains more information about E d .

Original: C. S AMPLING OF E t AND E d
Revised: C. S AMPLING OF E t AND E d

Original: We sample 5% of the total edges as the test set ( E t ) to evaluate the model’s performance on link prediction and sample another 5% as validation set for selecting the best model.
Revised: We sample 5% of the total edges as the test set ( E t ) to evaluate the model’s performance on link prediction and sample another 5% as validation set for selecting the best model.

Original: We propose two sampling strategies for sampling E d for edge deletion tasks: i) E d, OUT refers to randomly sampling edges outside the 2 -hop enclosing subgraph of E t , i.e., E d, OUT = { e | e / ∈ S 2 E t } ; and ii) E d, IN refers to randomly sampling edges from the 2 -hop enclosing subgraph of E t , i.e., E d, IN = { e | e ∈ S 2 E t } .
Revised: We propose two sampling strategies for sampling E d for edge deletion tasks: i) E d, OUT refers to randomly sampling edges outside the 2 -hop enclosing subgraph of E t , i.e., E d, OUT = { e | e / ∈ S 2 E t } ; and ii) E d, IN refers to randomly sampling edges from the 2 -hop enclosing subgraph of E t , i.e., E d, IN = { e | e ∈ S 2 E t } .

Original: We note that deleting E d, IN is more difficult than deleting E d, OUT as the deletion operation will have an impact on the local neighborhood of E d, IN , where E t is located.
Revised: We note that deleting E d, IN is more difficult than deleting E d, OUT as the deletion operation will have an impact on the local neighborhood of E d, IN , where E t is located.

Original: For comparison of these two sampling strategies, please refer to the Appendix Tables 8-9 for results on PubMed, Tables 10-11 on
Revised: For comparison of these two sampling strategies, please refer to the Appendix Tables 8-9 for results on PubMed, Tables 10-11 on

Original: DBLP, Tables 12-13 on OGB-Collab, and Tables 14-15 on WordNet18.
Revised: DBLP, Tables 12-13 on OGB-Collab, and Tables 14-15 on WordNet18.

Original: C. E VALUATED DATASETS
Revised: C. E VALUATED DATASETS

Original: Table 5 presents the size of the graphs and the maximum number of deleted edges in the experiments, ranging from small to large scales. 
Revised: Table 5 presents the size of the graphs and the maximum number of deleted edges in the experiments, ranging from small to large scales. [Table]

Original: C. M ODEL EFFICIENCY
Revised: C. M ODEL EFFICIENCY

Original: Space efficiency is reflected by the number of trainable parameters a model has.
Revised: Space efficiency is reflected by the number of trainable parameters a model has.

Original: We report number of parameters for all methods in Table 7.
Revised: We report number of parameters for all methods in Table 7.

Original: We can observe that GNND ELETE has the smallest model size, which does not scale with respect to the size of the graph.
Revised: We can observe that GNND ELETE has the smallest model size, which does not scale with respect to the size of the graph.

Original: This proves the efficiency and scalability of GNND ELETE .
Revised: This proves the efficiency and scalability of GNND ELETE .

Original: It is also significantly smaller than G RAPH E RASER , where we usually have to divide the original graph into 10 or 20 shards, each requiring a separate GNN model.
Revised: It is also significantly smaller than G RAPH E RASER , where we usually have to divide the original graph into 10 or 20 shards, each requiring a separate GNN model.

Original: [Table]
Revised: [Table]

Original: D F ULL RESULTS
Revised: D F ULL RESULTS

Original: We detail all the results from different graphs, GNN architectures, models, and deletion ratios
Revised: We detail all the results from different graphs, GNN architectures, models

Original: [Table]
Revised: [Table]

Revised: and deletion ratios

Original: [Table] [Table] 
Revised: [Table] [Table] [Table]

Original: R ETRAIN
Revised: R ETRAIN

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table][Table]
Revised: [Table] [Table]

Original: [Table][Figure] [Table] [Figure]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table][Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table] [Figure] 
Revised: [Table] [Figure] [Table]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]  [Table]
Revised: [Table] [Figure] [Table]

Original:  [Table]
Revised: [Figure] [Table] [Table] [Table]

Original: [Table]
Revised: [Table]

Original: 5. R ETRAIN
Revised: 5. R ETRAIN

Original: [Table]
Revised: [Table]