Original: C OUNTERFACTUAL G RAPH L EARNING FOR L INK P REDICTION
Revised: C OUNTERFACTUAL G RAPH L EARNING FOR L INK P REDICTION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Learning to predict missing links is important for many graph-based applications.
Revised: Learning to predict missing links is important for many graph-based applications.

Original: Existing methods were designed to learn the association between two sets of variables: (1) the observed graph structure (e.g., clustering effect) and (2) the existence of link between a pair of nodes.
Revised: Existing methods were designed to learn the association between two sets of variables: (1) the observed graph structure (e.g., clustering effect) and (2) the existence of link between a pair of nodes.

Original: However, the causal relationship between these variables was ignored.
Revised: However, the causal relationship between these variables was ignored.

Original: We visit the possibility of learning it by asking a counterfactual question: “ would the link exist or not if the observed graph structure became different? ”
Revised: We visit the possibility of learning it by asking a counterfactual question: “ would the link exist or not if the observed graph structure became different? ”

Original: To answer this question, we leverage causal models considering the information of the node pair (i.e., learned graph representations) as context, global graph structural properties as treatment, and link existence as outcome.
Revised: To answer this question, we leverage causal models considering the information of the node pair (i.e., learned graph representations) as context, global graph structural properties as treatment, and link existence as outcome.

Original: In this work, we propose a novel link prediction method that enhances graph learning by counterfactual inference.
Revised: In this work, we propose a novel link prediction method that enhances graph learning by counterfactual inference.

Original: It creates counterfactual links from the observed ones, and learns representations from both the observed and counterfactual links.
Revised: It creates counterfactual links from the observed ones, and learns representations from both the observed and counterfactual links.

Original: Experiments on benchmark datasets show that this novel graph learning method achieves state-of-the-art performance on link prediction.
Revised: Experiments on benchmark datasets show that this novel graph learning method achieves state-of-the-art performance on link prediction.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Link prediction seeks to predict the likelihood of edge existence between node pairs based on the observed graph.
Revised: Link prediction seeks to predict the likelihood of edge existence between node pairs based on the observed graph.

Original: Given the omnipresence of graph-structured data, link prediction has copious applications such as movie recommendation (Bennett et al., 2007), chemical interaction prediction (Stanﬁeld et al., 2017), and knowledge graph completion (Kazemi & Poole, 2018).
Revised: Given the omnipresence of graph-structured data, link prediction has copious applications such as movie recommendation (Bennett et al., 2007), chemical interaction prediction (Stanﬁeld et al., 2017), and knowledge graph completion (Kazemi & Poole, 2018).

Original: Graph machine learning methods have been widely applied to solve this problem.
Revised: Graph machine learning methods have been widely applied to solve this problem.

Original: Their standard scheme is to ﬁrst learn the representation vectors of nodes and then learn the association between the representations of a pair of nodes and the existence of the link between them.
Revised: Their standard scheme is to ﬁrst learn the representation vectors of nodes and then learn the association between the representations of a pair of nodes and the existence of the link between them.

Original: For example, graph neural networks (GNNs) use neighborhood aggregation to create the representation vectors: the representation vector of a node is computed by recursively aggregating and transforming representation vectors of its neighboring nodes (Kipf & Welling, 2016a; Hamilton et al., 2017; Wu et al., 2020).
Revised: For example, graph neural networks (GNNs) use neighborhood aggregation to create the representation vectors: the representation vector of a node is computed by recursively aggregating and transforming representation vectors of its neighboring nodes (Kipf & Welling, 2016a; Hamilton et al., 2017; Wu et al., 2020).

Original: Then the vectors are fed into a binary classiﬁcation model to learn the association .
Revised: Then the vectors are fed into a binary classiﬁcation model to learn the association .

Original: GNN methods have shown predominance in the task of link prediction (Kipf & Welling, 2016b; Zhang et al., 2020a).
Revised: GNN methods have shown predominance in the task of link prediction (Kipf & Welling, 2016b; Zhang et al., 2020a).

Original: Unfortunately, the causal relationship between graph structure and link existence was largely ignored in previous work.
Revised: Unfortunately, the causal relationship between graph structure and link existence was largely ignored in previous work.

Original: Existing methods that learn from association only were  not able to capture essential factors to accurately predict missing links in the test data .
Revised: Existing methods that learn from association only were are not able to capture essential factors to accurately predict missing links in the test data .

Original: Take social network as an example.
Revised: Take social network as an example.

Original: Suppose Alice and Adam live in the same neighborhood and they are close friends.
Revised: Suppose Alice and Adam live in the same neighborhood and they are close friends.

Original: The association between neighborhood belonging and friend closeness could be too strong to discover the essential factors of the friendship such as common interests or family relationship which could be the cause of being living in the same neighborhood.
Revised: The association between neighborhood belonging and friend closeness could be too strong to discover the essential factors of the friendship such as common interests or family relationship which could be the cause of being living in the same neighborhood.

Original: So, our idea is asking a counterfactual question: “ would Alice and Adam still be close friends if they were not living in the same neighborhood? ”
Revised: So, our idea is asking a counterfactual question: “ would Alice and Adam still be close friends if they were not living in the same neighborhood? ”

Original: If a graph learning model could learn the causal relationship by answering the counterfactual questions, it would improve the accuracy of link prediction with the novel knowledge it captured.
Revised: If a graph learning model could learn the causal relationship by answering the counterfactual questions, it would improve the accuracy of link prediction with the novel knowledge it captured.

Original: Generally, the questions can be described as “ would the link exist or not if the graph structure became different? ”
Revised: Generally, the questions can be described as “ would the link exist or not if the graph structure became different? ”

Original: As known to many, counterfactual question is a key component of causal inference and have been well deﬁned in the literature.
Revised: As known to many, counterfactual question is a key component of causal inference and have been well deﬁned in the literature.

Original: A counterfactual question is usually framed with three factors: context (as a data point), manipulation (e.g., treatment, intervention, action, strategy), and outcome (van der
Revised: A counterfactual question is usually framed with three factors: context (as a data point), manipulation (e.g., treatment, intervention, action, strategy), and outcome (van der

Original: Laan & Petersen, 2007; Johansson et al., 2016).
Revised: Laan & Petersen, 2007; Johansson et al., 2016).

Original: (To simplify the language, we use “treatment” to refer to the manipulation in this paper, as readers might be familiar more with the word “treatment.”)
Revised: (To simplify the language, we use “treatment” to refer to the manipulation in this paper, as readers might be familiar more with the word “treatment.”)

Original: [Figure]
Revised: [Figure]

Original: Factual link:
Revised: Factual link:

Original: [Figure]
Revised: [Figure]

Original: Decoder Decoder
Revised: Decoder Decoder

Original: [Figure]
Revised: [Figure]

Original: In this example, the link predictor would be trainedto estimate the individual treatment effect (ITE) as 1 − 1 = 

Revised: trained individual treatment effect (ITE) as 1 − 1 =

Original: so it looks for factors other than neighborhood to predict the factuallink.
Revised: so it looks for factors other than neighborhood to predict the factual

Original: Given certain data context, it asks what the outcome would have been if the treatment had not been the observed value.
Revised: Given certain data context, it asks what the outcome would have been if the treatment had not been the observed value.

Original: In the scenario of link prediction, we consider the information of a pair of nodes as context, graph structural properties as treatment, and link existence as outcome.
Revised: In the scenario of link prediction, we consider the information of a pair of nodes as context, graph structural properties as treatment, and link existence as outcome.

Original: Recall the social network example.
Revised: Recall the social network example.

Original: The context is the representations of Alice and Adam that are learned from their personal attributes and relationships with others on the network.
Revised: The context is the representations of Alice and Adam that are learned from their personal attributes and relationships with others on the network.

Original: The treatment is living in the same neighborhood, which can be identiﬁed by community detection.
Revised: The treatment is living in the same neighborhood, which can be identiﬁed by community detection.

Original: And the outcome is their friendship.
Revised: And the outcome is their friendship.

Original: In this work, we present a c ounter f actual graph learning method for l ink p rediction (CFLP) that trains graph learning models to answer the counterfactual questions.
Revised: In this work, we present a c ounter f actual graph learning method for l ink p rediction (CFLP) that trains graph learning models to answer the counterfactual questions.

Original: Figure 1 illustrates this twostep method.
Revised: Figure 1 illustrates this twostep method.

Original: Suppose the treatment variable is deﬁned as one type of global graph structure, e.g., the neighborhood assignment discovered by spectral clustering or community detection algorithms.
Revised: Suppose the treatment variable is deﬁned as one type of global graph structure, e.g., the neighborhood assignment discovered by spectral clustering or community detection algorithms.

Original: We are wondering how likely the neighborhood distribution makes a difference on the link (non- )existence for each pair of nodes.
Revised: We are wondering how likely the neighborhood distribution makes a difference on the link (non- )existence for each pair of nodes.

Original: So, given a pair of nodes (like Alice and Adam) and the treatment value on this pair (in the same neighborhood), we ﬁnd a pair of nodes (like Helen and Bob) that satisﬁes two conditions: (1) it has a different treatment (in different neighborhoods) and (2) it is the most similar pair with the given pair of nodes.
Revised: So, given a pair of nodes (like Alice and Adam) and the treatment value on this pair (in the same neighborhood), we ﬁnd a pair of nodes (like Helen and Bob) that satisﬁes two conditions: (1) it has a different treatment (in different neighborhoods) and (2) it is the most similar pair with the given pair of nodes.

Original: We call these matched pair of nodes as “counterfactual links.”
Revised: We call these matched pair of nodes as “counterfactual links.”

Original: Note that the outcome of the counterfactual link can be either 1 or 0, depending on whether there exists an edge between the matched pair of nodes (Helen and Bob).
Revised: Note that the outcome of the counterfactual link can be either 1 or 0, depending on whether there exists an edge between the matched pair of nodes (Helen and Bob).

Original: The counterfactual link provides unobserved outcome to the given pair of nodes (Alice and Adam) under a counterfactual condition (in different neighborhoods).
Revised: The counterfactual link provides unobserved outcome to the given pair of nodes (Alice and Adam) under a counterfactual condition (in different neighborhoods).

Original: After counterfactual links are created for all (positive and negative) training examples, CFLP trains a link predictor (which can be GNN-based) to learn the representation vectors of nodes to predict both the observed factual links and the counterfactual links.
Revised: After counterfactual links are created for all (positive and negative) training examples, CFLP trains a link predictor (which can be GNN-based) to learn the representation vectors of nodes to predict both the observed factual links and the counterfactual links.

Original: In this Alice-Adam example, the link predictor is trained to estimate the individual treatment effect (ITE) of neighborhood assignment as 1 − 1 = 0 .
Revised: In this Alice-Adam example, the link predictor is trained to estimate the individual treatment effect (ITE) of neighborhood assignment as 1 − 1 = 0 , where ITE is a metric for the effect of treatment on the outcome and zero indicates the given treatment has no effect on the outcome.

Original: So, the learner will try to discover the essential factors on the friendship between Alice and Adam.
Revised: So, the learner will try to discover the essential factors on the friendship between Alice and Adam.

Original: CFLP leverages causal models to ﬁnd these factors for graph learning models to accurately predict missing links.
Revised: CFLP leverages causal models to ﬁnd these factors for graph learning models to accurately predict missing links.

Original: Contributions.
Revised: Contributions.

Original: Our main contributions can be summarized as follows.
Revised: Our main contributions can be summarized as follows.

Original: (1) This is the ﬁrst work that proposes to improve link prediction by causal inference, speciﬁcally, learning to answer counterfactual questions about link existence.
Revised: (1) This is the ﬁrst work that proposes to improve link prediction by causal inference, speciﬁcally, learning to answer counterfactual questions about link existence.

Original: (2) This work introduces CFLP that trains GNN-based link predictors to predict both factual and counterfactual links.
Revised: (2) This work introduces CFLP that trains GNN-based link predictors to predict both factual and counterfactual links.

Original: It learns the causal relationship between global graph structure and link existence.
Revised: It learns the causal relationship between global graph structure and link existence.

Original: (3) CFLP outperforms competitive baseline methods on several benchmark datasets.
Revised: (3) CFLP outperforms competitive baseline methods on several benchmark datasets.

Original: We analyze the impact of counterfactual links as well as the choice of treatment variable.
Revised: We analyze the impact of counterfactual links as well as the choice of treatment variable.

Original: This work sheds insights for improving graph machine learning with causal analysis, which has not been extensively studied yet, when the other direction (machine learning for causal inference) has been studied for a long time.
Revised: This work sheds insights for improving graph machine learning with causal analysis, which has not been extensively studied yet, when the other direction (machine learning for causal inference) has been studied for a long time.

Original:  P ROBLEM D EFINITION
Revised:  P ROBLEM D EFINITION

Original: Notations Let G =
Revised: Notations Let G =

Original: ( V , E ) be an undirected graph of N nodes, where V = { v 1 , v 2 , . .
Revised: ( V , E ) be an undirected graph of N nodes, where V = { v 1 , v 2 , . .

Original: . , v N } is the set of nodes and E ⊆
Revised: . , v N } is the set of nodes and E ⊆

Original: V × V is the set of observed links.
Revised: V × V is the set of observed links.

Original: We denote the adjacency matrix as
Revised: We denote the adjacency matrix as

Original: A ∈ { 0 , 1 } N × N , where A i,j = 1 indicates nodes v i and v j are connected and vice versa.
Revised: A ∈ { 0 , 1 } N × N , where A i,j = 1 indicates nodes v i and v j are connected and vice versa.

Original: We denote the node feature matrix as X ∈ R N × F , where F is the number of node features and x i (bolded) indicates the feature vector of node v i (the i -th row of X ).
Revised: We denote the node feature matrix as X ∈ R N × F , where F is the number of node features and x i (bolded) indicates the feature vector of node v i (the i -th row of X ).

Original: In this work, we follow the commonly accepted problem deﬁnition of link prediction on graph data (Zhang & Chen, 2018; Zhang et al., 2020a;
Revised: In this work, we follow the commonly accepted problem deﬁnition of link prediction on graph data (Zhang & Chen, 2018; Zhang et al., 2020a;

Original: Cai et al., 2021):
Revised: Cai et al., 2021):

Original: Given an observed graph G (with validation and testing links masked off), predict the link existence between every pair of nodes.
Revised: Given an observed graph G (with validation and testing links masked off), predict the link existence between every pair of nodes.

Original: More speciﬁcally, for the GNN-based link prediction methods, they learn low-dimensional node representations Z ∈ R N × H , where H is the dimensional size of latent space such that H (cid:28) F , and then use Z for the prediction of link existence.
Revised: More speciﬁcally, for the GNN-based link prediction methods, they learn low-dimensional node representations Z ∈ R N × H , where H is the dimensional size of latent space such that H (cid:28) F , and then use Z for the prediction of link existence.

Original:  P ROPOSED M ETHOD 3. I MPROVING G RAPH L EARNING WITH C AUSAL M ODEL
Revised:  P ROPOSED M ETHOD 3. I MPROVING G RAPH L EARNING WITH C AUSAL M ODEL

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: Leveraging Causal Model Counterfactual causal inference aims to ﬁnd out the causal relationship between treatment and outcomes by asking the counterfactual questions such as ”would the outcome be different if the treatment was different?”
Revised: Leveraging Causal Model(s) Counterfactual causal inference aims to ﬁnd out the causal relationship between treatment and outcomes by asking the counterfactual questions such as ”would the outcome be different if the treatment was different?”

Original: (Morgan & Winship, 2015).
Revised: (Morgan & Winship, 2015).

Original: Figure 2 is a typical example, in which we denote the context (confounder) as Z , treatment as T , and the outcome as Y .
Revised: Figure 2 is a typical example, in which we denote the context (confounder) as Z , treatment as T , and the outcome as Y .

Original: Given the context, treatments, and their corresponding outcomes, counterfactual inference methods aim to ﬁnd the effect of treatment on the outcome, which is usually measured by individual treatment effect (ITE) and its expectation averaged treatment effect (ATE) (van der Laan & Petersen, 2007;
Revised: Given the context, treatments, and their corresponding outcomes, counterfactual inference methods aim to ﬁnd the effect of treatment on the outcome, which is usually measured by individual treatment effect (ITE) and its expectation averaged treatment effect (ATE) (van der Laan & Petersen, 2007;

Original: Weiss et al., 2015).
Revised: Weiss et al., 2015).

Original: For a binary treatment variable T = { 0 , 1 } , denoting g ( z , T ) as the outcome of z given the treatment T , we have
Revised: For a binary treatment variable T = { 0 , 1 } , denoting g ( z , T ) as the outcome of z given the treatment T , we have

Original: ITE ( z ) =
Revised: ITE ( z ) =

Original: g ( z , 1) − g
Revised: g ( z , 1) − g

Original: ( z , 0) , and ATE =
Revised: ( z , 0) , and ATE =

Original: E z ∼ Z ITE ( z ) .
Revised: E z ∼ Z ITE ( z ) .

Original: Ideally, we need all possible outcomes of the contexts under all kinds of treatments to study the causal relationships (Morgan & Winship, 2015).
Revised: Ideally, we need all possible outcomes of the contexts under all kinds of treatments to study the causal relationships (Morgan & Winship, 2015).

Original: However, in reality, the fact that we can only observe one potential outcome under one particular treatment prevents the ITE from being known (Johansson et al., 2016).
Revised: However, in reality, the fact that we can only observe one potential outcome under one particular treatment prevents the ITE from being known (Johansson et al., 2016).

Original: Traditional causal inference methods hence statistical learning approaches such as Neyman–Rubin casual model (BCM) and propensity score matching (PSM) to predict the value of ATE (Rubin, 1974; 2005).
Revised: Traditional causal inference methods use statistical learning approaches such as Neyman–Rubin casual model (BCM) and propensity score matching (PSM) to predict the value of ATE (Rubin, 1974; 2005).

Original: In this work, we look at link prediction with graph learning, which is essentially learning the best node representations Z for the prediction of link existence.
Revised: In this work, we look at link prediction with graph learning, which is essentially learning the best node representations Z for the prediction of link existence.

Original: Therefore, as shown in Figure 3, where the outcome Y is the link existence, the objective is different from classic causal inference.
Revised: Therefore, as shown in Figure 3, where the outcome Y is the link existence, the objective is different from classic causal inference.

Original: In graph learning, we can estimate the effect of treatment on the outcome (ATE ( Y | T ) ), and we want to improve the learning of Z with the estimation.
Revised: In graph learning, we can estimate the effect of treatment on the outcome (ATE ( Y | T ) ), and we want to improve the learning of Z with the estimation.

Original: More speciﬁcally, in graph learning for link prediction, for each pair of nodes ( v i , v j ) , its ITE can be estimated with
Revised: More speciﬁcally, in graph learning for link prediction, for each pair of nodes ( v i , v j ) , its ITE can be estimated with

Original: [Equation]
Revised: [Equation]

Original: and we use this information to improve the learning of Z , i.e., P ( Z | Y ) .
Revised: and we use this information to improve the learning of Z , i.e., P ( Z | Y ) .

Original: We denote the observed adjacency matrix as the factual outcomes A and the unobserved adjacency matrix when the treatment is different as the counterfactual outcomes A CF .
Revised: We denote the observed adjacency matrix as the factual outcomes A and the unobserved adjacency matrix when the treatment is different as the counterfactual outcomes A CF .

Original: We denote
Revised: We denote

Original: T ∈ { 0 , 1 } N × N as the binary factual treatment matrix, where T i,j indicates the treatment of the node pair ( v i , v j ) .
Revised: T ∈ { 0 , 1 } N × N as the binary factual treatment matrix, where T i,j indicates the treatment of the node pair ( v i , v j ) .

Original: We denote T CF as the counterfactual treatment matrix where T CF i,j = 1 − T
Revised: We denote T CF as the counterfactual treatment matrix where T CF i,j = 1 − T

Original: i,j .
Revised: i,j .

Original: We are interested in (1) estimating the counterfactual outcomes A CF via observed data, (2) learning with the counterfactual adjacency matrix A CF to enhance link prediction, and (3) learning the causal relationship between graph structural information (treatment) and link existence (outcome).
Revised: We are interested in (1) estimating the counterfactual outcomes A CF via observed data, (2) learning with the counterfactual adjacency matrix A CF to enhance link prediction, and (3) learning the causal relationship between graph structural information (treatment) and link existence (outcome).

Original: Treatment Variable Previous work on graph machine learning (Velickovic et al., 2019; Park et al., 2020) showed that the graph’s global structural information could improve the quality of representa- tion vectors of nodes learned by GNNs.
Revised: Treatment Variable Previous work on graph machine learning (Velickovic et al., 2019; Park et al., 2020) showed that the graph’s global structural information could improve the quality of representa- tion vectors of nodes learned by GNNs.

Original: This is because the message passing-based GNNs aggregate local information in the algorithm of representation vector generation and the global structural information is complementary with the aggregated information.
Revised: This is because the message passing-based GNNs aggregate local information in the algorithm of representation vector generation and the global structural information is complementary with the aggregated information.

Original: Therefore, for a pair of nodes, one option of deﬁning the treatment variable is its global structural role in the graph.
Revised: Therefore, for a pair of nodes, one option of deﬁning the treatment variable is its global structural role in the graph.

Original: Without the loss of generality, we use Louvain (Blondel et al., 2008), an unsupervised approach that has been widely used for community detection, as an example.
Revised: Without the loss of generality, we use Louvain (Blondel et al., 2008), an unsupervised approach that has been widely used for community detection, as an example.

Original: Louvain discovers community structure of a graph and assigns each node to one community.
Revised: Louvain discovers community structure of a graph and assigns each node to one community.

Original: Then we can deﬁne the binary treatment variable as whether these two nodes in the pair belong to the same community.
Revised: Then we can deﬁne the binary treatment variable as whether these two nodes in the pair belong to the same community.

Original: Let c : V → N be any graph mining/clustering method that outputs the index of community/cluster/neighborhood that each node belongs to.
Revised: Let c : V → N be any graph mining/clustering method that outputs the index of community/cluster/neighborhood that each node belongs to.

Original: The treatment matrix T is deﬁned as T i,j = 

Revised: The treatment matrix T is deﬁned as T i,j =

Original: if c ( v i ) =
Revised: if c ( v i ) =

Original: c ( v j ) , and T i,j = 0 otherwise.
Revised: c ( v j ) , and T i,j = 0 otherwise.

Original: For the choice of c , we suggest methods that group nodes based on global graph structural information, including but not limited to Louvain (Blondel et al., 2008), K-core (Bader & Hogue, 2003), and spectral clustering (Ng et al., 2001).
Revised: For the choice of c , we suggest methods that group nodes based on global graph structural information, including but not limited to Louvain (Blondel et al., 2008), K-core (Bader & Hogue, 2003), and spectral clustering (Ng et al., 2001).

Original: 3. C OUNTERFACTUAL L INKS
Revised: 3. C OUNTERFACTUAL L INKS

Original: To implement the solution based on above idea, we propose counterfactual links.
Revised: To implement the solution based on above idea, we propose counterfactual links.

Original: As aforementioned, for each node pair, the observed data contains only the factual treatment and outcome, meaning that the link existence for the given node pair with an opposite treatment is unknown.
Revised: As aforementioned, for each node pair, the observed data contains only the factual treatment and outcome, meaning that the link existence for the given node pair with an opposite treatment is unknown.

Original: Therefore, we use the outcome from the nearest observed context as a substitute.
Revised: Therefore, we use the outcome from the nearest observed context as a substitute.

Original: This  idea has been adopted by many causal inference literature (Johansson et al., 2016; Alaa & Van Der Schaar, 2019).
Revised: This matchingbased idea has been adopted by many causal inference literature (Johansson et al., 2016; Alaa & Van Der Schaar, 2019).

Original: That is, we want to ﬁnd the nearest neighbor with the opposite treatment for each observed node pairs and use the nearest neighbor’s outcome as a counterfactual link .
Revised: That is, we want to ﬁnd the nearest neighbor with the opposite treatment for each observed node pairs and use the nearest neighbor’s outcome as a counterfactual link .

Original: Formally, ∀ ( v i , v j ) ∈ V × V , we want to ﬁnd its counterfactual link ( v a , v b ) as below:
Revised: Formally, ∀ ( v i , v j ) ∈ V × V , we want to ﬁnd its counterfactual link ( v a , v b ) as below:

Original: [Equation]
Revised: [Equation]

Original: where d ( · , · ) is a metric of measuring the distance between a pair of node pairs (a pair of contexts).
Revised: where h ( · , · ) is a metric of measuring the distance between a pair of node pairs (a pair of contexts).

Original: Nevertheless, ﬁnding the nearest neighbors by computing the distance between all pairs of node pairs is extremely inefﬁcient and infeasible in application, which takes O ( N 4 ) comparisons (as there are totally O
Revised: Nevertheless, ﬁnding the nearest neighbors by computing the distance between all pairs of node pairs is extremely inefﬁcient and infeasible in application, which takes O ( N 4 ) comparisons (as there are totally O

Original: ( N 2 ) node pairs).
Revised: ( N 2 ) node pairs).

Original: Hence we implement Eq.
Revised: Hence we implement Eq.

Original: (2) using node-level embeddings.
Revised: (2) using node-level embeddings.

Original: Speciﬁcally, considering that we want to ﬁnd the nearest node pair based on not only the raw node features but also structural features, we take the state-of-the-art unsupervised graph representation learning method MVGRL (Hassani & Khasahmadi, 2020) to learn the node embeddings ˜ X ∈ R N × ˜ F from the observed graph.
Revised: Speciﬁcally, considering that we want to ﬁnd the nearest node pair based on not only the raw node features but also structural features, we take the state-of-the-art unsupervised graph representation learning method MVGRL (Hassani & Khasahmadi, 2020) to learn the node embeddings ˜ X ∈ R N × ˜ F from the observed graph (with validation and testing links masked off).

Original: We use ˜ X to ﬁnd the nearest neighbors of node pairs.
Revised: We use ˜ X to ﬁnd the nearest neighbors of node pairs.

Original: Therefore, ∀ ( v i , v j ) ∈ V ×V , we deﬁne its counterfactual link ( v a , v b ) as
Revised: Therefore, ∀ ( v i , v j ) ∈ V × V , we deﬁne its counterfactual link ( v a , v b ) as

Original: [Equation]
Revised: [Equation]

Original: where d ( · , · ) is speciﬁed as the Euclidean distance on the embedding space of ˜ X , and γ is a hyperparameter that deﬁnes the maximum distance that two nodes are considered as similar.
Revised: where d ( · , · ) is speciﬁed as the Euclidean distance on the embedding space of ˜ X , and γ is a hyperparameter that deﬁnes the maximum distance that two nodes are considered as similar.

Original: When no node pair satisﬁes the above equation, we do not assign any nearest neighbor for a given node pair to ensure all the neighbors are similar enough (as substitutes) in the feature space.
Revised: When no node pair satisﬁes the above equation (i.e., there does not exist any node pair with opposite treatment that is close enough to the target node pair), we do not assign any nearest neighbor for the given node pair to ensure all the neighbors are similar enough (as substitutes) in the feature space.

Original: Thus, the counterfactual treatment matrix T CF and the counterfactual adjacency matrix A CF are deﬁned as
Revised: Thus, the counterfactual treatment matrix T CF and the counterfactual adjacency matrix A CF are deﬁned as

Original: [Equation]
Revised: [Equation]

Original: It is worth noting that the node embeddings ˜ X and the nearest neighbors are computed only once and do not change during the learning process.
Revised: It is worth noting that the node embeddings ˜ X and the nearest neighbors are computed only once and do not change during the learning process.

Original: ˜ X is only used for ﬁnding the nearest neighbors.
Revised: ˜ X is only used for ﬁnding the nearest neighbors.

Original: Learning from Counterfactual Distributions Let P F be the factual distribution of the observed contexts and treatments, and P CF be the counterfactual distribution that is composed of the observed contexts and opposite treatments.
Revised: Learning from Counterfactual Distributions Let P F be the factual distribution of the observed contexts and treatments, and P CF be the counterfactual distribution that is composed of the observed contexts and opposite treatments.

Original: We deﬁne the empirical factual distribution ˆ P F ∼ P F as ˆ P F = { ( v i , v j , T i,j ) } Ni,j =1 , and deﬁne the empirical counterfactual distribution ˆ P CF ∼ P CF as ˆ P CF =
Revised: We deﬁne the empirical factual distribution ˆ P F ∼ P F as ˆ P F = { ( v i , v j , T i,j ) } Ni,j =1 , and deﬁne the empirical counterfactual distribution ˆ P CF ∼ P CF as ˆ P CF =

Original: { ( v i , v j , T CFi,j ) } Ni,j =1 .
Revised: { ( v i , v j , T CFi,j ) } Ni,j =1 .

Original: Unlike traditional link prediction methods that take only ˆ P F as input and use the observed outcomes A as the training target, the idea of counterfactual graph learning is to take advantage of the counterfactual distribution by having ˆ P CF as a complementary input and use the counterfactual outcomes A CF as the training target for the counterfactual data samples.
Revised: Unlike traditional link prediction methods that take only ˆ P F as input and use the observed outcomes A as the training target, the idea of counterfactual graph learning is to take advantage of the counterfactual distribution by having ˆ P CF as a complementary input and use the counterfactual outcomes A CF as the training target for the counterfactual data samples.

Original: 3. T HE C OUNTERFACTUAL G RAPH L EARNING M ODEL
Revised: 3. T HE C OUNTERFACTUAL G RAPH L EARNING M ODEL

Original: In this subsection, we present the design of our model as well as the training method.
Revised: In this subsection, we present the design of our model as well as the training method.

Original: The input of the model in CFLP includes (1) the observed graph data A and raw feature matrix X , (2) the factual treatments T F and counterfactual treatments T CF , and (3) the counterfactual graph data A CF .
Revised: The input of the model in CFLP includes (1) the observed graph data A and raw feature matrix X , (2) the factual treatments T F and counterfactual treatments T CF , and (3) the counterfactual graph data A CF .

Original: The output contains link prediction logits in (cid:98) A and (cid:98) A CF for the factual and counterfactual adjacency matrices A and A CF , respectively.
Revised: The output contains link prediction logits in (cid:98) A and (cid:98) A CF for the factual and counterfactual adjacency matrices A and A CF , respectively.

Original: Graph Learning Model
Revised: Graph Learning Model

Original: The model consist of two trainable components: a graph encoder f and a link decoder g .
Revised: The model consist of two trainable components: a graph encoder f and a link decoder g .

Original: The graph encoder generates representation vectors of nodes from graph data
Revised: The graph encoder generates representation vectors of nodes from graph data

Original: G .
Revised: G .

Original: And the link decoder projects the representation vectors of node pairs into the link prediction logits.
Revised: And the link decoder projects the representation vectors of node pairs into the link prediction logits.

Original: The choice of the graph encoder f can be any end-to-end GNN model.
Revised: The choice of the graph encoder f can be any end-to-end GNN model.

Original: Without the loss of generality, here we use the commonly used graph convolutional network (GCN) (Kipf & Welling, 2016a).
Revised: Without the loss of generality, here we use the commonly used graph convolutional network (GCN) (Kipf & Welling, 2016a).

Original: Each layer of GCN is deﬁned as
Revised: Each layer of GCN is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: where l is the layer index,
Revised: where l is the layer index,

Original: ˜ A =
Revised: ˜ A =

Original: A
Revised: A

Original: + I is the adjacency matrix with added self-loops, ˜ D is the diagonal degree matrix ˜ D ii
Revised: + I is the adjacency matrix with added self-loops, ˜ D is the diagonal degree matrix ˜ D ii

Original: =
Revised: =

Original: (cid:80)
Revised: (cid:80)

Original: j ˜
Revised: j ˜

Original: A ij , H (0) =
Revised: A ij , H (0) =

Original: X , W ( l ) is the learnable weight matrix at the l -th layer, and σ ( · ) denotes a nonlinear activation such as ReLU.
Revised: X , W ( l ) is the learnable weight matrix at the l -th layer, and σ ( · ) denotes a nonlinear activation such as ReLU.

Original: We denote Z =
Revised: We denote Z =

Original: f
Revised: f

Original: ( A , X ) ∈ R N × H as the output from the encoder’s last layer, i.e., the H -dimensional representation vectors of nodes.
Revised: ( A , X ) ∈ R N × H as the output from the encoder’s last layer, i.e., the H -dimensional representation vectors of nodes.

Original: Following previous work (Zhang et al., 2020a), we compute the representation of a node pair as the Hadamard product of the vectors of the two nodes.
Revised: Following previous work (Zhang et al., 2020a), we compute the representation of a node pair as the Hadamard product of the vectors of the two nodes.

Original: That is, the representation for the node pair ( v i , v j ) is z i (cid:12) z
Revised: That is, the representation for the node pair ( v i , v j ) is z i (cid:12) z

Original: j ∈ R H , where (cid:12) stands for the Hadamard product.
Revised: j ∈ R H , where (cid:12) stands for the Hadamard product.

Original: For the link decoder that predicts whether a link exists between a pair of nodes, we opt for simplicity and adopt a simple decoder based on multi-layer perceptron (MLP), given the representations of node pairs and their treatments.
Revised: For the link decoder that predicts whether a link exists between a pair of nodes, we opt for simplicity and adopt a simple decoder based on multi-layer perceptron (MLP), given the representations of node pairs and their treatments.

Original: That is, the decoder g is deﬁned as (cid:98)
Revised: That is, the decoder g is deﬁned as (cid:98)

Original: A =
Revised: A =

Original: g ( Z , T ) ,
Revised: g ( Z , T ) ,

Original: where (cid:98) A i,j = MLP ([ z i (cid:12)
Revised: where (cid:98) A i,j = MLP ([ z i (cid:12)

Original: z j , T i,j ]) , (6) (cid:98)
Revised: z j , T i,j ]) , (6) (cid:98)

Original: A CF =
Revised: A CF =

Original: g ( Z , T CF ) , where (cid:98) A CFi,j = MLP ([ z i (cid:12) z j , T CFi,j ]) , (7) where [ · , · ] stands for the concatenation of vectors, and (cid:98) A and (cid:98)
Revised: g ( Z , T CF ) , where (cid:98) A CFi,j = MLP ([ z i (cid:12) z j , T CFi,j ]) , (7) where [ · , · ] stands for the concatenation of vectors, and (cid:98) A and (cid:98)

Original: A CF can be used for estimating the observed ITE as aforementioned in Eq.
Revised: A CF can be used for estimating the observed ITE as aforementioned in Eq.

Original: During the training process, data samples from the empirical factual distribution ˆ P F and the empirical counterfactual distribution ˆ P CF are fed into decoder g and optimized towards A and A CF , respectively.
Revised: During the training process, data samples from the empirical factual distribution ˆ P F and the empirical counterfactual distribution ˆ P CF are fed into decoder g and optimized towards A and A CF , respectively.

Original: That is, for the two distributions, the loss functions are as follows:
Revised: That is, for the two distributions, the loss functions are as follows:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Balancing Counterfactual Learning In the training process, the above loss minimizations train the model on both the empirical factual distribution ˆ P F ∼ P F and empirical counterfactual distribution ˆ P CF ∼ P CF that are not necessarily equal – the training examples (node pairs) do not have to be aligned.
Revised: Balancing Counterfactual Learning In the training process, the above loss minimizations train the model on both the empirical factual distribution ˆ P F ∼ P F and empirical counterfactual distribution ˆ P CF ∼ P CF that are not necessarily equal – the training examples (node pairs) do not have to be aligned.

Original: However, at the stage of inference, the test data contains only observed (factual) samples.
Revised: However, at the stage of inference, the test data contains only observed (factual) samples.

Original: Such a gap between the training and testing data distributions exposes the model in the risk of covariant shift, which is a common issue in counterfactual learning (Johansson et al., 2016;
Revised: Such a gap between the training and testing data distributions exposes the model in the risk of covariant shift, which is a common issue in counterfactual learning (Johansson et al., 2016;

Original: Assaad et al., 2021).
Revised: Assaad et al., 2021).

Original: To force the distributions of representations of factual distributions and counterfactual distributions to be similar, we use the discrepancy distance (Mansour et al., 2009; Johansson et al., 2016) as another objective to regularize the representation learning.
Revised: To force the distributions of representations of factual distributions and counterfactual distributions to be similar, we use the discrepancy distance (Mansour et al., 2009; Johansson et al., 2016) as another objective to regularize the representation learning.

Original: That is, we use the following loss term to minimize the distance between the learned representations from ˆ P F and ˆ P CF :
Revised: That is, we use the following loss term to minimize the distance between the learned representations from ˆ P F and ˆ P CF :

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where || · || F denotes the Frobenius Norm, and ˆ P Ff and ˆ P CFf denote the node pair representations learned by graph encoder f from factual distribution and counterfactual distribution, respectively.
Revised: where || · || F denotes the Frobenius Norm, and ˆ P Ff and ˆ P CFf denote the node pair representations learned by graph encoder f from factual distribution and counterfactual distribution, respectively.

Original: That is, the learned representations for ( v i , v j , T i,j ) and ( v i , v j , T CFi,j ) are [ z i (cid:12)
Revised: That is, the learned representations for ( v i , v j , T i,j ) and ( v i , v j , T CFi,j ) are [ z i (cid:12)

Original: z
Revised: z

Original: j , T i,j ] (Eq.
Revised: j , T i,j ] (Eq.

Original: and [ z i (cid:12) z j , T CFi,j ] (Eq.
Revised: and [ z i (cid:12) z j , T CFi,j ] (Eq.

Original: (7)), respectively.
Revised: (7)), respectively.

Original: Training During the training of CFLP, we want the model to be optimized towards three targets: (1) accurate link prediction on the observed outcomes (Eq.
Revised: Training During the training of CFLP, we want the model to be optimized towards three targets: (1) accurate link prediction on the observed outcomes (Eq.

Original: (2) accurate estimation on the counterfactual outcomes (Eq.
Revised: (2) accurate estimation on the counterfactual outcomes (Eq.

Original: (9)), and (3) regularization on the representation spaces learned from ˆ P F and ˆ P CF (Eq.
Revised: (9)), and (3) regularization on the representation spaces learned from ˆ P F and ˆ P CF (Eq.

Original: Therefore, the overall training loss of our proposed CFLP is
Revised: Therefore, the overall training loss of our proposed CFLP is

Original: [Equation]
Revised: [Equation]

Original: where α and β are hyperparameters to control the weights of counterfactual link prediction (outcome estimation) loss and discrepancy loss.
Revised: where α and β are hyperparameters to control the weights of counterfactual link prediction (outcome estimation) loss and discrepancy loss.

Original: Input : f , g , A , X , n epochs , n epoch ft 1 Compute T as presented in Section 3.1 ;
Revised: Input : f , g , A , X , n epochs , n epoch ft 1 Compute T as presented in Section 3.1 ;

Original: [Equation]
Revised: [Equation]

Original: Summary Algorithm 1 summarizes the whole process of CFLP.
Revised: Summary Algorithm 1 summarizes the whole process of CFLP.

Original: The ﬁrst step is to compute the factual and counterfactual treatments T , T CF as well as the counterfactual outcomes
Revised: The ﬁrst step is to compute the factual and counterfactual treatments T , T CF as well as the counterfactual outcomes

Original: A CF .
Revised: A CF .

Original: Then, the second step trains the graph learning model on both the observed factual data and created counterfactual data with the integrated loss function (Eq.
Revised: Then, the second step trains the graph learning model on both the observed factual data and created counterfactual data with the integrated loss function (Eq.

Original: Note that the discrepancy loss (Eq.
Revised: Note that the discrepancy loss (Eq.

Original: (10)) is computed on the representations of node pairs learned by the graph encoder f , so the decoder g is trained with data from both ˆ P F and ˆ P CF without balancing the constraints.
Revised: (10)) is computed on the representations of node pairs learned by the graph encoder f , so the decoder g is trained with data from both ˆ P F and ˆ P CF without balancing the constraints.

Original: Therefore, after the model is sufﬁciently trained, we freeze the graph encoder f and ﬁne-tune g with only the factual data.
Revised: Therefore, after the model is sufﬁciently trained, we freeze the graph encoder f and ﬁne-tune g with only the factual data.

Original: Finally, after the decoder is sufﬁciently ﬁnetuned, we output the link prediction logits for both the factual and counterfactual adjacency matrices.
Revised: Finally, after the decoder is sufﬁciently ﬁnetuned, we output the link prediction logits for both the factual and counterfactual adjacency matrices.

Original: Complexity
Revised: Complexity

Original: The complexity of the ﬁrst step (ﬁnding counterfactual links with nearest neighbors) is proportional to the number of node pairs.
Revised: The complexity of the ﬁrst step (ﬁnding counterfactual links with nearest neighbors) is proportional to the number of node pairs.

Original: When γ is set as a small value to obtain indeed similar node pairs, this step (Eq. (3)) uses constant time.
Revised: When γ is set as a small value to obtain indeed similar node pairs, this step (Eq. (3)) uses constant time.

Original: Moreover, the computation in Eq.
Revised: Moreover, the computation in Eq.

Original: (3) can be parallelized.
Revised: (3) can be parallelized.

Original: Therefore, the time complexity is O ( N 2 /C )
Revised: Therefore, the time complexity is O ( N 2 /C )

Original: where C is the number of processes.
Revised: where C is the number of processes.

Original: For the complexity of the second step (training counterfactual learning model), the GNN encoder has time complexity of O ( LH 2 N + LH |E| )
Revised: For the complexity of the second step (training counterfactual learning model), the GNN encoder has time complexity of O ( LH 2 N + LH |E| )

Original: (Wu et al., 2020), where L is the number of GNN layers and H is the size of node representations.
Revised: (Wu et al., 2020), where L is the number of GNN layers and H is the size of node representations.

Original: Given that we sample the same number of non-existing links as that of observed links during training, the complexity of a three-layer MLP decoder is O ((( H + 1) · d h + d h · 1) |E| ) =
Revised: Given that we sample the same number of non-existing links as that of observed links during training, the complexity of a three-layer MLP decoder is O ((( H + 1) · d h + d h · 1) |E| ) =

Original: O ( d h ( H + 2) |E| ) , where d h is the number of neurons in the hidden layer.
Revised: O ( d h ( H + 2) |E| ) , where d h is the number of neurons in the hidden layer.

Original: Therefore, the second step has linear time complexity w.r.t.
Revised: Therefore, the second step has linear time complexity w.r.t.

Original: the sum of node and edge counts.
Revised: the sum of node and edge counts.

Original: Limitations First, as mentioned above, the computation of ﬁnding counterfactual links has a worst-case complexity of O ( N 2 ) .
Revised: Limitations First, as mentioned above, the computation of ﬁnding counterfactual links has a worst-case complexity of O ( N 2 ) .

Original: Second, CFLP performs counterfactual prediction with only a single treatment; however, there are quite a few kinds of graph structural information that can be considered as treatments.
Revised: Second, CFLP performs counterfactual prediction with only a single treatment; however, there are quite a few kinds of graph structural information that can be considered as treatments.

Original: Future work can leverage the rich structural information by bundled treatments (Zou et al., 2020) in counterfactual graph learning.
Revised: Future work can leverage the rich structural information by bundled treatments (Zou et al., 2020) in counterfactual graph learning.

Original: [Table]
Revised: [Table]

Original:  E XPERIMENTS 4. E XPERIMENTAL S ETUP
Revised:  E XPERIMENTS 4. E XPERIMENTAL S ETUP

Original: We conduct experiments on ﬁve benchmark datasets including citation networks (C ORA , C ITES EER , P UB M ED (Yang et al., 2016)), social network (F ACEBOOK (McAuley & Leskovec, 2012)), and drug-drug interaction network (OGB- DDI (Wishart et al., 2018)) from the Open Graph Benchmark (OGB) (Hu et al., 2020).
Revised: We conduct experiments on ﬁve benchmark datasets including citation networks (C ORA , C ITES EER , P UB M ED (Yang et al., 2016)), social network (F ACEBOOK (McAuley & Leskovec, 2012)), and drug-drug interaction network (OGB- DDI (Wishart et al., 2018)) from the Open Graph Benchmark (OGB) (Hu et al., 2020).

Original: For the ﬁrst four datasets, we randomly select 10%/20% of the links and the same numbers of disconnected node pairs as validation/test samples.
Revised: For the ﬁrst four datasets, we randomly select 10%/20% of the links and the same numbers of disconnected node pairs as validation/test samples.

Original: The links in the validation and test sets are masked off from the training graph.
Revised: The links in the validation and test sets are masked off from the training graph.

Original: For OGB- DDI , we used the OGB ofﬁcial train/validation/test splits.
Revised: For OGB- DDI , we used the OGB ofﬁcial train/validation/test splits.

Original: Statistics and details for the datasets are given in Appendix.
Revised: Statistics and details for the datasets are given in Appendix.

Original: We use K-core (Bader & Hogue, 2003) clusters as the default treatment variable.
Revised: We use K-core (Bader & Hogue, 2003) clusters as the default treatment variable.

Original: We evaluate CFLP on three commonly used GNN encoders: GCN (Kipf & Welling, 2016a), GSAGE (Hamilton et al., 2017), and JKNet (Xu et al., 2018).
Revised: We evaluate CFLP on three commonly used GNN encoders: GCN (Kipf & Welling, 2016a), GSAGE (Hamilton et al., 2017), and JKNet (Xu et al., 2018).

Original: We compare the link prediction performance of CFLP against
Revised: We compare the link prediction performance of CFLP against

Original: Node2Vec (Grover & Leskovec, 2016), MVGRL (Hassani & Khasahmadi, 2020), VGAE (Kipf &
Revised: Node2Vec (Grover & Leskovec, 2016), MVGRL (Hassani & Khasahmadi, 2020), VGAE (Kipf &

Original: Welling, 2016b), SEAL (Zhang & Chen, 2018), LGLP (Cai et al., 2021), and GNNs with MLP decoder.
Revised: Welling, 2016b), SEAL (Zhang & Chen, 2018), LGLP (Cai et al., 2021), and GNNs with MLP decoder.

Original: We report averaged test performance and their standard deviation over 20 runs with different random parameter initializations.
Revised: We report averaged test performance and their standard deviation over 20 runs with different random parameter initializations.

Original: Other than the most commonly used of Area Under ROC Curve (AUC), we report Hits@20 (one of the primary metrics on OGB leaderboard) as a more challenging metric, as it expects models to rank positive edges higher than nearly all negative edges.
Revised: Other than the most commonly used of Area Under ROC Curve (AUC), we report Hits@20 (one of the primary metrics on OGB leaderboard) as a more challenging metric, as it expects models to rank positive edges higher than nearly all negative edges.

Original: Besides performance comparison on link prediction, we will answer two questions to suggest a way of choosing a treatment variable for creating counterfactual links: (Q1) Does CFLP sufﬁciently learn the observed averaged treatment effect (ATE) derived from the counterfactual links? (Q2)
Revised: Besides performance comparison on link prediction, we will answer two questions to suggest a way of choosing a treatment variable for creating counterfactual links: (Q1) Does CFLP sufﬁciently learn the observed averaged treatment effect (ATE) derived from the counterfactual links? (Q2)

Original: What is the relationship between the estimated ATE learned in the method and the prediction performance?
Revised: What is the relationship between the estimated ATE learned in the method and the prediction performance?

Original: If the answer to Q1 is yes, then the answer to Q2 will indicate how to choose treatment based on observed ATE.
Revised: If the answer to Q1 is yes, then the answer to Q2 will indicate how to choose treatment based on observed ATE.

Original: To answer the Q1, we calculate the observed ATE ( (cid:100) ATE obs ) by comparing the observed links in A and created counterfactual links A CF that have opposite treatments.
Revised: To answer the Q1, we calculate the observed ATE ( (cid:100) ATE obs ) by comparing the observed links in A and created counterfactual links A CF that have opposite treatments.

Original: And we calculate the estimated ATE ( (cid:100) ATE est ) by comparing the predicted links in (cid:98) A and predicted counterfactual links (cid:98) A CF .
Revised: And we calculate the estimated ATE ( (cid:100) ATE est ) by comparing the predicted links in (cid:98) A and predicted counterfactual links (cid:98) A CF .

Original: Formally, (cid:100) ATE obs and (cid:100) ATE est are deﬁned as
Revised: Formally, (cid:100) ATE obs and (cid:100) ATE est are deﬁned as

Original: [Equation]
Revised: [Equation]

Original: The treatment variables we will investigate are usually graph clustering or community detection methods, such as K-core (Bader & Hogue, 2003), stochastic block model (SBM) (Karrer & Newman, 2011),
Revised: The treatment variables we will investigate are usually graph clustering or community detection methods, such as K-core (Bader & Hogue, 2003), stochastic block model (SBM) (Karrer & Newman, 2011),

Original: spectral clustering (SpecC)
Revised: spectral clustering (SpecC)

Original: (Ng et al., 2001), propagation clustering (PropC) (Raghavan et al., 2007), Louvain (Blondel et al., 2008), common neighbors (CommN), Katz index, and hierarchical clustering (Ward)
Revised: (Ng et al., 2001), propagation clustering (PropC) (Raghavan et al., 2007), Louvain (Blondel et al., 2008), common neighbors (CommN), Katz index, and hierarchical clustering (Ward)

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Implementation details and supplementary experimental results (e.g., sensitivity on γ , ablation study on L CF and L disc ) can be found in Appendix.
Revised: Implementation details and supplementary experimental results (e.g., sensitivity on γ , ablation study on L CF and L disc ) can be found in Appendix.

Original: Source code is available in supplementary material.
Revised: Source code is available in supplementary material.

Original: 4. E XPERIMENTAL R ESULTS
Revised: 4. E XPERIMENTAL R ESULTS

Original: Link Prediction Tables 1 and 2 show the link prediction performance of Hits@20 and AUC by all methods.
Revised: Link Prediction Tables 1 and 2 show the link prediction performance of Hits@20 and AUC by all methods.

Original: LGLP on P UB M ED and OGB- DDI are missing due to the out of memory error when running the code package from the authors.
Revised: LGLP on P UB M ED and OGB- DDI are missing due to the out of memory error when running the code package from the authors.

Original: We observe that our CFLP on different graph encoders achieve similar or better performances compared with baselines.
Revised: We observe that our CFLP on different graph encoders achieve similar or better performances compared with baselines.

Original: The only exception is the AUC on F ACEBOOK where most methods have close-to-perfect AUC.
Revised: The only exception is the AUC on F ACEBOOK where most methods have close-to-perfect AUC.

Original: As AUC is a relatively easier metric comparing with Hits@20, most methods achieved good performance on AUC.
Revised: As AUC is a relatively easier metric comparing with Hits@20, most methods achieved good performance on AUC.

Original: We observe that
Revised: We observe that

Original: CFLP with JKNet almost consistently achieves the best performance and outperforms baselines signiﬁcantly on Hits@20.
Revised: CFLP with JKNet almost consistently achieves the best performance and outperforms baselines signiﬁcantly on Hits@20.

Original: Speciﬁcally, comparing with the best baseline, CFLP improves relatively by 16.4% and 0.8% on Hits@20 and AUC, respectively.
Revised: Speciﬁcally, comparing with the best baseline, CFLP improves relatively by 16.4% and 0.8% on Hits@20 and AUC, respectively.

Original: Comparing with the best performing baselines, which are also GNN-based, CFLP beneﬁts from learning with both observed link existence ( A ) and our deﬁned counterfactual links ( A CF ).
Revised: Comparing with the best performing baselines, which are also GNN-based, CFLP beneﬁts from learning with both observed link existence ( A ) and our deﬁned counterfactual links ( A CF ).

Original: ATE with Different Treatments Tables 3 and 4 show the link prediction performance, (cid:100) ATE obs , and (cid:100) ATE est of CFLP (with JKNet) when using different treatments.
Revised: ATE with Different Treatments Tables 3 and 4 show the link prediction performance, (cid:100) ATE obs , and (cid:100) ATE est of CFLP (with JKNet) when using different treatments.

Original: The treatments in Tables 3 andare sorted by the Hits@20 performance.
Revised: The treatments in Tables 3 andare sorted by the Hits@20 performance.

Original: Bigger ATE indicates stronger causal relationship between the treatment and outcome, and vice versa.
Revised: Bigger ATE indicates stronger causal relationship between the treatment and outcome, and vice versa.

Original: We observe: (1)  (cid:100) ATE est values are generally close to (cid:100) ATE obs  , showing that CFLP was sufﬁciently trained to learn the causal relationship between graph structure information and link existence; (2) (cid:100) ATE obs and (cid:100) ATE est are both negatively correlated with the link prediction performance, showing that we can pick a proper treatment prior to training a model with CFLP.
Revised: We observe: (1) the rankings of (cid:100) ATE est and (cid:100) ATE obs are positively correlated with Kendell’s ranking coefﬁcient (Abdi, 2007) of 0.67 and 0.57 for C ORA and C ITE S EER , respectively. Hence, CFLP was sufﬁciently trained to learn the causal relationship between graph structure information and link existence; (2) (cid:100) ATE obs and (cid:100) ATE est are both negatively correlated with the link prediction performance, showing that we can pick a proper treatment prior to training a model with CFLP.

Original: Using the treatment that has the weakest causal relationship with link existence is likely to train the model to capture more essential factors on the outcome, in a way similar to denoising the unrelated information from the representations.
Revised: Using the treatment that has the weakest causal relationship with link existence is likely to train the model to capture more essential factors on the outcome, in a way similar to denoising the unrelated information from the representations.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Link Prediction With its wide applications, link prediction has draw attention from many research communities including statistical machine learning and data mining.
Revised: Link Prediction With its wide applications, link prediction has draw attention from many research communities including statistical machine learning and data mining.

Original: Stochastic generative methods based on stochastic block models (SBM) are developed to generate links (Mehta et al., 2019).
Revised: Stochastic generative methods based on stochastic block models (SBM) are developed to generate links (Mehta et al., 2019).

Original: In data mining, matrix factorization (Menon & Elkan, 2011), heuristic methods (Philip et al., 2010; Mart´ınez et al., 2016), and graph embedding methods (Cui et al., 2018) have been applied to predict links in the graph.
Revised: In data mining, matrix factorization (Menon & Elkan, 2011), heuristic methods (Philip et al., 2010; Mart´ınez et al., 2016), and graph embedding methods (Cui et al., 2018) have been applied to predict links in the graph.

Original: Heuristic methods compute the similarity score of nodes based on their neighborhoods.
Revised: Heuristic methods compute the similarity score of nodes based on their neighborhoods.

Original: These methods can be generally categorized into ﬁrst-order, second-order, and high-order heuristics based on the maximum distance of the neighbors.
Revised: These methods can be generally categorized into ﬁrst-order, second-order, and high-order heuristics based on the maximum distance of the neighbors.

Original: Graph embedding methods learn latent node features via embedding lookup and use them for link prediction (Perozzi et al., 2014; Tang et al., 2015; Grover & Leskovec, 2016; Wang et al., 2016).
Revised: Graph embedding methods learn latent node features via embedding lookup and use them for link prediction (Perozzi et al., 2014; Tang et al., 2015; Grover & Leskovec, 2016; Wang et al., 2016).

Original: In the past few years, GNNs have showed promising results on various graph-based tasks with their ability of learning from features and custom aggregations on structures, (Kipf & Welling, 2016a; Hamilton et al., 2017; Xu et al., 2018; Wu et al., 2020).
Revised: In the past few years, GNNs have showed promising results on various graph-based tasks with their ability of learning from features and custom aggregations on structures, (Kipf & Welling, 2016a; Hamilton et al., 2017; Xu et al., 2018; Wu et al., 2020).

Original: With node pair representations and an attached MLP or inner-product decoder, GNNs can be used for link prediction (Zhang et al., 2020a;
Revised: With node pair representations and an attached MLP or inner-product decoder, GNNs can be used for link prediction (Zhang et al., 2020a;

Original: Davidson et al., 2018;
Revised: Davidson et al., 2018;

Original: Yang et al., 2018).
Revised: Yang et al., 2018).

Original: For example, VGAE used GCN to learn node representations and reconstruct the graph structure (Kipf & Welling, 2016b).
Revised: For example, VGAE used GCN to learn node representations and reconstruct the graph structure (Kipf & Welling, 2016b).

Original: SEAL extracted a local subgraph around each target node pair and then learned graph representation from local subgraph for link prediction (Zhang & Chen, 2018).
Revised: SEAL extracted a local subgraph around each target node pair and then learned graph representation from local subgraph for link prediction (Zhang & Chen, 2018).

Original: Following the scheme of SEAL, Cai & Ji (2020) proposed to improve local subgraph representation learning by multi-scale graph representation learning.
Revised: Following the scheme of SEAL, Cai & Ji (2020) proposed to improve local subgraph representation learning by multi-scale graph representation learning.

Original: And
Revised: And

Original: LGLP inverted the local subgraphs to line graphs before learning representations (Cai et al., 2021).
Revised: LGLP inverted the local subgraphs to line graphs before learning representations (Cai et al., 2021).

Original: However, very limited work has studied to use causal inference for improving link prediction.
Revised: However, very limited work has studied to use causal inference for improving link prediction.

Original: Counterfactual Prediction As a mean of learning the causality between treatment and outcome, counterfactual prediction has been used for a variety of applications such as recommender systems (Wang et al., 2020; Xu et al., 2020), health care (Alaa & van der Schaar, 2017), vision-language tasks (Zhang et al., 2020b; Parvaneh et al., 2020), and decision making (Coston et al., 2020; Pitis et al., 2020; Kusner et al., 2017).
Revised: Counterfactual Prediction As a mean of learning the causality between treatment and outcome, counterfactual prediction has been used for a variety of applications such as recommender systems (Wang et al., 2020; Xu et al., 2020), health care (Alaa & van der Schaar, 2017; Pawlowski et al., 2020), vision-language tasks (Zhang et al., 2020b; Parvaneh et al., 2020), and decision making (Coston et al., 2020; Pitis et al., 2020; Kusner et al., 2017).

Original: To infer the causal relationships, previous work usually estimated the ITE via function ﬁtting models (Gelman & Hill, 2006; Chipman et al., 2010; Wager & Athey, 2018; Assaad et al., 2021).Peysakhovich et al.
Revised: To infer the causal relationships, previous work usually estimated the ITE via function ﬁtting models (Gelman & Hill, 2006; Chipman et al., 2010; Wager & Athey, 2018; Assaad et al., 2021). Peysakhovich et al.

Original: (2019) and Zou et al.
Revised: (2019) and Zou et al.

Original: (2020) studied counterfactual prediction with multiple agents and bundled treatments, respectively.
Revised: (2020) studied counterfactual prediction with multiple agents and bundled treatments, respectively.

Original: Pawlowski et al. (2020) proposed a deep structural causal model for tractable counterfactual inference.
Revised: Sherman & Shpitser (2020) modeled social networks with causal DAGs.

Original: Causal Inference Causal inference methods usually re-weighted samples based on propensity score (Rosenbaum & Rubin, 1983; Austin, 2011; Kuang et al., 2017a;b) to remove confounding bias from binary treatments.
Revised: Causal Inference Causal inference methods usually re-weighted samples based on propensity score (Rosenbaum & Rubin, 1983; Austin, 2011; Kuang et al., 2017a;b) to remove confounding bias from binary treatments.

Original: Recently, several works studied about learning treatment invariant representation to predict the counterfactual outcomes (Hassanpour & Greiner, 2019b;a; Shalit et al., 2017;  Yao et al., 2018; Bica et al., 2020; Hassanpour & Greiner, 2019a; Li & Fu, 2017).
Revised: Recently, several works studied about learning treatment invariant representation to predict the counterfactual outcomes (Shalit et al., 2017; Li & Fu, 2017; Yao et al., 2018; Yoon et al., 2018; Hassanpour & Greiner, 2019b;a; Bica et al., 2020).

Original: When part of unobserved outcomes may mislead the counterfactual prediction, Louizos et al.
Revised: When part of unobserved outcomes may mislead the counterfactual prediction, Louizos et al.

Original: (2017) attempted to infer the outcomes from proxies, and Hartford et al.
Revised: (2017) attempted to infer the outcomes from proxies, and Hartford et al.

Original: (2017) introduced instrumental variable.
Revised: (2017) introduced instrumental variable.

Original: SITE preserved local similarity to balance the distributions of control and treated groups (Yao et al., 2018).Yoon et al.
Revised: SITE preserved local similarity to balance the distributions of control and treated groups (Yao et al., 2018).

Original: (2018) estimated ITE with generative adversarial networks (GANs).

Original: Assaad et al.

Original: (2021) discussed the trade-off between achieving balance and predictive power.

Original: Bevilacqua et al.
Revised: Bevilacqua et al.

Original: (2021) tackled graph classiﬁcation under a counterfactually-driven data shift.
Revised: (2021) tackled graph classiﬁcation under a counterfactually-driven data shift.

Original:  C ONCLUSION AND F UTURE W ORK
Revised:  C ONCLUSION AND F UTURE W ORK

Original: In this work, we presented a counterfactual graph learning method for link prediction (CFLP).
Revised: In this work, we presented a counterfactual graph learning method for link prediction (CFLP).

Original: We introduced the idea of counterfactual prediction to improve link prediction on graphs.
Revised: We introduced the idea of counterfactual prediction to improve link prediction on graphs.

Original: CFLP accurately predicted the missing links by exploring the causal relationship between global graph structure and link existence.
Revised: CFLP accurately predicted the missing links by exploring the causal relationship between global graph structure and link existence.

Original: Extensive experiments demonstrated that CFLP achieved the stateof-the-art performance on benchmark datasets.
Revised: Extensive experiments demonstrated that CFLP achieved the state-of-the-art performance on benchmark datasets.

Original: This work sheds insights that a good use of causal models (even basic ones) can greatly improve the performance of (graph) machine learning tasks, which in our case is link prediction.
Revised: This work sheds insights that a good use of causal models (even basic ones) can greatly improve the performance of (graph) machine learning tasks, which in our case is link prediction.

Original: We note that the use of more sophistically designed causal models may lead to larger improvements for other machine learning tasks, which could be a valuable future research direction for the community.
Revised: We note that the use of more sophistically designed causal models may lead to larger improvements for other machine learning tasks, which could be a valuable future research direction for the community.

Revised: Other than our use of global graph structure as treatment, other treatments choices (with both empirical and theoretical analyses) are also worth exploring.

Revised: Moreover, as CFLP ﬁrst generates counterfactual links and then learns from both observed and counterfactual link existence, investigating the relationship between counterfactual graph learning and graph data augmentation is also a possible future research direction.

Original: A PPENDICES A A DDITIONAL D ATASET D ETAILS
Revised: A PPENDICES A A DDITIONAL D ATASET D ETAILS

Original: In this section, we provide some additional dataset details.
Revised: In this section, we provide some additional dataset details.

Original: All the datasets used in this work are publicly available.
Revised: All the datasets used in this work are publicly available.

Original: Statistics for the datasets are shown in Table 5.
Revised: Statistics for the datasets are shown in Table 5.

Original: Citation Networks
Revised: Citation Networks

Original: C ORA , C ITE S EER , and P UB M ED are citation networks that were ﬁrst used by Yang et al.
Revised: C ORA , C ITE S EER , and P UB M ED are citation networks that were ﬁrst used by Yang et al.

Original: and then commonly used as benchmarks in GNN-related literature (Kipf &
Revised: and then commonly used as benchmarks in GNN-related literature (Kipf &

Original: Welling, 2016a; Veli ˇ ckovi ´ c et al., 2017).
Revised: Welling, 2016a; Veli ˇ ckovi ´ c et al., 2017).

Original: In these citation networks, the nodes are published papers and features are bag-of-word vectors extracted from the corresponding paper.
Revised: In these citation networks, the nodes are published papers and features are bag-of-word vectors extracted from the corresponding paper.

Original: Links represent the citation relation between papers.
Revised: Links represent the citation relation between papers.

Original: We loaded the datasets with the DGL 1 package.
Revised: We loaded the datasets with the DGL 1 package.

Original: Social Network The F ACEBOOK dataset 2 is a social network constructed from friends lists from
Revised: Social Network The F ACEBOOK dataset 2 is a social network constructed from friends lists from

Original: Facebook (McAuley & Leskovec, 2012).
Revised: Facebook (McAuley & Leskovec, 2012).

Original: The nodes are Facebook users and links indicate the friendship relation on Facebook.
Revised: The nodes are Facebook users and links indicate the friendship relation on Facebook.

Original: The node features were constructed from the user proﬁles and anonymized by McAuley & Leskovec (2012).
Revised: The node features were constructed from the user proﬁles and anonymized by McAuley & Leskovec (2012).

Original: Drug-Drug Interaction Network The OGB- DDI dataset was constructed from a public Drug database (Wishart et al., 2018) and provided by the Open Graph Benchmark (OGB)
Revised: Drug-Drug Interaction Network The OGB- DDI dataset was constructed from a public Drug database (Wishart et al., 2018) and provided by the Open Graph Benchmark (OGB)

Original: (Hu et al., 2020).
Revised: (Hu et al., 2020).

Original: Each node in this graph represents an FDA-approved or experimental drug and edges represent the existence of unexpected effect when the two drugs are taken together.
Revised: Each node in this graph represents an FDA-approved or experimental drug and edges represent the existence of unexpected effect when the two drugs are taken together.

Original: This dataset does not contain any node features, and it can be downloaded with the dataloader 3 provided by OGB.
Revised: This dataset does not contain any node features, and it can be downloaded with the dataloader 3 provided by OGB.

Original: B D ETAILS ON I MPLEMENTATION AND H YPERPARAMETERS
Revised: B D ETAILS ON I MPLEMENTATION AND H YPERPARAMETERS

Original: All the experiments in this work were conducted on a Linux server with Intel Xeon Gold
Revised: All the experiments in this work were conducted on a Linux server with Intel Xeon Gold

Original: Processor (16 Cores @2.1Ghz), 96 GB of RAM, and 4 RTX 2080Ti cards (11 GB of RAM each).
Revised: Processor (16 Cores @2.1Ghz), 96 GB of RAM, and 4 RTX 2080Ti cards (11 GB of RAM each).

Original: Our method are implemented with Python 3.8.5 with PyTorch .
Revised: Our method are implemented with Python 3.8.5 with PyTorch .

Revised: Source code is available in the supplementary materials.

Original: A list of used packages can be found in requirements.txt .
Revised: A list of used packages can be found in requirements.txt .

Original: Baseline Methods For baseline methods, we use ofﬁcial code packages from the authors for MVGRL 4 (Hassani & Khasahmadi, 2020), SEAL 5 (Zhang & Chen, 2018), and LGLP 

Revised: Baseline Methods For baseline methods, we use ofﬁcial code packages from the authors for MVGRL 4 (Hassani & Khasahmadi, 2020), SEAL 5 (Zhang & Chen, 2018), and LGLP

Original: (Cai et al., 2021).
Revised: (Cai et al., 2021).

Original: We use a public implementation for VGAE 7 (Kipf & Welling, 2016b) and OGB implementationsfor Node2Vec and baseline GNNs.
Revised: We use a public implementation for VGAE 7 (Kipf & Welling, 2016b) and OGB implementationsfor Node2Vec and baseline GNNs.

Original: For fair comparison, we set the size of node/link representations to be 256 of all methods.
Revised: For fair comparison, we set the size of node/link representations to be 256 of all methods.

Original: [Table]
Revised: [Table]

Original: CFLP
Revised: CFLP

Original: We use the Adam optimizer with a simple cyclical learning rate scheduler (Smith, 2017), in which the learning rate waves cyclically between the given learning rate ( lr ) and 1e-4 in every 70 epochs (50 warmup steps and 20 annealing steps).
Revised: We use the Adam optimizer with a simple cyclical learning rate scheduler (Smith, 2017), in which the learning rate waves cyclically between the given learning rate ( lr ) and 1e-4 in every 70 epochs (50 warmup steps and 20 annealing steps).

Original: We implement the GNN encoders with torch_geometric 9 (Fey & Lenssen, 2019).
Revised: We implement the GNN encoders with torch_geometric 9 (Fey & Lenssen, 2019).

Original: Same with the baselines, we set the size of all hidden layers and node/link representations of CFLP as 256.
Revised: Same with the baselines, we set the size of all hidden layers and node/link representations of CFLP as 256.

Original: The graph encoders all have three layers and JKNet has mean pooling for the ﬁnal aggregation layer.
Revised: The graph encoders all have three layers and JKNet has mean pooling for the ﬁnal aggregation layer.

Original: The decoder is a 3-layer MLP with a hidden layer of size 64 and ELU as the nonlinearity.
Revised: The decoder is a 3-layer MLP with a hidden layer of size 64 and ELU as the nonlinearity.

Original: As the Euclidean distance used in Eq.
Revised: As the Euclidean distance used in Eq.

Original: (3) has a range of [0 , ∞ ) , the value of γ depends on the distribution of all-pair node embedding distances, which varies for different datasets.
Revised: (3) has a range of [0 , ∞ ) , the value of γ depends on the distribution of all-pair node embedding distances, which varies for different datasets.

Original: Therefore, we set the value of γ as the γ pct -percentile of all-pair node embedding distances.
Revised: Therefore, we set the value of γ as the γ pct -percentile of all-pair node embedding distances.

Original: Commands for reproducing the experiments are included in
Revised: Commands for reproducing the experiments are included in

Original: README.md .
Revised: README.md .

Original: Hyperparameter Searching Space
Revised: Hyperparameter Searching Space

Original: We manually tune the following hyperparameters over range: lr ∈ { 0 . 005 , 0 . 01 , 0 . 05 , 0 . 1 , 0 . 2 } , α ∈ { 0 . 001 , 0 . 01 , 0 . 1 , 1 , 2 } , β ∈ { 0 . 001 , 0 . 01 , 0 . 1 , 1 , 2 } , γ pct ∈ { 10 , 20 , 30 } .
Revised: We manually tune the following hyperparameters over range: lr ∈ { 0 . 005 , 0 . 01 , 0 . 05 , 0 . 1 , 0 . 2 } , α ∈ { 0 . 001 , 0 . 01 , 0 . 1 , 1 , 2 } , β ∈ { 0 . 001 , 0 . 01 , 0 . 1 , 1 , 2 } , γ pct ∈ { 10 , 20 , 30 } .

Original: Treatments For the graph clustering or community detection methods we used as treatments, we use the implementation from scikit-network 10 for Louvain (Blondel et al., 2008), SpecC (Ng et al., 2001),
Revised: Treatments For the graph clustering or community detection methods we used as treatments, we use the implementation from scikit-network 10 for Louvain (Blondel et al., 2008), SpecC (Ng et al., 2001),

Original: PropC (Raghavan et al., 2007), and Ward (Ward Jr, 1963).
Revised: PropC (Raghavan et al., 2007), and Ward (Ward Jr, 1963).

Original: We used implementation of
Revised: We used implementation of

Original: K-core (Bader & Hogue, 2003) from networkx .
Revised: K-core (Bader & Hogue, 2003) from networkx .

Original: 11 We used SBM (Karrer & Newman, 2011) from a public implementation by Funke & Becker (2019).
Revised: 11 We used SBM (Karrer & Newman, 2011) from a public implementation by Funke & Becker (2019).

Original: For CommN and Katz, we set T i,j = 1 if the number of common neighbors or Katz index between v i and v j are greater or equal to 2 or 2 times the average of all Katz index values, respectively.
Revised: For CommN and Katz, we set T i,j = 1 if the number of common neighbors or Katz index between v i and v j are greater or equal to 2 or 2 times the average of all Katz index values, respectively.

Original: For SpecC, we set the number of clusters as 16.
Revised: For SpecC, we set the number of clusters as 16.

Original: For SBM, we set the number of communities as 16.
Revised: For SBM, we set the number of communities as 16.

Original: These settings are ﬁxed for all datasets.
Revised: These settings are ﬁxed for all datasets.

Original: C A DDITIONAL E XPERIMENTAL R ESULTS 
Revised: C A DDITIONAL E XPERIMENTAL R ESULTS AND D ISCUSSIONS

Original: Link Prediction Tables 6 and 7 show the link prediction performance of Hits@50 and Average
Revised: Link Prediction Tables 6 and 7 show the link prediction performance of Hits@50 and Average

Original: Precision (AP) by all methods.
Revised: Precision (AP) by all methods.

Original: LGLP on P UB M ED and OGB- DDI are missing due to the out of memory error when running the code package from the authors.
Revised: LGLP on P UB M ED and OGB- DDI are missing due to the out of memory error when running the code package from the authors.

Original: Similar to the results in Tablesand 2, we observe that our CFLP on different graph encoders achieve similar or better performances compared with baselines, with the only exception of AP on F ACEBOOK where most methods have close-to-perfect AP.
Revised: Similar to the results in Tablesand 2, we observe that our CFLP on different graph encoders achieve similar or better performances compared with baselines, with the only exception of AP on F ACEBOOK where most methods have close-to-perfect AP.

Original: We observe that CFLPwith JKNet almost consistently achieves the best performance and outperforms baselines signiﬁcantly on Hits@50.
Revised: From Tables 1, 2, 6 and 7, we observe that CFLP achieves improvement over all GNN architectures (averaged across datasets).

Original: Speciﬁcally,compared with the best baseline, CFLP improves relatively by 6.8% and 0.9% on Hits@50 and AP, respectively.
Revised: Speciﬁcally, CFLP improves 25.6% (GCN), 12.0% (GSAGE), and 36.3% (JKNet) on Hits@20, 9.6% (GCN), 5.0% (GSAGE), and 17.8% (JKNet) [Table]

Original: [Table]
Revised: [Table]

Revised: on Hits@50, 5.6% (GCN), 1.6% (GSAGE), and 1.9% (JKNet) on AUC

Original: [Table]
Revised: [Table]

Revised: and 0.8% (GCN), 0.8% (GSAGE), and 1.8% (JKNet) on AP.

Revised: We note that CFLP with JKNet almost consistently achieves the best performance and outperforms baselines signiﬁcantly on Hits@50.

Revised: Speciﬁcally, compared with the best baseline, CFLP improves relatively by 6.8% and 0.9% on Hits@50 and AP, respectively.

Original: Ablation Study  For the ablative studies of L CF (Eq.
Revised: Ablation Study on Losses For the ablative studies of L CF (Eq.

Original: (9)) and L disc (Eq.
Revised: (9)) and L disc (Eq.

Original: , we show their effect by removing them from the integrated loss function (Eq.
Revised: , we show their effect by removing them from the integrated loss function (Eq.

Original: Table 8 shows the results of CFLP on C ORA and C ITE S EER under different settings ( α = 0 , β = 0 , α = β = 0 , and original setting).
Revised: Table 8 shows the results of CFLP on C ORA and C ITE S EER under different settings ( α = 0 , β = 0 , α = β = 0 , and original setting).

Original: We observe that CFLP in the original setting achieves the best performance.
Revised: We observe that CFLP in the original setting achieves the best performance.

Original: The performance drops signiﬁcantly when having α = 0 , i.e., not using any counterfactual data during training.
Revised: The performance drops signiﬁcantly when having α = 0 , i.e., not using any counterfactual data during training.

Original: We note that having β = 0 , i.e., not using the discrepancy loss, also lowers the performance.
Revised: We note that having β = 0 , i.e., not using the discrepancy loss, also lowers the performance.

Original: Therefore, both L CF and L disc are essential for improving the link prediction performance.
Revised: Therefore, both L CF and L disc are essential for improving the link prediction performance.

Original: Sensitivity Analysis of α andβ
Revised: Ablation Study on Node Embedding ˜ X As the node embedding ˜ X is used in the early step of CFLP for ﬁnding the counterfactual links, the quality of ˜ X may affect the later learning process. Therefore, we also evaluate CFLP with different state-of-the-art unsupervised graph representation learning methods: MVGRL (Hassani & Khasahmadi, 2020), DGI (Velickovic et al., 2019), and

Revised: GRACE (Zhu et al., 2020).

Original: Figure 4 shows the AUC performance of CFLP  on C ORA  with different combinations of α and β .
Revised: Table 9 shows the link prediction performance of CFLP (w/ JKNet) on C ORA and C ITE S EER with different node embeddings.

Original: We observe that the performance is the poorest when α = β =and gradually improves and gets stable asα and β increase, showing that CFLP is generally robust to the hyperparameters α and β , and the optimal values are easy to locate.
Revised: We observe that the choice of the method for learning ˜ X does have an impact on the later learning process as well as the link prediction performance. Nevertheless, Table 9 shows CFLP’s advantage can be consistently observed with different choices of methods for learning ˜ X , as CFLP with ˜ X learned from all three methods showed promising link prediction performance.

Original: Sensitivity Analysis of γ
Revised: Sensitivity Analysis of α and β

Original: Figure 5 shows theHits@20 and AUC performanceon link prediction of CFLP(with JKNet) on C ORAand C ITE S EER with different treatments and γ pct .
Revised: Figure 4 shows the AUC performance of CFLP on C ORA with different combinations of α and β .

Original: We observe that the performance is generally good when 10 ≤ γ pct ≤ 20 and gradually get worse when the value of γ pct is too small or too large, showing that CFLP is  robust to γ and the optimal γ is easy to ﬁnd.
Revised: We observe that the performance is the poorest when α = β =and gradually improves and gets stable as α and β increase, showing that CFLP is generally robust to the hyperparameters α and β , and the optimal values are easy to locate. [Table] [Figure] [Table]

Original: [Figure]
Revised: [Figure]

Revised: [Table]

Original: [Figure] 
Revised: [Figure] [Table] [Figure] [Table]

Original: [Figure] 
Revised: [Figure] [Table] [Figure] Analysis of

Original: [Figure]
Revised: Figure 5 shows the Hits@20 and AUC performance on link prediction of CFLP (with JKNet) on C ORA and C ITE S EER with different treatments and γ pct . We observe that the performance is generally good when 10 ≤ γ pct ≤ 20 and gradually get worse when the value of γ pct is too small or too large, showing that CFLP is robust to γ and the optimal γ is easy to ﬁnd. Sensitivity to Noisy Data We note that robustness w.r.t. noisy data is not within our claim of technical contributions.

Revised: Nevertheless, CFLP is not more vulnerable than other link prediction baselines.

Revised: We conduct experiments with random attacks on the Cora dataset (randomly removing links and adding noisy links).

Revised: Table 10 shows the AUC performances of our proposed CFLP (w/ JKNet) compared to the strongest baseline methods under different levels of random attacks (0%, 2%, 5%, and 10%).

Revised: We can observe that as the attack strength goes up, the link prediction performance of all methods go down.

Revised: We also note that our proposed CFLP still outperforms the baseline methods.

Revised: Generalization to Graphs with Weighted Edges As our proposed CFLP uses GNN as the graph encoder and GNNs are usually able to take weighted graph as input (e.g., the adjacency matrix A for GCN can be weighted), the model should be able to handle weighted graphs as given.

Revised: Note the link prediction losses (Eqs.

Revised: (8) and (9)) need to be slightly modiﬁed considering the task.

Revised: When the task is to predict the link existence, the label adjacency matrix used in Eqs.

Revised: (8) and (9) must be of binary values.

Revised: When the task is to predict the link weights, the BCE loss functions (Eqs.

Revised: (8) and (9)) need to be changed to regression loss functions such as MSE.

Original: [Figure] 
Revised: [Figure] [Figure] [Figure] [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: SEAL 92.55 ± 0.

Revised: [Table]