Original: D UAL G RAPH C OMPLEMENTARY N ETWORK
Revised: D UAL G RAPH C OMPLEMENTARY N ETWORK

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: As a powerful representation learning method on graph data, graph neural networks (GNNs) have shown great popularity in tackling graph analytic problems.
Revised: As a powerful representation learning method on graph data, graph neural networks (GNNs) have shown great popularity in tackling graph analytic problems.

Original: Although many attempts have been made in literatures to ﬁnd strategies about extracting better embedding of the target nodes, few of them consider this issue from a comprehensive perspective.
Revised: Although many attempts have been made in literatures to ﬁnd strategies about extracting better embedding of the target nodes, few of them consider this issue from a comprehensive perspective.

Original: Most of current GNNs usually employ some single method which can commendably extract a certain kind of feature but some equally important features are often ignored.
Revised: Most of current GNNs usually employ some single method which can commendably extract a certain kind of feature but some equally important features are often ignored.

Original: In this paper, we develop a novel dual graph complementary network (DGCN) to learn representation complementarily.
Revised: In this paper, we develop a novel dual graph complementary network (DGCN) to learn representation complementarily.

Original: We use two different branches, and inputs of the two branches are the same, which are composed of structure and feature information.
Revised: We use two different branches, and inputs of the two branches are the same, which are composed of structure and feature information.

Original: At the same time, there is also a complementary relationship between the two branches.
Revised: At the same time, there is also a complementary relationship between the two branches.

Original: Beyond that, our extensive experiments show that DGCN outperforms state-of-the-art methods on ﬁve public benchmark datasets.
Revised: Beyond that, our extensive experiments show that DGCN outperforms state-of-the-art methods on ﬁve public benchmark datasets.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Although many attempts have been made in literatures to ﬁnd a better strategy to learn the target node representation, the feature extraction capabilities of most methods are still far from optimal, especially when only a small amount of data is labeled.
Revised: Although many attempts have been made in literatures to ﬁnd a better strategy to learn the target node representation, the feature extraction capabilities of most methods are still far from optimal, especially when only a small amount of data is labeled.

Original: However, in fact, compared with the expensive and laborious acquisition of labeled data, unlabeled data is much easier to obtain.
Revised: However, in fact, compared with the expensive and laborious acquisition of labeled data, unlabeled data is much easier to obtain.

Original: Therefore, how to learn more useful representations with limited label information is the key direct of representation learning study.
Revised: Therefore, how to learn more useful representations with limited label information is the key direct of representation learning study.

Original: Methods of this issue, commonly referred to as semi-supervised learning, which essentially believe that the similar points have similar outputs.
Revised: Methods of this issue, commonly referred to as semi-supervised learning, which essentially believe that the similar points have similar outputs.

Original: Thus, it can properly utilize the consistency of data to make full use of the rich information of unsupervised data.
Revised: Thus, it can properly utilize the consistency of data to make full use of the rich information of unsupervised data.

Original: In the real world, it is common that we have data with speciﬁc topological structures which usually called graph data.
Revised: In the real world, it is common that we have data with speciﬁc topological structures which usually called graph data.

Original: The graph structure is usually expressed as the connection between nodes.
Revised: The graph structure is usually expressed as the connection between nodes.

Original: By aggregating the features of neighborhood and performing appropriate linear transformation, graph neural networks (GNNs) can convert graph data into a low-dimensional, compact, and continuous feature space.
Revised: By aggregating the features of neighborhood and performing appropriate linear transformation, graph neural networks (GNNs) can convert graph data into a low-dimensional, compact, and continuous feature space.

Original: Nevertheless, most of them only care about a single aggregation strategy, which is counter intuitive: for example, as far as social networks are concerned, the relationship between people is very complex, while, most of the traditional GNNs only consider the single connection between nodes and ignore other implicit information.
Revised: Nevertheless, most of them only care about a single aggregation strategy, which is counter intuitive: for example, as far as social networks are concerned, the relationship between people is very complex, while, most of the traditional GNNs only consider the single connection between nodes and ignore other implicit information.

Original: In this paper, our work focuses on learning node representations by GNNs in a semi-supervised way.
Revised: In this paper, our work focuses on learning node representations by GNNs in a semi-supervised way.

Original: Despite there are already many graph-based semi-supervised learning methods (Kipf & Welling, 2016;
Revised: Despite there are already many graph-based semi-supervised learning methods (Kipf & Welling, 2016;

Original: Yang et al., 2016; Khan & Blumenstock, 2019), most of them can only ﬁnd a single relationship between nodes.
Revised: Yang et al., 2016; Khan & Blumenstock, 2019), most of them can only ﬁnd a single relationship between nodes.

Original: As a result, some information in unsupervised data is usually ignored.
Revised: As a result, some information in unsupervised data is usually ignored.

Original: To overcome this problem, we develop a novel dual graph complementary network (DGCN) to extract information from both feature and topology spaces.
Revised: To overcome this problem, we develop a novel dual graph complementary network (DGCN) to extract information from both feature and topology spaces.

Original: An intuition of our method is to learn based on disagreement: network performance is largely related to the quality of the graph, which usually emphasizes the relevance of an attribute of instances.
Revised: An intuition of our method is to learn based on disagreement: network performance is largely related to the quality of the graph, which usually emphasizes the relevance of an attribute of instances.

Original: So, since we don’t know what attributes are most important, we consider both of them in the model design.
Revised: So, since we don’t know what attributes are most important, we consider both of them in the model design.

Original: Compared with the traditional GNN-based methods, we perform two different aggregate strategies which emphasize different attributes in each branch, one from the perspective of node feature, and the other from the topological structure.
Revised: Compared with the traditional GNN-based methods, we perform two different aggregate strategies which emphasize different attributes in each branch, one from the perspective of node feature, and the other from the topological structure.

Original: Then, to further utilize implicit information, we employ two networks with different structures to extract embedding from input feature.
Revised: Then, to further utilize implicit information, we employ two networks with different structures to extract embedding from input feature.

Original: By doing so, nodes’ information can be propagated in different ways.
Revised: By doing so, nodes’ information can be propagated in different ways.

Original: Then, the supervised loss (cid:96) sup and diversity constraint (cid:96) div are used to guide the training.
Revised: Then, the supervised loss ℓ sup and diversity constraint ℓ div are used to guide the training.

Original: We use two different branches to extract common information in topology and feature spaces.
Revised: We use two different branches to extract common information in topology and feature spaces.

Original: By utilizing disagreements between the two branches, model can gain information that may be ignored by single branch.
Revised: By utilizing disagreements between the two branches, model can gain information that may be ignored by single branch.

Original: To prove the effectiveness of our method, we conducted experiments on ﬁve public benchmark datasets.
Revised: To prove the effectiveness of our method, we conducted experiments on ﬁve public benchmark datasets.

Original: The contributions of our work are summarized as follows:
Revised: The contributions of our work are summarized as follows:

Original: • We propose a novel dual graph complementary network (DGCN) to fuse complementary information, which utilizes different graphs to aggregate nodes that are similar in certain attributes in a complementary way.
Revised: • We propose a novel dual graph complementary network (DGCN) to fuse complementary information, which utilizes different graphs to aggregate nodes that are similar in certain attributes in a complementary way.

Original: • By comparing with algorithms that use non-single graphs, it proves that our complementary architecture can extract richer information • Through extensive evaluation on multiple datasets, we demonstrate DGCN effectiveness over state-of-the-art baselines.
Revised: • By comparing with algorithms that use non-single graphs, it proves that our complementary architecture can extract richer information • Through extensive evaluation on multiple datasets, we demonstrate DGCN effectiveness over state-of-the-art baselines.

Original:  R ELATED W ORK 2. S EMI - SUPERVISED LEARNING
Revised:  R ELATED W ORK 2. S EMI - SUPERVISED LEARNING

Original: Semi-supervised learning is usually aimed at the case of insufﬁcient data labels.
Revised: Semi-supervised learning is usually aimed at the case of insufﬁcient data labels.

Original: X ∈ R n × d is the feature of input nodes.
Revised: X ∈ R n × d is the feature of input nodes.

Original: Y =
Revised: Y =

Original: [ y ij ] ∈ R n × k is the label matrix, where k is the class number.
Revised: [ y ij ] ∈ R n × k is the label matrix, where k is the class number.

Original: y ij means that the i -th node belongs to the j -th class.
Revised: y ij means that the i -th node belongs to the j -th class.

Original: Then split data points into labeled and unlabeled points.
Revised: Then split data points into labeled and unlabeled points.

Original: Accordingly, x L and x U express a feature of labeled and unlabeled instance, respectively.
Revised: Accordingly, x L and x U express a feature of labeled and unlabeled instance, respectively.

Original: Moreover, the ground-truth label of the label nodes is available only.
Revised: Moreover, the ground-truth label of the label nodes is available only.

Original: The main objective of semi-supervised learning is to extract supervised information from labeled dataset whilst adequately utilizing data distribution information contained in X .
Revised: The main objective of semi-supervised learning is to extract supervised information from labeled dataset whilst adequately utilizing data distribution information contained in X .

Original: There are four categories of semi-supervised learning algorithms:
Revised: There are four categories of semi-supervised learning algorithms:

Original: Self-training semi-supervised learning (Lee, 2013):
Revised: Self-training semi-supervised learning (Lee, 2013):

Original: It utilizes high-conﬁdence pseudo labels to expand label set.
Revised: It utilizes high-conﬁdence pseudo labels to expand label set.

Original: Ideally, it can continuously improve network performance, but is usually limited by the quality of pseudo labels.
Revised: Ideally, it can continuously improve network performance, but is usually limited by the quality of pseudo labels.

Original: Graph-based semi-supervised learning
Revised: Graph-based semi-supervised learning

Original: : It propagates information between instances according to edges in graph.
Revised: : It propagates information between instances according to edges in graph.

Original: It’s an inductive learning method, of which the performance mainly depends on the aggregation algorithm.
Revised: It’s an inductive learning method, of which the performance mainly depends on the aggregation algorithm.

Original: Low-density separation methods (Joachims, 1999):
Revised: Low-density separation methods (Joachims, 1999):

Original: They assume that the decision hyperplane is consistent with the data distribution, so so it should pass through the sparse region of the data.
Revised: They assume that the decision hyperplane is consistent with the data distribution, so so it should pass through the sparse region of the data.

Original: Pretrain semi-supervised learning: such as autoencoder (Vincent et al., 2008; Rifai et al., 2011), trains the model based on reconstruction error and then ﬁne tune it using labeled data.
Revised: Pretrain semi-supervised learning: such as autoencoder (Vincent et al., 2008; Rifai et al., 2011), trains the model based on reconstruction error and then ﬁne tune it using labeled data.

Original: However, semi-supervised learning tasks prefer to obtain information related to data distribution rather than all information of samples.
Revised: However, semi-supervised learning tasks prefer to obtain information related to data distribution rather than all information of samples.

Original: In this paper, we mainly focus on the graph-based semisupervised learning.
Revised: In this paper, we mainly focus on the graph-based semisupervised learning

Original:  2. G RAPH - BASED S EMI -S UPERVISED L EARNING
Revised: . 2. G RAPH - BASED S EMI -S UPERVISED L EARNING

Original: In addition to features, graph-based semi-supervised learning methods (Kipf & Welling, 2016) represent the topological edge connection between different instances.
Revised: In addition to features, graph-based semi-supervised learning methods (Kipf & Welling, 2016) represent the topological edge connection between different instances.

Original: For many datasets, graph is given as a feature.
Revised: For many datasets, graph is given as a feature.

Original: If the features of the dataset do not contain the relationships between different samples, a graph can also be constructed by measuring the similarity between the features of the instances (Zhu et al., 2003).
Revised: If the features of the dataset do not contain the relationships between different samples, a graph can also be constructed by measuring the similarity between the features of the instances (Zhu et al., 2003).

Original: Actually, the graph is a measure of whether the instances are closely connected.
Revised: Actually, the graph is a measure of whether the instances are closely connected.

Original: Then, according to this graph, information exchange between instances can be carried out, so that the information of unlabeled data can be effectively utilized.
Revised: Then, according to this graph, information exchange between instances can be carried out, so that the information of unlabeled data can be effectively utilized.

Original: Network performance is largely related to the quality of the graph.
Revised: Network performance is largely related to the quality of the graph.

Original: When the attributes emphasized in the graph do not match the expectations of the task objective, misjudgments are often caused.
Revised: When the attributes emphasized in the graph do not match the expectations of the task objective, misjudgments are often caused.

Original: Usually, it is difﬁcult to ﬁnding what really matters.
Revised: Usually, it is difﬁcult to ﬁnding what really matters.

Original: The traditional graph-based semi-supervised learning methods usually uses a single graph for node aggregation, which causes a single attribute to be emphatically considered, but when this attribute does not match the task goal, it will mislead the training instead.
Revised: The traditional graph-based semi-supervised learning methods usually uses a single graph for node aggregation, which causes a single attribute to be emphatically considered, but when this attribute does not match the task goal, it will mislead the training instead.

Original:  DGCN ARCHITECTURE
Revised:  DGCN ARCHITECTURE

Original: In this section, we will present the overall framework of DGCN, see Fig.
Revised: In this section, we will present the overall framework of DGCN, see Fig.

Original: The main idea of DGCN is that information exchange under the control of graphs emphasizing different attributes can extract more abundant features.
Revised: The main idea of DGCN is that information exchange under the control of graphs emphasizing different attributes can extract more abundant features.

Original: To this end, we use two branches to extract information from two inputs at the same time.
Revised: To this end, we use two branches to extract information from two inputs at the same time.

Original: The node features of these two inputs are the same, the only difference is the graphs that control the information exchange.
Revised: The node features of these two inputs are the same, the only difference is the graphs that control the information exchange.

Original: In addition, in order to further expand the difference between branches, we use a diversity loss (cid:96) div .
Revised: In addition, in order to further expand the difference between branches, we use a diversity loss ℓ div .

Original: 3. N OTATION & PROBLEM STATEMENT
Revised: 3. N OTATION & PROBLEM STATEMENT

Original: Let G =
Revised: Let G =

Original: ( V , A, X ) be an undirected graph.
Revised: ( V , A, X ) be an undirected graph.

Original: V is the set of nodes on the graph, which is composed of unlabeled ( V u ) and labeled ( V l ) nodes with the number of nodes is n u and n l respectively.
Revised: V is the set of nodes on the graph, which is composed of unlabeled ( V u ) and labeled ( V l ) nodes with the number of nodes is n u and n l respectively.

Original: n = n l + n u is the number of nodes.
Revised: n = n l + n u is the number of nodes.

Original: A =
Revised: A =

Original: [ a ij ] ∈ R n × n is the adjacency matrix.
Revised: [ a ij ] ∈ R n × n is the adjacency matrix.

Original: a ij = 1 represents that node i and node j are closely related in an attribute, otherwise, a ij = 0 .
Revised: a ij = 1 represents that node i and node j are closely related in an attribute, otherwise, a ij = 0 .

Original: 3. B RANCHES
Revised: 3. B RANCHES

Original: In order to capture different characteristics by the two branches (also called viewer), we use different network structures for each branch: GCN (Kipf & Welling, 2016) and GAT (Veliˇckovi´c et al., 2017).
Revised: In order to capture different characteristics by the two branches (also called viewer), we use different network structures for each branch: GCN (Kipf & Welling, 2016) and GAT (Veliˇckovi´c et al., 2017).

Original: Given a graph G =
Revised: Given a graph G =

Original: ( V , A, X ) , both GCN and GAT intend to extract richer features at a vertex by aggregating features of vertices from its neighborhood (Li et al., 2019).
Revised: ( V , A, X ) , both GCN and GAT intend to extract richer features at a vertex by aggregating features of vertices from its neighborhood (Li et al., 2019).

Original: So the node representation of the l -th layer H l can be deﬁned by:
Revised: So the node representation of the l -th layer H l can be deﬁned by:

Original: [Equation]
Revised: [Equation]

Original: where Θ agg l and Θ update l are the learnable weights of aggregation and update functions of the l -th layer respectively and the initial H 

Revised: where (cid:2) agg l and (cid:2) update l are the learnable weights of aggregation and update functions of the l -th layer respectively and the initial H

Original: = X .
Revised: = X .

Original: The aggregation and update functions are the essential components of GNNs, and obviously the features extracted by different aggregation functions will have certain differences.
Revised: The aggregation and update functions are the essential components of GNNs, and obviously the features extracted by different aggregation functions will have certain differences.

Original: Thus, we take advantage of two different networks, GCN and GAT, to obtain node representation.
Revised: Thus, we take advantage of two different networks, GCN and GAT, to obtain node representation.

Original: The node features output by the l -th GCN layer can be expressed as:
Revised: The node features output by the l -th GCN layer can be expressed as:

Original: [Equation]
Revised: [Equation]

Original: where I ∈ R n × n indicates the identity matrix,
Revised: where I ∈ R n × n indicates the identity matrix,

Original: A + I means adding self-loop in the graph, ˜ D is the diagonal degree matrix of A
Revised: A + I means adding self-loop in the graph, ~ D is the diagonal degree matrix of A

Original: + I , and σ ( · ) is the activation function.
Revised: + I , and σ ( · ) is the activation function.

Original: It can be seen from equation 2 that GCN aggregates neighbor features by weighting the value of symmetric normalized laplacian.
Revised: It can be seen from equation 2 that GCN aggregates neighbor features by weighting the value of symmetric normalized laplacian.

Original: Next, we introduce the algorithm GAT that uses the attention mechanism to calculate the neighbor weight.
Revised: Next, we introduce the algorithm GAT that uses the attention mechanism to calculate the neighbor weight.

Original: Through a learnable coefﬁcient a , GAT can assign learnable weights to each neighbor of the node.
Revised: Through a learnable coefﬁcient a , GAT can assign learnable weights to each neighbor of the node.

Original: For node i , the weight α ij between it and its neighbor node j can be expressed as:
Revised: For node i , the weight α ij between it and its neighbor node j can be expressed as:

Original: [Equation]
Revised: [Equation]

Original: where · (cid:62) is the transposition operation and (cid:107) represents concatenation.
Revised: where · ⊤ is the transposition operation and ∥ represents concatenation.

Original: Then the forward propagation process of node v in l -th layer can be represented as:
Revised: Then the forward propagation process of node v in l -th layer can be represented as:

Original: [Equation]
Revised: [Equation]

Original:  where, h l,i is the embeding of node i in the l -th layer.
Revised: (4) where, h l;i is the embeding of node i in the l -th layer.

Original: M is the number of independent attention mechanisms.
Revised: M is the number of independent attention mechanisms.

Original: σ is activation function of GAT.
Revised: σ is activation function of GAT.

Original: α mij is the normalized attention coefﬁcients computed by the m -th attention mechanism , see equation 3.
Revised: α mij is the normalized attention coefﬁcients computed by the m -th attention mechanism , see equation 3.

Original: As can be seen from equation 4, the weights
Revised: As can be seen from equation 4, the weights

Original: GAT assigns to a node’s neighbors are learnable.
Revised: GAT assigns to a node’s neighbors are learnable.

Original: Thus we can assign adaptive weights to different neighbors.
Revised: Thus we can assign adaptive weights to different neighbors.

Original: Although these two methods are based on the existence of connection between points as the premise of aggregation.
Revised: Although these two methods are based on the existence of connection between points as the premise of aggregation.

Original: Both the GCN and GAT models we use have their own advantages and disadvantages.
Revised: Both the GCN and GAT models we use have their own advantages and disadvantages.

Original: The former considers the relationship between nodes (probability conduction matrix), but can’t learn neighbor weights dynamically.
Revised: The former considers the relationship between nodes (probability conduction matrix), but can’t learn neighbor weights dynamically.

Original: Although the latter can assign dynamic weights to neighbors, it ignores the inherent structure of the graph space.
Revised: Although the latter can assign dynamic weights to neighbors, it ignores the inﬂuence of degree attribute of node on aggregation.

Original: Therefore, using these two branches, we can extract more complementary features from the input.
Revised: Therefore, using these two branches, we can extract more complementary features from the input.

Original: 3. F ORWARD PROPAGATION
Revised: 3. F ORWARD PROPAGATION

Original: In this subsection, we introduce the input used by the network and the speciﬁc forward propagation strategy.
Revised: In this subsection, we introduce the input used by the network and the speciﬁc forward propagation strategy.

Original: In order to consider different attributes when aggregating, we use different graphs for training, but adopt the same features.
Revised: In order to consider different attributes when aggregating, we use different graphs for training, but adopt the same features.

Original: In this paper, the datasets used in our experiment are graphstructured which have two characteristics, one is the feature of the instance itself, which is not affected by other instances, and the other reﬂects the relationship with other instances.
Revised: In this paper, the datasets used in our experiment are graphstructured which have two characteristics, one is the feature of the instance itself, which is not affected by other instances, and the other reﬂects the relationship with other instances.

Original: For example, dataset ACM (Wang et al., 2019) which extracted from ACM dataset containspapers.
Revised: For example, dataset ACM (Wang et al., 2019) which extracted from ACM dataset containspapers.

Original: It has two properties: one is a bag-of-words that indicates whether the keyword exists, and the other indicates which papers are written by the same author.
Revised: It has two properties: one is a bag-of-words that indicates whether the keyword exists, and the other indicates which papers are written by the same author.

Original: Obviously, if we only base whether the paper is written by the same author as the basis for aggregation, we will inevitably ignore the situation where the same author has written different types of papers and the same type of papers belong to different authors, thus mistakenly aggregate articles of different categories together.
Revised: Obviously, if we only base whether the paper is written by the same author as the basis for aggregation, we will inevitably ignore the situation where the same author has written different types of papers and the same type of papers belong to different authors, thus mistakenly aggregate articles of different categories together.

Original: Therefore, we also construct a graph based on another attribute of the dataset: bag-of-words, so that information can be transferred between instances with similar keywords.
Revised: Therefore, we also construct a graph based on another attribute of the dataset: bag-of-words, so that information can be transferred between instances with similar keywords.

Original: In order to measure the similarity of instances’ features, we ﬁnd the cosine similarity between the features of all instances:
Revised: In order to measure the similarity of instances’ features, we ﬁnd the cosine similarity between the features of all instances:

Original: [Equation]
Revised: [Equation]

Original: where s ij denotes the cosine similarity between the feature x i of node i and the feature x j of node j ∈ V .
Revised: where s ij denotes the cosine similarity between the feature x i of node i and the feature x j of node j ∈ V .

Original: Notice that j (cid:54) = i .
Revised: Notice that j ̸ = i .

Original: For node i , we choose the t largest s ij and let the corresponding j as the neighbors of i .
Revised: For node i , we choose the t largest s ij and let the corresponding j as the neighbors of i .

Original: Then if j is the neighbor of node i , obviously i is the neighbor of node j too.
Revised: Then if j is the neighbor of node i , obviously i is the neighbor of node j too.

Original: As above-mentioned, we can get a new graph constructed from features.
Revised: As above-mentioned, we can get a new graph constructed from features.

Original: We use A 1 and A 2 to represent the inherent graph structure of the data and the graph constructed according to the feature, respectively.
Revised: We use A 1 and A 2 to represent the inherent graph structure of the data and the graph constructed according to the feature, respectively.

Original: Therefor, by inputting A 1 and A 2 for each branch, we can get four different outputs.
Revised: Therefor, by inputting A 1 and A 2 for each branch, we can get four different outputs.

Original: according to equation 2 and equation 4 the forward propagation of CDAN can be represented as:[Equation] [Equation]
Revised: according to equation 2 and equation 4 the forward propagation of DGCN can be represented as:

Original: [Equation]
Revised: [Equation]

Original: where, v = 1 represents that the graph is A 1 , while v = 2 corresponding to A 2 .
Revised: where, v = 1 represents that the graph is A 1 , while v = 2 corresponding to A 2 .

Original: σ and σ are the activation function.
Revised: σ and σ are the activation function.

Original: α kv,i,j is the normalized attention coefﬁcients.
Revised: α kv;i;j is the normalized attention coefﬁcients.

Original: Θ v,l and W kv,l are the weights of linear transformations.
Revised: (cid:2) v;l and W kv;l are the weights of linear transformations.

Original: For the GAT branch, h gat n,l,i means the representation of node i in the l -th layer with the input graph is A v .
Revised: For the GAT branch, h gat n;l;i means the representation of node i in the l -th layer with the input graph is A v .

Original: Similarly, H gcn v,l corresponds to the l -layer embedding matrix of the GCN branch when the input graph is A v .
Revised: Similarly, H gcn v;l corresponds to the l -layer embedding matrix of the GCN branch when the input graph is A v .

Original: For these four embeddings, we ﬁrst use the attention mechanism to combine the embeddings generated by different graphs of the same branch: 
Revised: For these four embeddings, we ﬁrst use the attention mechanism to combine the embeddings generated by different graphs of the same branch: [Equation] [Equation]

Original: [Equation]
Revised: [Equation]

Original: Then, we apply the attention mechanism again to combine H gcn1 ,l , H gcn2 ,l , H gat1 ,l , H gat2 ,l , H gcn c and H gat c .
Revised: Then, we apply the attention mechanism again to combine H gcn1 ;l , H gcn2 ;l , H gat1 ;l , H gat2 ;l , H gcn c and H gat c .

Original: Through these two attention operations, we can dynamically assign weights to different embedding to ﬁnd attributes that better match the task goal.
Revised: Through these two attention operations, we can dynamically assign weights to different embedding to ﬁnd attributes that better match the task goal.

Original: 3. L OSS F UNCTIONS OF DGCN
Revised: 3. L OSS F UNCTIONS OF DGCN

Original: The objective function of DGCN consists of two parts: the supervised loss (cid:96) sup and the diversity loss (cid:96) div .
Revised: The objective function of DGCN consists of two parts: the supervised loss ℓ sup and the diversity loss ℓ div .

Original: 3.4. S UPERVISED LOSS
Revised: 3.4. S UPERVISED LOSS

Original: Given a graph G =
Revised: Given a graph G =

Original: ( V , A, X ) , as V =
Revised: ( V , A, X ) , as V =

Original: V l ∪V u , the corresponding label of V l is Y l .
Revised: V l ∪V u , the corresponding label of V l is Y l .

Original: In order to utilize the supervision information, we use the cross-entropy loss function to guide the training:
Revised: In order to utilize the supervision information, we use the cross-entropy loss function to guide the training:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where y ij is the ground-truth label of node i and p ij is the model predicted label.
Revised: where y ij is the ground-truth label of node i and p ij is the model predicted label.

Original: k is the number of classes.
Revised: k is the number of classes.

Original: 3.4. D IVERSITY LOSS
Revised: 3.4. D IVERSITY LOSS

Original: In order to further expand the differences between branches and capture richer node features, we use L div to add a diversity constraint on H gcn c and H gat c .
Revised: In order to further expand the differences between branches and capture richer node features, we use L div to add a diversity constraint on H gcn c and H gat c .

Original: First, we use L 2 -normalization to normalize H gcnc and H gatc output by the attention module.
Revised: First, we use L 2 -normalization to normalize H gcnc and H gatc output by the attention module.

Original: The normalized results are ˆ H rmgcn and ˆ H gat respectively.
Revised: The normalized results are ^ H gcn and ^ H gat respectively.

Original: Using the above results, we can capture the similarity of node embedding:
Revised: Using the above results, we can capture the similarity of node embedding:

Original: [Equation]
Revised: [Equation]

Original: Then, the diversity loss can be deﬁned by:
Revised: Then, the diversity loss can be deﬁned by:

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original:  where n is the number of nodes.
Revised: 12) where n is the number of nodes.

Original: Through this diversity constraint, we can expand the difference between the branches to learn complementary features.
Revised: Through this diversity constraint, we can expand the difference between the branches to learn complementary features.

Original: Therefore, our ﬁnal optimization object can be expressed as:
Revised: Therefore, our ﬁnal optimization object can be expressed as:

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original:  where γ is parameter of the disparity constraint terms.
Revised: 13) where γ is parameter of the disparity constraint terms.

Original: Using this objective function, we can optimize the proposed model through back propagation and learn the node embedding for classiﬁcation.
Revised: Using this objective function, we can optimize the proposed model through back propagation and learn the node embedding for classiﬁcation.

Original: [Table]
Revised: [Table]

Original:  E XPERIMENT 4. D ATASETS
Revised:  E XPERIMENT 4. D ATASETS

Original: For evaluating the effectiveness of DGCN, we evaluate on several semi-supervised classiﬁcation benchmarks.
Revised: For evaluating the effectiveness of DGCN, we evaluate on several semi-supervised classiﬁcation benchmarks.

Original: Following the experimental setup of Wang et al.
Revised: Following the experimental setup of Wang et al.

Original: (2020), we evaluate on ﬁve datasets.
Revised: (2020), we evaluate on ﬁve datasets.

Original: • ACM (Wang et al., 2019):
Revised: • ACM (Wang et al., 2019):

Original: This dataset is extracted from the ACM dataset, where the nodes represent the papers, the edges represent that the connected two papers belong to the same author, and the feature is the word bag representation of paper’s keywords.
Revised: This dataset is extracted from the ACM dataset, where the nodes represent the papers, the edges represent that the connected two papers belong to the same author, and the feature is the word bag representation of paper’s keywords.

Original: • UAI

Revised: • UAI

Original: (Wang et al., 2018):
Revised: (Wang et al., 2018):

Original: This dataset has 3067 nodes and 19 classes.
Revised: This dataset has 3067 nodes and 19 classes.

Original: • Citeseer (Kipf & Welling, 2016):
Revised: • Citeseer (Kipf & Welling, 2016):

Original: In the CiteSeer dataset, papers are divided into six categories, containing a total of 3312 papers, which record citation information between papers.
Revised: In the CiteSeer dataset, papers are divided into six categories, containing a total of 3312 papers, which record citation information between papers.

Original: And the feature is the word bag representation of the paper.
Revised: And the feature is the word bag representation of the paper.

Original: • BlogCatalog (Meng et al., 2019):
Revised: • BlogCatalog (Meng et al., 2019):

Original: This is a network of social relationships from the BlogCatalog websitewhere the nodes are bloggers and edges are their social relationships.
Revised: This is a network of social relationships from the BlogCatalog websitewhere the nodes are bloggers and edges are their social relationships.

Original: Node attributes are the short descriptions of users’ blogs provided by users.
Revised: Node attributes are the short descriptions of users’ blogs provided by users.

Original: The labels represent the topic categories provided by the authors which can be divided into 6 classes.
Revised: The labels represent the topic categories provided by the authors which can be divided into 6 classes.

Original: • Flickr (Meng et al., 2019):
Revised: • Flickr (Meng et al., 2019):

Original: This network is built from proﬁle and relation data of users in Flickr.
Revised: This network is built from proﬁle and relation data of users in Flickr.

Original: We treat each user as a node, relationships between two user as an edge.
Revised: We treat each user as a node, relationships between two user as an edge.

Original: The labels represent the interest groups of the users.
Revised: The labels represent the interest groups of the users.

Original: The detailed descriptions of the datasets used here are shown in Table 1.
Revised: The detailed descriptions of the datasets used here are shown in Table 1.

Original: [Table]
Revised: [Table]

Original: AM-GCN 70.
Revised: AM-GCN 70.

Original: CDAN 72.
Revised: DGCN 72.

Original: Citeseer
Revised: Citeseer

Original: DeepWalk 43.
Revised: DeepWalk 43.

Original: LINE 32.
Revised: LINE 32.

Original: GCN 70.
Revised: GCN 70.

Original: kNN-GCN 61.
Revised: kNN-GCN 61.

Original: GAT 72.
Revised: GAT 72.

Original: DEMO-Net 69.
Revised: DEMO-Net 69.

Original: MixHop 71.
Revised: MixHop 71.

Original: AM-GCN 73.
Revised: AM-GCN 73.

Original: CDAN 74.
Revised: DGCN 74.

Original: BlogCatalog
Revised: BlogCatalog

Original: DeepWalk 38.
Revised: DeepWalk 38.

Original: LINE 58.
Revised: LINE 58.

Original: GCN 69.
Revised: GCN 69.

Original: kNN-GCN 75.
Revised: kNN-GCN 75.

Original: GAT 64.
Revised: GAT 64.

Original: DEMO-Net 54.
Revised: DEMO-Net 54.

Original: MixHop 65.
Revised: MixHop 65.

Original: AM-GCN 81.
Revised: AM-GCN 81.

Original: CDAN 88.
Revised: DGCN 88.

Original: Flickr
Revised: Flickr

Original: DeepWalk 24.
Revised: DeepWalk 24.

Original: LINE 33.
Revised: LINE 33.

Original: GCN 41.
Revised: GCN 41.

Original: kNN-GCN 69.
Revised: kNN-GCN 69.

Original: GAT 38.
Revised: GAT 38.

Original: DEMO-Net 34.
Revised: DEMO-Net 34.

Original: MixHop 39.
Revised: MixHop 39.

Original: AM-GCN 75.
Revised: AM-GCN 75.

Original: CDAN 74.
Revised: DGCN 74.

Original: 4. B ASELINES
Revised: 4. B ASELINES

Original: We compare with some state-of-art baselines to verﬁfy the effectiveness of the proposed DGCN.
Revised: We compare with some state-of-art baselines to verﬁfy the effectiveness of the proposed DGCN.

Original: • DeepWalk (Perozzi et al., 2014) is a random walk based network embedding method, learning feature by treating truncated random walks in a graph as the equivalent of sentences.
Revised: • DeepWalk (Perozzi et al., 2014) is a random walk based network embedding method, learning feature by treating truncated random walks in a graph as the equivalent of sentences.

Original: • LINE
Revised: • LINE

Original: (Tang et al., 2015) is a large-scale embedding method that retains both the local network structure and the global network structure.
Revised: (Tang et al., 2015) is a large-scale embedding method that retains both the local network structure and the global network structure.

Original: • GCN (Kipf & Welling, 2016) is a variant of convolutional neural networks which aggregates information of nodes to get node characteristics.
Revised: • GCN (Kipf & Welling, 2016) is a variant of convolutional neural networks which aggregates information of nodes to get node characteristics.

Original: • kNN-GCN.
Revised: • kNN-GCN.

Original: The network structure of kNN-GCN is the same as that of GCN.
Revised: The network structure of kNN-GCN is the same as that of GCN.

Original: But the graph we use here is the aforementioned A 2 , see Section 3.3.
Revised: But the graph we use here is the aforementioned A 2 , see Section 3.3.

Original: • GAT (Veliˇckovi´c et al., 2017) is a graph attention based method which can assign different weights to nodes during aggregation.
Revised: • GAT (Veliˇckovi´c et al., 2017) is a graph attention based method which can assign different weights to nodes during aggregation.

Original: • DEMO-Net
Revised: • DEMO-Net

Original: (Wu et al., 2019) assumes that nodes with the same degree value will share the same graph convolution, and the feature aggregation is expressed as a multi-task learning problem according to the degree value of the node.
Revised: (Wu et al., 2019) assumes that nodes with the same degree value will share the same graph convolution, and the feature aggregation is expressed as a multi-task learning problem according to the degree value of the node.

Original: • MixHop (Abu-El-Haija et al., 2019) can learn the neighbor mixture relationship by repeatedly mixing the feature representations of neighbors at various distances.
Revised: • MixHop (Abu-El-Haija et al., 2019) can learn the neighbor mixture relationship by repeatedly mixing the feature representations of neighbors at various distances.

Original: • AM-GCN (Wang et al., 2020) extracts embeddings from node features, topological structures and their combinations, and uses the attention mechanism to learn the adaptive importance weights of embeddings
Revised: • AM-GCN (Wang et al., 2020) extracts embeddings from node features, topological structures and their combinations, and uses the attention mechanism to learn the adaptive importance weights of embeddings

Original: 4. R ESULTS
Revised: 4. R ESULTS

Original: We train the DGCN network described in Section 3 on ﬁve public datasets and evaluate the prediction accuracy on a test set of 1,000 labeled examples, and experiments on all datasets are optimized using
Revised: We train the DGCN network described in Section 3 on ﬁve public datasets and evaluate the prediction accuracy on a test set of 1,000 labeled examples, and experiments on all datasets are optimized using

Original: Adam optimizer.
Revised: Adam optimizer.

Original: The model adopts GCN and GAT branches with a layer number of 2, and the quantitative analysis results can be seen in Table 2.
Revised: The model adopts GCN and GAT branches with a layer number of 2, and the quantitative analysis results can be seen in Table 2.

Original: • It can be seen from the Table 2 that DGCN can exceed the baseline in most of the accuracy rates of all datasets, which proves the effectiveness of our method.
Revised: • It can be seen from the Table 2 that DGCN can exceed the baseline in most of the accuracy rates of all datasets, which proves the effectiveness of our method.

Original: • On most datasets, the performance of DGCN is better than AM-GCN using two graphs and GCN, kNN-GCN, GAT using one graph, which fully proves that DGCN can capture more information that meets the task objectives.
Revised: On most datasets, the performance of DGCN is better than AM-GCN using two graphs and GCN, kNNGCN, GAT using one graph, which fully proves that DGCN can capture more information that meets the task objectives.

Original: • In addition, by comparing with AM-GCN, which also uses different graphs for learning, our DGCN can learn better node embeddings through its complementary learning mechanism.
Revised: In addition, by comparing with AM-GCN, which also uses different graphs for learning, our DGCN can learn better node embeddings through its complementary learning mechanism.

Revised: • The main difference between GCN and KNN-GCN is that the structure graph and cosine graph are used respectively.

Revised: For a dataset, a graph that is more relevant to its classiﬁcation goal(Nodes connected by edges are more likely to belong to the same class) will perform better.

Revised: As we can see, on UAI2010, BlogCatelog and Flickr kNN-GCN seem to be significantly better than GCN, but the other two datasets are opposite.

Revised: This means that on the UAI2010, BlogCatelog and Flickr cosine graph is closer to the classiﬁcation target than structure graph.

Original:  CONCLUSION
Revised:  CONCLUSION

Original: In this paper, aiming at the problem of semi-supervised graph node classiﬁcation, we proposes a novel dual graph complementary network (DGCN), which can utilize graphs that emphasize the different attributes of the input to guide the aggregation process.
Revised: In this paper, aiming at the problem of semi-supervised graph node classiﬁcation, we proposes a novel dual graph complementary network (DGCN), which can utilize graphs that emphasize the different attributes of the input to guide the aggregation process.

Original: In addition, in order to further capture richer information, we use two different branches to perform feature learning separately.
Revised: In addition, in order to further capture richer information, we use two different branches to perform feature learning separately.

Original: At the same time, the disparity constraint is used between the two branches to further expand the difference.
Revised: At the same time, the disparity constraint is used between the two branches to further expand the difference.

Original: However, just using the diversity loss may retain too much unnecessary redundant information, which will interfere with the really important information.
Revised: However, just using the diversity loss may retain too much unnecessary redundant information, which will interfere with the really important information.

Original: Therefore, our future work will try to emphasize the common attributes in the embedding while expanding the differences between branches.
Revised: Therefore, our future work will try to emphasize the common attributes in the embedding while expanding the differences between branches.

Original: The extensive experiments on several datasets further demonstrate the effectiveness of our
Revised: The extensive experiments on several datasets further demonstrate the effectiveness of our

Original: DGCN algorithm.
Revised: DGCN algorithm.

Original: In the future, we can use contrast learning to do unsupervised learning with DGCN architecture.
Revised: In the future we will further study the correlation measurement of graphs and training objectives and further enrich our model with them.