Original: G RAPH I NFERENCE L EARNING FOR SEMI - SUPERVISED C LASSIFICATION
Revised: G RAPH I NFERENCE L EARNING FOR SEMI - SUPERVISED C LASSIFICATION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: In this work, we address the semi-supervised classiﬁcation of graph data, where the categories of those unlabeled nodes are inferred from labeled nodes as well as graph structures.
Revised: In this work, we address the semi-supervised classiﬁcation of graph data, where the categories of those unlabeled nodes are inferred from labeled nodes as well as graph structures.

Original: Recent works often solve this problem with the advanced graph convolution in a conventional supervised manner, but the performance could be heavily affected when labeled data is scarce.
Revised: Recent works often solve this problem with the advanced graph convolution in a conventional supervised manner, but the performance could be heavily affected when labeled data is scarce.

Original: Here we propose a Graph Inference
Revised: Here we propose a Graph Inference

Original: Learning (GIL) framework to boost the performance of node classiﬁcation by learning the inference of node labels on graph topology.
Revised: Learning (GIL) framework to boost the performance of node classiﬁcation by learning the inference of node labels on graph topology.

Original: To bridge the connection of two nodes, we formally deﬁne a structure relation by encapsulating node attributes, between-node paths and local topological structures together, which can make inference conveniently deduced from one node to another node.
Revised: To bridge the connection of two nodes, we formally deﬁne a structure relation by encapsulating node attributes, between-node paths and local topological structures together, which can make inference conveniently deduced from one node to another node.

Original: For learning the inference process, we further introduce meta-optimization on structure relations from training nodes to validation nodes, such that the learnt graph inference capability can be better self-adapted into test nodes.
Revised: For learning the inference process, we further introduce meta-optimization on structure relations from training nodes to validation nodes, such that the learnt graph inference capability can be better self-adapted into test nodes.

Original: Comprehensive evaluations on four benchmark datasets (including Cora, Citeseer, Pubmed and NELL) demonstrate the superiority of our GIL when compared with other state-of-the-art methods in the semi-supervised node classiﬁcation task.
Revised: Comprehensive evaluations on four benchmark datasets (including Cora, Citeseer, Pubmed and NELL) demonstrate the superiority of our GIL when compared with other state-of-the-art methods in the semi-supervised node classiﬁcation task.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Graph, which comprises a set of vertices/nodes together with connected edges, is an efﬁcient structured representation of non-regular data.
Revised: Graph, which comprises a set of vertices/nodes together with connected edges, is an efﬁcient structured representation of non-regular data.

Original: Due to the strong representation ability, it advocates many potential applications, e.g., social network Orsini et al. (2017), world wide web data Page et al. (1999), knowledge graph Xu et al. and protein-interaction network Borgwardt et al.
Revised: Due to the strong representation ability, it advocates many potential applications, e.g., social network (Orsini et al., 2017), world wide web data (Page et al., 1999), knowledge graph (Xu et al., 2017) and protein-interaction network (Borgwardt et al., 2007).

Original: Among these, semi-supervised node classiﬁcation on graph is one of the most interesting also popular topics.
Revised: Among these, semi-supervised node classiﬁcation on graph is one of the most interesting also popular topics.

Original: Given a graph in which some nodes are labeled, the aim of the semi-supervised classiﬁcation is to estimate the categories of those remaining unlabeled nodes by using various priors of graph.
Revised: Given a graph in which some nodes are labeled, the aim of the semi-supervised classiﬁcation is to estimate the categories of those remaining unlabeled nodes by using various priors of graph.

Original: While there have been numerous previous works Brandes et al. ; Zhou et al. ; Zhu et al. ; Yang et al.
Revised: While there have been numerous previous works (Brandes et al., 2008; Zhou et al., 2004; Zhu et al., 2003; Yang et al., 2016) devoted to semi-supervised node classiﬁcation based on explicit graph

Original: (2016) devoted to semi-supervised node classiﬁcation based on explicit graph Laplacian regularization, it is hard to efﬁciently boost the performance of label prediction due to the strict assumption that connected nodes are likely to share the same label information in the graph data.
Revised: Laplacian regularization, it is hard to efﬁciently boost the performance of label prediction due to the strict assumption that connected nodes are likely to share the same label information in the graph data.

Original: With the progress of deep learning in grid-shaped images/videos He et al.

Original: (2016), a few of graph convolutional neural networks (CNN) based methods, including spectral Kipf & Welling (2017) and spatial methods Niepert et al. ; Pan et al. Yu et al. (2018), are proposed to learn local convolution ﬁlters on graph in order to extract more discriminative node representation.
Revised: With the progress of deep learning in grid-shaped images/videos (He et al., 2016), a few of graph convolutional neural networks (CNN) based methods, including spectral (Kipf & Welling, 2017) and spatial methods (Niepert et al., 2016; Pan et al., 2018; Yu et al., 2018), are proposed to learn local convolution ﬁlters on graph in order to extract more discriminative node representation.

Original: Although graph CNN based methods have achieved considerable capabilities of graph embedding by optimizing ﬁlters, they are limited in the conventional semi-supervised framework and lack of an efﬁcient inference mechanism on graph.
Revised: Although graph CNN based methods have achieved considerable capabilities of graph embedding by optimizing ﬁlters, they are limited in the conventional semi-supervised framework and lack of an efﬁcient inference mechanism on graph.

Original: Especially, in the case of few-shot learning, where a small number of training nodes are labeled, this kind of methods would signiﬁcantly decline the performance.
Revised: Especially, in the case of few-shot learning, where a small number of training nodes are labeled, this kind of methods would signiﬁcantly decline the performance.

Original: For example, the Pubmed graph dataset Sen et al. (2008) consists of 19,717 nodes and 44,338 edges, but only 0.3% nodes are labeled for the semi-supervised node classiﬁcation task.
Revised: For example, the Pubmed graph dataset (Sen et al., 2008) consists of 19,717 nodes and 44,338 edges, but only 0.3% nodes are labeled for the semi-supervised node classiﬁcation task.

Original: These above previous works usually boil it down to a general classiﬁcation problem, where the model is learnt on training set and selected by checking validate set.
Revised: These above previous works usually boil it down to a general classiﬁcation problem, where the model is learnt on training set and selected by checking validate set.

Original: However, they do not put great efforts on how to learn to infer from one node to another node on topological graph, especially in the few-shot regime.
Revised: However, they do not put great efforts on how to learn to infer from one node to another node on topological graph, especially in the few-shot regime.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: f
Revised: th

Original: [Figure]
Revised: [Figure]

Original:  t th d
Revised: bilit f d t th d

Original: In this paper we propose a graph inference learning (GIL) framework to teach the model to adaptively infer from reference labeled nodes to these query unlabeled nodes and ﬁnally boost the performance of semi-supervised node classiﬁcation in the case of a few number of labeled samples.
Revised: In this paper we propose a graph inference learning (GIL) framework to teach the model to adaptively infer from reference labeled nodes to these query unlabeled nodes and ﬁnally boost the performance of semi-supervised node classiﬁcation in the case of a few number of labeled samples.

Original: Given an input graph data, GIL attempts to infer the unlabeled nodes from those observed nodes by building betweennode relations.
Revised: Given an input graph data, GIL attempts to infer the unlabeled nodes from those observed nodes by building betweennode relations.

Original: The between-node relations are structured as the integration of node attributes, connection paths as well as graph topological structure.
Revised: The between-node relations are structured as the integration of node attributes, connection paths as well as graph topological structure.

Original: It means that the similarity between two nodes is decided from three aspects: the consistency of node attributes, the consistency of local topological structures and the between-node path reachability, as shown in Fig.
Revised: It means that the similarity between two nodes is decided from three aspects: the consistency of node attributes, the consistency of local topological structures and the between-node path reachability, as shown in Fig.

Original: The local structures anchored around each node as well as the attributes of nodes therein are jointly encoded with graph convolution Defferrard et al. for the sake of high-level feature extraction.
Revised: The local structures anchored around each node as well as the attributes of nodes therein are jointly encoded with graph convolution (Defferrard et al., 2016) for the sake of high-level feature extraction.

Original: For the betweennode path reachability, we adopt the random walk algorithm to obtain the characteristics from a labeled reference node v i to a query unlabeled node v j in a graph data.
Revised: For the betweennode path reachability, we adopt the random walk algorithm to obtain the characteristics from a labeled reference node v i to a query unlabeled node v j in a graph data.

Original: Based on the computed node representation and between-node reachability, the structure relations can be obtained by computing the similar scores/relationships from reference nodes to unlabeled nodes in a graph.
Revised: Based on the computed node representation and between-node reachability, the structure relations can be obtained by computing the similar scores/relationships from reference nodes to unlabeled nodes in a graph.

Original: Inspired by the recent meta-learning strategy Finn et al. (2017), we learn to infer the structure relations from training set to validation set, which can beneﬁt the generalization capability of the learned model.
Revised: Inspired by the recent meta-learning strategy (Finn et al., 2017), we learn to infer the structure relations from training set to validation set, which can beneﬁt the generalization capability of the learned model.

Original: In other words, our GIL attempts to learn some transferable knowledge underlying in structure relations from training samples to validation samples, such that the learned structure relations can be better self-adapted to the new testing stage.
Revised: In other words, our GIL attempts to learn some transferable knowledge underlying in structure relations from training samples to validation samples, such that the learned structure relations can be better self-adapted to the new testing stage.

Original: We summarize the main contributions as three folds:
Revised: We summarize the main contributions as three folds:

Original: • We propose a novel graph inference learning framework by building structure relations to infer unknown node labels from those labeled nodes in an end-to-end way.
Revised: • We propose a novel graph inference learning framework by building structure relations to infer unknown node labels from those labeled nodes in an end-to-end way.

Original: The structure relations are well deﬁned by jointly considering node attributes, between-node paths and graph topological structures.
Revised: The structure relations are well deﬁned by jointly considering node attributes, between-node paths and graph topological structures.

Original: • To make the inference model better generalize into test nodes, we introduce meta-learning to optimize structure relations, which should be the ﬁrst time to graph node classiﬁcation to our knowledge.
Revised: • To make the inference model better generalize into test nodes, we introduce meta-learning to optimize structure relations, which should be the ﬁrst time to graph node classiﬁcation to our knowledge.

Original: • Comprehensive evaluations on three citation network datasets (including Cora, Citeseer and Pubmed), and one knowledge graph data (i
Revised: • Comprehensive evaluations on three citation network datasets (including Cora, Citeseer and Pubmed), and one knowledge graph data (i

Original: e., NELL) demonstrate the superiority of our proposed GIL when compared with other state-of-the-art methods in the semi-supervised classiﬁcation problem.
Revised: e., NELL) demonstrate the superiority of our proposed GIL when compared with other state-of-the-art methods in the semi-supervised classiﬁcation problem.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Graph CNNs: With the rapid development of deep learning methods, various graph convolution neural networks Kashima et al. ; Morris et al. ; Shervashidze et al. ; Yanardag & 
Revised: Graph CNNs: With the rapid development of deep learning methods, various graph convolution neural networks (Kashima et al., 2003; Morris et al., 2017; Shervashidze et al., 2009; Yanardag & Vishwanathan, 2015) have been employed to analyze the irregular graph-structured data.

Original: Vishwanathan (2015) have been employed to analyze the irregular graph-structured data.

Original: For better extending the general convolutional neural networks to graph domains, two broad strategies have been proposed, including spectral and spatial convolution methods.
Revised: For better extending the general convolutional neural networks to graph domains, two broad strategies have been proposed, including spectral and spatial convolution methods.

Original: Speciﬁcally, spectral ﬁltering methods Henaff et al. ; Kipf & Welling (2017) develop convolution-like operators in the spectral domain, and then perform a series of spectral ﬁlters by decomposing graph Laplacian.
Revised: Speciﬁcally, spectral ﬁltering methods (Henaff et al., 2015; Kipf & Welling, 2017) develop convolution-like operators in the spectral domain, and then perform a series of spectral ﬁlters by decomposing graph Laplacian.

Original: Unfortunately, the spectral-based approaches often lead to a high computation complex due to the operation of eigenvalue decomposition, especially for a large amount of graph nodes.
Revised: Unfortunately, the spectral-based approaches often lead to a high computation complex due to the operation of eigenvalue decomposition, especially for a large amount of graph nodes.

Original: To alleviate the computation burden, the local spectral ﬁltering methods Defferrard et al. (2016) are then proposed by parameterizing the frequency responses as Chebyshev polynomial approximation.
Revised: To alleviate the computation burden, the local spectral ﬁltering methods (Defferrard et al., 2016) are then proposed by parameterizing the frequency responses as Chebyshev polynomial approximation.

Original: Another type of graph CNNs, namely spatial methods Li et al. ; Niepert et al. (2016), can perform the ﬁltering operation by deﬁning the spatial structures of adjacent vertices.
Revised: Another type of graph CNNs, namely spatial methods (Li et al., 2016; Niepert et al., 2016), can perform the ﬁltering operation by deﬁning the spatial structures of adjacent vertices.

Original: Various approaches can be employed to aggregate or sort neighbor vertices, such as diffusion CNNs Atwood & Towsley (2016),
Revised: Various approaches can be employed to aggregate or sort neighbor vertices, such as diffusion CNNs (Atwood & Towsley, 2016),

Original: GraphSAGE Hamilton et al. PSCN Niepert et al. and NgramCNN Luo et al.
Revised: GraphSAGE (Hamilton et al., 2017), PSCN (Niepert et al., 2016) and NgramCNN (Luo et al., 2017).

Original: Semi-supervised node classiﬁcation: Among various graph-related applications, semi-supervised node classiﬁcation has achieved increasing attentions recently, and various approaches has been proposed to deal with this problem, including explicit graph Laplacian regularization and graphembedding approaches.
Revised: Semi-supervised node classiﬁcation: Among various graph-related applications, semi-supervised node classiﬁcation has achieved increasing attentions recently, and various approaches has been proposed to deal with this problem, including explicit graph Laplacian regularization and graphembedding approaches.

Original: Several classic algorithms with graph Laplacian regularization contain the label propagation method using Gaussian random ﬁelds Zhu et al. (2003), the regularization framework by relying on the local and global consistency Zhou et al. and the random walkbased sampling algorithm for getting the context information Yang et al. Several graph convolution network methods Abu-El-Haija et al.
Revised: Several classic algorithms with graph Laplacian regularization contain the label propagation method using Gaussian random ﬁelds (Zhu et al., 2003), the regularization framework by relying on the local and global consistency (Zhou et al., 2004) and the random walkbased sampling algorithm for getting the context information (Yang et al., 2016).

Original:  Du et al. ; Thekumparampil et al. (2018); Velickovic et al. (2018); Zhuang & Ma (2018) are then developed to obtain discriminative representations of input graph data.
Revised: Several graph convolution network methods (Abu-El-Haija et al., 2018; Du et al., 2017; Thekumparampil et al., 2018; Velickovic et al., 2018; Zhuang & Ma, 2018) are then developed to obtain discriminative representations of input graph data.

Original: For example, Kipf et.al Kipf & Welling (2017) propose a scalable graph CNN model, which can scale linearly in the number of graph edges and learn graph representations by encoding both local graph structure and node attributes.
Revised: For example, Kipf et.al (Kipf & Welling, 2017) propose a scalable graph CNN model, which can scale linearly in the number of graph edges and learn graph representations by encoding both local graph structure and node attributes.

Original: Graph attention networks (GAT) Velickovic et al. (2018) are proposed to compute hidden representations of each node for attending over its neighbors with a self-attention strategy.
Revised: Graph attention networks (GAT) (Velickovic et al., 2018) are proposed to compute hidden representations of each node for attending over its neighbors with a self-attention strategy.

Original: By jointly considering the local- and globalconsistency information, dual graph convolutional networks Zhuang & Ma (2018) are presented to deal with the semi-supervised classiﬁcation.
Revised: By jointly considering the local- and globalconsistency information, dual graph convolutional networks (Zhuang & Ma, 2018) are presented to deal with the semi-supervised classiﬁcation.

Original: The main difference between our GIL and these previous semi-supervised classiﬁcation methods is to take a graph inference strategy by deﬁning structure relations on graph and then uses meta-optimization to learn the inference model, which should be the ﬁrst time to our knowledge, while the existing graph CNNs take it as a general classiﬁcation problem.
Revised: The main difference between our GIL and these previous semi-supervised classiﬁcation methods is to take a graph inference strategy by deﬁning structure relations on graph and then uses meta-optimization to learn the inference model, which should be the ﬁrst time to our knowledge, while the existing graph CNNs take it as a general classiﬁcation problem.

Original:  T HE M ODEL 3. P ROBLEM D EFINITION
Revised:  T HE M ODEL 3. P ROBLEM D EFINITION

Original: Formally, we denote an undirected/directed graph as G =
Revised: Formally, we denote an undirected/directed graph as G =

Original: {V , E , X , Y} ,
Revised: {V , E , X , Y} ,

Original: where V = { v i } ni =1 is the ﬁnite set of n (or |V| ) vertices, E ∈ R n × n deﬁnes the adjacency relationships (i.e., edges) between vertices representing the topology of G , X ∈ R n × d records the explicit/implicit attributes/signals of vertices, and Y ∈ R n is the vertex labels of C classes.
Revised: where V = { v i } ni =1 is the ﬁnite set of n (or |V| ) vertices, E ∈ R n × n deﬁnes the adjacency relationships (i.e., edges) between vertices representing the topology of G , X ∈ R n × d records the explicit/implicit attributes/signals of vertices, and Y ∈ R n is the vertex labels of C classes.

Original: The edge E ij = E ( v i , v j ) = 

Revised: The edge E ij = E ( v i , v j ) =

Original: if and only if vertices v i , v j are not connected, otherwise E ij (cid:54) = 0 .
Revised: if and only if vertices v i , v j are not connected, otherwise E ij (cid:54) = 0 .

Original: The attribute matrix X is attached to the vertex set V , whose i -th row X v i (or X i · ) represents the attribute of the i -th vertex v i .
Revised: The attribute matrix X is attached to the vertex set V , whose i -th row X v i (or X i · ) represents the attribute of the i -th vertex v i .

Original: It means that v i ∈ V carries a vector of d -dimension signals.
Revised: It means that v i ∈ V carries a vector of d -dimension signals.

Original: Associated with each node v i ∈ V , there is a discrete label y i ∈ { 1 , 2 , · · · , C } .
Revised: Associated with each node v i ∈ V , there is a discrete label y i ∈ { 1 , 2 , · · · , C } .

Original: We consider the task of semi-supervised learning classiﬁcation over graph data, where only a small number of vertices are labeled for the model learning, i.e., |V Label | (cid:28) |V| .
Revised: We consider the task of semi-supervised learning classiﬁcation over graph data, where only a small number of vertices are labeled for the model learning, i.e., |V Label | (cid:28) |V| .

Original: Generally, we have three node sets: a training set V tr , a validation set V val and a testing set V te .
Revised: Generally, we have three node sets: a training set V tr , a validation set V val and a testing set V te .

Original: In the standard protocol of prior literatures Yang et al. (2016), the three node sets share the same label space.
Revised: In the standard protocol of prior literatures (Yang et al., 2016), the three node sets share the same label space.

Original: We follow but not limit this protocol for our proposed method.
Revised: We follow but not limit this protocol for our proposed method.

Original: Given the training and validation node sets, the aim is to predict the node labels of testing nodes by using node attributes as well as edge connections.
Revised: Given the training and validation node sets, the aim is to predict the node labels of testing nodes by using node attributes as well as edge connections.

Original: A sophisticated machine learning technique used in most existing methods Kipf & Welling (2017);
Revised: A sophisticated machine learning technique used in most existing methods (Kipf & Welling, 2017;

Original: Zhou et al. (2004) is to choose an optimal classiﬁer (trained on training set) after checking the performance on validate set.
Revised: Zhou et al., 2004) is to choose an optimal classiﬁer (trained on training set) after checking the performance on validate set.

Original: However, these methods essentially ignore how to extract transferable knowledge from these known labeled nodes to unlabeled nodes, as the graph structure itself implies node connectivity/reachability.
Revised: However, these methods essentially ignore how to extract transferable knowledge from these known labeled nodes to unlabeled nodes, as the graph structure itself implies node connectivity/reachability.

Original: Moreover, due to the scarcity of labeled samples, the performance of such a classiﬁer is usually not satisfactory.
Revised: Moreover, due to the scarcity of labeled samples, the performance of such a classiﬁer is usually not satisfactory.

Original: To address these problems, we introduce the meta-learning mechanism Finn et al. Ravi & Larochelle (2017); Sung et al. (2017) to learn to infer node labels on graph.
Revised: To address these problems, we introduce the meta-learning mechanism (Finn et al., 2017; Ravi & Larochelle, 2017; Sung et al., 2017) to learn to infer node labels on graph.

Original: Speciﬁcally, the graph structure, between-node path reachability and node attributes are jointly modeled into the learning process.
Revised: Speciﬁcally, the graph structure, between-node path reachability and node attributes are jointly modeled into the learning process.

Original: Our aim is learning to infer from labeled nodes to unlabeled nodes, so that the learner can perform better on validation set and thus classify testing set more accurately.
Revised: Our aim is learning to infer from labeled nodes to unlabeled nodes, so that the learner can perform better on validation set and thus classify testing set more accurately.

Original: 3. S TRUCTURE R ELATION
Revised: 3. S TRUCTURE R ELATION

Original: For convenient inference, we speciﬁcally build a structure relation between two nodes on the topology graph.
Revised: For convenient inference, we speciﬁcally build a structure relation between two nodes on the topology graph.

Original: The labeled vertices (in training set) are viewed as the reference nodes, and their information can be propagated into those unlabeled vertices for improving the label prediction.
Revised: The labeled vertices (in training set) are viewed as the reference nodes, and their information can be propagated into those unlabeled vertices for improving the label prediction.

Original: Formally, given a reference node v i ∈ V Label , we deﬁne the score of a query node v j similar to v i as s i → j = f r ( f e ( G v i ) , f e ( G v j ) , f P ( v i , v j , E )) , (1) where G v i and G v j may be understood as the centralized subgraphs around v i and v j , respectively.
Revised: Formally, given a reference node v i ∈ V Label , we deﬁne the score of a query node v j similar to v i as s i → j = f r ( f e ( G v i ) , f e ( G v j ) , f P ( v i , v j , E )) , (1) where G v i and G v j may be understood as the centralized subgraphs around v i and v j , respectively.

Original: f e , f r , f P are three abstract functions we explain as follows:
Revised: f e , f r , f P are three abstract functions we explain as follows:

Original: • Node representation f e ( G v i ) −→ R d v , encodes the local representation of the centralized subgraph G v i around node v i , and thus may be understood as a local ﬁlter function on graph.
Revised: • Node representation f e ( G v i ) −→ R d v , encodes the local representation of the centralized subgraph G v i around node v i , and thus may be understood as a local ﬁlter function on graph.

Original: This function should not only take the signals of nodes therein as input, but also consider the local topological structure of the subgraph for more accurate similarity computation.
Revised: This function should not only take the signals of nodes therein as input, but also consider the local topological structure of the subgraph for more accurate similarity computation.

Original: To this end, we perform the spectral graph convolution on subgraphs to learn discriminative node features, analogical to the pixel-level feature extraction from convolution maps of gridded images.
Revised: To this end, we perform the spectral graph convolution on subgraphs to learn discriminative node features, analogical to the pixel-level feature extraction from convolution maps of gridded images.

Original: The details of feature extraction f e are described in Section 4.
Revised: The details of feature extraction f e are described in Section 4.

Original: • Path reachability f P ( v i , v j , E ) −→ R d p , represents the characteristics of path reachability from v i to v j .
Revised: • Path reachability f P ( v i , v j , E ) −→ R d p , represents the characteristics of path reachability from v i to v j .

Original: As there usually exist multiple traversal paths between two nodes, we choose the function as reachable probabilities of different lengths of walks from v i to v j .
Revised: As there usually exist multiple traversal paths between two nodes, we choose the function as reachable probabilities of different lengths of walks from v i to v j .

Original: More details will be introduced in Section 4.
Revised: More details will be introduced in Section 4.

Original: • Structure relation f r ( R d v , R d v , R d p ) −→ R , is a relational function computing the score of v j similar to v i .
Revised: • Structure relation f r ( R d v , R d v , R d p ) −→ R , is a relational function computing the score of v j similar to v i .

Original: This function is not exchangeable for different orders of two nodes, due to the asymmetric
Revised: This function is not exchangeable for different orders of two nodes, due to the asymmetric

Original: reachable relationship f P .
Revised: reachable relationship f P .

Original: If necessary, we may easily revise it as a symmetry function, e.g., summarizing two traversal directions.
Revised: If necessary, we may easily revise it as a symmetry function, e.g., summarizing two traversal directions.

Original: The score function depends on triple inputs: the local representations extracted from the subgraphs w.r.t.
Revised: The score function depends on triple inputs: the local representations extracted from the subgraphs w.r.t.

Original: f e ( G v i ) , f e ( G v j ) respectively, and the path reachability from v i to v j .
Revised: f e ( G v i ) , f e ( G v j ) respectively, and the path reachability from v i to v j .

Original: In semi-supervised node classiﬁcation, we take the training node set V tr as the reference samples, and the validation set V val as the query samples during the training stage.
Revised: In semi-supervised node classiﬁcation, we take the training node set V tr as the reference samples, and the validation set V val as the query samples during the training stage.

Original: Given a query node v j ∈ V val , we can derive the class similarity score of v j w.r.t.
Revised: Given a query node v j ∈ V val , we can derive the class similarity score of v j w.r.t.

Original: the c -th ( c = 1 , · · · , C ) category by weighting the reference samples C c
Revised: the c -th ( c = 1 , · · · , C ) category by weighting the reference samples C c

Original: = { v k | y v k = c } .
Revised: = { v k | y v k = c } .

Original: Formally, we can further revise Eqn.
Revised: Formally, we can further revise Eqn.

Original: (1) and deﬁne the class-to-node relationship function as
Revised: (1) and deﬁne the class-to-node relationship function as

Original: [Equation]
Revised: [Equation]

Original: where the function φ w maps a reachable vector f P ( v i , v j , E ) into a weight value, the function φ r computes the similar score between v j and the c -th class nodes, and the normalization factor F C c → v j of the c -th category w.r.t v j is deﬁned as
Revised: where the function φ w maps a reachable vector f P ( v i , v j , E ) into a weight value, the function φ r computes the similar score between v j and the c -th class nodes, and the normalization factor F C c → v j of the c -th category w.r.t v j is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: For the relation function φ r and the weight function φ w , we may choose some subnetworks to instantiate them in practice.
Revised: For the relation function φ r and the weight function φ w , we may choose some subnetworks to instantiate them in practice.

Original: The detailed implementation of our model can be found in Section 4.
Revised: The detailed implementation of our model can be found in Section 4.

Original: 3. I NFERENCE L EARNING
Revised: 3. I NFERENCE L EARNING

Original: According to the class-to-node relationship function in Eqn.
Revised: According to the class-to-node relationship function in Eqn.

Original: (2), given the query node v j , we can obtain a score vector s C→ j =
Revised: (2), given the query node v j , we can obtain a score vector s C→ j =

Original: [ s C 1 → j , · · · , s
Revised: [ s C 1 → j , · · · , s

Original: C C
Revised: C C

Original: → j ] (cid:124) ∈ R C after computing the relations to all classes .
Revised: → j ] (cid:124) ∈ R C after computing the relations to all classes .

Original: The indexed category with the maximum score is assumed to the estimated label.
Revised: The indexed category with the maximum score is assumed to the estimated label.

Original: Thus, we can deﬁne the loss function based on cross entropy as follows
Revised: Thus, we can deﬁne the loss function based on cross entropy as follows

Original: [Equation]
Revised: [Equation]

Original: where y j,c is a binary indicator (i.e., 0 or 1) of class label c for the node v j , and the softmax operation is imposed on s C c
Revised: where y j,c is a binary indicator (i.e., 0 or 1) of class label c for the node v j , and the softmax operation is imposed on s C c

Original: → j , i.e., ˆ y C c → j = exp( s C c
Revised: → j , i.e., ˆ y C c → j = exp( s C c

Original: → j ) / (cid:80) Ck =1 exp( s C k
Revised: → j ) / (cid:80) Ck =1 exp( s C k

Original: → j ) .
Revised: → j ) .

Original: Other error functions may be chosen as the loss function, e.g., mean square error.
Revised: Other error functions may be chosen as the loss function, e.g., mean square error.

Original: In the general classiﬁcation, the cross entropy loss is a standard one that performs well.
Revised: In the general classiﬁcation, the cross entropy loss is a standard one that performs well.

Original: Given the training set V tr , we expect the best performance can be obtained on the validate set V val after optimizing the model on V tr .
Revised: Given the training set V tr , we expect the best performance can be obtained on the validate set V val after optimizing the model on V tr .

Original: Given a trained/pretrained model Θ = { f e , φ w , φ r } , we perform iteratively gradient updates on the training set V tr to obtain the new model, formally,
Revised: Given a trained/pretrained model Θ = { f e , φ w , φ r } , we perform iteratively gradient updates on the training set V tr to obtain the new model, formally,

Original: [Equation]
Revised: [Equation]

Original: where α is the update rate.
Revised: where α is the update rate.

Original: Note that, in the computation of class scores, since the reference node and query node might be both from the training set V tr , we set the computation weight w i → j = 0 if i = j in Eqn.
Revised: Note that, in the computation of class scores, since the reference node and query node might be both from the training set V tr , we set the computation weight w i → j = 0 if i = j in Eqn.

Original: After several iterates of gradient descent on V tr , we expect a better performance on the validate set V val , i.e., min Θ L val (Θ (cid:48) ) .
Revised: After several iterates of gradient descent on V tr , we expect a better performance on the validate set V val , i.e., min Θ L val (Θ (cid:48) ) .

Original: Thus, we can perform the gradient update as follows
Revised: Thus, we can perform the gradient update as follows

Original: [Equation]
Revised: [Equation]

Original: where β is the learning rate of meta optimization Finn et al.
Revised: where β is the learning rate of meta optimization (Finn et al., 2017).

Original: During the training process, we may perform the batch sampling from training nodes and validate nodes, instead of taking all one time.
Revised: During the training process, we may perform the batch sampling from training nodes and validate nodes, instead of taking all one time.

Original: In the testing stage, we may take all training nodes and perform the model update according to Eqn.
Revised: In the testing stage, we may take all training nodes and perform the model update according to Eqn.

Original: (6) like the training process.
Revised: (6) like the training process.

Original: The updated model is used as the ﬁnal model and is then fed into Eqn.
Revised: The updated model is used as the ﬁnal model and is then fed into Eqn.

Original: (2) to infer class labels for the query nodes.
Revised: (2) to infer class labels for the query nodes.

Original:  M ODULES
Revised:  M ODULES

Original: In this section, we instantiate each modules (i.e., functions) of the aforementioned structure relation.
Revised: In this section, we instantiate each modules (i.e., functions) of the aforementioned structure relation.

Original: The implementation details can be found in the following.
Revised: The implementation details can be found in the following.

Original: Node Representation f e ( G v i ) :
Revised: Node Representation f e ( G v i ) :

Original: The local representation at the vertex v i can be extracted by performing the graph convolution operation on the subgraph G v i .
Revised: The local representation at the vertex v i can be extracted by performing the graph convolution operation on the subgraph G v i .

Original: Similar to gridded images/videos, on which local convolution kernels are deﬁned as multiple lattices with various receptive ﬁelds, the spectral graph convolution is used to encode the local representations of input graph in our work.
Revised: Similar to gridded images/videos, on which local convolution kernels are deﬁned as multiple lattices with various receptive ﬁelds, the spectral graph convolution is used to encode the local representations of input graph in our work.

Original: Given a graph sample G =
Revised: Given a graph sample G =

Original: {V , E , X} , the normalized graph Laplacian matrix is L
Revised: {V , E , X} , the normalized graph Laplacian matrix is L

Original: = I n −
Revised: = I n −

Original: D − 1 / 2 ED − 1 / 2 =
Revised: D − 1 / 2 ED − 1 / 2 =

Original: U Λ U T , with a diagonal matrix of its eigenvalues Λ .
Revised: U Λ U T , with a diagonal matrix of its eigenvalues Λ .

Original: The spectral graph convolutions can be deﬁned as the multiplication of the signal X with a ﬁlter g θ (Λ) =
Revised: The spectral graph convolutions can be deﬁned as the multiplication of the signal X with a ﬁlter g θ (Λ) =

Original: diag( θ ) parametrized by θ in the Fourier domain: conv ( X ) =
Revised: diag( θ ) parametrized by θ in the Fourier domain: conv ( X ) =

Original: g θ
Revised: g θ

Original: ( L ) ∗ X
Revised: ( L ) ∗ X

Original: =
Revised: =

Original: U g θ (Λ) U T X , where the parameter θ ∈ R n is a vector of Fourier coefﬁcients.
Revised: U g θ (Λ) U T X , where the parameter θ ∈ R n is a vector of Fourier coefﬁcients.

Original: To reduce the computation complexity and obtain the local information, we use an approximate local ﬁlter of the Chebyshev polynomial Defferrard et al. g θ (Λ) = (cid:80) K − 1 k =

Revised: To reduce the computation complexity and obtain the local information, we use an approximate local ﬁlter of the Chebyshev polynomial (Defferrard et al., 2016), g θ (Λ) = (cid:80) K − 1 k =

Original: θ k T k
Revised: θ k T k

Original: (ˆΛ) , where the parameter θ ∈ R K is a vector of Chebyshev coefﬁcients and T k (ˆΛ) ∈ R n × n is the Chebyshev polynomial of order k evaluated at ˆΛ =
Revised: (ˆΛ) , where the parameter θ ∈ R K is a vector of Chebyshev coefﬁcients and T k (ˆΛ) ∈ R n × n is the Chebyshev polynomial of order k evaluated at ˆΛ =

Original: 2Λ /λ max − I n , a diagonal matrix of scaled eigenvalues.
Revised: 2Λ /λ max − I n , a diagonal matrix of scaled eigenvalues.

Original: The graph ﬁltering operation can then be expressed as g θ (Λ) ∗ X
Revised: The graph ﬁltering operation can then be expressed as g θ (Λ) ∗ X

Original: = (cid:80) K − 1 k =

Revised: = (cid:80) K − 1 k =

Original: θ k T k
Revised: θ k T k

Original: (ˆ L ) X , where T k (ˆ L ) ∈ R n × n is the Chebyshev polynomial of order k evaluated at the scaled Laplacian
Revised: (ˆ L ) X , where T k (ˆ L ) ∈ R n × n is the Chebyshev polynomial of order k evaluated at the scaled Laplacian

Original: ˆ L = 2 L /λ max − I n .
Revised: ˆ L = 2 L /λ max − I n .

Original: Further, we can construct multi-scale receptive ﬁelds for each vertex based on the Laplacian matrix L , where each receptive ﬁeld records hopping neighborhood relationships around the reference vertex v i , and forms a local centralized subgraph.
Revised: Further, we can construct multi-scale receptive ﬁelds for each vertex based on the Laplacian matrix L , where each receptive ﬁeld records hopping neighborhood relationships around the reference vertex v i , and forms a local centralized subgraph.

Original: Path Reachability f P ( v i , v j , E ) :
Revised: Path Reachability f P ( v i , v j , E ) :

Original: Here we compute the probabilities of paths from vertex i to vertex j by employing random walk on graph, which refers to traversing the graph from v i to v j according to the probability matrix P .
Revised: Here we compute the probabilities of paths from vertex i to vertex j by employing random walk on graph, which refers to traversing the graph from v i to v j according to the probability matrix P .

Original: For the input graph data G with n vertices, the random-walk transition matrix can be deﬁned as P = D − 1 E , where D ∈ R n × n is the diagonal degree matrix with D ii =
Revised: For the input graph data G with n vertices, the random-walk transition matrix can be deﬁned as P = D − 1 E , where D ∈ R n × n is the diagonal degree matrix with D ii =

Original: (cid:80) i E ij .
Revised: (cid:80) i E ij .

Original: That is to say, each element P ij is the probability of going from vertex i to vertex j in one step.
Revised: That is to say, each element P ij is the probability of going from vertex i to vertex j in one step.

Original: The sequence of nodes from vertex i to vertex j is a random walk on the graph, which can be modeled as a classical Markov chain by considering the set of graph vertices.
Revised: The sequence of nodes from vertex i to vertex j is a random walk on the graph, which can be modeled as a classical Markov chain by considering the set of graph vertices.

Original: To represent this formulation, we show that P t ij is the probability of getting from vertex v i to vertex v j in t steps.
Revised: To represent this formulation, we show that P t ij is the probability of getting from vertex v i to vertex v j in t steps.

Original: This fact is easily exhibited by considering a t -step path from vertex v i to vertex v j as ﬁrst taking a single step to some vertex h , and then taking t − 1 steps to v j .
Revised: This fact is easily exhibited by considering a t -step path from vertex v i to vertex v j as ﬁrst taking a single step to some vertex h , and then taking t − 1 steps to v j .

Original: The transition probability P t in t steps can be formulated as
Revised: The transition probability P t in t steps can be formulated as

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: where each matrix entry P tij denotes the probability that starting at vertex i and ending at vertex j in t steps.
Revised: where each matrix entry P tij denotes the probability that starting at vertex i and ending at vertex j in t steps.

Original: Finally, the node reachability from v i to v j can be formated as a d p -dimension vector,
Revised: Finally, the node reachability from v i to v j can be formated as a d p -dimension vector,

Original: [Equation]
Revised: [Equation]

Original: where d p refers to the step length of the longest path from v i to v j .
Revised: where d p refers to the step length of the longest path from v i to v j .

Original: Class-to-Node Relationship s C c → j : To deﬁne the node relationship s i → j from v i to v j , we simultaneously consider the property of path reachability f P ( v i , v j , E ) , local representations f e ( G v i ) and f e ( G v j ) of nodes v i , v j .
Revised: Class-to-Node Relationship s C c → j : To deﬁne the node relationship s i → j from v i to v j , we simultaneously consider the property of path reachability f P ( v i , v j , E ) , local representations f e ( G v i ) and f e ( G v j ) of nodes v i , v j .

Original: The function φ w ( f P ( v i , v j , E )) in Eqn.
Revised: The function φ w ( f P ( v i , v j , E )) in Eqn.

Original: (3), which is to map the reachable vector f P ( v i , v j , E ) ∈ R d p into a weight value, can be implemented with two 16-dimensional fully connected layers in our experiments.
Revised: (3), which is to map the reachable vector f P ( v i , v j , E ) ∈ R d p into a weight value, can be implemented with two 16-dimensional fully connected layers in our experiments.

Original: The computed value w i → j can be further used to weight the local features at the node v i , f e ( G v i ) ∈ R d v .
Revised: The computed value w i → j can be further used to weight the local features at the node v i , f e ( G v i ) ∈ R d v .

Original: For obtaining the similar score between v j and the c -th class nodes C c in Eqn.
Revised: For obtaining the similar score between v j and the c -th class nodes C c in Eqn.

Original: (2), we perform a concatenation of two input features, where one refers to the weighted features of vertex v i , another is the local features of vertex v j .
Revised: (2), we perform a concatenation of two input features, where one refers to the weighted features of vertex v i , another is the local features of vertex v j .

Original: One fully connected layer (w.r.t.
Revised: One fully connected layer (w.r.t.

Original: φ r ) with C -dimensions is ﬁnally adopted to obtain the relation regression score.
Revised: φ r ) with C -dimensions is ﬁnally adopted to obtain the relation regression score.

Original:  E XPERIMENTS 5. E XPERIMENTAL S ETTINGS
Revised:  E XPERIMENTS 5. E XPERIMENTAL S ETTINGS

Original: We evaluate our proposed GIL method on four citation network datasets: Cora, Citeseer and
Revised: We evaluate our proposed GIL method on four citation network datasets: Cora, Citeseer and

Original: Pubmed Sen et al. (2008), and one knowledge graph NELL dataset Carlson et al.
Revised: Pubmed (Sen et al., 2008), and one knowledge graph NELL dataset (Carlson et al., 2010).

Original: The statistical properties of graph data are summarized in Table 1.
Revised: The statistical properties of graph data are summarized in Table 1.

Original: Followed the previous protocol in Kipf & Welling (2017); Zhuang & Ma (2018), we split the graph data into training set, validation set, and testing set.
Revised: Followed the previous protocol in Kipf & Welling (2017); Zhuang & Ma (2018), we split the graph data into training set, validation set, and testing set.

Original: Taking the graph convolution and pooling modules, we may alternately stack them into a multi-layer Graph convolution network.
Revised: Taking the graph convolution and pooling modules, we may alternately stack them into a multi-layer Graph convolution network.

Original: The GIL model consists of two graph convolution layers, each of which followed by a mean-pooling layer, a class-to-node relationship regression module and a ﬁnal softmax layer.
Revised: The GIL model consists of two graph convolution layers, each of which followed by a mean-pooling layer, a class-to-node relationship regression module and a ﬁnal softmax layer.

Original: We have given the detailed conﬁguration of the relationship regression module in the class-to-node relationship of Section 4.
Revised: We have given the detailed conﬁguration of the relationship regression module in the class-to-node relationship of Section 4.

Original: The parameter d p in Eqn.
Revised: The parameter d p in Eqn.

Original: (9) is set to the mean length of between-node reachability paths in the input graph data.
Revised: (9) is set to the mean length of between-node reachability paths in the input graph data.

Original: The channels of the 1-st and 2-nd convolutional layer are set to 128 and 256, respectively.
Revised: The channels of the 1-st and 2-nd convolutional layer are set to 128 and 256, respectively.

Original: The scale of respective ﬁled is 2 in both convolutional layers.
Revised: The scale of respective ﬁled is 2 in both convolutional layers.

Original: Dropout rate is set to 0.5 in the convolution and fully connected layers to avoid over-ﬁtting, and the ReLU unit is leveraged as a nonlinear activation function.
Revised: Dropout rate is set to 0.5 in the convolution and fully connected layers to avoid over-ﬁtting, and the ReLU unit is leveraged as a nonlinear activation function.

Original: We pre-train our GIL model for 200 iterations with the training set, where its initial learning rate, decay factor and momentum are set to 0.05, 0.95 and 0.9, respectively.
Revised: We pre-train our GIL model for 200 iterations with the training set, where its initial learning rate, decay factor and momentum are set to 0.05, 0.95 and 0.9, respectively.

Original: Here we train the GIL model using the stochastic gradient descent algorithm with the batch size of 100.
Revised: Here we train the GIL model using the stochastic gradient descent algorithm with the batch size of 100.

Original: We further improve the inference learning capability of GIL model for 1200 iterations with the validation set, where the meta-learning rates α and β are set to 0.001.
Revised: We further improve the inference learning capability of GIL model for 1200 iterations with the validation set, where the meta-learning rates α and β are set to 0.001.

Original: 5. C OMPARISON WITH THE STATE - OF - THE - ARTS
Revised: 5. C OMPARISON WITH THE STATE - OF - THE - ARTS

Original: We compare the GIL approach with several state-of-the-art methods Monti et al. ; Kipf &
Revised: We compare the GIL approach with several state-of-the-art methods (Monti et al., 2017; Kipf &

Original: Welling (2017); Zhou et al. ; Zhuang & Ma (2018) over four graph datasets, including Cora,
Revised: Welling, 2017; Zhou et al., 2004; Zhuang & Ma, 2018) over four graph datasets, including Cora,

Original: Citeseer, Pubmed and NELL.
Revised: Citeseer, Pubmed and NELL.

Original: The classiﬁcation accuracies for all methods are reported in Table 2.
Revised: The classiﬁcation accuracies for all methods are reported in Table 2.

Original: Our proposed GIL can signiﬁcantly outperform these graph Laplacian regularized methods on four graph datasets, including Deep walk Zhou et al. (2004), modularity clustering Brandes et al.
Revised: Our proposed GIL can signiﬁcantly outperform these graph Laplacian regularized methods on four graph datasets, including Deep walk (Zhou et al., 2004), modularity clustering (Brandes et al., 2008),

Original: Gaussian ﬁelds Zhu et al. (2003), and graph embedding Yang et al. (2016) methods.
Revised: Gaussian ﬁelds (Zhu et al., 2003), and graph embedding (Yang et al., 2016) methods.

Original: For example, we can achieve much higher performance than the deepwalk method Zhou et al. (2004), e.g., 43.2% vs 74.1% on Citeseer dataset, 65.3% vs 83.1% on Pubmed dataset and 58.1% vs 78.9% on NELL
Revised: For example, we can achieve much higher performance than the deepwalk method (Zhou et al., 2004), e.g., 43.2% vs 74.1% on Citeseer dataset, 65.3% vs 83.1% on Pubmed dataset and 58.1% vs 78.9% on NELL

Original: [Table] [Table]
Revised: [Table] [Table]

Original: dataset.
Revised: dataset.

Original: We ﬁnd that the graph embedding method Yang et al.
Revised: We ﬁnd that the graph embedding method (Yang [Table]

Original: (2016), which have considered both label information and graph structure during sampling, can obtain lower accuracies than our proposed
Revised: al., 2016), which have considered both label information and graph structure during sampling, can obtain lower accuracies than our proposed

Original: GIL by 9.4% on Citeseer dataset and 10.5% on Cora dataset.
Revised: GIL by 9.4% on Citeseer dataset and 10.5% on Cora dataset.

Original: This indicates that our GIL can optimize better structure relations and improve the network generality.
Revised: This indicates that our GIL can optimize better structure relations and improve the network generality.

Original: We further compare our GIL with several existing deep graph embedding methods, including graph attention network Velickovic et al. dual graph convolutional networks Zhuang & Ma (2018), topology adaptive graph convolutional networks Du et al. (2017), Multi-scale graph convolution Abu-El-Haija et al. (2018), etc.
Revised: We further compare our GIL with several existing deep graph embedding methods, including graph attention network (Velickovic et al., 2018), dual graph convolutional networks (Zhuang & Ma, 2018), topology adaptive graph convolutional networks (Du et al., 2017), Multi-scale graph convolution (Abu-El-Haija et al., 2018), etc.

Original: For example, our GIL method achieves a very large gain, e.g., 86.2% vs 83.3% Du et al. (2017) on Cora dataset, 78.9% vs 66.0% Kipf & Welling (2017) on NELL dataset.
Revised: For example, our GIL method achieves a very large gain, e.g., 86.2% vs 83.3% (Du et al., 2017) on Cora dataset, 78.9% vs 66.0% (Kipf & Welling, 2017) on NELL dataset.

Original: We evaluate our proposed
Revised: We evaluate our proposed

Original: GIL method on a large graph dataset with a lower label rate, which can signiﬁcantly outperform existing baselines on Pubmed dataset: 3.1% over DGCN Zhuang & Ma (2018), 4.1% over classic
Revised: GIL method on a large graph dataset with a lower label rate, which can signiﬁcantly outperform existing baselines on Pubmed dataset: 3.1% over DGCN (Zhuang & Ma, 2018), 4.1% over classic

Original: GCN Kipf & Welling (2017) and TAGCN Du et al. (2017), 3.2% over AGNN Thekumparampil et al. (2018), and 3.6% over N-GCN Abu-El-Haija et al.
Revised: GCN (Kipf & Welling, 2017) and TAGCN (Du et al., 2017), 3.2% over AGNN (Thekumparampil et al., 2018), and 3.6% over N-GCN (Abu-El-Haija et al., 2018).

Original: It demonstrate that our GIL method performs very well on various graph datasets by building the graph inference learning process, where the limited label information and graph structure can be well employed in the predicted framework.
Revised: It demonstrate that our GIL method performs very well on various graph datasets by building the graph inference learning process, where the limited label information and graph structure can be well employed in the predicted framework.

Original: 5. A LGORITHM ANALYSIS
Revised: 5. A LGORITHM ANALYSIS

Original: Meta-optimization: As can be seen in Table 3, we report the classiﬁcation accuracies of semisupervised classiﬁcation with several variants of our GIL and the classical GCN method Kipf &
Revised: Meta-optimization: As can be seen in Table 3, we report the classiﬁcation accuracies of semisupervised classiﬁcation with several variants of our GIL and the classical GCN method (Kipf &

Original: Welling (2017) when evaluating on Cora dataset.
Revised: Welling, 2017) when evaluating on Cora dataset.

Original: For analyzing the performance improvement of our
Revised: For analyzing the performance improvement of our

Original: GIL with the graph inference learning process, we report the classiﬁcation accuracies of GCN Kipf &
Revised: GIL with the graph inference learning process, we report the classiﬁcation accuracies of GCN (Kipf &

Original: Welling (2017) and our GIL on Cora dataset under two different situations, including “only learning with training set V tr " and “with jointly learning on training set V tr and validation set V val ".
Revised: Welling, 2017) and our GIL on Cora dataset under two different situations, including “only learning with training set V tr " and “with jointly learning on training set V tr and validation set V val ".

Original: “GCN /w jointly learning on V tr & V val " achieves a better result than “GCN /w learning on V tr " by 3.6%, which demonstrates that the network performance can be improved by employing validation samples.
Revised: “GCN /w jointly learning on V tr & V val " achieves a better result than “GCN /w learning on V tr " by 3.6%, which demonstrates that the network performance can be improved by employing validation samples.

Original: When using structure relation, “GIL /w learning on V tr " obtains an improvement of 1.9% (over “GCN /w learning on V tr ”), which can be attributed to the building connection of nodes.
Revised: When using structure relation, “GIL /w learning on V tr " obtains an improvement of 1.9% (over “GCN /w learning on V tr ”), which can be attributed to the building connection of nodes.

Original: The meta-optimization strategy (“GIL /w meta-train from V tr → V val " vs “GIL /w learning on V tr ”) has a gain of 2.9%, which indicates a good inference capability can be learnt through meta-optimization.
Revised: The meta-optimization strategy (“GIL /w meta-train from V tr → V val " vs “GIL /w learning on V tr ”) has a gain of 2.9%, which indicates a good inference capability can be learnt through meta-optimization.

Original: 11 step 0. a cc u r a c y our GIL GCN label rate 0.30% 0.60% 0.90% 1.20% 1.50% 1.80% GCN 0. 0.834 GIL(ours) 0. 1x 2x 3x 4x 5x 6x 77.0% 79.0% 81.0% 83.0% 85.0% 1x 2x 3x 4x 5x 6x
Revised: [Table]

Original: [Figure]
Revised: [Table]

Original: [Figure] [Figure] [Figure] Network settings: We explore the effectiveness of our proposed GIL with the same mean pooling mechanism, but with different number of convolutional layers, i.e., “GIL + mean pooling" with one, two and three convolutional layers, respectively.
Revised: Network settings: We explore the effectiveness of our proposed GIL with the same mean pooling mechanism, but with different number of convolutional layers, i.e., “GIL + mean pooling" with one, two and three convolutional layers, respectively.

Original: As can be seen in Table 3, the GIL with two convolutional layers can obtain a better performance on Cora data than other two network settings (i.e., GIL with one or three convolutional layers).
Revised: As can be seen in Table 3, the GIL with two convolutional layers can obtain a better performance on Cora data than other two network settings (i.e., GIL with one or three convolutional layers).

Original: For example, the performance of ‘GIL /w 1 conv.
Revised: For example, the performance of ‘GIL /w 1 conv.

Original: layer
Revised: layer

Original: + mean pooling" is slightly decreased by 1.7% over “GIL /w 2 conv.
Revised: + mean pooling" is slightly decreased by 1.7% over “GIL /w 2 conv.

Original: layers + mean pooling" on
Revised: layers + mean pooling" on

Original: Cora dataset.
Revised: Cora dataset.

Original: Furthermore, we report the classiﬁcation results of our proposed GIL by using mean and max-pooling mechanisms, respectively.
Revised: Furthermore, we report the classiﬁcation results of our proposed GIL by using mean and max-pooling mechanisms, respectively.

Original: The GIL with mean pooling (i.e., “GIL /w 2 conv layers + mean pooling") can get a better result than GIL method with max-pooling method (i.e., “GIL /wconv layers + max-pooling"), e.g., 86.2% vs 85.2% on Cora graph dataset.
Revised: The GIL with mean pooling (i.e., “GIL /w 2 conv layers + mean pooling") can get a better result than GIL method with max-pooling method (i.e., “GIL /wconv layers + max-pooling"), e.g., 86.2% vs 85.2% on Cora graph dataset.

Original: The reason may be that the graph network with two convolutional layers and the mean pooling mechanism can obtain the optimal graph embeddings, but when increasing the network layers, more parameters of graph model need to be optimized which may lead to the over-ﬁtting problem.
Revised: The reason may be that the graph network with two convolutional layers and the mean pooling mechanism can obtain the optimal graph embeddings, but when increasing the network layers, more parameters of graph model need to be optimized which may lead to the over-ﬁtting problem.

Original: Inﬂuence of different between-node steps: We compare the classiﬁcation performance within different between-node steps for our GIL and GCN Kipf & Welling (2017), as illustrated in Fig.
Revised: Inﬂuence of different between-node steps: We compare the classiﬁcation performance within different between-node steps for our GIL and GCN (Kipf & Welling, 2017), as illustrated in Fig.

Original: The length of between-node steps can be computed with the shortest path between reference nodes and query nodes.
Revised: The length of between-node steps can be computed with the shortest path between reference nodes and query nodes.

Original: When the step between nodes is smaller, both GIL and GCN methods can predict the category information for a small part of unlabeled nodes in the testing set.
Revised: When the step between nodes is smaller, both GIL and GCN methods can predict the category information for a small part of unlabeled nodes in the testing set.

Original: The reason may be that the node category information may be disturbed by its nearest neighbor nodes with different labels and less nodes are within 1 or 2 steps in the testing set.
Revised: The reason may be that the node category information may be disturbed by its nearest neighbor nodes with different labels and less nodes are within 1 or 2 steps in the testing set.

Original: The GIL and GCN methods can infer the category information for a part of unlabeled nodes by adopting node attributes, when two nodes are not connected in the graph (i.e., step= ∞ ).
Revised: The GIL and GCN methods can infer the category information for a part of unlabeled nodes by adopting node attributes, when two nodes are not connected in the graph (i.e., step= ∞ ).

Original: With increasing the length of reachability path, the inference capacity of the GIL method would become difﬁcult and more graph structure information may be employed in the predicted process.
Revised: With increasing the length of reachability path, the inference capacity of the GIL method would become difﬁcult and more graph structure information may be employed in the predicted process.

Original: The GIL can outperform the classic GCN by analyzing the accuracies within different between-node steps, which indicates that our GIL has a better reference capability than GCN by using the meta-optimization from training nodes to validation nodes.
Revised: The GIL can outperform the classic GCN by analyzing the accuracies within different between-node steps, which indicates that our GIL has a better reference capability than GCN by using the meta-optimization from training nodes to validation nodes.

Original: Inﬂuence of Different label rates: We also explore the performance comparisons of GIL method with different label rates, and the detailed results on Pubmed dataset can be shown in Fig.
Revised: Inﬂuence of Different label rates: We also explore the performance comparisons of GIL method with different label rates, and the detailed results on Pubmed dataset can be shown in Fig.

Original: When label rates increase by multiplication, the performances of GIL and GCN are improved, but the relative gain becomes narrow.
Revised: When label rates increase by multiplication, the performances of GIL and GCN are improved, but the relative gain becomes narrow.

Original: The reason is that, the reachable path lengths between unlabeled nodes and labeled nodes will be reduced with the increase of labeled nodes, which will weaken the effect of inference learning.
Revised: The reason is that, the reachable path lengths between unlabeled nodes and labeled nodes will be reduced with the increase of labeled nodes, which will weaken the effect of inference learning.

Original: In the extreme case, labels of unlabeled nodes could be determined by those neighbors with the 1 ∼ 2 step reachability.
Revised: In the extreme case, labels of unlabeled nodes could be determined by those neighbors with the 1 ∼ 2 step reachability.

Original: In summary, our proposed GIL method prefers small ratio labeled nodes in the semi-supervised node classiﬁcation task.
Revised: In summary, our proposed GIL method prefers small ratio labeled nodes in the semi-supervised node classiﬁcation task.

Original: Inference learning process: Classiﬁcation errors of different epochs on the validation set of Cora dataset can be illustrated in Fig.
Revised: Inference learning process: Classiﬁcation errors of different epochs on the validation set of Cora dataset can be illustrated in Fig.

Original: Classiﬁcation errors are rapidly decreasing as the number of iteration increases from the beginning to 400 iterations, while they are with a slow descent fromiterations to 1200 iterations.
Revised: Classiﬁcation errors are rapidly decreasing as the number of iteration increases from the beginning to 400 iterations, while they are with a slow descent fromiterations to 1200 iterations.

Original: It demonstrates that the learned knowledge from the training samples can be transferred for inferring node category information from these reference labeled nodes.
Revised: It demonstrates that the learned knowledge from the training samples can be transferred for inferring node category information from these reference labeled nodes.

Original: The performance of semi-supervised classiﬁcation can be further increased by improving the generalized capability of Graph CNN model.
Revised: The performance of semi-supervised classiﬁcation can be further increased by improving the generalized capability of Graph CNN model.

Original: Module analysis: We evaluate the effectiveness of different modules in our GIL framework, including node representation f e , path reachability f P and structure relation f r .
Revised: Module analysis: We evaluate the effectiveness of different modules in our GIL framework, including node representation f e , path reachability f P and structure relation f r .

Original: Note that the last one f r deﬁnes on the former two ones, thus we consider the cases in Table 4 by adding modules.
Revised: Note that the last one f r deﬁnes on the former two ones, thus we consider the cases in Table 4 by adding modules.

Original: When no using all modules, only original attributes of nodes are used to predict labels.
Revised: When no using all modules, only original attributes of nodes are used to predict labels.

Original: The case of only using f e belongs to the GCN method, which can achieve 81.5% on Cora dataset.
Revised: The case of only using f e belongs to the GCN method, which can achieve 81.5% on Cora dataset.

Original: The large gain of using the relation module f r (i.e., from 81.5% to 85.0%) may be contributed to the ability of inference learning on attributes as well as local topology structures which are implicitly encoded in f e .
Revised: The large gain of using the relation module f r (i.e., from 81.5% to 85.0%) may be contributed to the ability of inference learning on attributes as well as local topology structures which are implicitly encoded in f e .

Original: The path information f P can further boost the performance by 1.2%, e.g., 86.2% vs 85.0%.
Revised: The path information f P can further boost the performance by 1.2%, e.g., 86.2% vs 85.0%.

Original: It demonstrates that three different modules of our method can improve the graph inference learning capability.
Revised: It demonstrates that three different modules of our method can improve the graph inference learning capability.

Original: Computation complexity: For the computation complexity of our proposed GIL, the cost mainly spends on the computation of node representation, between-node reachability and class-to-node relationship, which are about O (( n tr + n te ) ∗ e ∗ d in ∗ d out ) , O (( n tr + n te ) ∗ e ∗ P ) and O ( n tr ∗ n te d 2 out ) .
Revised: Computation complexity: For the computation complexity of our proposed GIL, the cost mainly spends on the computation of node representation, between-node reachability and class-to-node relationship, which are about O (( n tr + n te ) ∗ e ∗ d in ∗ d out ) , O (( n tr + n te ) ∗ e ∗ P ) and O ( n tr ∗ n te d 2 out ) .

Original: n tr and n te refer to the number of training and testing nodes, d in and d out denote the input and output dimension of node representation, e is about the average degree of graph node and P is the step length of node reachability.
Revised: n tr and n te refer to the number of training and testing nodes, d in and d out denote the input and output dimension of node representation, e is about the average degree of graph node and P is the step length of node reachability.

Original: Compared with these classic Graph CNNs Kipf & Welling (2017), our GIL has a slightly higher cost due to an extra inference learning process, but can ﬁnish the testing process with several seconds for these benchmark sets.
Revised: Compared with these classic Graph CNNs (Kipf & Welling, 2017), our GIL has a slightly higher cost due to an extra inference learning process, but can ﬁnish the testing process with several seconds for these benchmark sets.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this work, we tackle the semi-supervised node classiﬁcation problem with a graph inference learning method, which can better predict the categories of these unlabeled nodes in an end-to-end framework.
Revised: In this work, we tackle the semi-supervised node classiﬁcation problem with a graph inference learning method, which can better predict the categories of these unlabeled nodes in an end-to-end framework.

Original: We can build a structure relation for obtaining the connection of any two graph nodes, where node attributes, between-node paths and graph structure information can be encapsulated together.
Revised: We can build a structure relation for obtaining the connection of any two graph nodes, where node attributes, between-node paths and graph structure information can be encapsulated together.

Original: For better capturing the transferable knowledge, our method can further learn to transfer these mined knowledge from the training samples to validation set and ﬁnally improve the performance of predicting unlabeled nodes in the testing set.
Revised: For better capturing the transferable knowledge, our method can further learn to transfer these mined knowledge from the training samples to validation set and ﬁnally improve the performance of predicting unlabeled nodes in the testing set.

Original: Extensive experimental results clearly demonstrate the effectiveness of our proposed GIL for solving the semi-supervised learning problem, even in the few-shot regime.
Revised: Extensive experimental results clearly demonstrate the effectiveness of our proposed GIL for solving the semi-supervised learning problem, even in the few-shot regime.

Original: In the future, we will extent the graph reasoning method to other graph-related tasks, such as graph generation and social network analysis.
Revised: In the future, we will extent the graph reasoning method to other graph-related tasks, such as graph generation and social network analysis.