Original: G RAPH WAVELET NEURAL NETWORK
Revised: G RAPH WAVELET NEURAL NETWORK

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We present graph wavelet neural network (GWNN), a novel graph convolutional neural network (CNN), leveraging graph wavelet transform to address the shortcoming of previous spectral graph CNN methods that depend on graph Fourier transform.
Revised: We present graph wavelet neural network (GWNN), a novel graph convolutional neural network (CNN), leveraging graph wavelet transform to address the shortcomings of previous spectral graph CNN methods that depend on graph Fourier transform.

Original: Different from graph Fourier transform, graph wavelet transform can be obtained via a fast algorithm without requiring matrix eigendecomposition with high computational cost.
Revised: Different from graph Fourier transform, graph wavelet transform can be obtained via a fast algorithm without requiring matrix eigendecomposition with high computational cost.

Original: Moreover,  wavelets are sparse and localized in vertex domain, offering high efﬁciency and good interpretability for  convolution.
Revised: Moreover, graph wavelets are sparse and localized in vertex domain, offering high efﬁciency and good interpretability for graph convolution.

Original: The proposed GWNN signiﬁcantly outperforms previous spectral graph CNNs in the task of graph semi-supervised classiﬁcation on three benchmark datasets: Cora, Citeseer and Pubmed
Revised: The proposed GWNN signiﬁcantly outperforms previous spectral graph CNNs in the task of graph-based semi-supervised classiﬁcation on three benchmark datasets: Cora, Citeseer and Pubmed.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Convolutional neural networks (CNNs) (LeCun et al., 1998) have been successfully used in many machine learning problems, such as image classiﬁcation (He et al., 2016) and speech recognition (Hinton et al., 2012), where there is an underlying Euclidean structure.
Revised: Convolutional neural networks (CNNs) (LeCun et al., 1998) have been successfully used in many machine learning problems, such as image classiﬁcation (He et al., 2016) and speech recognition (Hinton et al., 2012), where there is an underlying Euclidean structure.

Original: The success of CNNs lies in their ability to leverage the statistical properties of Euclidean data, e.g., translation invariance.
Revised: The success of CNNs lies in their ability to leverage the statistical properties of Euclidean data, e g., translation invariance.

Original: However, in many research areas, data are naturally located in a non-Euclidean space, with graph or network being one typical case.
Revised: However, in many research areas, data are naturally located in a non-Euclidean space with graph or network being one typical case.

Original: The non-Euclidean nature of graph is the main obstacle or challenge when we attempt to generalize CNNs to graph.
Revised: The non-Euclidean nature of graph is the main obstacle or challenge when we attempt to generalize CNNs to graph.

Original: For example, convolution is not well deﬁned in graph, due to that the size of neighborhood for each node varies dramatically (Bronstein et al., 2017).
Revised: For example, convolution is not well deﬁned in graph, due to that the size of neighborhood for each node varies dramatically (Bronstein et al., 2017).

Original: Existing methods attempting to generalize CNNs to graph data fall into two categories, spatial methods and spectral methods, according to the way that convolution is deﬁned.
Revised: Existing methods attempting to generalize CNNs to graph data fall into two categories, spatial methods and spectral methods, according to the way that convolution is deﬁned.

Original: Spatial methods deﬁne convolution directly on the vertex domain, following the practice of the conventional CNN.
Revised: Spatial methods deﬁne convolution directly on the vertex domain, following the practice of the conventional CNN.

Original: For each vertex, convolution is deﬁned as a weighted average function over all vertices located in its neighborhood, with the weighting function characterizing the inﬂuence exerting to the target vertex by its neighbors (Monti et al., 2017).
Revised: For each vertex, convolution is deﬁned as a weighted average function over all vertices located in its neighborhood, with the weighting function characterizing the inﬂuence exerting to the target vertex by its neighbors (Monti et al., 2017).

Original: The main challenge is to deﬁne a convolution operator that can handle neighborhood with different sizes and maintain the weight sharing property of CNN.
Revised: The main challenge is to deﬁne a convolution operator that can handle neighborhood with different sizes and maintain the weight sharing property of CNN.

Original: Although spatial methods gain some initial success and offer us a ﬂexible framework to generalize CNNs to graph, it is still elusive to determine appropriate neighborhood.
Revised: Although spatial methods gain some initial success and offer us a ﬂexible framework to generalize CNNs to graph, it is still elusive to determine appropriate neighborhood.

Original: Spectral methods deﬁne convolution via graph Fourier transform and convolution theorem.
Revised: Spectral methods deﬁne convolution via graph Fourier transform and convolution theorem.

Original: Spectral methods leverage graph Fourier transform to convert signals deﬁned in vertex domain into spectral domain, e.g., the space spanned by the eigenvectors of the graph Laplacian matrix, and then ﬁlter is deﬁned in spectral domain, maintaining the weight sharing property of CNN.
Revised: Spectral methods leverage graph Fourier transform to convert signals deﬁned in vertex domain into spectral domain, e.g., the space spanned by the eigenvectors of the graph Laplacian matrix, and then ﬁlter is deﬁned in spectral domain, maintaining the weight sharing property of CNN.

Original: As the pioneering work of spectral methods, spectral CNN (Bruna et al., 2014) exploited graph data with the graph Fourier transform to implement convolution operator using convolution theorem.
Revised: As the pioneering work of spectral methods, spectral CNN (Bruna et al., 2014) exploited graph data with the graph Fourier transform to implement convolution operator using convolution theorem.

Original: Some subsequent works make spectral methods spectrum-free (Defferrard et al., 2016;
Revised: Some subsequent works make spectral methods spectrum-free (Defferrard et al., 2016;

Original: Kipf & Welling, 2017; Khasanova &
Revised: Kipf & Welling, 2017; Khasanova &

Original: Frossard, 2017), achieving locality in spatial domain and avoiding high computational cost of the eigendecomposition of Laplacian matrix.
Revised: Frossard, 2017), achieving locality in spatial domain and avoiding high computational cost of the eigendecomposition of Laplacian matrix.

Original: In this paper, we present graph wavelet neural network to implement efﬁcient convolution on graph data.
Revised: In this paper, we present graph wavelet neural network to implement efﬁcient convolution on graph data.

Original: We take graph wavelets instead of the eigenvectors of graph Laplacian as a set of bases, and deﬁne the convolution operator via wavelet transform and convolution theorem.
Revised: We take graph wavelets instead of the eigenvectors of graph Laplacian as a set of bases, and deﬁne the convolution operator via wavelet transform and convolution theorem.

Original: Graph wavelet neural network distinguishes itself from spectral CNN by its three desirable properties: (1) Compared with graph Fourier transform, graph wavelets can be obtained via a fast algorithm without requiring the eigendecomposition of Laplacian matrix, and thus is efﬁcient; (2) Graph wavelets are sparse, while eigenvectors of Laplacian matrix are dense.
Revised: Graph wavelet neural network distinguishes itself from spectral CNN by its three desirable properties: (1) Graph wavelets can be obtained via a fast algorithm without requiring the eigendecomposition of Laplacian matrix, and thus is efﬁcient; (2) Graph wavelets are sparse, while eigenvectors of Laplacian matrix are dense.

Original: As a result, graph wavelet transform is much more efﬁcient than graph Fourier transform; (3) Graph wavelets are localized in vertex domain, reﬂecting the information diffusion centered at each node (Tremblay & Borgnat, 2014).
Revised: As a result, graph wavelet transform is much more efﬁcient than graph Fourier transform; (3) Graph wavelets are localized in vertex domain, reﬂecting the information diffusion centered at each node (Tremblay & Borgnat, 2014).

Original: This property eases the understanding of graph convolution deﬁned by graph wavelets.
Revised: This property eases the understanding of graph convolution deﬁned by graph wavelets.

Original: We develop an efﬁcient implementation of the proposed graph wavelet neural network.
Revised: We develop an efﬁcient implementation of the proposed graph wavelet neural network.

Original: Convolution in conventional CNN learns an individual convolution kernel for each pair of input feature and output feature, causing a huge number of parameters especially when the number of features is high.
Revised: Convolution in conventional CNN learns an individual convolution kernel for each pair of input feature and output feature, causing a huge number of parameters especially when the number of features is high.

Original: We detach the feature transformation from convolution and learn a sole convolution kernel among all features, substantially reducing the number of parameters.
Revised: We detach the feature transformation from convolution and learn a sole convolution kernel among all features, substantially reducing the number of parameters.

Original: Finally, we validate the effectiveness of the proposed graph wavelet neural network by applying it to graph semi-supervised classiﬁcation.
Revised: Finally, we validate the effectiveness of the proposed graph wavelet neural network by applying it to graph-based semi-supervised classiﬁcation.

Original: Experimental results demonstrate that our method consistently outperforms previous spectral CNNs on three benchmark datasets, i.e., Cora, Citeseer, and Pubmed.
Revised: Experimental results demonstrate that our method consistently outperforms previous spectral CNNs on three benchmark datasets, i.e., Cora, Citeseer, and Pubmed.

Original:  O UR METHOD 2. P RELIMINARY
Revised:  O UR METHOD 2. P RELIMINARY

Original: Let G =
Revised: Let G =

Original: { V , E , A } be an undirected graph, where V is the set of nodes with | V | = n , E is the set of edges, and A is adjacency matrix with A i,j =
Revised: { V , E , A } be an undirected graph, where V is the set of nodes with | V | = n , E is the set of edges, and A is adjacency matrix with A i,j =

Original: A j,i to deﬁne the connection between node i and node j .
Revised: A j,i to deﬁne the connection between node i and node j .

Original: The graph Laplacian matrix L is deﬁned as L = D − A where D is a diagonal degree matrix with
Revised: The graph Laplacian matrix L is deﬁned as L = D − A where D is a diagonal degree matrix with

Original: D i,i = (cid:80)
Revised: D i,i = (cid:80)

Original: j A i,j , and the normalized Laplacian matrix is L
Revised: j A i,j , and the normalized Laplacian matrix is L

Original: = I n − D − 1 / 2 AD − 1 / 2 where I n is the identity matrix.
Revised: = I n − D − 1 / 2 AD − 1 / 2 where I n is the identity matrix.

Original: Since L is a real symmetric matrix, it has a complete set of orthonormal eigenvectors
Revised: Since L is a real symmetric matrix, it has a complete set of orthonormal eigenvectors

Original: U =
Revised: U =

Original: ( u 1 , u 2 , ..., u n ) , known as Laplacian eigenvectors.
Revised: ( u 1 , u 2 , ..., u n ) , known as Laplacian eigenvectors.

Original: These eigenvectors have associated real, non-negative eigenvalues { λ l } nl =1 , identiﬁed as the frequencies of graph.
Revised: These eigenvectors have associated real, non-negative eigenvalues { λ l } nl =1 , identiﬁed as the frequencies of graph.

Original: Eigenvectors associated with smaller eigenvalues carry slow varying signals, indicating that  nodesthat are neighbors share similar values.
Revised: Eigenvectors associated with smaller eigenvalues carry slow varying signals, indicating that connected nodes share similar values.

Original: In contrast, eigenvectors associated with larger eigenvalues carry faster varying signals across connected nodes.
Revised: In contrast, eigenvectors associated with larger eigenvalues carry faster varying signals across connected nodes.

Original: 2. G RAPH F OURIER TRANSFORM
Revised: 2. G RAPH F OURIER TRANSFORM

Original: Taking the eigenvectors of normalized Laplacian matrix as a set of bases, graph Fourier transform of a signal x ∈ R n on graph G is deﬁned as ˆ x = U (cid:62) x , and the inverse graph Fourier transform is x =
Revised: Taking the eigenvectors of normalized Laplacian matrix as a set of bases, graph Fourier transform of a signal x ∈ R n on graph G is deﬁned as ˆ x = U (cid:62) x , and the inverse graph Fourier transform is x =

Original: U ˆ x (Shuman et al., 2013).
Revised: U ˆ x (Shuman et al., 2013).

Original: Graph Fourier transform, according to convolution theorem, offers us a way to deﬁne the graph convolution operator, denoted as ∗ G .
Revised: Graph Fourier transform, according to convolution theorem, offers us a way to deﬁne the graph convolution operator, denoted as ∗ G .

Original: Denoting with y the convolution kernel, ∗ G is deﬁned as
Revised: Denoting with y the convolution kernel, ∗ G is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: where (cid:12) is the element-wise Hadamard product.
Revised: where (cid:12) is the element-wise Hadamard product.

Original: Replacing the vector U (cid:62) y by a diagonal matrix g θ , then Hadamard product can be written in the form of matrix multiplication.
Revised: Replacing the vector U (cid:62) y by a diagonal matrix g θ , then Hadamard product can be written in the form of matrix multiplication.

Original: Filtering the signal x by the ﬁlter g θ , we obtain the Equation (1) as U g θ U (cid:62) x .
Revised: Filtering the signal x by the ﬁlter g θ , we can write Equation (1) as U g θ U (cid:62) x .

Original: However, there are some limitations when using Fourier transform to implement graph convolution: (1) Eigendecomposition of Laplacian matrix to obtain Fourier basis U is of high computational cost with O ( n 3 ) ; (2) Graph Fourier transform is inefﬁcient, since it involves the multiplication between a dense matrix U and the signal x ; (3) Graph convolution deﬁned through Fourier transform is not localized in vertex domain, i.e., the inﬂuence to the signal on one node is not localized in its neighborhood.
Revised: However, there are some limitations when using Fourier transform to implement graph convolution: (1) Eigendecomposition of Laplacian matrix to obtain Fourier basis U is of high computational cost with O ( n 3 ) ; ( 2) Graph Fourier transform is inefﬁcient, since it involves the multiplication between a dense matrix U and the signal x ; (3) Graph convolution deﬁned through Fourier transform is not localized in vertex domain, i.e., the inﬂuence to the signal on one node is not localized in its neighborhood.

Original: To address these limitations, ChebyNet (Defferrard et al., 2016) restricts convolution kernel g θ to a polynomial expansion
Revised: To address these limitations, ChebyNet (Defferrard et al., 2016) restricts convolution kernel g θ to a polynomial expansion

Original: [Equation]
Revised: [Equation]

Original: where K is a hyper-parameter to determine the range of node neighborhoods via the shortest path distance, θ ∈ R K is a vector of polynomial coefﬁcients, and Λ =
Revised: where K is a hyper-parameter to determine the range of node neighborhoods via the shortest path distance, θ ∈ R K is a vector of polynomial coefﬁcients, and Λ =

Original: diag ( { λ l } nl =1 ) .
Revised: diag ( { λ l } nl =1 ) .

Original: However, such a polynomial approximation limits the ﬂexibility to deﬁne appropriate convolution on graph, i.e., with a smaller K , it’s hard to approximate the diagonal matrix g θ with n free parameters.
Revised: However, such a polynomial approximation limits the ﬂexibility to deﬁne appropriate convolution on graph, i.e., with a smaller K , it’s hard to approximate the diagonal matrix g θ with n free parameters.

Original: While with a larger K , locality is no longer guaranteed.
Revised: While with a larger K , locality is no longer guaranteed.

Original: Different from ChebyNet, we address the aforementioned three limitations through replacing graph Fourier transform with graph wavelet transform.
Revised: Different from ChebyNet, we address the aforementioned three limitations through replacing graph Fourier transform with graph wavelet transform.

Original: 2. G RAPH WAVELET TRANSFORM
Revised: 2. G RAPH WAVELET TRANSFORM

Original: Similar to graph Fourier transform, graph wavelet transform projects graph signal from vertex domain into spectral domain.
Revised: Similar to graph Fourier transform, graph wavelet transform projects graph signal from vertex domain into spectral domain.

Original: Graph wavelet transform employs a set of wavelets as basis, deﬁned as ψ s = ( ψ s 1 , ψ s 2 , ..., ψ sn ) , where each wavelet ψ si corresponds to a signal on graph diffused away from node i and s is a scaling parameter.
Revised: Graph wavelet transform employs a set of wavelets as bases, deﬁned as ψ s = ( ψ s 1 , ψ s 2 , ..., ψ sn ) , where each wavelet ψ si corresponds to a signal on graph diffused away from node i and s is a scaling parameter.

Original: Mathematically, ψ si can be written as
Revised: Mathematically, ψ si can be written as

Original: [Equation]
Revised: [Equation]

Original: ) where U is Laplacian eigenvectors, G s =diag (cid:0) g ( sλ 1 ) , ..., g ( sλ n ) (cid:1) is a scaling matrix and g ( sλ i )
Revised: where U is Laplacian eigenvectors, G s =diag (cid:0) g ( sλ 1 ) , ..., g ( sλ n ) (cid:1) is a scaling matrix and g ( sλ i )

Original: =
Revised: =

Original: e λ i s .
Revised: e λ i s .

Original: Using graph wavelets as basis, graph wavelet transform of a signal x on graph is deﬁned as ˆ x = ψ − 1 s x and the inverse graph wavelet transform is x = ψ s ˆ x .
Revised: Using graph wavelets as bases, graph wavelet transform of a signal x on graph is deﬁned as ˆ x = ψ − 1 s x and the inverse graph wavelet transform is x = ψ s ˆ x .

Original: Note that ψ − 1 s can be obtained by simply replacing the g ( sλ i ) in ψ s with g ( − sλ i ) corresponding to a heat kernel (Donnat et al., 2018).
Revised: Note that ψ − 1 s can be obtained by simply replacing the g ( sλ i ) in ψ s with g ( − sλ i ) corresponding to a heat kernel (Donnat et al., 2018).

Original: Replacing the graph Fourier transform in Equation (1) with graph wavelet transform, we obtain the graph convolution as
Revised: Replacing the graph Fourier transform in Equation (1) with graph wavelet transform, we obtain the graph convolution as

Original: [Equation]
Revised: [Equation]

Original: Compared to graph Fourier transform, graph wavelet transform has the following beneﬁts when being used to deﬁne graph convolution:
Revised: Compared to graph Fourier transform, graph wavelet transform has the following beneﬁts when being used to deﬁne graph convolution:

Original: High efﬁciency: graph wavelets can be obtained via a fast algorithm without requiring the eigendecomposition of Laplacian matrix.
Revised: High efﬁciency: graph wavelets can be obtained via a fast algorithm without requiring the eigendecomposition of Laplacian matrix.

Original: In Hammond et al.
Revised: In Hammond et al

Original: (2011), a method is proposed to use Chebyshev polynomials to efﬁciently approximate ψ s and ψ − 1 s , with the computationalcomplexity O
Revised: ), a method is proposed to use Chebyshev polynomials to efﬁciently approximate ψ s and ψ − 1 s , with the computational

Original:  ( m × | E | ) , where | E | is the number of edges and m is the order of Chebyshev polynomials.
Revised: complexity O ( m × | E | ) , where | E | is the number of edges and m is the order of Chebyshev polynomials

Original: High spareness: the matrix ψ s and ψ − 1 s are both sparse for real world networks, given that these networks are usually sparse.
Revised: High spareness: the matrix ψ s and ψ − 1 s are both sparse for real world networks, given that these networks are usually sparse.

Original: Therefore, graph wavelet transform is much more computationally efﬁcient than graph Fourier transform.
Revised: Therefore, graph wavelet transform is much more computationally efﬁcient than graph Fourier transform.

Original: For example, in the Cora dataset, more than 97% elements in ψ − 1 s are zero while only less than 1% elements in U (cid:62) are zero (Table 4).
Revised: For example, in the Cora dataset, more than 97% elements in ψ − 1 s are zero while only less than 1% elements in U (cid:62) are zero (Table 4).

Original: Localized convolution: each wavelet corresponds to a signal on graph diffused away from a centered node, highly localized in vertex domain.
Revised: Localized convolution: each wavelet corresponds to a signal on graph diffused away from a centered node, highly localized in vertex domain.

Original: As a result, the graph convolution deﬁned in Equation (4) is localized in vertex domain.
Revised: As a result, the graph convolution deﬁned in Equation (4) is localized in vertex domain.

Original: We show the localization property of graph convolution in Appendix A.
Revised: We show the localization property of graph convolution in Appendix A.

Original: It is the localization property that explains why graph wavelet transform outperforms Fourier transform in deﬁning graph convolution and the associated tasks like graph-based semisupervised learning.
Revised: It is the localization property that explains why graph wavelet transform outperforms Fourier transform in deﬁning graph convolution and the associated tasks like graph-based semisupervised learning [Figure]

Original: Flexible neighborhood: graph wavelets are more ﬂexible to adjust node neighborhoods.
Revised: Flexible neighborhood: graph wavelets are more ﬂexible to adjust node’s neighborhoods.

Original: Different from previous methods which constrain neighborhoods by the discrete shortest path distance, our method leverages a continuous manner, i.e., varying the scaling parameter s .
Revised: Different from previous methods which constrain neighborhoods by the discrete shortest path distance, our method leverages a continuous manner, i.e., varying the scaling parameter s .

Original: A small value of s generally corresponds to a smaller neighborhood.
Revised: A small value of s generally corresponds to a smaller neighborhood.

Original: Figure 1 shows two wavelet bases at different scale on an example network, depicted using GSP toolbox (Perraudin et al., 2014).[Figure]
Revised: Figure 1 shows two wavelet bases at different scale on an example network, depicted using GSP toolbox (Perraudin et al., 2014).

Original: 2. G RAPH WAVELET NEURAL NETWORK
Revised: 2. G RAPH WAVELET NEURAL NETWORK

Original: Replacing Fourier transform with wavelet transform, graph wavelet neural network (GWNN) is a multi-layer convolutional neural network.
Revised: Replacing Fourier transform with wavelet transform, graph wavelet neural network (GWNN) is a multi-layer convolutional neural network.

Original: The structure of the m -th layer is
Revised: The structure of the m -th layer is

Original: [Equation]
Revised: [Equation]

Original: where ψ s is wavelet bases, ψ − 1 s is the graph wavelet transform matrix at scale s which projects signal in vertex domain into spectral domain, X m [: ,i ] with dimensions n × 1 is the i -th column of
Revised: where ψ s is wavelet bases, ψ − 1 s is the graph wavelet transform matrix at scale s which projects signal in vertex domain into spectral domain, X m [: ,i ] with dimensions n × 1 is the i -th column of

Original: X m , F mi,j is a diagonal ﬁlter matrix learned in spectral domain, and h is a non-linear activation function.
Revised: X m , F mi,j is a diagonal ﬁlter matrix learned in spectral domain, and h is a non-linear activation function.

Original: This layer transforms an input tensor X m with dimensions n × p into an output tensor
Revised: This layer transforms an input tensor X m with dimensions n × p into an output tensor

Original: X m +1 with dimensions n × q .
Revised: X m +1 with dimensions n × q .

Original: In the paper, we consider a two-layer GWNN for semi-supervised node classiﬁcation on graph.
Revised: In this paper, we consider a two-layer GWNN for semi-supervised node classiﬁcation on graph.

Original: The formulation of our model is:
Revised: The formulation of our model is ﬁrst

Revised: [Equation]

Revised: second layer :

Original: [Equation]
Revised: [Equation]

Original: where c is the number of classes in node classiﬁcation, Z of dimensions n × c is the prediction result.
Revised: where c is the number of classes in node classiﬁcation, Z of dimensions n × c is the prediction result.

Original: The loss function is the cross-entropy error over all labeled examples:
Revised: The loss function is the cross-entropy error over all labeled examples:

Original: [Equation]
Revised: [Equation]

Original: where y L is the labeled node set, Y li
Revised: where y L is the labeled node set, Y li

Original: = 1 if the label of node l is i , and Y li = 0 otherwise.
Revised: = 1 if the label of node l is i , and Y li = 0 otherwise.

Original: The weights F are trained using gradient descent.
Revised: The weights F are trained using gradient descent.

Original: 2. R EDUCING P ARAMETER C OMPLEXITY
Revised: 2. R EDUCING P ARAMETER C OMPLEXITY

Original: In Equation (5), the parameter complexity of each layer is O ( n × p × q ) , where n is the number of nodes, p is the number of features of each vertex in current layer, and q is the number of features of each vertex in next layer.
Revised: In Equation (5), the parameter complexity of each layer is O ( n × p × q ) , where n is the number of nodes, p is the number of features of each vertex in current layer, and q is the number of features of each vertex in next layer.

Original: Conventional CNN methods learn convolution kernel for each pair of input feature and output feature.
Revised: Conventional CNN methods learn convolution kernel for each pair of input feature and output feature.

Original: This results in a huge number of parameters and generally requires huge training data for parameter learning.
Revised: This results in a huge number of parameters and generally requires huge training data for parameter learning.

Original: This is prohibited for graph-based semi-supervised learning.
Revised: This is prohibited for graph-based semi-supervised learning.

Original: To combat this issue, we detach the feature transformation from graph convolution.
Revised: To combat this issue, we detach the feature transformation from graph convolution.

Original: Each layer in
Revised: Each layer in

Original: GWNN is divided into two components: feature transformation and graph convolution.
Revised: GWNN is divided into two components: feature transformation and graph convolution.

Original: Spectially, we have
Revised: Spectially, we have

Original: [Equation]
Revised: [Equation]

Original: where W ∈ R p × q is the parameter matrix for feature transformation, X m (cid:48) with dimensions n × q is the feature matrix after feature transformation, F m is the diagonal matrix for graph convolution kernel, and h is a non-linear activation function.
Revised: where W ∈ R p × q is the parameter matrix for feature transformation, X m (cid:48) with dimensions n × q is the feature matrix after feature transformation, F m is the diagonal matrix for graph convolution kernel, and h is a non-linear activation function.

Original: After detaching feature transformation from graph convolution, the parameter complexity is reduced from O ( n × p × q ) to O ( n + p × q ) .
Revised: After detaching feature transformation from graph convolution, the parameter complexity is reduced from O ( n × p × q ) to O ( n + p × q ) .

Original: The reduction of parameters are particularly valuable fro graphbased semi-supervised learning where labels are quite limited.
Revised: The reduction of parameters is particularly valuable fro graphbased semi-supervised learning where labels are quite limited.

Original:  R ELATED WORKS
Revised:  R ELATED WORKS

Original: Graph convolutional neural networks on graphs.
Revised: Graph convolutional neural networks on graphs.

Original: The success of CNNs when dealing with images, videos, and speeches motivates researchers to design graph convolutional neural network on graphs.
Revised: The success of CNNs when dealing with images, videos, and speeches motivates researchers to design graph convolutional neural network on graphs.

Original: The key of generalizing CNNs to graphs is deﬁning convolution operator on graphs.
Revised: The key of generalizing CNNs to graphs is deﬁning convolution operator on graphs.

Original: Existing methods are classiﬁed into two categories, i.e., spectral methods and spatial methods.
Revised: Existing methods are classiﬁed into two categories, i.e., spectral methods and spatial methods.

Original: Spectral methods deﬁne convolution via convolution theorem.
Revised: Spectral methods deﬁne convolution via convolution theorem.

Original: Spectral CNN (Bruna et al., 2014) is the ﬁrst attempt at implementing CNNs on graphs, leveraging graph Fourier transform and deﬁning convolution kernel in spectral domain.
Revised: Spectral CNN (Bruna et al., 2014) is the ﬁrst attempt at implementing CNNs on graphs, leveraging graph Fourier transform and deﬁning convolution kernel in spectral domain.

Original: Boscaini et al.
Revised: Boscaini et al.

Original: (2015) developed a local spectral CNN approach based on the graph Windowed Fourier Transform.
Revised: (2015) developed a local spectral CNN approach based on the graph Windowed Fourier Transform.

Original: Defferrard et al.
Revised: Defferrard et al.

Original: (2016) introduced a
Revised: (2016) introduced a

Original: Chebyshev polynomial parametrization for spectral ﬁlter, offering us a fast localized spectral ﬁltering method.
Revised: Chebyshev polynomial parametrization for spectral ﬁlter, offering us a fast localized spectral ﬁltering method.

Original: Kipf & Welling (2017) provided a simpliﬁed version of ChebyNet, gaining success in graph semi-supervised learning task.
Revised: Kipf & Welling (2017) provided a simpliﬁed version of ChebyNet, gaining success in graph-based semi-supervised learning task.

Original: Khasanova & Frossard (2017) represented images as signals on graph and learn its transformation invariant representations.
Revised: Khasanova & Frossard (2017) represented images as signals on graph and learned their transformation invariant representations.

Original: They use Chebyshev approximations to implement graph convolution, avoiding matrix eigendecomposition.
Revised: They used Chebyshev approximations to implement graph convolution, avoiding matrix eigendecomposition.

Original: Levie et al.
Revised: Levie et al.

Original: (2017) used rational functions instead of polynomials and created anisotropic spectral ﬁlters on manifolds.
Revised: (2017) used rational functions instead of polynomials and created anisotropic spectral ﬁlters on manifolds.

Original: Spatial methods deﬁne convolution as a weighted average function over neighborhood of target vertex.
Revised: Spatial methods deﬁne convolution as a weighted average function over neighborhood of target vertex.

Original: GraphSAGE takes one-hop neighbor as neighborhood and deﬁnes the weighting function as various aggregators over neighborhood (Hamilton et al., 2017).
Revised: GraphSAGE takes one-hop neighbors as neighborhoods and deﬁnes the weighting function as various aggregators over neighborhood (Hamilton et al., 2017).

Original: Graph attention network (GAT) proposes to learn the weighting function via self-attention mechanism (Velickovic et al., 2017).
Revised: Graph attention network (GAT) proposes to learn the weighting function via self-attention mechanism (Velickovic et al., 2017).

Original: MoNet offers us a general framework for design spatial methods, taking convolution as the weighted average of multiple weighting functions deﬁned over neighborhood (Monti et al., 2017).
Revised: MoNet offers us a general framework for design spatial methods, taking convolution as the weighted average of multiple weighting functions deﬁned over neighborhood (Monti et al., 2017).

Original: Some works devote to making graph convolutional networks more powerful.
Revised: Some works devote to making graph convolutional networks more powerful.

Original: Monti et al.
Revised: Monti et al.

Original: (2018) alternated convolutions on vertices and edges, generalizing GAT and leading to better performance.
Revised: (2018) alternated convolutions on vertices and edges, generalizing GAT and leading to better performance.

Original: GraphsGAN (Ding et al., 2018) generalizes GANs to graph, and generates fake samples in low-density areas between subgraphs to improve the performance on graph-based semi-supervised learning.
Revised: GraphsGAN (Ding et al., 2018) generalizes GANs to graph, and generates fake samples in low-density areas between subgraphs to improve the performance on graph-based semi-supervised learning.

Original: Graph wavelets.
Revised: Graph wavelets.

Original: Sweldens (1998) presented a lifting scheme, a simple construction of wavelets that can be adapted to graphs without learning process.
Revised: Sweldens (1998) presented a lifting scheme, a simple construction of wavelets that can be adapted to graphs without learning process.

Original: Hammond et al.
Revised: Hammond et al.

Original: (2011) proposed a method to construct wavelet transform on graphs.
Revised: (2011) proposed a method to construct wavelet transform on graphs.

Original: Moreover, they designed an efﬁcient way to bypass the eigendecomposition of the Laplacian and approximated wavelets with Chebyshev polynomials.
Revised: Moreover, they designed an efﬁcient way to bypass the eigendecomposition of the Laplacian and approximated wavelets with Chebyshev polynomials.

Original: Tremblay & Borgnat (2014) leveraged graph wavelets for multi-scale community mining by modulating a scaling parameter.
Revised: Tremblay & Borgnat (2014) leveraged graph wavelets for multi-scale community mining by modulating a scaling parameter.

Original: Owing to the property of describing information diffusion, Donnat et al.
Revised: Owing to the property of describing information diffusion, Donnat et al.

Original: (2018) learned structural node embeddings via wavelets.
Revised: (2018) learned structural node embeddings via wavelets.

Original: All these works prove that graph wavelets are not only local and sparse but also valuable for signal processiong on graph.
Revised: All these works prove that graph wavelets are not only local and sparse but also valuable for signal processiong on graph.

Original:  E XPERIMENTS 4. D ATASETS
Revised:  E XPERIMENTS 4. D ATASETS

Original: To evaluate the proposed GWNN, we apply GWNN on semi-supervised node classiﬁcation, and conduct experiments on three benchmark datasets, namely, Cora, Citeseer and Pubmed (Sen et al., 2008).
Revised: To evaluate the proposed GWNN, we apply GWNN on semi-supervised node classiﬁcation, and conduct experiments on three benchmark datasets, namely, Cora, Citeseer and Pubmed (Sen et al., 2008).

Original: In these citation network datasets, nodes represent documents and edges are citation links.
Revised: In the three citation network datasets, nodes represent documents and edges are citation links.

Original: Details of these datasets are demonstrated in Table 1.
Revised: Details of these datasets are demonstrated in Table 1.

Original: Here, the label rate denotes the proportion of labeled nodes used for training.
Revised: Here, the label rate denotes the proportion of labeled nodes used for training.

Original: Following the experimental setup of GCN (Kipf & Welling, 2017), we fetch 20 labeled nodes per class in each dataset to train the model.[Table]
Revised: Following the experimental setup of GCN (Kipf & Welling, 2017), we fetch 20 labeled nodes per class in each dataset to train the model.

Original: [Table]
Revised: [Table]

Original:  B ASELINES
Revised: 4. B ASELINES

Original: We compare with several traditional semi-supervised learning methods, including label propagation (LP) (Zhu et al., 2003), semi-supervised embedding (SemiEmb) (Weston et al., 2012), manifold regularization (ManiReg) (Belkin et al., 2006), graph embeddings (DeepWalk) (Perozzi et al., 2014), iterative classiﬁcation algorithm (ICA)
Revised: We compare with several traditional semi-supervised learning methods, including label propagation (LP) (Zhu et al., 2003), semi-supervised embedding (SemiEmb) (Weston et al., 2012), manifold regularization (ManiReg) (Belkin et al., 2006), graph embeddings (DeepWalk) (Perozzi et al., 2014), iterative classiﬁcation algorithm (ICA)

Original: (Lu & Getoor, 2003) and Planetoid (Yang et al., 2016).
Revised: (Lu & Getoor, 2003) and Planetoid (Yang et al., 2016).

Original: Furthermore, along with the development of deep learning on graph, graph convolutional networks are proved to be effective in semi-supervised learning.
Revised: Furthermore, along with the development of deep learning on graph, graph convolutional networks are proved to be effective in semi-supervised learning.

Original: Since our method is a spectral method based on convolution theorem, we compare it with the Spectral CNN (Bruna et al., 2014).
Revised: Since our method is a spectral method based on convolution theorem, we compare it with the Spectral CNN (Bruna et al., 2014).

Original: ChebyNet (Defferrard et al., 2016) and GCN (Kipf & Welling, 2017) are also included as our baselines, which are the variant of Spectral CNN.
Revised: ChebyNet (Defferrard et al., 2016) and GCN (Kipf & Welling, 2017), two variants of the Spectral CNN, are also included as our baselines.

Original: Considering spatial methods, we take MoNet (Monti et al., 2017) as our baseline, which also depends on Laplacian matrix.
Revised: Considering spatial methods, we take MoNet (Monti et al., 2017) as our baseline, which also depends on Laplacian matrix.

Original: 4. E XPERIMENTAL S ETTINGS
Revised: 4. E XPERIMENTAL S ETTINGS

Original: We train a two-layer graph wavelet neural network with 16 hidden units, and prediction accuracy is evaluated on a test set of 1000 labeled samples.
Revised: We train a two-layer graph wavelet neural network with 16 hidden units, and prediction accuracy is evaluated on a test set of 1000 labeled samples.

Original: The partition of datasets is the same as
Revised: The partition of datasets is the same as

Original: GCN (Kipf & Welling, 2017) with an additional validation set of 500 labeled samples to determine hyper-parameters.
Revised: GCN (Kipf & Welling, 2017) with an additional validation set of 500 labeled samples to determine hyper-parameters.

Original: Weights are initialized following Glorot & Bengio (2010).
Revised: Weights are initialized following Glorot & Bengio (2010).

Original: We adopt the Adam optimizer (Kingma & Ba, 2014) for parameter optimization with an initial learning rate lr = 0 .
Revised: We adopt the Adam optimizer (Kingma & Ba, 2014) for parameter optimization with an initial learning rate lr = 0 .

Original: For computational efﬁciency, we set the elements of ψ s and ψ − 1 s smaller than a threshold t to 0.
Revised: For computational efﬁciency, we set the elements of ψ s and ψ − 1 s smaller than a threshold t to 0.

Original: We ﬁnd the optimal hyper-parameters s and t through grid search, and the detailed discussion about the two hyperparameters is introduced in Appendix B. For Cora, s = 1 . 0 and t = 1 e − 4 .
Revised: We ﬁnd the optimal hyper-parameters s and t through grid search, and the detailed discussion about the two hyperparameters is introduced in Appendix B. For Cora, s = 1 . 0 and t = 1 e − 4 .

Original: For Citeseer, s = 0 .and
Revised: For Citeseer, s = 0 .and

Original: t = 1 e − 5 .
Revised: t = 1 e − 5 .

Original: For Pubmed, s = 0 .
Revised: For Pubmed, s = 0 .

Original: 5 and t = 1 e − 7 .
Revised: 5 and t = 1 e − 7 .

Original: To avoid overﬁtting, dropout (Srivastava et al., 2014) is applied.
Revised: To avoid overﬁtting, dropout (Srivastava et al., 2014) is applied.

Original: Meanwhile, we terminate the training if the validation loss does not decrease for 100 consecutive epochs.
Revised: Meanwhile, we terminate the training if the validation loss does not decrease for 100 consecutive epochs.

Original: 4. A NALYSIS ON DETACHING FEATURE TRANSFORMATION FROM CONVOLUTION
Revised: 4. A NALYSIS ON DETACHING FEATURE TRANSFORMATION FROM CONVOLUTION

Original: Since the parameters of  undetached version of GWNN is O ( n × p × q ) , we can hardly implement this version in the case of networks with a large number n of nodes and a huge number p of input features.
Revised: Since the number of parameters for the undetached version of GWNN is O ( n × p × q ) , we can hardly implement this version in the case of networks with a large number n of nodes and a huge number p of input features.

Original: Here, we validate the effectiveness of detaching feature transformation form convolution on ChebyNet (introduced in Section 2.2), whose parameter complexity is O
Revised: Here, we validate the effectiveness of detaching feature transformation form convolution on ChebyNet (introduced in Section 2.2), whose parameter complexity is O

Original: ( K × p × q ) .
Revised: ( K × p × q ) .

Original: For ChebyNet of detaching feature transformation from graph convolution, the number of parameters is reduced to O
Revised: For ChebyNet of detaching feature transformation from graph convolution, the number of parameters is reduced to O

Original: ( K + p × q ) .
Revised: ( K + p × q ) .

Original: Table 2 shows the performance and the number of parameters on three datasets.
Revised: Table 2 shows the performance and the number of parameters on three datasets.

Original: Here, the reported performance is the optimal performance varying the order K = 2 , 3 , 4 .
Revised: Here, the reported performance is the optimal performance varying the order K = 2 , 3 , 4 .

Original: [Table]
Revised: [Table]

Original: As demonstrated in Table

Original: [Table] with fewer parameters [Table] we improve the accuracy on Pubmed by a large margin.
Revised: As demonstrated in Table 2, with fewer parameters, we improve the accuracy on Pubmed by a large margin.

Original: This  due to that the label rate of Pubmed is only 0 .003 .
Revised: This is due to that the label rate of Pubmed is only 0 .

Original: By detaching feature transformation from convolution, the parameter complexity is signiﬁcantly reduced, alleviating overﬁtting in semisupervised learning and thus greatly improving prediction accuracy.
Revised: By detaching feature transformation from convolution, the parameter complexity is signiﬁcantly reduced, alleviating overﬁtting in semi-supervised learning and thus remarkably improving prediction accuracy.

Original: On Citeseer, there is a little drop on the accuracy
Revised: On Citeseer, there is a little drop on the accuracy.

Original: [Table] which implies that reducing the number of parameters may restrict the modeling capacity to some degree.[Table]
Revised: One possible explanation is that reducing the number of parameters may restrict the modeling capacity to some degree.

Original: 4. P ERFORMANCE OF GWNN
Revised: 4. P ERFORMANCE OF GWNN

Original: We now validate the effectiveness of GWNN with detaching technique on node classiﬁcation.
Revised: We now validate the effectiveness of GWNN with detaching technique on node classiﬁcation.

Original: Experimental results are reported in Table 3.
Revised: Experimental results are reported in Table 3.

Original: GWNN improves the classiﬁcation accuracy on all the three  datasets.
Revised: GWNN improves the classiﬁcation accuracy on all the three [Table] datasets.

Original: In particular, replacing Fourier transform with wavelet transform, the proposed GWNN is comfortably ahead of Spectral CNN, achieving 10% improvement on Cora and Citeseer, and 5% improvement in Pubmed.
Revised: In particular, replacing Fourier transform with wavelet transform [Table] the proposed GWNN is comfortably ahead of Spectral CNN, achieving 10% improvement on Cora and Citeseer, and 5% improvement on Pubmed.

Original: The large improvement could be explained from two perspectives: (1)
Revised: The large improvement could be explained from two perspectives: (1)

Original: Convolution in Spectral CNN is non-local in vertex domain, and thus, the range of feature diffusion is not restricted to neighboring nodes. The scaling parameter s of wavelet transform is ﬂexible to adjust the diffusion range to suit different applications and different networks.
Revised: Convolution in Spectral CNN is non-local in vertex domain, and thus the range of feature diffusion is not restricted to neighboring nodes; (2) The scaling parameter s of wavelet transform is ﬂexible to adjust the diffusion range to suit different applications and different networks.

Original: GWNN consistently outperforms ChebyNet, since it has enough degree of freedom to learn the convolution kernel, while
Revised: GWNN consistently outperforms ChebyNet, since it has enough degree of freedom to learn the convolution kernel, while

Original: ChebyNet is a kind of approximation with limited degree of freedom.
Revised: ChebyNet is a kind of approximation with limited degree of freedom.

Original: Furthermore, our GWNN also performs better than GCN and MoNet, reﬂecting that it is promising to design appropriate bases for spectral methods to achieve good performance.
Revised: Furthermore, our GWNN also performs better than GCN and MoNet, reﬂecting that it is promising to design appropriate bases for spectral methods to achieve good performance.

Original: 4. A NALYSIS ON SPARSITY
Revised: 4. A NALYSIS ON SPARSITY

Original: Besides the improvement on prediction accuracy, wavelet transform with localized and sparse transform matrix holds sparsity in both spatial domain and spectral domain.
Revised: Besides the improvement on prediction accuracy [Table] wavelet transform with localized and sparse transform matrix holds sparsity in both spatial domain and spectral domain.

Original: Here, we take Cora as an example to illustrate the sparsity of graph wavelet transform.
Revised: Here, we take Cora as an example to illustrate the sparsity of graph wavelet transform.

Original: The sparsity of transform matrix.
Revised: The sparsity of transform matrix.

Original: There are 2,708 nodes in Cora.
Revised: There are 2,708 nodes in Cora.

Original: Thus, the wavelet transform matrix ψ − 1 s and the Fourier transform matrix U (cid:62) both belong to R 2 , 708 × 2 , 708 .
Revised: Thus, the wavelet transform matrix ψ − 1 s and the Fourier transform matrix U (cid:62) both belong to R 2 , 708 × 2 , 708 .

Original: The ﬁrst two rows in Table 4 demonstrate that ψ − 1 s is much sparser than U (cid:62) .
Revised: The ﬁrst two rows in Table 4 demonstrate that ψ − 1 s is much sparser than U (cid:62) .

Original: Sparse wavelet basis not only accelerates the computation, but also well captures the neighboring topology centered at each node.
Revised: Sparse wavelets not only accelerate the computation, but also well capture the neighboring topology centered at each node.

Original: The sparsity of projected signal.
Revised: The sparsity of projected signal.

Original: As mentioned above, each node in Cora represents a document and has a sparse bag-of-words feature.
Revised: As mentioned above, each node in Cora represents a document and has a sparse bag-of-words feature.

Original: The input feature X ∈ R n × p is a binary matrix, and X [ i,j ]
Revised: The input feature X ∈ R n × p is a binary matrix, and X [ i,j ]

Original: = 1 when the i -th document contains the j -th word in the bag of words, it equals 0 otherwise.
Revised: = 1 when the i -th document contains the j -th word in the bag of words, it equals 0 otherwise.

Original: Here,
Revised: Here,

Original: X [: ,j ] denotes the j -th column of X , and it represents the feature vector of a word.
Revised: X [: ,j ] denotes the j -th column of X , and each column represents the feature vector of a word.

Original: Considering a speciﬁc signal X [: , 984] , we project the spatial signal into spectral domain with domain transform, and get the projected vector of each basis matrix.
Revised: Considering a speciﬁc signal X [: , 984] , we project the spatial signal into spectral domain, and get its projected vector.

Original: Here, p = ψ − 1 s X
Revised: Here, p = ψ − 1 s X

Original: [: , 984] denotes the projected vector of wavelet basis, q = U (cid:62)
Revised: [: , 984] denotes the projected vector via wavelet transform, q = U (cid:62)

Original: X
Revised: X

Original: [: , 984] denotes the projected vector of Fourier basis, and p , q ∈ R 2 , 708 .
Revised: [: , 984] denotes the projected vector via Fourier transform, and p , q ∈ R 2 , 708 .

Original: The last row in Table 4 lists the numbers of non-zero elements in p and q .
Revised: The last row in Table 4 lists the numbers of non-zero elements in p and q .

Original: As shown in Table 4, with wavelet transform, the projected signal is much sparser.
Revised: As shown in Table 4, with wavelet transform, the projected signal is much sparser.

Original: [Table]
Revised: [Table]

Original: 4. A NALYSIS ON I NTERPRETABILITY
Revised: 4. A NALYSIS ON I NTERPRETABILITY

Original: Compare with graph  Fourier transform, GWNN provides good interpretability.
Revised: Compare with graph convolution network using Fourier transform, GWNN provides good interpretability.

Original: Here, we show the interpretability with speciﬁc examples in Cora.
Revised: Here, we show the interpretability with speciﬁc examples in Cora.

Original: Each feature, i.e. word in the bag of words, has a projected vector, and each item in this vector is associated with a spectral wavelet basis.
Revised: Each feature, i.e. word in the bag of words, has a projected vector, and each element in this vector is associated with a spectral wavelet basis.

Original: Here, each basis is centered at a node, corresponding to a document.
Revised: Here, each basis is centered at a node, corresponding to a document.

Original: The value can be regarded as the relation between the word and the document.
Revised: The value can be regarded as the relation between the word and the document.

Original: Thus, each value in p can be interpreted as the relation between Word 984 and a document.
Revised: Thus, each value in p can be interpreted as the relation between Word 984 and a document.

Original: In order to elaborate the interpretability of wavelet transform, we analyze the projected values of different feature as following.
Revised: In order to elaborate the interpretability of wavelet transform, we analyze the projected values of different feature as following.

Original: Considering two features Word 984 and Word 1177 , we select the top-10 active bases, which have the 10 largest projected values of each feature.
Revised: Considering two features Word 984 and Word 1177 , we select the top-10 active bases, which have the 10 largest projected values of each feature.

Original: As illustrated in Figure 2, to present clearly, we magnify the local structure of corresponding nodes and marked them with bold rims.
Revised: As illustrated in Figure 2, for clarity, we magnify the local structure of corresponding nodes and marked them with bold rims.

Original: The central network in each subgraph denotes the dataset Cora, each node represents a document, and 7 different colors classes
Revised: The central network in each subgraph denotes the dataset Cora, each node represents a document, and 7 different colors represent 7 classes.

Original: These  are  al., 2011) 
Revised: These nodes are clustered by OpenOrd (Martin et al., 2011) based on the adjacency matrix.

Original: Figure 2a shows the top- 10 active bases of Word 984 .
Revised: Figure 2a shows the top- 10 active bases of Word 984 .

Original: In Cora, this word only appears 8 times, and all the documents containing Word 984 belong to the class “ Case-Based ”.
Revised: In Cora, this word only appears 8 times, and all the documents containing Word 984 belong to the class “ Case-Based ”.

Original: Consistently, all top- 10 nodes activated by Word 984 are concentrated and belong to the class “ Case-Based ”.
Revised: Consistently, all top- 10 nodes activated by Word 984 are concentrated and belong to the class “ Case-Based ”.

Original: And, the frequencies of Word 1177 appearing in different classes are similar, indicating that Word 1177 is a universal word.
Revised: And, the frequencies of Word 1177 appearing in different classes are similar, indicating that Word 1177 is a universal word.

Original: In concordance with our expectation, the top- 10 active bases of Word 1177 are discrete and belong to different classes in Figure 2b.
Revised: In concordance with our expectation, the top- 10 active bases of Word 1177 are discrete and belong to different classes in Figure 2b.

Original: [Figure]
Revised: [Figure]

Original:  Owing to the properties of graph wavelets, which describe the neighboring topology centered at each node the projected values of wavelet transform  be 
Revised: also belong to different classes. Owing to the properties of graph wavelets, which describe the neighboring topology centered at each node, the projected values of wavelet transform can be explained as the correlation between features and nodes.

Revised: These properties provide an interpretable domain transformation and ease the understanding of graph convolution.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: Replacing graph Fourier transform with  wavelet transform we proposed GWNN.
Revised: Replacing graph Fourier transform with graph wavelet transform, we proposed GWNN.

Original: Graph wavelet transform has three desirable properties: (1) Graph wavelets are local and sparse; (2) Scaling parameter can ﬂexibly modulate the range of feature diffusion; (3) Convolution is localized in vertex domain while retaining the ﬂexibility of kernel.
Revised: Graph wavelet transform has three desirable properties: (1) Graph wavelets are local and sparse; (2) Graph wavelet transform is computationally efﬁcient; (3) Convolution is localized in vertex domain.

Original: These advantages make the whole learning processbecome more interpretable and efﬁcient.
Revised: These advantages make the whole learning process interpretable and efﬁcient.

Original: Moreover, to reduce the number of parameters and the dependence on huge training data, we detached the feature transformation from convolution.
Revised: Moreover, to reduce the number of parameters and the dependence on huge training data, we detached the feature transformation from convolution.

Original: This pratice makes GWNN applicable to large graphs, with remarkable performance improvement on graph-based semi-supervised learning
Revised: This practice makes GWNN applicable to large graphs, with remarkable performance improvement on graph-based semi-supervised learning.  A CKNOWLEDGEMENTS

Revised: This work is funded by the National Natural Science Foundation of China under grant numbers 61425016, 61433014, and 91746301.

Revised: Huawei Shen is also funded by K.C. Wong Education Foundation and the Youth Innovation Promotion Association of the Chinese Academy of Sciences.

Original: A PPENDIX A L OCALIZED GRAPH CONVOLUTION VIA WAVELET TRANSFORM
Revised: A PPENDIX A L OCALIZED GRAPH CONVOLUTION VIA WAVELET TRANSFORM

Original: We use a diagonal matrix Θ to represent the learned kernel transformed by wavelets ψ − 1 s y , and replace the Hadamard product with matrix muplication.
Revised: We use a diagonal matrix Θ to represent the learned kernel transformed by wavelets ψ − 1 s y , and replace the Hadamard product with matrix muplication.

Original: Then Equation (4) is:
Revised: Then Equation (4) is:

Original: [Equation]
Revised: [Equation]

Original: We set ψ s = ( ψ s 1 , ψ s 2 , ..., ψ sn ) , ψ − 1 s = ( ψ ∗ s 1 , ψ ∗ s 2 , ..., ψ ∗ sn ) , and Θ = diag( { θ k } nk =1 ) .
Revised: We set ψ s = ( ψ s 1 , ψ s 2 , ..., ψ sn ) , ψ − 1 s = ( ψ ∗ s 1 , ψ ∗ s 2 , ..., ψ ∗ sn ) , and Θ = diag( { θ k } nk =1 ) .

Original: Equation (11) becomes :
Revised: Equation (11) becomes :

Original: [Equation]
Revised: [Equation]

Original: As proved by Hammond et al.
Revised: As proved by Hammond et al.

Original: (2011), both ψ s and ψ − 1 s are local in small scale ( s ).
Revised: (2011), both ψ s and ψ − 1 s are local in small scale ( s ).

Original: Figure 3 shows the locality of ψ s 1 and ψ ∗ s 1 , i.e., the ﬁrst column in ψ s and ψ − 1 s when s = 3 .
Revised: Figure 3 shows the locality of ψ s 1 and ψ ∗ s 1 , i.e., the ﬁrst column in ψ s and ψ − 1 s when s = 3 .

Original: Each column in ψ s and ψ − 1 s describes the neighboring topology of target node, which means that ψ s and ψ − 1 s are local.
Revised: Each column in ψ s and ψ − 1 s describes the neighboring topology of target node, which means that ψ s and ψ − 1 s are local.

Original: The locality of ψ sk and ψ ∗ sk leads to the locality of the resulting matrix of multiplication between the column vector ψ sk and row vector ( ψ ∗ sk ) (cid:62) .
Revised: The locality of ψ sk and ψ ∗ sk leads to the locality of the resulting matrix of multiplication between the column vector ψ sk and row vector ( ψ ∗ sk ) (cid:62) .

Original: For convenience, we set M k =
Revised: For convenience, we set M k =

Original: ψ sk ( ψ ∗ sk ) (cid:62) , M k
Revised: ψ sk ( ψ ∗ sk ) (cid:62) , M k

Original: [ i,j ] >only when ψ sk [ i ] > 0 and ( ψ ∗ sk ) (cid:62) [ j ] > 0 .
Revised: [ i,j ] >only when ψ sk [ i ] > 0 and ( ψ ∗ sk ) (cid:62) [ j ] > 0 .

Original: In other words, if M k [ i,j ] > 0 , vertex i and vertex j can correlate with each other through vertex k .
Revised: In other words, if M k [ i,j ] > 0 , vertex i and vertex j can correlate with each other through vertex k .

Original: [Figure]
Revised: [Figure]

Original: Since each M k is local, for any convolution kernel Θ , ψ s Θ ψ − 1 s is local, and it means that convolution is localized in vertex domain.
Revised: Since each M k is local, for any convolution kernel Θ , ψ s Θ ψ − 1 s is local, and it means that convolution is localized in vertex domain.

Original: By replacing Θ with an identity matrix in Equation (12), we get x ∗ G y =
Revised: By replacing Θ with an identity matrix in Equation (12), we get x ∗ G y =

Original: (cid:80)
Revised: (cid:80)

Original: nk
Revised: nk

Original: =1 M k x .
Revised: =1 M k x .

Original: We deﬁne H
Revised: We deﬁne H

Original: =
Revised: =

Original: (cid:80)
Revised: (cid:80)

Original: nk =1 M k , and Figure 4 shows H [1 , :] in different scaling, i.e., correlation between the ﬁrst node and other nodes during convolution.
Revised: nk =1 M k , and Figure 4 shows H [1 , :] in different scaling, i.e., correlation between the ﬁrst node and other nodes during convolution.

Original: The locality of H suggests that graph convolution is localized in vertex domain.
Revised: The locality of H suggests that graph convolution is localized in vertex domain.

Original: Moreover, as the scaling parameter s becomes larger, the range of feature diffusion becomes larger.
Revised: Moreover, as the scaling parameter s becomes larger, the range of feature diffusion becomes larger.

Original: [Figure]
Revised: [Figure]

Revised: [Figure]

Original: Locality of H suggests that graph convolution is localized in vertex domain.
Revised: Locality of H suggests that graph convolution is localized in vertex domain.

Original: Moreover, with scaling parameter s becoming larger, the range of feature diffusion becomes larger.
Revised: Moreover, with scaling parameter s becoming larger, the range of feature diffusion becomes larger.

Original: A PPENDIX B I NFLUENCE OF H YPER - PARAMETERS
Revised: A PPENDIX B I NFLUENCE OF H YPER - PARAMETERS

Original: GWNN leverages graph wavelets to implement graph convolution, where s is used to modulate the range of neighborhoods.
Revised: GWNN leverages graph wavelets to implement graph convolution, where s is used to modulate the range of neighborhoods.

Original: From Figure 5, as s becomes larger starting from 0, the range of neighboring nodes becomes large, resulting the increase of accuracy.
Revised: From Figure 5, as s becomes larger starting from 0, the range of neighboring nodes becomes large, resulting the increase of accuracy on Cora.

Original: However when s becomes too large, some irrelevant nodes are included, leading to decreasing of accuracy.
Revised: However when s becomes too large, some irrelevant nodes are included, leading to decreasing of accuracy.

Original: The hyperparameter t only used for computational efﬁciency, has any slight inﬂuence on its performance.
Revised: The hyperparameter t only used for computational efﬁciency, has any slight inﬂuence on its performance.

Original: For experiments on speciﬁc dataset, s and t are choosen via grid search using validation.
Revised: For experiments on speciﬁc dataset, s and t are choosen via grid search using validation.

Original: Generally, a appropriate s is in the range of [0 . 5 , 1] , which can not only capture the graph structure but also guarantee the locality of convolution, and t is less insensive to dataset.
Revised: Generally, a appropriate s is in the range of [0 . 5 , 1] , which can not only capture the graph structure but also guarantee the locality of convolution, and t is less insensive to dataset.

Original: A PPENDIX C P ARAMETER COMPLEXITY OF N ODE C LASSIFICATION
Revised: A PPENDIX C P ARAMETER COMPLEXITY OF N ODE C LASSIFICATION

Original: We show the parameter complexity of node classiﬁcation in Table 5.
Revised: We show the parameter complexity of node classiﬁcation in Table 5.

Original: The high parameter complexity O ( n ∗ p ∗ q ) of Spectral CNN makes it difﬁcult to generalize to real world networks.
Revised: The high parameter complexity O ( n ∗ p ∗ q ) of Spectral CNN makes it difﬁcult to generalize to real world networks.

Original: ChebyNet approximates the convolution kernel via polynomial function of the diagonal matrix of Laplacian eigenvalues, reducing parameter complexity to O ( K ∗ p ∗ q ) with K being the order of polynomial function.
Revised: ChebyNet approximates the convolution kernel via polynomial function of the diagonal matrix of Laplacian eigenvalues, reducing parameter complexity to O ( K ∗ p ∗ q ) with K being the order of polynomial function.

Original: GCN simpliﬁes ChebyNet via setting K =1.
Revised: GCN simpliﬁes ChebyNet via setting K =1.

Original: We detach feature transformation from graph convolution to implement GWNN and Spectral CNN in our experiments, which can reduce parameter to O ( n + p ∗ q ) .[Table]
Revised: We detach feature transformation from graph convolution to implement GWNN and Spectral CNN in our experiments, which can reduce parameter to O ( n + p ∗ q ) .

Original: [Table]
Revised: [Table]

Original: In Cora and Citeseer, with smaller parameter complexity, GWNN achieves better performance than
Revised: In Cora and Citeseer, with smaller parameter complexity, GWNN achieves better performance than

Original: ChebyNet, reﬂecting that it is promising to implement convolution via graph wavelet transform.
Revised: ChebyNet, reﬂecting that it is promising to implement convolution via graph wavelet transform.

Original: As
Revised: As

Original: Pubmed has a large number of nodes, the parameter complexity of GWNN is larger than ChebyNet.
Revised: Pubmed has a large number of nodes, the parameter complexity of GWNN is larger than ChebyNet.

Original: As future work, it is an interesting attempt to select wavelets associated with a subset of nodes, further reducing parameter complexity with potential loss of performance.
Revised: As future work, it is an interesting attempt to select wavelets associated with a subset of nodes, further reducing parameter complexity with potential loss of performance.

Original: A PPENDIX D F AST G RAPH WAVELETS WITH C HEBYSHEV POLYNOMIAL APPROXIMATION
Revised: A PPENDIX D F AST G RAPH WAVELETS WITH C HEBYSHEV POLYNOMIAL APPROXIMATION

Original: In Hammond et al.
Revised: Hammond et al.

Original: (2011), a method is proposed to use Chebyshev polynomials to efﬁciently approximate ψ s and ψ − 1 s , with the computational complexity O
Revised: (2011) proposed a method, using Chebyshev polynomials to efﬁciently approximate ψ s and ψ − 1 s .

Original:  ( m × | E | ) , where | E | is the number of edges and m is the order of Chebyshev polynomials.
Revised: The computational complexity is O ( m × | E | ) , where | E | is the number of edges and m is the order of Chebyshev polynomials.

Original: We give the detail of this approximation  in Hammond et al.
Revised: We give the details of the approximation proposed in Hammond et al.

Original: The Chebyshev polynomials T k ( y )  can be generated by the stable recurrence relation T k ( y ) =
Revised: With the stable recurrence relation T k ( y ) = 2 yT k − 1 ( y ) − T k − 2 ( y ) , we can generate the Chebyshev polynomials T k ( y ) .

Original: 2 yT k − 1 ( y ) − T k − 2 ( y ) , with T 0 = 1 and T 1 = y .
Revised: Here T 0 = 1 and T 1 = y .

Original: For y ∈ [ − 1 , 1] , they satisfy the trigonometric expression T k ( y ) =
Revised: For y sampled between -1 and 1, the trigonometric expression T k ( y ) =

Original: cos ( k arccos ( y )) , which shows thateach T k ( y ) is bounded between -1 and 1 for y ∈ [ − 1 , 1] .
Revised: cos ( k arccos ( y )) is satisﬁed. It shows that T k ( y ) ∈ [ − 1 , 1] when y ∈ [ − 1 , 1] .

Original: The Chebyshev polynomials form an orthogonal basis for  L 2 ([ − 1 , 1] , dy [Equation] 1 − 
Revised: Through the Chebyshev polynomials, an orthogonal basis for the Hilbert space of square integrable functions L 2 ([ − 1 , 1] , dy √ 1 − y 2 ) is formed.

Original: [Equation] 2 ) , the Hilbert space of squre integrable functions with respect to the measure dy/ (cid:112) 1 − y 2 . Every h ∈ L 2 ([ − 1 , 1] , dy √{+and the Chebyshev coefﬁcients c k = 
+}
Revised: For each h in this Hilbert space, we have a uniformly convergent Chebyshev series h ( y ) = 12 c 0 + (cid:80) ∞ k =1 c k T k ( y ) , {+and the Chebyshev coefﬁcients c k = 
+}

Original: [Equation]
Revised: π

Original:  y2 ) has a uniformly convergent Chebyshev series [Equation]
Revised: (cid:82) 1 − 1 T k ( y ) h ( y ) √ 1 − y 2 dy =

Original: 13) with Chebyshev coefﬁcients
Revised: 2 π (cid:82) π 0 cos ( kθ ) h ( cos ( θ )) dθ .

Original: [Equation] We now assume a ﬁxed scale s , approximating g ( sx ) for x ∈ [0 , λ max ]  can be done by shifting the domain using the transformation x
Revised: A ﬁxed scale s is assumed. To approximate g ( sx ) for x ∈ [0 , λ max ] , we can shift the domain through the transformation x

Original: = a ( y + 1) , with a = λ max 2 .
Revised: = a ( y + 1) , where a = λ max 2 .

Original: Denote the shifted Chebyshev polynomials T (cid:48) k
Revised: T (cid:48) k

Original: ( x ) =
Revised: ( x ) =

Original: T k ( x − aa )  with x − aa ∈ [ − 1 , 1] .
Revised: T k ( x − a a ) denotes the shifted Chebyshev polynomials, with x − a a ∈ [ − 1 , 1] .

Original: We then write
Revised: Then we have g ( sx ) = 12 c 0 + (cid:80) ∞ k

Original: [Equation]
Revised: =1 c k T (cid:48) k

Original: valid for x ∈ [0 , λ max ] , with{+c k = 
+}
Revised: ( x ) , and x ∈ [0 , λ max ] , {+c k = 
+}

Original: [Equation]
Revised: π (cid:82) π 0 cos ( kθ ) g ( s ( a ( cos ( θ ) + 1))) dθ .

Original: Polynomial approximation is achieved by truncating the Chebyshev expansion to m terms.
Revised: we truncate the Chebyshev expansion to m terms and achieve Polynomial approximation.

Original: Let us use the ψ − 1 sas an example and g ( sx ) =
Revised: Here we give the example of the ψ − 1 s and g ( sx ) =

Original: e − sx ,  f ∈ R n denotes the graph signal.
Revised: e − sx , the graph signal is f ∈ R n .

Original: Then  the fast approximation waveletsare given by 
Revised: Then we can give the fast approximation wavelets by ψ − 1 s f = 12 c 0 f

Revised: + (cid:80) mk =1 c k T (cid:48) k

Original: [Equation]
Revised: ( L ) f .

Original: Theutility of this approach relies on the efﬁcient computation of T (cid:48) k 
Revised: The efﬁcient computation of T (cid:48) k ( L ) determines the utility of this approach, where T (cid:48)

Original:  ( L ) , with
Revised: k ( L ) f = 2 a ( L − I )

Original: [Equation]
Revised: ( T (cid:48) k − 1 ( L ) f ) − T (cid:48) k − 2 ( L ) f .

Original: A PPENDIX E A NALYSIS ON SPASITY OF SPECTRAL TRANSFORM AND L APLACIAN MATRIX
Revised: A PPENDIX E A NALYSIS ON SPASITY OF SPECTRAL TRANSFORM AND L APLACIAN MATRIX

Original: The sparsity of the graph wavelets depends on the sparsity of the Laplacian matrix and the hyperparameter s , We show the sparsity of spectral transform matrix and Laplacian matrix in Table 6.
Revised: The sparsity of the graph wavelets depends on the sparsity of the Laplacian matrix and the hyperparameter s , We show the sparsity of spectral transform matrix and Laplacian matrix in Table 6.

Original: [Table]
Revised: [Table]

Original: The sparsity of Laplacian matrix is sparser than graph wavelets, and this property limits our method, i.e., the higher time complexity than some methods depending on Laplacian matrix and identity matrix, e.g., GCN.
Revised: The sparsity of Laplacian matrix is sparser than graph wavelets, and this property limits our method, i.e., the higher time complexity than some methods depending on Laplacian matrix and identity matrix, e.g., GCN.

Original: Speciﬁcally, both our method and GCN aim to improve Spectral CNN via designing localized graph convolution.
Revised: Speciﬁcally, both our method and GCN aim to improve Spectral CNN via designing localized graph convolution.

Original: GCN, as a simpliﬁed version of ChebyNet, leverages Laplacian matrix as weighted matrix and expresses the spectral graph convolution in spatial domain, acting as spatial-like method (Monti et al., 2017).
Revised: GCN, as a simpliﬁed version of ChebyNet, leverages Laplacian matrix as weighted matrix and expresses the spectral graph convolution in spatial domain, acting as spatial-like method (Monti et al., 2017).

Original: However, our method resorts to using graph wavelets as a new set of bases, directly designing localized spectral graph convolution.
Revised: However, our method resorts to using graph wavelets as a new set of bases, directly designing localized spectral graph convolution.

Original: GWNN offers a localized graph convolution via replacing graph Fourier transform with graph wavelet transform, ﬁnding good spectral basis with localization property and good interpretability.
Revised: GWNN offers a localized graph convolution via replacing graph Fourier transform with graph wavelet transform, ﬁnding good spectral basis with localization property and good interpretability.

Original: This distinguishes GWNN from
Revised: This distinguishes GWNN from

Original: ChebyNet and GCN, which express the graph convolution deﬁned via graph Fourier transform in 
Revised: ChebyNet and GCN, which express the graph convolution deﬁned via graph Fourier transform in vertex domain.