Original: O
Revised: O

Original: CTAVE G RAPH C ONVOLUTIONAL N ETWORK
Revised: CTAVE G RAPH C ONVOLUTIONAL N ETWORK

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Many variants of Graph Convolutional Networks (GCNs) for representation learning have been proposed recently and have achieved fruitful results in various domains.
Revised: Many variants of Graph Convolutional Networks (GCNs) for representation learning have been proposed recently and have achieved fruitful results in various domains.

Original: Among them, spectral-based GCNs are constructed via convolution theorem upon a theoretical foundation from the perspective of Graph Signal Processing (GSP).
Revised: Among them, spectral-based GCNs are constructed via convolution theorem upon a theoretical foundation from the perspective of Graph Signal Processing (GSP).

Original: However, despite most of them implicitly act as low-pass ﬁlters that generate smooth representations for each node, there is limited development on the full usage of underlying information from low-frequency components.
Revised: However, despite most of them implicitly act as low-pass ﬁlters that generate smooth representations for each node, there is limited development on the full usage of underlying information from low-frequency components.

Original: Here, we ﬁrst introduce the octave convolution on graphs in spectral domain.
Revised: Here, we ﬁrst introduce the octave convolution on graphs in spectral domain.

Original: Accordingly, we present Octave Graph Convolutional Network ( OctGCN ), a novel architecture that learns representations for different frequency components regarding weighted ﬁlters and graph wavelets bases.
Revised: Accordingly, we present Octave Graph Convolutional Network ( OctGCN ), a novel architecture that learns representations for different frequency components regarding weighted ﬁlters and graph wavelets bases.

Original: We empirically validate the importance of lowfrequency components in graph signals on semi-supervised node classiﬁcation and demonstrate that our model achieves state-of-the-art performance in comparison with both spectral-based and spatial-based baselines.
Revised: We empirically validate the importance of lowfrequency components in graph signals on semi-supervised node classiﬁcation and demonstrate that our model achieves state-of-the-art performance in comparison with both spectral-based and spatial-based baselines.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: The family of Graph Convolutional Networks (GCNs) (Zhang et al., 2018), which generalizes the traditional Convolutional Neural Networks (CNNs) from Euclidean structure data to graphs, has achieved a remarkable success in various application domains, including but not limited to social networks (Chen et al., 2018), computer vision (Kampffmeyer et al., 2018), text classiﬁcation (Yao et al., 2019) and applied chemistry (Liao et al., 2019).
Revised: The family of Graph Convolutional Networks (GCNs) (Zhang et al., 2018), which generalizes the traditional Convolutional Neural Networks (CNNs) from Euclidean structure data to graphs, has achieved a remarkable success in various application domains, including but not limited to social networks (Chen et al., 2018), computer vision (Kampffmeyer et al., 2018), text classiﬁcation (Yao et al., 2019) and applied chemistry (Liao et al., 2019).

Original: Existing methods of GCNs design falls into two categories: spatial-based methods and spectralbased methods (Wu et al., 2019).
Revised: Existing methods of GCNs design falls into two categories: spatial-based methods and spectralbased methods (Wu et al., 2019).

Original: On the surface, the spatial-based models directly perform information aggregation through graph topology.
Revised: On the surface, the spatial-based models directly perform information aggregation through graph topology.

Original: However, this aggregation can be viewed as a simpliﬁed convolution operation on spectral domain with the theoretical foundation in Graph Signal Processing (GSP).
Revised: However, this aggregation can be viewed as a simpliﬁed convolution operation on spectral domain with the theoretical foundation in Graph Signal Processing (GSP).

Original: GSP extends the concepts in Discrete Signal Processing (DSP) and focuses on analyzing and processing data points whose relations are modeled as graph (Shuman et al., 2013; Ortega et al., 2018).
Revised: GSP extends the concepts in Discrete Signal Processing (DSP) and focuses on analyzing and processing data points whose relations are modeled as graph (Shuman et al., 2013; Ortega et al., 2018).

Original: In standard signal processing problems, the underlying "real signal" is usually assumed to have low frequencies (Rabiner & Gold, 1975).
Revised: In standard signal processing problems, the underlying "real signal" is usually assumed to have low frequencies (Rabiner & Gold, 1975).

Original: Recent works (Wu et al., 2019; Maehara, 2019) reveal that the spectral-based GCNs can be viewed as an implicit low-pass-type ﬁlter based denoising mechanism on the spectral domain.
Revised: Recent works (Wu et al., 2019; Maehara, 2019) reveal that the spectral-based GCNs can be viewed as an implicit low-pass-type ﬁlter based denoising mechanism on the spectral domain.

Original: However, there is still a lack of the explicit learning architecture of GCNs to extract the beneﬁcial information from low-frequency while making full use of the high-frequency under certain scenarios.
Revised: However, there is still a lack of the explicit learning architecture of GCNs to extract the beneﬁcial information from low-frequency while making full use of the high-frequency under certain scenarios.

Original: Considering the signal processing problem in computer vision, a natural image can be decomposed into a low spatial frequency component containing the smoothly changing structure, e.g. , background, and a high spatial frequency component describing the rapidly changing ﬁne details, e.g. , outlines.
Revised: Considering the signal processing problem in computer vision, a natural image can be decomposed into a low spatial frequency component containing the smoothly changing structure, e.g. , background, and a high spatial frequency component describing the rapidly changing ﬁne details, e.g. , outlines.

Original: To accommodate with this phenomenon, (Chen et al., 2019) proposed Octave Convolution (OctConv) to learn the octave feature representations, which factorizes convolutional feature maps into two groups at different spatial frequencies and process them with different convolutions at their corresponding frequency.
Revised: To accommodate with this phenomenon, (Chen et al., 2019) proposed Octave Convolution (OctConv) to learn the octave feature representations, which factorizes convolutional feature maps into two groups at different spatial frequencies and process them with different convolutions at their corresponding frequency.

Original: Similarly, the octave mechanism is observed in graph representational learning more naturally.
Revised: Similarly, the octave mechanism is observed in graph representational learning more naturally.

Original: The eigenvectors associated with small eigenvalues carry smoothly varying signal, encouraging nodes that are neighbors to share similar values.
Revised: The eigenvectors associated with small eigenvalues carry smoothly varying signal, encouraging nodes that are neighbors to share similar values.

Original: In contrast, the eigenvectors associated with large eigenvalues carry sharply varying signal across edges (Donnat et al., 2018).
Revised: In contrast, the eigenvectors associated with large eigenvalues carry sharply varying signal across edges (Donnat et al., 2018).

Original: Accordingly, extending octave convolution from images to graphs sheds light on the explicit learning of GCNs regarding the representation of different frequencies.
Revised: Accordingly, extending octave convolution from images to graphs sheds light on the explicit learning of GCNs regarding the representation of different frequencies.

Original: [Figure]
Revised: [Figure]

Original: Different from the scale-space theory (Lindeberg, 2013) utilized in OctConv (Chen et al., 2019) to deﬁne the low- and high-frequency spaces, graph signal processing (GSP) provides us a way to directly divide the low- and high-frequency components based on the ascending ordered eigenvalues of Laplacian.
Revised: Different from the scale-space theory (Lindeberg, 2013) utilized in OctConv (Chen et al., 2019) to deﬁne the low- and high-frequency spaces, graph signal processing (GSP) provides us a way to directly divide the low- and high-frequency components based on the ascending ordered eigenvalues of Laplacian.

Original: Inspired from this, we propose to consider the octave feature in the spectral domain to construct a new graph convolutional model: Octave Graph Convolutional Network OctGCN .
Revised: Inspired from this, we propose to consider the octave feature in the spectral domain to construct a new graph convolutional model: Octave Graph Convolutional Network OctGCN .

Original: In
Revised: In

Original: OctGCN , with a particular design of ﬁlters for different spectrum, we allocate different weights on low- and high-frequency.
Revised: OctGCN , with a particular design of ﬁlters for different spectrum, we allocate different weights on low- and high-frequency.

Original: Spectral graph wavelets are chosen as feature transformation bases due to their local and sparse property.
Revised: Spectral graph wavelets are chosen as feature transformation bases due to their local and sparse property.

Original: Two parameters are further introduced to construct the ﬁlters for reducing the parameter complexity to the same as (Kipf & Welling, 2017), which is critical when labels of training data are limited.
Revised: Two parameters are further introduced to construct the ﬁlters for reducing the parameter complexity to the same as (Kipf & Welling, 2017), which is critical when labels of training data are limited.

Original: Meanwhile, we employ the attention mechanism to learn the importance of low and high pass ﬁlters.
Revised: Meanwhile, we employ the attention mechanism to learn the importance of low and high pass ﬁlters.

Original: Figure 1 provides the overview of the design of OctGCN in spectral domain.
Revised: Figure 1 provides the overview of the design of OctGCN in spectral domain.

Original: We validate the effectiveness of our model via experiments on semi-supervised node classiﬁcation tasks, where the expressive power of GCNs is crucial to capture the underlying beneﬁcial information in graph signals.
Revised: We validate the effectiveness of our model via experiments on semi-supervised node classiﬁcation tasks, where the expressive power of GCNs is crucial to capture the underlying beneﬁcial information in graph signals.

Original: Our results conﬁrm the importance of low-frequency in graphs and bring interpretability to the innate character of GCNs.
Revised: Our results conﬁrm the importance of low-frequency in graphs and bring interpretability to the innate character of GCNs.

Original: In addition, empirical results show that our proposed method consistently rivals the state-of-art methods from both spectral-based and spatial-based baselines on real-world datasets.
Revised: In addition, empirical results show that our proposed method consistently rivals the state-of-art methods from both spectral-based and spatial-based baselines on real-world datasets.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Spectral convolutional networks on graphs.
Revised: Spectral convolutional networks on graphs.

Original: Existing methods of deﬁning a convolutional operation on graphs can be broadly divided into two categories: spectral based and spatial based methods (Zhang et al., 2018).
Revised: Existing methods of deﬁning a convolutional operation on graphs can be broadly divided into two categories: spectral based and spatial based methods (Zhang et al., 2018).

Original: We focus on the spectral graph convolutions in this paper.
Revised: We focus on the spectral graph convolutions in this paper.

Original: Spectral CNN (Bruna et al., 2014)
Revised: Spectral CNN (Bruna et al., 2014)

Original: ﬁrst attempts to generalize CNNs to graphs based on the spectrum of the graph Laplacian and deﬁnes the convolutional kernel in the spectral domain.
Revised: ﬁrst attempts to generalize CNNs to graphs based on the spectrum of the graph Laplacian and deﬁnes the convolutional kernel in the spectral domain.

Original: (Boscaini et al., 2015) further employs windowed Fourier transformation to deﬁne a local spectral CNN approach.
Revised: (Boscaini et al., 2015) further employs windowed Fourier transformation to deﬁne a local spectral CNN approach.

Original: ChebyNet (Defferrard et al., 2016) introduces a fast localized convolutional ﬁlter on graphs via
Revised: ChebyNet (Defferrard et al., 2016) introduces a fast localized convolutional ﬁlter on graphs via

Original: Chebyshev polynomial approximation.
Revised: Chebyshev polynomial approximation.

Original: Vanilla GCN (Kipf & Welling, 2017) further extends the spectral graph convolutions considering networks of signiﬁcantly larger scale by several simpliﬁcations.
Revised: Vanilla GCN (Kipf & Welling, 2017) further extends the spectral graph convolutions considering networks of signiﬁcantly larger scale by several simpliﬁcations.

Original: (Khasanova & Frossard, 2017) learns graph-based features on images that are inherently invariant to isometric transformations.
Revised: (Khasanova & Frossard, 2017) learns graph-based features on images that are inherently invariant to isometric transformations.

Original: Cayleynets (Levie et al., 2018) alternatively introduce Cayley polynomials allowing to efﬁciently compute spectral ﬁlters on graphs.
Revised: Cayleynets (Levie et al., 2018) alternatively introduce Cayley polynomials allowing to efﬁciently compute spectral ﬁlters on graphs.

Original: Lanczos algorithm is utilized in LanczosNet (Liao et al., 2019) to construct low-rank approximations of the graph Laplacian for convolution.
Revised: Lanczos algorithm is utilized in LanczosNet (Liao et al., 2019) to construct low-rank approximations of the graph Laplacian for convolution.

Original: SGC (Wu et al., 2019) further reduces the complexity of Vanilla GCN by successively removing the non-linearities and collapsing weights between consecutive layers.
Revised: SGC (Wu et al., 2019) further reduces the complexity of Vanilla GCN by successively removing the non-linearities and collapsing weights between consecutive layers.

Original: Despite their effective performance, all these convolution theorem based methods lack the strategy to explicitly treat low- and high-frequency components with different importance.
Revised: Despite their effective performance, all these convolution theorem based methods lack the strategy to explicitly treat low- and high-frequency components with different importance.

Original: Spectral graph wavelets.
Revised: Spectral graph wavelets.

Original: Theoretically, the lifting scheme is proposed for the construction of wavelets that can be adapted to irregular graphs in (Sweldens, 1998).
Revised: Theoretically, the lifting scheme is proposed for the construction of wavelets that can be adapted to irregular graphs in (Sweldens, 1998).

Original: (Hammond et al., 2011)
Revised: (Hammond et al., 2011)

Original: deﬁnes wavelet transforms appropriate for graphs and describes a fast algorithm for computation via fast
Revised: deﬁnes wavelet transforms appropriate for graphs and describes a fast algorithm for computation via fast

Original: Chebyshev polynomial approximation.
Revised: Chebyshev polynomial approximation.

Original: For applications, (Tremblay & Borgnat, 2014) utilizes graph wavelets for multi-scale community mining and obtains a local view of the graph from each node.
Revised: For applications, (Tremblay & Borgnat, 2014) utilizes graph wavelets for multi-scale community mining and obtains a local view of the graph from each node.

Original: (Donnat et al., 2018) introduces the property of graph wavelets that describes information diffusion and learns structural node embeddings accordingly.
Revised: (Donnat et al., 2018) introduces the property of graph wavelets that describes information diffusion and learns structural node embeddings accordingly.

Original: GWNN (Xu et al., 2019a)
Revised: GWNN (Xu et al., 2019a)

Original: ﬁrst attempts to construct graph neural networks with graph wavelets.
Revised: ﬁrst attempts to construct graph neural networks with graph wavelets.

Original: These works emphasize the local and sparse property of graph wavelets for graph signal processing both theoretically and practically.
Revised: These works emphasize the local and sparse property of graph wavelets for graph signal processing both theoretically and practically.

Original: Octave feature representation.
Revised: Octave feature representation.

Original: In computer vision, (Chen et al., 2019)
Revised: In computer vision, (Chen et al., 2019)

Original: ﬁrst deﬁnes octave feature representations based on scale-space theory and reduces spatial redundancy of vanilla CNN models.
Revised: ﬁrst deﬁnes octave feature representations based on scale-space theory and reduces spatial redundancy of vanilla CNN models.

Original: (Durall et al., 2019) further leverages octave convolutions for designing stabilizing GANs.
Revised: (Durall et al., 2019) further leverages octave convolutions for designing stabilizing GANs.

Original: To our knowledge, this is the ﬁrst time that octave feature representations are considered in irregular graph domain and established with graph convolutional neural networks.
Revised: To our knowledge, this is the ﬁrst time that octave feature representations are considered in irregular graph domain and established with graph convolutional neural networks.

Original:  P ROPOSED A PPROACH 3. P RELIMINARY
Revised:  P ROPOSED A PPROACH 3. P RELIMINARY

Original: We denote G =
Revised: We denote G =

Original: { V, E } as an undirected graph, where | V | =
Revised: { V, E } as an undirected graph, where | V | =

Original: n is the set of n nodes, and E is the set of edges.
Revised: n is the set of n nodes, and E is the set of edges.

Original: The adjacency matrix is deﬁned as A with A i,j =
Revised: The adjacency matrix is deﬁned as A with A i,j =

Original: A j,i describing the edge connecting node i and node j .
Revised: A j,i describing the edge connecting node i and node j .

Original: The graph Laplacian matrix L is deﬁned as the difference L = D − A , where
Revised: The graph Laplacian matrix L is deﬁned as the difference L = D − A , where

Original: D i,i = (cid:80)
Revised: D i,i = (cid:80)

Original: j A i,j is a diagonal degree matrix.
Revised: j A i,j is a diagonal degree matrix.

Original: The normalized graph Laplacian matrix is referred as L =
Revised: The normalized graph Laplacian matrix is referred as L =

Original: I n − D − 1 / 2 AD − 1 / 2 , where I n is the identity matrix.
Revised: I n − D − 1 / 2 AD − 1 / 2 , where I n is the identity matrix.

Original: The graph Laplacian L can be decomposed into its eigenvalue components, L = U Λ U (cid:62) , such that for the set of eigenvalues in ascending order { λ i } n − 1 i =

Revised: The graph Laplacian L can be decomposed into its eigenvalue components, L = U Λ U (cid:62) , such that for the set of eigenvalues in ascending order { λ i } n − 1 i =

Original: = λ 0 ≤ λ 1 ≤ · · · ≤ λ n − 1 , the diagonal eigenvalue matrix is deﬁned as Λ =
Revised: = λ 0 ≤ λ 1 ≤ · · · ≤ λ n − 1 , the diagonal eigenvalue matrix is deﬁned as Λ =

Original: diag
Revised: diag

Original: ( λ 0 , . . .
Revised: ( λ 0 , . . .

Original: , λ n − 1 ) and U =
Revised: , λ n − 1 ) and U =

Original: ( u 1 , u 2 , ..., u n ) is the eigenvector matrix.
Revised: ( u 1 , u 2 , ..., u n ) is the eigenvector matrix.

Original: Since L is a real symmetric matrix, it has real, non-negative eigenvalues { λ i } n − 1 i =

Revised: Since L is a real symmetric matrix, it has real, non-negative eigenvalues { λ i } n − 1 i =

Original: = λ 0 ≤ λ 1 ≤ · · · ≤ λ n − 1 , known as the frequencies of graph.
Revised: = λ 0 ≤ λ 1 ≤ · · · ≤ λ n − 1 , known as the frequencies of graph.

Original: These eigenvalues have associated a complete set of orthonormal eigenvectors in U , identiﬁed as Laplacian eigenvectors.
Revised: These eigenvalues have associated a complete set of orthonormal eigenvectors in U , identiﬁed as Laplacian eigenvectors.

Original: In Graph Signal Processing (GSP), we denote frequency components with small/large eigenvalues of Laplacian as low/high frequencies.
Revised: In Graph Signal Processing (GSP), we denote frequency components with small/large eigenvalues of Laplacian as low/high frequencies.

Original: Given a signal x and a graph Laplacian L , the Graph Fourier Transform (GFT) of x with respect to L is deﬁned as the signal ˜ x
Revised: Given a signal x and a graph Laplacian L , the Graph Fourier Transform (GFT) of x with respect to L is deﬁned as the signal ˜ x

Original: = U (cid:62) x , and the inverse (i)GFT of x with respect to L is x = U ˜ x
Revised: = U (cid:62) x , and the inverse (i)GFT of x with respect to L is x = U ˜ x

Original: (Shuman et al., 2013).
Revised: (Shuman et al., 2013).

Original: 3. S PECTRAL G RAPH C ONVOLUTION
Revised: 3. S PECTRAL G RAPH C ONVOLUTION

Original: The spectral convolution on graphs is normally deﬁned as the multiplication of a signal on every node x with a diagonal ﬁlter g θ = diag ( θ ) parameterized by θ in the Fourier domain:
Revised: The spectral convolution on graphs is normally deﬁned as the multiplication of a signal on every node x with a diagonal ﬁlter g θ = diag ( θ ) parameterized by θ in the Fourier domain:

Original: [Equation]
Revised: [Equation]

Original: The ﬁlters are usually understood as a function of the eigenvalues.
Revised: The ﬁlters are usually understood as a function of the eigenvalues.

Original: Inspired by (Maehara, 2019), we can decompose the spectral convolution process on graphs from the perspective of GSP as four steps: 1.
Revised: Inspired by (Maehara, 2019), we can decompose the spectral convolution process on graphs from the perspective of GSP as four steps: 1.

Original: Compute the graph bases U ; 2.
Revised: Compute the graph bases U ; 2.

Original: Graph spectral transform on signal x with U (cid:62) ; 3.
Revised: Graph spectral transform on signal x with U (cid:62) ; 3.

Original: Filtering with g θ ; 4.
Revised: Filtering with g θ ; 4.

Original: Reconstruct the signal features in the spatial domain with U .
Revised: Reconstruct the signal features in the spatial domain with U .

Original: In this sense, the design of ﬁlter g θ is essential to the performance of spectral convolution.
Revised: In this sense, the design of ﬁlter g θ is essential to the performance of spectral convolution.

Original: Broadly, the ﬁlter design can be divided into two categories: it is either learned by the neural network (Bruna et al., 2014; Xu et al., 2019a) or directly ﬁxed as the eigenvalues via approximation (Kipf & Welling, 2017; Wu et al., 2019).
Revised: Broadly, the ﬁlter design can be divided into two categories: it is either learned by the neural network (Bruna et al., 2014; Xu et al., 2019a) or directly ﬁxed as the eigenvalues via approximation (Kipf & Welling, 2017; Wu et al., 2019).

Original: In this paper, we will focus on the ﬁrst kind.
Revised: In this paper, we will focus on the ﬁrst kind.

Original: Spectral CNN (Bruna et al., 2014) generalizes the convolutional net by operating on the spectrum of weights, given by the ordered eigenvectors of its graph Laplacian.
Revised: Spectral CNN (Bruna et al., 2014) generalizes the convolutional net by operating on the spectrum of weights, given by the ordered eigenvectors of its graph Laplacian.

Original: The structure of k -th layer is constructed as:
Revised: The structure of k -th layer is constructed as:

Original: [Equation]
Revised: [Equation]

Original: 1 , · · · , q, (2) where X k ∈ R n × p is the signal with p input channels and X k +1 ∈ R n × q is the convolved signal matrix.
Revised: 1 , · · · , q, (2) where X k ∈ R n × p is the signal with p input channels and X k +1 ∈ R n × q is the convolved signal matrix.

Original: X k
Revised: X k

Original: [: ,i ] and X k +

Revised: [: ,i ] and X k +

Original: [: ,j ] are the i -th and j -th column of X k and X k +1 , respectively.
Revised: [: ,j ] are the i -th and j -th column of X k and X k +1 , respectively.

Original: g θ of k -th layer is deﬁned as F ki,j , which is a diagonal ﬁlter matrix to be learned for each input channel in spectral domain.
Revised: g θ of k -th layer is deﬁned as F ki,j , which is a diagonal ﬁlter matrix to be learned for each input channel in spectral domain.

Original: h is a real valued nonlinear activation function, e.g. , ReLU( · ) = max (0 , 1) .
Revised: h is a real valued nonlinear activation function, e.g. , ReLU( · ) = max (0 , 1) .

Original: Thus, the parameter complexity of Spectral CNN is O ( n × p × q ) , which generally demands a huge amount of training data for parameter learning.
Revised: Thus, the parameter complexity of Spectral CNN is O ( n × p × q ) , which generally demands a huge amount of training data for parameter learning.

Original: 3. W HY S PECTRAL G RAPH W AVELET ?
Revised: 3. W HY S PECTRAL G RAPH W AVELET ?

Original: Graph wavelet neural network (GWNN) (Xu et al., 2019a) expands the spectral convolution from
Revised: Graph wavelet neural network (GWNN) (Xu et al., 2019a) expands the spectral convolution from

Original: Fourier transformation to wavelet transformation.
Revised: Fourier transformation to wavelet transformation.

Original: Let g s ( λ ) =
Revised: Let g s ( λ ) =

Original: e − λs be a heat kernel ﬁlter with scaling parameter s .
Revised: e − λs be a heat kernel ﬁlter with scaling parameter s .

Original: In GSP (Hammond et al., 2011; Shuman et al., 2013), the spectral graph wavelet ψ si is deﬁned as the signal resulting from the modulation in the spectral domain of a signal x centered around the associated node i .
Revised: In GSP (Hammond et al., 2011; Shuman et al., 2013), the spectral graph wavelet ψ si is deﬁned as the signal resulting from the modulation in the spectral domain of a signal x centered around the associated node i .

Original: Then, the graph wavelet transform is conducted by employing a set of wavelets ψ s = ( ψ s 1 , ψ s 2 , . . .
Revised: Then, the graph wavelet transform is conducted by employing a set of wavelets ψ s = ( ψ s 1 , ψ s 2 , . . .

Original: , ψ sn ) as bases.
Revised: , ψ sn ) as bases.

Original: Formally, the spectral graph wavelets are given as:
Revised: Formally, the spectral graph wavelets are given as:

Original: [Equation]
Revised: [Equation]

Original: ) where U is Laplacian eigenvectors of L = D − A or normalized Laplacian L
Revised: where U is Laplacian eigenvectors of L = D − A or normalized Laplacian L

Original: =
Revised: =

Original: I n − D − 12 AD − 12 , g s = diag (cid:0) g s ( λ 1 ) , g s ( λ 2 ) , . .
Revised: I n − D − 12 AD − 12 , g s = diag (cid:0) g s ( λ 1 ) , g s ( λ 2 ) , . .

Original: . , g s ( λ n ) (cid:1) is a scaling matrix with heat kernel.
Revised: . , g s ( λ n ) (cid:1) is a scaling matrix with heat kernel.

Original: The inverse of graph wavelets ψ − 1 s is obtained by simply replacing the g s ( λ ) in ψ s with g s ( − λ ) corresponding to the heat kernel (Donnat et al., 2018).
Revised: The inverse of graph wavelets ψ − 1 s is obtained by simply replacing the g s ( λ ) in ψ s with g s ( − λ ) corresponding to the heat kernel (Donnat et al., 2018).

Original: Similarly, smaller indices in graph wavelets correspond to lowfrequency components and vice versa.
Revised: Similarly, smaller indices in graph wavelets correspond to lowfrequency components and vice versa.

Original: Similar to GFT, after replacing the Fourier bases with spectral graph wavelets, the graph wavelet transformation of a signal x on graph is deﬁned as ˆ x = ψ − 1 s x and the inverse graph wavelet transform is x = ψ s ˆ x .
Revised: Similar to GFT, after replacing the Fourier bases with spectral graph wavelets, the graph wavelet transformation of a signal x on graph is deﬁned as ˆ x = ψ − 1 s x and the inverse graph wavelet transform is x = ψ s ˆ x .

Original: Replacing the graph Fourier transform in spectral convolution (Equation 1) with graph wavelet transform, the graph wavelet convolution can be obtained as:
Revised: Replacing the graph Fourier transform in spectral convolution (Equation 1) with graph wavelet transform, the graph wavelet convolution can be obtained as:

Original: [Equation]
Revised: [Equation]

Original: The beneﬁts that spectral graph wavelet bases have over Fourier bases mainly fall into two aspects: 1.
Revised: The beneﬁts that spectral graph wavelet bases have over Fourier bases mainly fall into two aspects: 1.

Original: Given the sparse real-world networks, the graph wavelet bases are usually much more sparse than Fourier bases, e.g. , the density of ψ s is 2 .
Revised: Given the sparse real-world networks, the graph wavelet bases are usually much more sparse than Fourier bases, e.g. , the density of ψ s is 2 .

Original: 8% comparing with 99 .
Revised: 8% comparing with 99 .

Original: 1% of U (Xu et al., 2019a).
Revised: 1% of U (Xu et al., 2019a).

Original: The sparseness of graph wavelets makes them more computationally efﬁcient for use.
Revised: The sparseness of graph wavelets makes them more computationally efﬁcient for use.

Original: In spectral graph wavelets, the signal ψ s resulting from heat kernel ﬁlter g s is typically localized on the graph and in the spectral domain (Shuman et al., 2013).
Revised: In spectral graph wavelets, the signal ψ s resulting from heat kernel ﬁlter g s is typically localized on the graph and in the spectral domain (Shuman et al., 2013).

Original: By adjusting the scaling parameter s , one can easily constrain the range of localized neighborhood.
Revised: By adjusting the scaling parameter s , one can easily constrain the range of localized neighborhood.

Original: Smaller values of s generally associate with smaller neighborhoods.
Revised: Smaller values of s generally associate with smaller neighborhoods.

Original: Employing the same strategy as in Spectral CNN (Bruna et al., 2014), GWNN designs the same diagonal ﬁlter F ki,j to be learned for each input channel.
Revised: Employing the same strategy as in Spectral CNN (Bruna et al., 2014), GWNN designs the same diagonal ﬁlter F ki,j to be learned for each input channel.

Original: The structure ofthe k -th layer of GWNN is deﬁned as:
Revised: The structure of k -th layer of GWNN is:

Original: [Equation]
Revised: [Equation]

Original:  As mentioned before, the parameter complexity of Spectral CNN is large since each pair of input and output channel requires learning an individual diagonal ﬁlter matrix F ki,j . GWNN further reduces the parameter complexity by dividing each layer into two components: feature transformation and graph convolution:
Revised: Note that both Spectral CNN and GWNN employ the same ﬁlters for learning full frequency components. As mentioned before, the parameter complexity of Spectral CNN is large since each pair of input and output channel requires learning an individual diagonal ﬁlter matrix F ki,j . GWNN further reduces the parameter complexity by dividing each layer into two components: feature transformation and graph convolution:

Original: [Equation]
Revised: [Equation]

Original: where W k ∈ R p × q is the feature transformation parameter matrix similar to Vanilla GCN (Kipf & Welling, 2017).
Revised: where W k ∈ R p × q is the feature transformation parameter matrix similar to Vanilla GCN (Kipf & Welling, 2017).

Original: In such a way, the feature transformation operation is detached from graph convolution and the parameter complexity is decreased from from O ( n × p × q ) to O ( n + p × q ) .
Revised: In such a way, the feature transformation operation is detached from graph convolution and the parameter complexity is decreased from from O ( n × p × q ) to O ( n + p × q ) .

Original: 3. O CTAVE C ONVOLUTIONAL L AYER
Revised: 3. O CTAVE C ONVOLUTIONAL L AYER

Original: In contrast to the scale-space theory-based octave feature representation utilized in computer vision (Lindeberg, 2013), Graph Signal Processing provides us a more principle way in the spectral domain.
Revised: In contrast to the scale-space theory-based octave feature representation utilized in computer vision (Lindeberg, 2013), Graph Signal Processing provides us a more principle way in the spectral domain.

Original: Based on the  low- and high-components of graph and combining the beneﬁts of graph wavelets, we can naturally construct each layer in an octave convolution manner with two components as: 
Revised: To better capture the different importance of low- and high- frequency components and combining the beneﬁts of graph wavelets, we can naturally construct each layer in an octave convolution manner by learning two different ﬁlters as: feature transformation

Original: [Equation]
Revised: [Equation]

Original:  convolution
Revised: graph convolution

Original: [Equation]
Revised: [Equation]

Original: where F kL ∈ R d × d and F kH ∈ R ( n − d ) × ( n − d ) are the diagonal ﬁlter matrix for graph convolution to be learned with different weights for low- and high- components, respectively.
Revised: where F kL ∈ R d × d and F kH ∈ R ( n − d ) × ( n − d ) are the diagonal ﬁlter matrix for graph convolution to be learned with different weights for low- and high- components, respectively.

Original: d is the hyper-parameter  to select the proportion d/n of low frequency components.
Revised: d is the hyper-parameter [Table] to select the proportion d/n of low frequency components.

Original: ψ sL and ψ sL are the corresponding low-
Revised: ψ sL and ψ sL are the corresponding lowand

Original: [Table]

Original: and high- frequency graph wavelet bases.
Revised: high- frequency graph wavelet bases.

Original: Further, with a pooling operation on the outputs and non-linear activation function, the structure of the k -th layer in our structure can be deﬁned as
Revised: Further, with a pooling operation on the outputs and non-linear activation function, the structure of the k -th layer in our structure can be deﬁned as

Original: [Equation]
Revised: [Equation]

Original: We refer this proposed architecture as Octave Graph Convolutional Network ( OctGCN ).
Revised: We refer this proposed architecture as Octave Graph Convolutional Network ( OctGCN ).

Original: Since the parameters in the diagonal convolution ﬁltering kernels could be huge especially for large graphs, the graph-based semi-supervised learning might prohibit the parameter learning due to the limited amount of training data.
Revised: Since the parameters in the diagonal convolution ﬁltering kernels could be huge especially for large graphs, the graph-based semi-supervised learning might prohibit the parameter learning due to the limited amount of training data.

Original: To mitigate this issue, we further reduce the parameter complexity by constructing the graph convolution kernel F k with two parameters α L and α H , and keeping the same weight matrix W shared between low- and high- frequency components as
Revised: To mitigate this issue, we further reduce the parameter complexity by constructing the graph convolution kernel F k with two parameters α L and α H , and keeping the same weight matrix W shared between low- and high- frequency components as

Original: [Equation]
Revised: [Equation]

Original: For the learning of weights of low- and high-frequency components α L and α H , we adopt the attention strategy to constraint them within the scale of (0 , 
Revised: For the learning of weights of low- and high-frequency components α L and α H , we adopt the attention strategy to constraint them within the scale of (0 , 1) :

Original: [Equation]
Revised: [Equation]

Original: [Equation] Hence, we introduce three more hyper-parameters to be tuned:parameter α L and α H controls the importance of low and high frequency components, and parameter d specify the ratio of low frequencies we expect to represent the graph.
Revised: Hence, we introduce three more hyper-parameters to be tuned: α L and α H control the importance of low and high frequency components, and parameter d specify the ratio of low frequencies we expect to represent the graph.

Original: In this way, we reduces the parameter complexity from O ( n + p × q ) in GWNN (Xu et al., 2019a) to O ( p × q ) , which is the same as Vanilla GCN (Kipf & Welling, 2017).
Revised: In this way, we reduces the parameter complexity from O ( n + p × q ) in GWNN (Xu et al., 2019a) to O ( p × q ) , which is the same as Vanilla GCN (Kipf & Welling, 2017).

Original: 3. F AST S PECTRAL G RAPH W AVELET A PPROXIMATION VIA C HEBYSHEV P OLYNOMIALS
Revised: 3. F AST S PECTRAL G RAPH W AVELET A PPROXIMATION VIA C HEBYSHEV P OLYNOMIALS

Original: Directly computing the transformation according to Equation 3 is intensive for large graphs, since diagonalizing Laplacian L commonly requires O ( n 3 ) operations.
Revised: Directly computing the transformation according to Equation 3 is intensive for large graphs, since diagonalizing Laplacian L commonly requires O ( n 3 ) operations.

Original: Luckily, (Hammond et al., 2011) provides us a method to fast approximate the spectral graph wavelet via Chebyshev polynomials.
Revised: Luckily, (Hammond et al., 2011) provides us a method to fast approximate the spectral graph wavelet via Chebyshev polynomials.

Original: Let s be the ﬁxed scaling parameter in the heat ﬁlter kernel g s ( λ ) =
Revised: Let s be the ﬁxed scaling parameter in the heat ﬁlter kernel g s ( λ ) =

Original: e − λs and M be the degree of the Chebyshev polynomial approximations for the scaled wavelet (Larger value of M yields more accurate approximations but higher computational cost in opposite), the graph wavelet is given by
Revised: e − λs and M be the degree of the Chebyshev polynomial approximations for the scaled wavelet (Larger value of M yields more accurate approximations but higher computational cost in opposite), the graph wavelet is given by

Original: [Equation]
Revised: [Equation]

Original: (12) where ˜ L = 2 λ max L − I n and J i ( − s ) is the Bessel function of the ﬁrst kind.
Revised: (12) where ˜ L = 2 λ max L − I n and J i ( − s ) is the Bessel function of the ﬁrst kind.

Original: The proof can be referred to (Hammond et al., 2011).
Revised: The proof can be referred to (Hammond et al., 2011).

Original: With this Chebyshev polynomial approximation, the computational cost of spectral graph wavelets is decreased to O ( M (cid:107) E (cid:107) + M × n ) .
Revised: With this Chebyshev polynomial approximation, the computational cost of spectral graph wavelets is decreased to O ( M (cid:107) E (cid:107) + M × n ) .

Original: Due the real world graphs are usually sparse, this computational difference can be very signiﬁcant.
Revised: Due the real world graphs are usually sparse, this computational difference can be very signiﬁcant.

Original:  E XPERIMENTS 4. D ATASETS
Revised:  E XPERIMENTS 4. D ATASETS

Original: We evaluate our proposed OctGCN on semi-supervised node classiﬁcation task.
Revised: We evaluate our proposed OctGCN on semi-supervised node classiﬁcation task.

Original: The experimental setup is closely followed (Yang et al., 2016; Kipf & Welling, 2017). Statistical overview of
Revised: The experimental setup is closely followed (Yang et al.

Original: [Table]
Revised: [Table]

Original: (this paper)
Revised: (this paper)

Original: [Table]
Revised: [Table]

Original: datasets is given in Table 1.
Revised: datasets is given in Table 1.

Original: Three real-world datasets are chosen as benchmarks: Citeseer, Cora and Pubmed (Sen et al., 2008).
Revised: Three real-world datasets are chosen as benchmarks: Citeseer, Cora and Pubmed (Sen et al., 2008).

Original: In these citation networks, nodes are documents with corresponding bag-of-words features and edges are citation links.
Revised: In these citation networks, nodes are documents with corresponding bag-of-words features and edges are citation links.

Original: Label rate denotes the ratio of labeled nodes fetched in training process.
Revised: Label rate denotes the ratio of labeled nodes fetched in training process.

Original: We keep the label rate consistent with the classic public split, which is 20 labeled nodes per class in each dataset for training.
Revised: We keep the label rate consistent with the classic public split, which is 20 labeled nodes per class in each dataset for training.

Original: Meantime, the test set contains 1000 labeled samples for prediction accuracy evaluation, and the validation set includes 500 labeled samples for determining hyper-parameters.
Revised: Meantime, the test set contains 1000 labeled samples for prediction accuracy evaluation, and the validation set includes 500 labeled samples for determining hyper-parameters.

Original: 4. B ASELINES
Revised: 4. B ASELINES

Original: We ﬁrst compare against traditional baselines, i.e. , label propagation (LP) (Zhu et al., 2003), iterative classiﬁcation algorithm (ICA)
Revised: We ﬁrst compare against traditional baselines, i.e. , label propagation (LP) (Zhu et al., 2003), iterative classiﬁcation algorithm (ICA)

Original: (Lu & Getoor, 2003), manifold regularization (ManiReg) (Belkin et al., 2006), semi-supervised embedding (SemiEmb) (Weston et al., 2012), skip-gram based graph embeddings (DeepWalk) (Perozzi et al., 2014) and Planetoid (Yang et al., 2016).
Revised: (Lu & Getoor, 2003), manifold regularization (ManiReg) (Belkin et al., 2006), semi-supervised embedding (SemiEmb) (Weston et al., 2012), skip-gram based graph embeddings (DeepWalk) (Perozzi et al., 2014) and Planetoid (Yang et al., 2016).

Original: Then we compare the most recent and state-of-the-art baselines from both spectral and spatial graph neural networks, since they are shown effective for semi-supervised settings.
Revised: Then we compare the most recent and state-of-the-art baselines from both spectral and spatial graph neural networks, since they are shown effective for semi-supervised settings.

Original: For spectral approaches based on convolution theorem, we compare our OctGCN with the Spectral CNN (Bruna et al., 2014), ChebyNet (Defferrard et al., 2016), Vanilla GCN (Kipf & Welling, 2017), GWNN (Xu et al., 2019a), LNet/AdaLNet (Liao et al., 2019) and SGC (Wu et al., 2019).
Revised: For spectral approaches based on convolution theorem, we compare our OctGCN with the Spectral CNN (Bruna et al., 2014), ChebyNet (Defferrard et al., 2016), Vanilla GCN (Kipf & Welling, 2017), GWNN (Xu et al., 2019a), LNet/AdaLNet (Liao et al., 2019) and SGC (Wu et al., 2019).

Original: For spatial based methods, we select the MoNet (Monti et al., 2017), GAT (Veliˇckovi´c et al., 2018),
Revised: For spatial based methods, we select the MoNet (Monti et al., 2017), GAT (Veliˇckovi´c et al., 2018),

Original: GIN (Xu et al., 2019b) and DGI (Velickovic et al., 2019) as comparisons.
Revised: GIN (Xu et al., 2019b) and DGI (Velickovic et al., 2019) as comparisons.

Original: 4. E XPERIMENTAL SETUP
Revised: 4. E XPERIMENTAL SETUP

Original: For all experiments, a 2-layer network of our model is constructed using TensorFlow (Abadi et al., 2015) with 64 hidden units.
Revised: For all experiments, a 2-layer network of our model is constructed using TensorFlow (Abadi et al., 2015) with 64 hidden units.

Original: We train our model utilizing the Adam optimizer (Kingma & Ba, 2014) with an initial learning rate lr = 0 .
Revised: We train our model utilizing the Adam optimizer (Kingma & Ba, 2014) with an initial learning rate lr = 0 .

Original: We terminate training if validation accuracy does not improve for 100 consecutive steps, and most runs ﬁnish in less than 200 steps as expected.
Revised: We terminate training if validation accuracy does not improve for 100 consecutive steps, and most runs ﬁnish in less than 200 steps as expected.

Original: We initialize the weights matrix following (Glorot & Bengio, 2010), employ 5 × 10 − 4 L2 regularization on weights and dropout input and hidden layers to prevent overﬁtting (Srivastava et al., 2014).
Revised: We initialize the weights matrix following (Glorot & Bengio, 2010), employ 5 × 10 − 4 L2 regularization on weights and dropout input and hidden layers to prevent overﬁtting (Srivastava et al., 2014).

Original: For hyper-parameters for constructing wavelets ψ s , we adopt the selection of the scaling parameter s and sparseness threshold t (the elements of ψ s are set to 0 when smaller than t ) as in (Xu et al., 2019a), i.e. , s = 0 .
Revised: For hyper-parameters for constructing wavelets ψ s , we adopt the selection of the scaling parameter s and sparseness threshold t (the elements of ψ s are set to 0 when smaller than t ) as in (Xu et al., 2019a), i.e. , s = 0 .

Original: 7 t = 1 × 10 − 5 for Citeseer, s = 1 . 

Revised: 7 t = 1 × 10 − 5 for Citeseer, s = 1 .

Original: t = 1 × 10 − 4 for Cora and s = 0 .t
Revised: t = 1 × 10 − 4 for Cora and s = 0 .t

Original: = 1 × 10 − 7 for Pubmed, since both smaller s and t are shown not sensitive to datasets.
Revised: = 1 × 10 − 7 for Pubmed, since both smaller s and t are shown not sensitive to datasets.

Original: For
Revised: For

Original: 0.8 Fraction of low-frequency components 0.
Revised: 0.8 Fraction of low-frequency components 0.

Original: C l a ss i f
Revised: C l a ss i f

Original: i c a t i on
Revised: i c a t i on

Original: A cc u r a cy
Revised: A cc u r a cy

Original: Best Fraction citeseer 0.
Revised: Best Fraction citeseer 0.

Original: 0.8 Fraction of low-frequency components 0.
Revised: 0.8 Fraction of low-frequency components 0.

Original: C l a ss i f
Revised: C l a ss i f

Original: i c a t i on
Revised: i c a t i on

Original: A cc u r a cy
Revised: A cc u r a cy

Original: Best Fraction cora 0.
Revised: Best Fraction cora 0.

Original: 0.8 Fraction of low-frequency components 0.
Revised: 0.8 Fraction of low-frequency components 0.

Original: C l a ss i f
Revised: C l a ss i f

Original: i c a t i on
Revised: i c a t i on

Original: A cc u r a cy
Revised: A cc u r a cy

Original: Best Fraction pubmed
Revised: Best Fraction pubmed

Original: [Table] [Table]
Revised: [Table] [Table]

Original: the only hyper-parameter of OctGCN , the optimal proportion d/n of low-frequency components for each dataset, is determined through grid search and studied in next Section.
Revised: the only hyper-parameter of OctGCN , the optimal proportion d/n of low-frequency components for each dataset, is determined through grid search and studied in next Section.

Original: The weights of lowand high-frequency components α L and α H are both initialized with 1 and learned automatically.
Revised: The weights of lowand high-frequency components α L and α H are both initialized with 1 and learned automatically.

Original: In experiments, Max - pooling is chosen to demonstrate the importance the low-frequency components.
Revised: In experiments, Max - pooling is chosen to demonstrate the importance the low-frequency components.

Original: 4. E XPERIMENTAL R ESULTS 4.4. P ERFORMANCE OF OctGCN ON N ODE C LASSIFICATION
Revised: 4. E XPERIMENTAL R ESULTS 4.4. P ERFORMANCE OF OctGCN ON N ODE C LASSIFICATION

Original: In Table 2, we demonstrate how our model performs on public splits taken from (Yang et al., 2016).
Revised: In Table 2, we demonstrate how our model performs on public splits taken from (Yang et al., 2016).

Original: The results of baselines are strictly consistent with the numbers from literature.
Revised: The results of baselines are strictly consistent with the numbers from literature.

Original: With the limited information given in semi-supervised learning, We achieve a average test accuracy of 72 .
Revised: With the limited information given in semi-supervised learning, We achieve a average test accuracy of 72 .

Original: 5% , and 80 .
Revised: 5% , and 80 .

Original: 5% on Citeseer, Cora and Pubmed, respectively.
Revised: 5% on Citeseer, Cora and Pubmed, respectively.

Original: As OctGCN learned the octave feature representations for graph in spectral domain, it can demonstrate the meaningful information extracted from the underlying "true signal" from low-frequency over high-frequency.
Revised: As OctGCN learned the octave feature representations for graph in spectral domain, it can demonstrate the meaningful information extracted from the underlying "true signal" from low-frequency over high-frequency.

Original: This is the main reason that explains why OctGCN outperforms other baseline methods.
Revised: This is the main reason that explains why OctGCN outperforms other baseline methods.

Original: 4.4. A NALYSIS ON INTERPRETABILITY
Revised: 4.4. A NALYSIS ON INTERPRETABILITY

Original: In Figure 2, how the proportion d/n of low-frequency components affect the performance is studied.
Revised: In Figure 2, how the proportion d/n of low-frequency components affect the performance is studied.

Original: We ﬁne-tune the proportion in a range of { 0% , 5% , · · · , 95% } .
Revised: We ﬁne-tune the proportion in a range of { 0% , 5% , · · · , 95% } .

Original: The best proportion of low-frequency components are 15% , 5% , and 10% for Citeseer, Cora and Pubmed, respectively.
Revised: The best proportion of low-frequency components are 15% , 5% , and 10% for Citeseer, Cora and Pubmed, respectively.

Original: The learned weights of low- and high-frequency components α L and α H w.r.t the best proportion for each dataset are demonstrated in Table 3, accordingly.
Revised: The learned weights of low- and high-frequency components α L and α H w.r.t the best proportion for each dataset are demonstrated in Table 3, accordingly.

Original: It’s clearly to note that the small proportion of lowfrequency components are essential to the learning octave feature representation.
Revised: It’s clearly to note that the small proportion of lowfrequency components are essential to the learning octave feature representation.

Original: The results are in line with the importance of low-frequency in GSP and bring interpretability to the nature of GCNs.
Revised: The results are in line with the importance of low-frequency in GSP and bring interpretability to the nature of GCNs.

Original: [Figure]
Revised: [Figure]

Original: 4.4. T -SNE VISUALIZATION OF LEARNED EMBEDDINGS
Revised: 4.4. T -SNE VISUALIZATION OF LEARNED EMBEDDINGS

Original: Table 4 presents the mean Silhouette Coefﬁcient (Rousseeuw, 1987) over all learned samples, larger the silhouette score is, better the clustering performs.
Revised: Table 4 presents the mean Silhouette Coefﬁcient (Rousseeuw, 1987) over all learned samples, larger the silhouette score is, better the clustering performs.

Original: We choose two representative baseline methods, i.e. , Vanilla GCN (Kipf & Welling, 2017) and GWNN (Xu et al., 2019a) for comparison.
Revised: We choose two representative baseline methods, i.e. , Vanilla GCN (Kipf & Welling, 2017) and GWNN (Xu et al., 2019a) for comparison.

Original: We can indicate that OctGCN achieves the best quality of embeddings.
Revised: We can indicate that OctGCN achieves the best quality of embeddings.

Original: Figure 3 depicts the t-SNE visualization (Maaten & Hinton, 2008) of learned embeddings on all three citation datasets.
Revised: Figure 3 depicts the t-SNE visualization (Maaten & Hinton, 2008) of learned embeddings on all three citation datasets.

Original: We can visualize the local and sparse property of spectral graph wavelets that utilized in GWNN and
Revised: We can visualize the local and sparse property of spectral graph wavelets that utilized in GWNN and

Original: OctGCN .
Revised: OctGCN .

Original: Further, the intersections of different classes are more separated in the results our OctGCN , since the octave feature embeddings learned from our model tend to capture the importance information in low-frequency components and effectively alleviate the noise from high-frequency.
Revised: Further, the intersections of different classes are more separated in the results our OctGCN , since the octave feature embeddings learned from our model tend to capture the importance information in low-frequency components and effectively alleviate the noise from high-frequency.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we propose OctGCN , a novel spectral-based graph convolutional neural network to learn the representation of graph with respect to different frequency components.
Revised: In this paper, we propose OctGCN , a novel spectral-based graph convolutional neural network to learn the representation of graph with respect to different frequency components.

Original: By distinct design of ﬁlters for low- and high-frequency, our model can effectively capture the octave feature representations and enhance the interpretability of GCNs.
Revised: By distinct design of ﬁlters for low- and high-frequency, our model can effectively capture the octave feature representations and enhance the interpretability of GCNs.

Original: To the best of our knowledge, this is the ﬁrst attempt on octave convolution for graphs.
Revised: To the best of our knowledge, this is the ﬁrst attempt on octave convolution for graphs.

Original: An interesting direction for future work is to extend the deﬁnition of octave convolution from spectral domain to spatial domain, in order to pursue more efﬁcient architectures for learning with graphs.
Revised: An interesting direction for future work is to extend the deﬁnition of octave convolution from spectral domain to spatial domain, in order to pursue more efﬁcient architectures for learning with graphs.

Revised:  A PPENDIX 6. L OW - FREQUENCY COMPONENTS IMPLY BETTER SMOOTHNESS AND I NFORMATION - TO - NOISE R ATIO

Revised: The different importance of low- and high-frequency components of graphs that contributes to the learning of modern GNNs is observed recently (Donnat et al., 2018; Maehara, 2019).

Revised: Concretely, the low-frequency components in graphs usually indicate smooth varying signals which can reﬂect the locality property (the neighbor nodes trend to be similar with each other) in graphs, thus they capture more information than the high-frequency components and should be more beneﬁcial for the representational learning as revealed in our experiments.

Revised: To this end, we investigate two measures in (Chen et al., 2019): Mean Average Distance (MAD) and Information-to-Noise Ratio (INR) w.r.t different spectrum basis.

Revised: Different spectrum basis has a corresponding node in graph, which we refer as low- and high- frequency node.

Revised: MAD reﬂects the smoothness of node representation.

Revised: Given a node v and the feature x v on it as signal, we take its eigenvector u v to multiply x v as the transformed feature ˜ x v in spectral domain.

Revised: Then the MAD of v is calculated by taking the average of cosine distances between the transformed feature of v and that of its 1-hop neighbors.

Revised: The lower MAD indicates better smoothness.

Revised: INR is deﬁned as the proportion of nodes from the same class as v through the 1-hop neighborhood of v .

Revised: INR reﬂects the information contained in nodes and the higher INR indicates the richer information.

Revised: For each dataset, we compute the average MAD and INR of top- 50% low-frequency nodes ( MAD L and INR L ) and rest high-frequency nodes ( MAD H and INR H ) and report the gap between lowand

Revised: high- frequency nodes as in Table 5.

Revised: We can observe that the low-frequency components in real graphs indeed are smoother than that of high-frequency (lower MAD).

Revised: Meanwhile, they have higher INR, which indicates they contain richer information.

Revised: In the vein, we argue that the low-frequency components may carry more information than that of the high-frequency components and should be more beneﬁcial for the representational learning.

Revised: Therefore, comparing with all other spectral-based methods that treat both low- and high-frequency components identically during training, our proposed octave convolutional structure model could gain more from this octave nature existing in graphs.