Title: OCTAVE GRAPH CONVOLUTIONAL NETWORK

Abstract: Many variants of Graph Convolutional Networks (GCNs) for representation learning have been proposed recently and have achieved fruitful results in various domains. Among them, spectral-based GCNs are constructed via convolution theorem upon a theoretical foundation from the perspective of Graph Signal Processing (GSP). However, despite most of them implicitly act as low-pass filters that generate smooth representations for each node, there is limited development on the full usage of underlying information from low-frequency components. Here, we first introduce the octave convolution on graphs in spectral domain. Accordingly, we present Octave Graph Convolutional Network (OctGCN), a novel architecture that learns representations for different frequency components regarding weighted filters and graph wavelets bases. We empirically validate the importance of lowfrequency components in graph signals on semi-supervised node classification and demonstrate that our model achieves state-of-the-art performance in comparison with both spectral-based and spatial-based baselines.

1. INTRODUCTION:

The family of Graph Convolutional Networks (GCNs) (Zhang et al., 2018) , which generalizes the traditional Convolutional Neural Networks (CNNs) from Euclidean structure data to graphs, has achieved a remarkable success in various application domains, including but not limited to social networks (Chen et al., 2018) , computer vision (Kampffmeyer et al., 2018 ), text classification (Yao et al., 2019) and applied chemistry (Liao et al., 2019) .

Existing methods of GCNs design falls into two categories: spatial-based methods and spectralbased methods (Wu et al., 2019) . On the surface, the spatial-based models directly perform information aggregation through graph topology. However, this aggregation can be viewed as a simplified convolution operation on spectral domain with the theoretical foundation in Graph Signal Processing (GSP). GSP extends the concepts in Discrete Signal Processing (DSP) and focuses on analyzing and processing data points whose relations are modeled as graph (Shuman et al., 2013; Ortega et al., 2018) . In standard signal processing problems, the underlying "real signal" is usually assumed to have low frequencies (Rabiner & Gold, 1975) . Recent works (Wu et al., 2019; Maehara, 2019) reveal that the spectral-based GCNs can be viewed as an implicit low-pass-type filter based denoising mechanism on the spectral domain. However, there is still a lack of the explicit learning architecture of GCNs to extract the beneficial information from low-frequency while making full use of the high-frequency under certain scenarios.

Considering the signal processing problem in computer vision, a natural image can be decomposed into a low spatial frequency component containing the smoothly changing structure, e.g., background, and a high spatial frequency component describing the rapidly changing fine details, e.g., outlines. To accommodate with this phenomenon, (Chen et al., 2019) proposed Octave Convolution (OctConv) to learn the octave feature representations, which factorizes convolutional feature maps into two groups at different spatial frequencies and process them with different convolutions at their corresponding frequency. Similarly, the octave mechanism is observed in graph representational learning more naturally. The eigenvectors associated with small eigenvalues carry smoothly varying signal, encouraging nodes that are neighbors to share similar values. In contrast, the eigenvectors associated with large eigenvalues carry sharply varying signal across edges (Donnat et al., 2018) . Accordingly, extending octave convolution from images to graphs sheds light on the explicit learning of GCNs regarding the representation of different frequencies.

Low-Frequency Components:



High-Frequency components:



Max Pooling:



Low-Frequency representations:



High-Frequency representations:



Octave Convolution:



Input Graph:

𝑓(𝑊 $ , 𝛼 $ ) 𝑓(𝑊 ( , 𝛼 ( ) Figure 1 : The overview of octave convolutional learning on graphs in spectral domain.

Different from the scale-space theory (Lindeberg, 2013) utilized in OctConv (Chen et al., 2019) to define the low-and high-frequency spaces, graph signal processing (GSP) provides us a way to directly divide the low-and high-frequency components based on the ascending ordered eigenvalues of Laplacian. Inspired from this, we propose to consider the octave feature in the spectral domain to construct a new graph convolutional model: Octave Graph Convolutional Network OctGCN. In OctGCN, with a particular design of filters for different spectrum, we allocate different weights on low-and high-frequency. Spectral graph wavelets are chosen as feature transformation bases due to their local and sparse property. Two parameters are further introduced to construct the filters for reducing the parameter complexity to the same as (Kipf & Welling, 2017) , which is critical when labels of training data are limited. Meanwhile, we employ the attention mechanism to learn the importance of low and high pass filters. Figure 1 provides the overview of the design of OctGCN in spectral domain. We validate the effectiveness of our model via experiments on semi-supervised node classification tasks, where the expressive power of GCNs is crucial to capture the underlying beneficial information in graph signals. Our results confirm the importance of low-frequency in graphs and bring interpretability to the innate character of GCNs. In addition, empirical results show that our proposed method consistently rivals the state-of-art methods from both spectral-based and spatial-based baselines on real-world datasets.

2. RELATED WORK:

Spectral convolutional networks on graphs. Existing methods of defining a convolutional operation on graphs can be broadly divided into two categories: spectral based and spatial based methods (Zhang et al., 2018) . We focus on the spectral graph convolutions in this paper. Spectral CNN (Bruna et al., 2014) first attempts to generalize CNNs to graphs based on the spectrum of the graph Laplacian and defines the convolutional kernel in the spectral domain. (Boscaini et al., 2015) further employs windowed Fourier transformation to define a local spectral CNN approach. ChebyNet (Defferrard et al., 2016) introduces a fast localized convolutional filter on graphs via Chebyshev polynomial approximation. Vanilla GCN (Kipf & Welling, 2017) further extends the spectral graph convolutions considering networks of significantly larger scale by several simplifications. (Khasanova & Frossard, 2017) learns graph-based features on images that are inherently invariant to isometric transformations. Cayleynets (Levie et al., 2018) alternatively introduce Cayley polynomials allowing to efficiently compute spectral filters on graphs. Lanczos algorithm is utilized in LanczosNet (Liao et al., 2019) to construct low-rank approximations of the graph Laplacian for convolution. SGC (Wu et al., 2019) further reduces the complexity of Vanilla GCN by successively removing the non-linearities and collapsing weights between consecutive layers. Despite their effective performance, all these convolution theorem based methods lack the strategy to explicitly treat low-and high-frequency components with different importance.

Spectral graph wavelets. Theoretically, the lifting scheme is proposed for the construction of wavelets that can be adapted to irregular graphs in (Sweldens, 1998) . (Hammond et al., 2011) defines wavelet transforms appropriate for graphs and describes a fast algorithm for computation via fast Chebyshev polynomial approximation. For applications, (Tremblay & Borgnat, 2014) utilizes graph wavelets for multi-scale community mining and obtains a local view of the graph from each node. (Donnat et al., 2018) introduces the property of graph wavelets that describes information diffusion and learns structural node embeddings accordingly. GWNN (Xu et al., 2019a) first attempts to construct graph neural networks with graph wavelets. These works emphasize the local and sparse property of graph wavelets for graph signal processing both theoretically and practically.

Octave feature representation. In computer vision, (Chen et al., 2019) first defines octave feature representations based on scale-space theory and reduces spatial redundancy of vanilla CNN models. (Durall et al., 2019) further leverages octave convolutions for designing stabilizing GANs. To our knowledge, this is the first time that octave feature representations are considered in irregular graph domain and established with graph convolutional neural networks.

3. PROPOSED APPROACH:



3.1. PRELIMINARY:

We denote G = {V, E} as an undirected graph, where |V | = n is the set of n nodes, and E is the set of edges. The adjacency matrix is defined as A with A i,j = A j,i describing the edge connecting node i and node j. The graph Laplacian matrix L is defined as the difference L = D -A, where D i,i = j A i,j is a diagonal degree matrix. The normalized graph Laplacian matrix is referred as

L = I n -D -1/2 AD -1/2

, where I n is the identity matrix. The graph Laplacian L can be decomposed into its eigenvalue components, L = U ΛU , such that for the set of eigenvalues in ascending order

{λ i } n-1 i=0 = λ 0 ≤ λ 1 ≤ • • • ≤ λ n-1 , the diagonal eigenvalue matrix is defined as Λ = diag(λ 0 , . . . , λ n-1 ) and U = (u 1 , u 2 , ..., u n ) is the eigenvector matrix.

Since L is a real symmetric matrix, it has real, non-negative eigenvalues

{λ i } n-1 i=0 = λ 0 ≤ λ 1 ≤ • • • ≤ λ n-1 ,

known as the frequencies of graph. These eigenvalues have associated a complete set of orthonormal eigenvectors in U , identified as Laplacian eigenvectors. In Graph Signal Processing (GSP), we denote frequency components with small/large eigenvalues of Laplacian as low/high frequencies. Given a signal x and a graph Laplacian L, the Graph Fourier Transform (GFT) of x with respect to L is defined as the signal x = U x, and the inverse (i)GFT of x with respect to L is x = U x (Shuman et al., 2013).

3.2. SPECTRAL GRAPH CONVOLUTION:

The spectral convolution on graphs is normally defined as the multiplication of a signal on every node x with a diagonal filter g θ = diag(θ) parameterized by θ in the Fourier domain:

EQUATION

The filters are usually understood as a function of the eigenvalues. Inspired by (Maehara, 2019) , we can decompose the spectral convolution process on graphs from the perspective of GSP as four steps: 1. Compute the graph bases U ; 2. Graph spectral transform on signal x with U ; 3. Filtering with g θ ; 4. Reconstruct the signal features in the spatial domain with U . In this sense, the design of filter g θ is essential to the performance of spectral convolution. Broadly, the filter design can be divided into two categories: it is either learned by the neural network (Bruna et al., 2014; Xu et al., 2019a) or directly fixed as the eigenvalues via approximation (Kipf & Welling, 2017; Wu et al., 2019) . In this paper, we will focus on the first kind.

Spectral CNN (Bruna et al., 2014) generalizes the convolutional net by operating on the spectrum of weights, given by the ordered eigenvectors of its graph Laplacian. The structure of k-th layer is constructed as:

EQUATION

where X k ∈ R n×p is the signal with p input channels and X k+1 ∈ R n×q is the convolved signal matrix. X k [:,i] and X k+1 [:,j] are the i-th and j-th column of X k and X k+1 , respectively. g θ of k-th layer is defined as F k i,j , which is a diagonal filter matrix to be learned for each input channel in spectral domain. h is a real valued nonlinear activation function, e.g., ReLU(•) = max (0, 1). Thus, the parameter complexity of Spectral CNN is O(n × p × q), which generally demands a huge amount of training data for parameter learning.

3.3. WHY SPECTRAL GRAPH WAVELET?:

Graph wavelet neural network (GWNN) (Xu et al., 2019a) expands the spectral convolution from Fourier transformation to wavelet transformation. Let g s (λ) = e -λs be a heat kernel filter with scaling parameter s. In GSP (Hammond et al., 2011; Shuman et al., 2013) , the spectral graph wavelet ψ si is defined as the signal resulting from the modulation in the spectral domain of a signal x centered around the associated node i. Then, the graph wavelet transform is conducted by employing a set of wavelets ψ s = (ψ s1 , ψ s2 , . . . , ψ sn ) as bases. Formally, the spectral graph wavelets are given as:

EQUATION

where U is Laplacian eigenvectors of L = D -A or normalized Laplacian L = I n -D -1 2 AD -1 2 , g s = diag g s (λ 1 ), g s (λ 2 ), . . . , g s (λ n ) is a scaling matrix with heat kernel. The inverse of graph wavelets ψ -1 s is obtained by simply replacing the g s (λ) in ψ s with g s (-λ) corresponding to the heat kernel (Donnat et al., 2018) . Similarly, smaller indices in graph wavelets correspond to lowfrequency components and vice versa.

Similar to GFT, after replacing the Fourier bases with spectral graph wavelets, the graph wavelet transformation of a signal x on graph is defined as x = ψ -1 s x and the inverse graph wavelet transform is x = ψ s x. Replacing the graph Fourier transform in spectral convolution (Equation 1) with graph wavelet transform, the graph wavelet convolution can be obtained as:

EQUATION

The benefits that spectral graph wavelet bases have over Fourier bases mainly fall into two aspects:

1. Given the sparse real-world networks, the graph wavelet bases are usually much more sparse than Fourier bases, e.g., the density of ψ s is 2.8% comparing with 99.1% of U (Xu et al., 2019a) .

The sparseness of graph wavelets makes them more computationally efficient for use. 2. In spectral graph wavelets, the signal ψ s resulting from heat kernel filter g s is typically localized on the graph and in the spectral domain (Shuman et al., 2013) . By adjusting the scaling parameter s, one can easily constrain the range of localized neighborhood. Smaller values of s generally associate with smaller neighborhoods.

Employing the same strategy as in Spectral CNN (Bruna et al., 2014) , GWNN designs the same diagonal filter F k i,j to be learned for each input channel. The structure of k-th layer of GWNN is:

EQUATION

Note that both Spectral CNN and GWNN employ the same filters for learning full frequency components. As mentioned before, the parameter complexity of Spectral CNN is large since each pair of input and output channel requires learning an individual diagonal filter matrix F k i,j . GWNN further reduces the parameter complexity by dividing each layer into two components: feature transformation and graph convolution:

EQUATION

graph convolution :

EQUATION

)

where W k ∈ R p×q is the feature transformation parameter matrix similar to Vanilla GCN (Kipf & Welling, 2017) . In such a way, the feature transformation operation is detached from graph convolution and the parameter complexity is decreased from from O(n × p × q) to O(n + p × q).

3.4. OCTAVE CONVOLUTIONAL LAYER:

In contrast to the scale-space theory-based octave feature representation utilized in computer vision (Lindeberg, 2013) , Graph Signal Processing provides us a more principle way in the spectral domain. To better capture the different importance of low-and high-frequency components and combining the benefits of graph wavelets, we can naturally construct each layer in an octave convolution manner by learning two different filters as:

EQUATION

where -d) are the diagonal filter matrix for graph convolution to be learned with different weights for low-and high-components, respectively. d is the hyper-parameter to select the proportion d/n of low frequency components. ψ sL and ψ sL are the corresponding lowand high-frequency graph wavelet bases. Further, with a pooling operation on the outputs and non-linear activation function, the structure of the k-th layer in our structure can be defined as

F k L ∈ R d×d and F k H ∈ R (n-d)×(n

EQUATION

We refer this proposed architecture as Octave Graph Convolutional Network (OctGCN).

Since the parameters in the diagonal convolution filtering kernels could be huge especially for large graphs, the graph-based semi-supervised learning might prohibit the parameter learning due to the limited amount of training data. To mitigate this issue, we further reduce the parameter complexity by constructing the graph convolution kernel F k with two parameters α L and α H , and keeping the same weight matrix W shared between low-and high-frequency components as

X k+1 = h Pooling(ψ sL    α L . . . α L    ψ -1 sL X k W k , ψ sH    α H . . . α H    ψ -1 sH X k W k )

(11) For the learning of weights of low-and high-frequency components α L and α H , we adopt the attention strategy to constraint them within the scale of (0, 1):

α * = softmax(α * ) = exp (α * ) * exp(α * ) , * = L, H

Hence, we introduce three more hyper-parameters to be tuned: α L and α H control the importance of low and high frequency components, and parameter d specify the ratio of low frequencies we expect to represent the graph. In this way, we reduces the parameter complexity from O(n + p × q) in GWNN (Xu et al., 2019a) to O(p×q), which is the same as Vanilla GCN (Kipf & Welling, 2017) .

3.5. FAST SPECTRAL GRAPH WAVELET APPROXIMATION VIA CHEBYSHEV POLYNOMIALS:

Directly computing the transformation according to Equation 3 is intensive for large graphs, since diagonalizing Laplacian L commonly requires O(n 3 ) operations. Luckily, (Hammond et al., 2011) provides us a method to fast approximate the spectral graph wavelet via Chebyshev polynomials. Let s be the fixed scaling parameter in the heat filter kernel g s (λ) = e -λs and M be the degree of the Chebyshev polynomial approximations for the scaled wavelet (Larger value of M yields more accurate approximations but higher computational cost in opposite), the graph wavelet is given by

EQUATION

)

where L = 2 λmax L -I n and J i (-s) is the Bessel function of the first kind. The proof can be referred to (Hammond et al., 2011) . With this Chebyshev polynomial approximation, the computational cost of spectral graph wavelets is decreased to O(M E +M ×n). Due the real world graphs are usually sparse, this computational difference can be very significant.

4. EXPERIMENTS:



4.1. DATASETS:

We evaluate our proposed OctGCN on semi-supervised node classification task. The experimental setup is closely followed (Yang et al., 2016; Kipf & Welling, 2017) . Statistical overview of Table 2 : Experimental results (in percent) on semi-supervised node classification.

Model:

Citeseer Cora Pubmed LP (Zhu et al., 2003) 45.3 68.0 63.0 ICA (Lu & Getoor, 2003) 69.1 75.1 73.9 ManiReg (Belkin et al., 2006) 60.1 59.5 70.7 SemiEmb (Weston et al., 2012) 59.6 59.0 71.1 DeepWalk (Perozzi et al., 2014) 43.2 67.2 65.3 Planetoid (Yang et al., 2016) 64.7 75.7 77.2 Spectral CNN (Bruna et al., 2014) 58.9 73.3 73.9 ChebyNet (Defferrard et al., 2016) 69.8 81.2 74.4 Vanilla GCN (Kipf & Welling, 2017) 70.3 81.5 79.0 GWNN (Xu et al., 2019a) 71.7 82.8 79.1 LNet (Liao et al., 2019) 66.2 ± 1.9 79.5 ± 1.8 78.3 ± 0.3 AdaLNet (Liao et al., 2019) 68.7 ± 1.0 80.4 ± 1.1 78.1 ± 0.4 SGC (Wu et al., 2019) 71.9 ± 0.1 81.0 ± 0.0 78.9 ± 0.0 MoNet (Monti et al., 2017) -81.7 ± 0.5 78.8 ± 0.3 GAT (Veličković et al., 2018) 72.5 ± 0.7 83.0 ± 0.7 79.0 ± 0.3 GIN (Xu et al., 2019b) 66.1 ± 0.9 77.6 ± 1.1 77.0 ± 1.2 DGI (Velickovic et al., 2019) 71.8 ± 0.7 82.3 ± 0.6 76.8 ± 0.6 OctGCN (this paper)

72.1 ± 0.2 83.5 ± 0.2 80.5 ± 0.3 datasets is given in Table 1 . Three real-world datasets are chosen as benchmarks: Citeseer, Cora and Pubmed (Sen et al., 2008) . In these citation networks, nodes are documents with corresponding bag-of-words features and edges are citation links. Label rate denotes the ratio of labeled nodes fetched in training process. We keep the label rate consistent with the classic public split, which is 20 labeled nodes per class in each dataset for training. Meantime, the test set contains 1000 labeled samples for prediction accuracy evaluation, and the validation set includes 500 labeled samples for determining hyper-parameters.

4.2. BASELINES:

We first compare against traditional baselines, i.e., label propagation (LP) (Zhu et al., 2003) , iterative classification algorithm (ICA) (Lu & Getoor, 2003) , manifold regularization (ManiReg) (Belkin et al., 2006) , semi-supervised embedding (SemiEmb) (Weston et al., 2012) , skip-gram based graph embeddings (DeepWalk) (Perozzi et al., 2014) and Planetoid (Yang et al., 2016) .

Then we compare the most recent and state-of-the-art baselines from both spectral and spatial graph neural networks, since they are shown effective for semi-supervised settings. For spectral approaches based on convolution theorem, we compare our OctGCN with the Spectral CNN (Bruna et al., 2014) , ChebyNet (Defferrard et al., 2016) , Vanilla GCN (Kipf & Welling, 2017) , GWNN (Xu et al., 2019a) , LNet/AdaLNet (Liao et al., 2019) and SGC (Wu et al., 2019) . For spatial based methods, we select the MoNet (Monti et al., 2017) , GAT (Veličković et al., 2018) , GIN (Xu et al., 2019b) and DGI (Velickovic et al., 2019) as comparisons.

4.3. EXPERIMENTAL SETUP:

For all experiments, a 2-layer network of our model is constructed using TensorFlow (Abadi et al., 2015) with 64 hidden units. We train our model utilizing the Adam optimizer (Kingma & Ba, 2014) with an initial learning rate lr = 0.01. We terminate training if validation accuracy does not improve for 100 consecutive steps, and most runs finish in less than 200 steps as expected. We initialize the weights matrix following (Glorot & Bengio, 2010) , employ 5 × 10 -4 L2 regularization on weights and dropout input and hidden layers to prevent overfitting (Srivastava et al., 2014) .

For hyper-parameters for constructing wavelets ψ s , we adopt the selection of the scaling parameter s and sparseness threshold t (the elements of ψ s are set to 0 when smaller than t) as in (Xu et al., 2019a) , i.e., s = 0.7 t = 1 × 10 -5 for Citeseer, s = 1.0 t = 1 × 10 -4 for Cora and s = 0.5 t = 1 × 10 -7 for Pubmed, since both smaller s and t are shown not sensitive to datasets. For the only hyper-parameter of OctGCN, the optimal proportion d/n of low-frequency components for each dataset, is determined through grid search and studied in next Section. The weights of lowand high-frequency components α L and α H are both initialized with 1 and learned automatically. In experiments, Max-pooling is chosen to demonstrate the importance the low-frequency components.

4.4. EXPERIMENTAL RESULTS:



4.4.1. PERFORMANCE OF OctGCN ON NODE CLASSIFICATION:

In Table 2 , we demonstrate how our model performs on public splits taken from (Yang et al., 2016) . The results of baselines are strictly consistent with the numbers from literature. With the limited information given in semi-supervised learning, We achieve a average test accuracy of 72.1%, 83.5%, and 80.5% on Citeseer, Cora and Pubmed, respectively. As OctGCN learned the octave feature representations for graph in spectral domain, it can demonstrate the meaningful information extracted from the underlying "true signal" from low-frequency over high-frequency. This is the main reason that explains why OctGCN outperforms other baseline methods.

4.4.2. ANALYSIS ON INTERPRETABILITY:

In Figure 2 , how the proportion d/n of low-frequency components affect the performance is studied. We fine-tune the proportion in a range of {0%, 5%, visualization (Maaten & Hinton, 2008) of learned embeddings on all three citation datasets. We can visualize the local and sparse property of spectral graph wavelets that utilized in GWNN and OctGCN. Further, the intersections of different classes are more separated in the results our Oct-GCN, since the octave feature embeddings learned from our model tend to capture the importance information in low-frequency components and effectively alleviate the noise from high-frequency.

5. CONCLUSION:

In this paper, we propose OctGCN, a novel spectral-based graph convolutional neural network to learn the representation of graph with respect to different frequency components. By distinct design of filters for low-and high-frequency, our model can effectively capture the octave feature representations and enhance the interpretability of GCNs. To the best of our knowledge, this is the first attempt on octave convolution for graphs. An interesting direction for future work is to extend the definition of octave convolution from spectral domain to spatial domain, in order to pursue more efficient architectures for learning with graphs. The different importance of low-and high-frequency components of graphs that contributes to the learning of modern GNNs is observed recently (Donnat et al., 2018; Maehara, 2019) . Concretely, the low-frequency components in graphs usually indicate smooth varying signals which can reflect the locality property (the neighbor nodes trend to be similar with each other) in graphs, thus they capture more information than the high-frequency components and should be more beneficial for the representational learning as revealed in our experiments. To this end, we investigate two measures in (Chen et al., 2019) : Mean Average Distance (MAD) and Information-to-Noise Ratio (INR) w.r.t different spectrum basis. Different spectrum basis has a corresponding node in graph, which we refer as low-and high-frequency node.

MAD reflects the smoothness of node representation. Given a node v and the feature x v on it as signal, we take its eigenvector u v to multiply x v as the transformed feature xv in spectral domain.

Then the MAD of v is calculated by taking the average of cosine distances between the transformed feature of v and that of its 1-hop neighbors. The lower MAD indicates better smoothness. INR is defined as the proportion of nodes from the same class as v through the 1-hop neighborhood of v.

INR reflects the information contained in nodes and the higher INR indicates the richer information.

For each dataset, we compute the average MAD and INR of top-50% low-frequency nodes (M AD L and IN R L ) and rest high-frequency nodes (M AD H and IN R H ) and report the gap between lowand high-frequency nodes as in Table 5 .

:



Figures and Tables:

Figure 3: The t-SNE visualization of OctGCN comparing with spectral convolution based baselines. Each color corresponds to a different class that the embeddings belongs to.

The overview of dataset statistics. Dataset Nodes Edges Classes Features Label rate

Table content: <table><tr><td>Citeseer</td><td>3,327</td><td>4,732</td><td>6</td><td>3,703</td><td>0.036</td></tr><tr><td>Cora</td><td>2,708</td><td>5,429</td><td>7</td><td>1,433</td><td>0.052</td></tr><tr><td colspan="3">Pubmed 19,717 44,338</td><td>3</td><td>500</td><td>0.003</td></tr></table>

The performance of learned OctGCN w.r.t the proportion of low-frequency components. The best fraction is marked with the red vertical line.

Table content: <table><tr><td>Classification Accuracy</td><td>0.712 0.714 0.716 0.718 0.720 0.722</td><td>Best Fraction</td><td>citeseer</td><td>Classification Accuracy</td><td>0.810 0.815 0.820 0.825 0.830 0.835</td><td>Best Fraction</td><td>cora</td><td>Classification Accuracy</td><td>0.7925 0.7950 0.7975 0.8000 0.8025 0.8050 0.8075</td><td>Best Fraction</td><td>pubmed</td></tr><tr><td/><td/><td/><td/><td/><td>0.805</td><td/><td/><td/><td>0.7900</td><td/><td/></tr><tr><td/><td colspan="3">0.0 Fraction of low-frequency components 0.2 0.4 0.6 0.8</td><td/><td/><td colspan="2">0.0 Fraction of low-frequency components 0.2 0.4 0.6 0.8</td><td/><td/><td colspan="2">0.0 Fraction of low-frequency components 0.2 0.4 0.6 0.8</td></tr><tr><td colspan="2">Figure 2:</td><td/><td/><td/><td/><td/><td/><td/><td/><td/><td/></tr></table>

Learned weights α L and α H of OctGCN for low and high frequency w.r.t the best fraction of low frequency components d/n (number followed after the name of datasets).

Table content: <table><tr><td>Dataset</td><td colspan="2">Citeseer (15%)</td><td colspan="2">Cora (5%)</td><td colspan="2">Pubmed (10%)</td></tr><tr><td>Octave filter weights</td><td>α L</td><td>α H</td><td>α L</td><td>α H</td><td>α L</td><td>α H</td></tr><tr><td>Learned value</td><td colspan="5">0.838 0.162 0.722 0.278 0.860</td><td>0.140</td></tr></table>

The mean Silhouette Coefficient of learned samples. Larger is better.

Table content: <table><tr><td>Dataset</td><td/><td>Citeseer</td><td/><td/><td>Cora</td><td/><td/><td>Pubmed</td><td/></tr><tr><td>Model</td><td colspan="9">Vanilla GCN GWNN OctGCN Vanilla GCN GWNN OctGCN Vanilla GCN GWNN OctGCN</td></tr><tr><td>Silhouette score</td><td>0.038</td><td>0.050</td><td>0.083</td><td>0.119</td><td>0.153</td><td>0.220</td><td>0.110</td><td>0.130</td><td>0.171</td></tr></table>

presents the mean Silhouette Coefficient(Rousseeuw, 1987) over all learned samples, larger the silhouette score is, better the clustering performs. We choose two representative baseline methods, i.e., Vanilla GCN(Kipf & Welling, 2017) and GWNN(Xu et al., 2019a)  for comparison. We can indicate that OctGCN achieves the best quality of embeddings. Figure3depicts the t-SNE

Table content: <table/>

The average gap of MAD and INR between the low-and high-frequency components in benchmark datasets. Smaller MAD indicates better smoothness and larger INR indicates that the frequency components contain richer information. -M AD H -3.7 × 10 -4 -9.0 × 10 -5 -1.0 × 10 -5

Table content: <table><tr><td>Dataset</td><td>Citeseer</td><td>Cora</td><td>Pubmed</td></tr><tr><td>M AD L IN R L -IN R H</td><td>0.018</td><td>0.002</td><td>0.005</td></tr></table>

Peer review (iclr2020):
This paper extends the previous graph wavelet neural network with separate computation for low frequency and high frequency part. The idea is to bring the octave convolution in vision to the graph domain. Experiments on three benchmark node classification tasks show comparable performances as previous methods. Overall the paper is written in a coherent and self-contained way, where the paper clearly states the related work and the contribution of this newly proposed work. Also it is interesting to see that normalizing the diagonal filter, tying the weights for low/high frequency parts would make the generalization better. However, there are several major concerns with the paper: 1. The contribution is somewhat limited. The main component is based on GWNN. While the GWNN itself takes the full spectrum of basis already, the formulation in (8) and (9) should somehow capture the same information. I think the paper spends too much content on the reviews, while lacks the intuition or theoretical explanations of the proposed formulation. 2. Having marginal improvement on the three benchmarks is not that interesting. Also given the results are mixed with other baselines, I think more experiments (e.g., on large graphs, or graph-level supervised tasks, etc.) are necessary to demonstrate the empirical gain using the proposed formulation. 3. Regarding the experiments, is it true that d/n=0 and d/n=1 should have exactly the same results? 4. In Figure 2, I guess d/n=0 should be reduced to the GWNN. But it seems the performance is different than GWNN on citeseer and cora. Why there’s such inconsistency, or did I miss anything?