Graph Autoencoders with Deconvolutional Networks
Authors: Jia Li
Jianwei Yu
Da-Cheng Juan
HAN Zhichao
Arjun Gopalan
Hong Cheng
Andrew Tomkins
Venue:  (2021)
Keywords: graph autoencoders
graph deconvolutional networks

Abstract: Recent studies have indicated that Graph Convolutional Networks (GCNs) act as a $\textit{low pass}$ filter in spectral domain and encode smoothed node representations.  In this paper, we consider their opposite, namely Graph Deconvolutional Networks (GDNs) that reconstruct graph signals from smoothed node representations. We motivate the design of Graph Deconvolutional Networks via a combination of inverse filters in spectral domain and de-noising layers in wavelet domain, as the inverse operation results in a $\textit{high pass}$ filter and may amplify the noise.  Based on the proposed GDN, we further propose a graph autoencoder framework that first encodes smoothed graph representations with GCN and then decodes accurate graph signals with GDN.  We demonstrate the effectiveness of the proposed method on several tasks including unsupervised graph-level representation, social recommendation  and graph generation.

Under review as a conference paper at ICLR 2021
GRAPH AUTOENCODERS WITH DECONVOLUTIONAL NETWORKS
Anonymous authors Paper under double-blind review
ABSTRACT
Recent studies have indicated that Graph Convolutional Networks (GCNs) act as a low pass filter in spectral domain and encode smoothed node representations. In this paper, we consider their opposite, namely Graph Deconvolutional Networks (GDNs) that reconstruct graph signals from smoothed node representations. We motivate the design of Graph Deconvolutional Networks via a combination of in- verse filters in spectral domain and de-noising layers in wavelet domain, as the inverse operation results in a high pass filter and may amplify the noise. Based the proposed GDN, we further propose graph autoencoder framework that first encodes smoothed graph representations with GCN and then decodes accu- rate graph signals with GDN. We demonstrate the effectiveness of the proposed method on several tasks including unsupervised graph-level representation, social recommendation and graph generation.
INTRODUCTION
Autoencoders have demonstrated excellent performance tasks such as unsupervised representa- tion learning (Bengio| 2009) and de-noising (Vincent et al.] 2010). Recently, several studies (Zeiler & Fergus 2014 Long al. 2015) have demonstrated that the performance of autoencoders can be further improved by encoding with Convolutional Networks and decoding with Deconvolutional Networks ZZiler al. 2010). Notably. Noh et al. (2015) present a novel symmetric architec- ture that provides a bottom-up mapping from input signals to latent hierarchical feature space with (convolution, pooling) operations and then maps the latent representation back the input space with {deconvolution, unpooling} operations. While this architecture has been successful when pro- cessing features with structures existed in the Euclidean space (e.g.. images), recently there has been surging interest in applying such framework on Euc-lucli data like graphs. However, ex- tending this autoencoder framework to graph-stre rrahh-struttured data requires Graph Deconvolutional opera- tions, which remains open-ended and hasn't 't been well-studied as opposed to the large body of works that have already been proposed for Graph Convolutional Networks (Defferrard et al., 2016: Kipf) & Welling, 2017). In this paper, we study the characteristies of Graph Deconvolutional Networks (GDNs), and observe de-noising to be the key for effective deconvolutional operations. Therefore, propose wavelet-based module (Hammond al.) 2011) that serves as de-noising mechanism after the signals reconstructed in the spectral domain (Shuman et al., 2013) for deconvolutional networks.
Most GCNs proposed by prior arts, e.g., Cheby-GCN (Defferrard et al. 2016) and GCN (Kipf &
Welling 2017). exploit spectral graph convolutions (Shuman et al. 2013) and Chebyshev poly- nomials Hammmnd et al. (01T)) to retain coarse-grained information and avoid explicit eigen- decomposition of the graph Laplacian. Until recently. Wu al. (2019) and Donnat et al. (2018) have noticed that GCN acts as a low pass filter in spectral domain and retains smoothed represen- tations. Inspired by prior arts in the domain of signal deconvolution (Kundur & Hatzinakos] 1996). we propose to design GDN by using high pass filters as the counterpart of low pass filters embod- ied GCNs. Due to the nature signal deconvolution being ill-posed, several prior arts (Donoho & Johnstone 1994 Figueiredo & Nowak 2003) rely on transforming these signals into another domain (c.g.. spectral domain) where the problem can be better posed and resolved. Furthermore,
Neelamani al. (004) observe inverse filters in spectral domain may amplify the noise, and observe the same phenomenon for GDNs. Therefore, inspired by their proposed hybrid spectral- wavelet method-inverre signal reconstruction in spectral domain followed by de-noising step in

---

Under review as a conference paper at ICLR 2021
wavelet domain we introduce spectral-wavelet GDN to decode the smoothed representations into the imput graph signals. The proposed waaelee GDN employs spectral graph convolutions with high pass filter to obtain inversed signals and then de-noises the inversed signals in wavelet domain. In addition, we apply Maclaurin series as a fast approximation technique to compute both high pass filters and wavelet kernels (Donnat al. 2018).
With the proposed spectral-wavelet GDN, we further propose a graph autoencoder (GAE) frame- work that resembles the symmetric fashion of architectures (Noh al., 2015). We then evaluate the effectiveness of the proposed GAE framework with three popular and important tasks: unsuper- vised graph-level representation (Sun et al., 2020), social recommendation (Jamali & Ester, 2010) and graph generation. In the first task, the proposed GAE outperforms the state-of- of-the-arts on graph classification in an unsupervised fashion, along with significant improvement on running time. In the second task, the performance of our proposed GAE is on par with the tfateooftthe-an on the rec- ommendation accuracy; at the meantime, the proposed GAE demonstrates strong robustness against rating noises and achieves the best recommendation diversification (Ziegler et al. 2005). In the third task, our proposed GDN can enhance the generation performance of popular variational autoencoder frameworks including VGAE (Kipf & Welling, 2016 and Graphite (Grover et al. 2019
RELATED WORK
Deconvolutional networks The area of signal deconvolution (Kundur & Hatzinakos 1996) has a long history in the signal processing community and is about the process of estimating the true signals given the degraded or smoothed signal characteristics (Banham & Katsaggelos) 1997.. Later
deep learning studies (Zeiler et al., 2010; Noh al. 2015) consider deconvolutional networks as the opposite operation for Convolutional Neural Networks (CNNs) and have mainly focused on Euclidean structures, e.g., image. Some work (Dumoulin & Visin 2016) notices Zeiler et al. (2010) in essence a transposed convolution network as it differs from what is used in the signal processing community. For deconvolutional networks in -unEEuciddenn structures like graphs, the study is still sparse. Feizi et al.] (2013) propose the network deconvolution as inferring the true network given partially observed structure. It relies on explicit cigen-decomposition and cannot be used as the counterpart for GCN. Yang & Segarra (2018) formulate the deconvolution as a pre- processing step on the observed signals, in order to improve classification accuracy. Zhang al. (2020) consider recovering graph signals from the latent representation. However, it just adopts the filter design used in GCN and sheds little insight into the internal operation of GDN.
Graph autoencoders Since the introduction of Graph Neural Networks (GNNs) (Kipf & Welling
2017 Defferrard et al. (016) and autoencoders (AEs), many studies (Kipf & Welling 2016 Grover et al. 2019) have used GNNs and AEs to encode to and decode from latent representations. Recently graph pooling has emerged as research topic that also contributes to the development of graph autoencoders. Common practices include DIFFPOOL (Ying et al.) 2018), SAGPool (Lee et al., 2019), MinCut-Pool (Bianchi et al., 2020). Although some encouraging progress has been achieved, there is still no work about graph deconvolution that can up-sample latent feature maps to restore their original resolutions (Gao & Ji 2019). In this regard, current graph autoencoders bypass the difficulty via (1) ponmmparamttri decoders (Kipf & Welling 2016; Deng et al.] 2020 [i et al. 2020), (2) GCN decoders (Grover al. 2019 Gao & Ji 2019), and (3) multilayer perceptron (MIP)
decoders (Simonovsky & Komodakis] 2018).
3 GRAPH AUTOENCODER FRAMEWORK
Formally, we are given an undirected, unweighted graph G = A, is the node set and N |V] denotes the number of nodes. The adjacency matrix E N represents the graph structure. The feature matrix X R Nxd represents the node attributes. Our goal is to learn an encoder and a decoder to map between the space of graph G and their latent factors Groot = (V A pood Z).
We show schematic diagram of our proposed framework in Figure 1
2

---

Under review as a conference paper at ICLR 2021
Encoder
Decoder
Swetraal
Carheite
Sprcten
Spudrral
Eigcorabae
Eggarralsc
Egguralac
Engurabe
GCN
Pooling
Unpooling
GDN
Input graph
E R N
Grool
R
Output graph
Figure 1: Schematic diagram of the proposed graph autoencoder framework. We plot graphs in spatial domain together with their signal coefficients spectral domain. the encoding stage, we run GCN model to derive smoothed node representations and obtain coarse coarse-grained graph via pool layer. In the decoding stage, we upscale the coarsened graph back and reconstruct the graph signal via GDN model.
3.1 ENCODER
Our encoder consists of several layers of Graph Convolutional Networks (GCNs) (Kipf & Welling 2017) and a pooling layer, to produce coarser representations of input graphs.
The convolutional layers are used to derive smoothed node representations, such that nodes that are similar in topological space should be close enough Euclidean space.
Convolution
H GCN A, X);
where H E Nxe denotes smoothed node representations. Specifically, Wu et al. (2019) show that GCN is low pass filter in spectral domain with (A(A)) = X4 where {A: are the eigenvalues of the normalized Laplacian matrix Laym matrices of the input graph A respectively.
D $ LD 4, L and D are the Laplacian and degree
Pooling We follow Lee et al. (2019) and Li et al. (2119) by using the self attention mechanism to pool the fine-grained graph into coarse gaarsed representations,
S softmax (tanh (HW))
(2)
where W1 E Rx and W22 E Rd K are two weight matrices, W1 is used for feature transformation and W2 is used to infer the membership of each node with respect each cluster Vg.
Similar to Ying et al. (2018). we compute the coarsed graph structure Apoot
IR x and feature
representation Z E I K as follows:
= H:
Apoot = S AS.
(3)
Note here (2oo) is size invariant and permutation invariant, as pointed by Li et al. 2019
3.2 DECODER
Our decoder consists of an unpooling layer and several layers of Graph Deconvolutional Networks (GDNs), produce fine-grained graphs from the encoded Goool
Unpooling
We follow Bianchi et al. (022)) to upscale the coarsened graph back to its original size,
H' sz;
A'
S ST.
The deconvolutional layers are used to recover the original graph features given smoothed node representations,
Deconvolution
X' GDN(A',H'',
(5)
where X' e Nxd denotes the recovered graph features. We shall further discuss our design of GDN in Section
3

---

Under review as a conference paper at ICLR 2021
3.3 THE LOSS FUNCTION
The overall reconstruction loss is weighted sum of structure reconstruction loss and feature recon- struction loss.
L L= A))
(6)
where ((..) denotes differential distance metric, e.g., [(::) MSB(;,) for continuous input, and MSE(.,.) represents mean squared emror.
4 GRAPH DECONVOLUTIONAL NETWORKS
In this section, we present our design of Graph Deconvolutional Networks (GDNs), A naive de- convolutional nets can be obtained using the inverse operator 9. (A,) = - in spectral domain. Unfortunately, inverse operation results & Johnstone 1994] Figueiredo & Nowak (003). In this regard, we propose an efficient, hybrid epectral-waveler deconvolutional network that performs inverse signal recovery spectral domain first, and then conducts de-noising step in wavelet domain to remove the amplified noise (Neela-
high pass filter and may amplify the noise (Donoho
mani et al. 2004).
4.1 INVERSE OF GCN
In order recover graph signals from the latent representation computed by GCN encoder, proposed a naive approach-inverse GCN with the inverse filter as g. *>:)
A-A; in spectral
domain. The spectral graph convolution on signal RN is defined as:
9e = U'diag (A1), .e. (Ax())))U=== x= USe (A)UTa x,
(7)
where U is the eigenvector matrix of the normalized graph Laplacian Leym = UAUT. Then, we apply Maclaurin series approximation on ge (A) = En=o A and get a fast algorithm as below:
Je * U - A"U ==
x Lym
(8)
As in GCN (Kipf & Welling 2017), when the first order approximation used to address overfitting. (A:) = 1 + As which is apparently a high pass filter.
we derive spectral filter with 9c
Following GCN (Kipf & Welling 2017 7 a feature transformation is applied to increase filter
strength. Recap the GDN in Section
3.2 the inverse version of GCN can be written as:
M = (IN + L'yym)
where L'yy is the corresponding normalized graph Laplacian matrix for A', H' is the smoothed representations and W3 is the parameter set be learned.
Compared with directly using GCN for signal reconstruction in Zhang et al. (2020), the proposed inverse GCN demonstrates its efficacy in recovering the high frequency signals of the graph, as shown in Figure 2 (b) and (d). We shall further discuss this point in Section 4.3
4.2 WAVELET DE-NOISING
The proposed inverse GCN may over-amplifies the high frequency signals and introduce undesir- able noise into the output graph. Thus, de-noising process is necessary to separate the useful information and the noise. Wavelet-based methods have a strong impact the field of de-noising, especially in image restoration when noise is amplified by inverse filters (Neelamani et al.) 2004). Notably, coefficients in wavelet domain could allow the noise be easily separated from the useful information, while transformation into spaces such spectral domain does not share the same char- acteristics. the literature, many wavelet de-noising methods have been proposed, c.g., SureShrink (Donoho & Johnstonee] 1994), BayesShrink (Chang al. 2000), and they differ in how they estimate the separating threshold. Our method generalizes these threshold ideas and automatically separates the noise from the useful information with learning baaneng-bbsed approach.
Consider a set of wavelet bases V, = (V (W......, V.N), where each one V,, denotes a signal on graph diffused away from node and is scaling parameter (Xu et al. 2019a), the wavelet bases
4

---

Under review as a conference paper at ICLR 2021
Wavelet kemne g,(A)
Inverse wavelet kernel g, (()
Second Order
Coefficent
Coefficent
Inverse heat kernel
Order
666
First Order
-248
Second Order
Third Order
-2.00
Eigenvalue(A)
Eigenvalue(A)
Figure 2: Low-order Maclaurin Series well approximate wavelet kernel and inverse wavelet kernel with 8 1.
can be written as V (A)UT and (x(-) is a filter kernel. Following previous works GWNN (Xu al. 2019a) and GRAPHWAVE (Donnat et al. 2018 we use the heat kernel go (A.) e - as heat kernel admits easy calculation of inverse wavelet transform with g. (A,) = e*A;, In addition, apply Maclaurin series approximation heat kernel and neglect explicit eigendecomposi- tion by:
v = UE,, n S"AUT = En= (-1)" s" L sym
(10)
- AT Σ
L sym!
(11)
Usually truncated low order approximation is used in practice with n S 3 Defferrard et al. 2016).
To cope with the noise threshold estimation problem, we introduce a parameterized matrix W2 in wavelet domain and eliminate these noise coefficients via a ReLU function, in the hope of learning a separating threshold from the data itself. Take the noised representation M raised Section into consideration, we derive the reconstructed signal as:
X' Ve,RelLUV, MW,)Ws-
(12)
where W4 and W3 are trainable parameters.
We then contrast the difference between Wavelet Neural Network (WNN) in this work and other related WNNs including GWNN (Xu et al. 2019a) and GRAPHWAVE (Donnat et al. 2018).
Scalability An important issue in WNN is that needs to avoid explicit eigendecomposition to compute wavelet bases for large graphs. Both GRAPHWAVE and GWNN, though try to avoid eigendecomposition by exploiting Chebyshev polynomials, still rely integral operations (see Ap- pendix D in Xu et al. (2019a)) and there is no clear way that we can scale up. Differently, we use Maclaurin series to approximate wavelet bases, which has explicit polynomial coefficients and can resemble the heat kernel well when = 3. Please refer to Figure for more details.
De-noising The purpose of both GWNN and GRAPHWAVE to derive node presentations with localized graph convolution and flexible neighborhood, such that downstream tasks like classifica- tion can be simplified. On the contrary, our work implements wavelet neural network the purpose of detaching the useful information and the noise amplified by inverse GCN. Due to the different purpose, our work applies the activation function in wavelet domain while GWNN in the original vertex domain.
4.3 VISUALIZATION
In Figure we illustrate the difference between GDN and each component inside by visualizing the reconstructed road occupancy rates in a traffic network. The traffic network targets District of California collected from Caltrans Performance Measurement System (PeMS We select 4438
http::/pent dot. catt.cc..oo gov/

---

Under review as a conference paper at ICLR 2021
Angg.
Les Ange.
(a) original
(b) inverse GCN
Elpemvatus
Auge..
Auge..
model
(d) GCN
Figure 3: Illustration of road occupancy rates decoded by different approaches 5pm on June 24, 2020 in District of California. A darker color implies higher road occupancy rate. In each figure above, we plot the reconstructed signal spatial domain together with coefficients in spectral domain. (a) the original, (b) inverse GCN (RMSE 0.11), (c) GDN/our model (RMSE 0.07), (d) GCN (RMSE 0.09).
stations the node V and collect their road average occupancy rates at 5pm June 24, Li 2020. Following [i et al. (2018). we construct an adjacency matrix A by denoting Ai = 1 sensor freeway along the same direction. We reconstruct road occupancy
station Uj and u; are adjacent rate of each sensor station with three decoder variants: (b) inverse GCN, (c) GDN, (d) GCN. Here the three variants share the same encoders defined in Section 3.1
As can be seen, while both GDN and GCN decoders can resemble the input signal in low frequency, GDN decoder retains more high frequency information. For inverse GCN decoder, although keeping much high frequency information (mixed with noise), it drops lots of low frequencies, making the decoded signals less similar the input in Euclidean space.
5 EXPERIMENTS
We first validate both the proposed graph autoencoder framework and GDN in two tasks including unsupervised graph-level representation (Sun al.) 2020 Hassani Khasahmadij (020)) and social recommendation (Jamali & Ester 2010 Monti et al. [017 Berg et 20118), then test GDN in
graph generation tasks (Kipf & Welling 2016 Grover et al. 2019).
5.1 UNSUPERVISED GRAPH-LEVEL REPRESENTATION
Following recent methods (Sun et al. 2020 Hassani & Khasahmadi 2020), we evaluate the effec- tiveness of the unsupervised leae representations downstream graph classification tasks.

---

Under review as a conference paper at ICLR 2021
Table 1: Mean 10-fold cross validation accuracy five graph datasets for kernel and unsupervised methods
Datasets
IMDE-BIN
IMDE-MULTI
REDDIT-BIN
PROTEINS
DD
RW Gärtner 2003
50.7 +0.3
34.7
74.2 40.4
SP (Borgwardi Ancgel
2005
50.2
30.3
64.1 #0.1
75.07 =0.5
73,7 +3.9
Komel
GK
SSeervashizze el al 2009
61.9
43.9 +0.4
77.3 #0.2
71.67 #0.6
74,9 #3.8
W1.
(Shervasiiize
43.3
47.0
68.8
72.92
12.4
DGK (Yaniraag & Vishwana
2015
67.0
44.6
40.4
75.7 +0.5
MLG Konatr & Fan 2016
66.6 10.3
40.0
76.3 #0.7
Supervised
GCN (Kipf & Welling 2017
43.4
$1.9
+0.0
76.0 #3.2
GAT Veenkkovi et 2018
70.5
47.8
83,2
GIN-0 |
75.1
52.3
92.4
76.2 42.8
GIN- Xu ct al. 20196
15.1
52.1 #3.6
92.3
75.9 13.8
VGAE (Kipf & Welling 2016
64.9+0.38
38.920.46
72.4=0.42
76.3=0.34
SUB2VEE (Adiikuri et all 2018
+1.5
36.7 +0.8
71.5 +0.4
Unsspervised
GRAPH2VEC (Narayanan et all 2017
71.1 +0.5
50.4
75.8
73.3
INFOGiRAPTY (Sun
2020
73.0
49.7 40.5
82.5
75.0 +1.3
76.9 11.3
MVGRL (Hassami & Khasahmadr 2020)
74.2 +0.7
$1.2 +0.5
84.5 +0.6
75.9 +1.9
78.3 +1.7
ALATION-NVERSSE
11.9
50.
83.9
75.0 +1.
77.8
ALATION- OURS
76.0 +1.3
$1.5
86.0
76.1 #1.4
79.1 11.5
Data and baselines We five graph classification benchmarks including BMDB-Binaae IMDB- Multi, Reddit- Reddit-Binary, PROTEINS and DD (Narayanan et al.) 2017 Ying et al. 2018). For the detailed statisties, please refer to Table 4 We compare with six graph kernels: Random Walk (RW)
Gärtner et al.) 2003). Shortest Path Kernel (SP) (Borgwardt Kriegel, 2005), Graphlet Kernel
(GK) (Shervashidze al.) 2009), Weisfeiler-Lehman Sub-tree Kernel (WL) (Shervashidze et al.
2011 Deep Graph Kernels (DGK) (Yanardag & Vishwanathan (015) and Multi-Scale Laplacian
Kernel (ML.) (Kondor & Pan 2016). In addition, compare with four unsupervised graph-level
representation learning methods: SUB2VEC (Adhikari et al. 2018). GRAPH2VEC (Narayanan al. 2017). INFOGRAPH (Sun et al. 2020) and MVGRL (Hassani & Khasahmadi (2220). We also include the results of recent supervised graph classification models: GCN (Kipf & Welling,
2017). GAT (Veličković et al. 2018). GIN (Xu et al. (0196). We denote our framework using (1)
GCN
KKipf & Welling (017) in the decoders as ALATION-GON!
(2) inverse of GCN in Section
the decoders as ALATION-INVERSE-COM
Setup We adopt the same procedure of previous works (Sun et al. 2020 Hassani & Khasahmadij 2020) and report the mean 10-fold cross validation accuracy with standard deviation after 5 runs using LIBSVM (Chang & Lin 2011). We report results from previous papers with the same setup available. For the datasets PROTEINS and DD, we implement strong baselines including INFO- GRAPH and MVGRL, with a hyperparameter search according the papers.
We train our model using minibatch based Adam optimizer with a learning rate of 0.01. use cross-entropy loss to reconstruct the features. In our encoders, the best variants of GNN are chosen from GCN (Kipf & Welling 2017) and Heatts (Li et al.) 2020). For Heatts, we let s = for all experiments. For detailed hyperparameter settings, please refer Appendix
Results The classification accuracies on the five benchmarks are shown Table MLG, kernel method, performs well on PROTEINS. However, it suffers from long run time and takes more than day on two larger datasets, as observed INFOGRAPH. Our method achieves the best results in 4 out of 5 datasets compared with both kernel and unsupervised models, e.g., it achieves 1.5% improvement over previous state-of-the-ar on Seeditt-Binrry and 1.8% improvement on
which shows the superiority of our methods. When compared with supervised graph classification models, ours beats the best supervised classification model GIN on MDB-BIN on-par with GIN PROTEINS,
and only loses REDDIT.
note ALATION-GCN coincides with Zhang
2020

---

Under review as a conference paper at ICLR 2021
Table 2: Comparison of different methods social recommendation tasks
Datasets
Clao
Douban
ILS
RMSE
ILS
RMSE
sRGCNN (Monti ct al. 2017
1.63%
1.183
6.03%
0.801
GC-MC Berg al 2018
1.09%
1,061
3.90%
0.734
GraphRee Faa
1.24%
1.062
8.27%
0.754
OURS
0.48%
1.071
2.65%
0.745
Ablation study We investigate the role of each component in GDN. We let the three variants share the same depth of layers and parameter size, As observed in Table when decoding by pure inverse of GCN, the performance is just comparable to that of GCN decoders and outperformed by our GDN decoders in all datasets, indicating the effectiveness of this hybrid design.
Running time We observe that our model runs significantly faster than INFOGRAPE and MV- GRL. Our model takes 1Os to train one epoch of PORTEINS on Tesla P40 24G, while INFOGRAPH needs 127s and MVGRL needs 193s. This is because our model neglects the tedious process of neg- ative sampling used in both INFOGRAPH and MVGRL.
5.2 SOCIAL RECOMMENDATION
In model, we consider social recommendation as feature recovery graphs, i.e., user-item ratings are represented as the feature matrix and a social network among users is represented as the graph structure. The model trained incomplete version of feature matrix (training data) and is used to infer the potential ratings (test data).
Data and baselines We use two social recommendation benchmark datasets: Douban and Ciad Both datasets consist of user ratings for items and incorporate social network among users. For the preprocessed dataset provided by Monti et al. (2017). For Ciao. matrix of 7,317 users and 1,000 items. Dataset statistics are summarized in Table the Appendix. We compare with three baselines: sRGCNN (Monti al. 2017 GC-MC (Berg al. (008)) and GraphRec (Fan et al. (219)). Following Candes & Plan (20T)), we consider the few available entries (training data) should corrupted with noise in reality, and add random ratings the training data at different level E {0, 0.1, 1}. where p = 0 denotes no noise is added.
Douban,
sub-
RMSE on Doubbn
RMSE on Ciao
0.0
1.2
0.8
0.7
SRGONN
GraphRes OURS
SRGONN GC-MC
GraphRee OURS
GC-MC
0.9
0.6
0.0
0.4
0.6
0.8
1.0
Noise Level
Noise Level
Figure 4: Comparison of different methods on social recommendation tasks. The x-axis denotes the noise level while the y-axis denotes RMSE on (Ciao, Douban}.
Setup For social recommendation, we neglect pooling and unpooling operations as the predictive task is within a single graph. We use the Mean Squared Error (MSE) loss to reconstruct the rat- ings. Same as in GC-MC (Berg et al. 2018). we stack the output of the first GCN layer and the second GCN layer in our encoders, and use left normalization to preprocess adjacency matrix. For
hhtps//www..cia.co.uk
8

---

Under review as a conference paper at ICLR 2021
Table 3: The effect of GINN with various graph generation methods
Datasels
MUTAG
PTC-MR
ZINC
AUC
logp(A]2)
AUC
AP
logp(AA|2
AUC
VGAB Kipf & Welling 2016
-1.156
0.869
0.645
-1.366
0.566
0.433
-1.033
0.551
0.288
VGAE
-1.114
0.880
-1.351
0.602
Graphite (Gnver et al. 2019
-1.140
0.868
0.632
-1.362
0.564
0.437
-1.043
0.589
0.288
Graphite with GDN
-1.104
0.882
0.681
-1.347
0.773
0.613
-0.999
0.851
0.593
recommendation accuracy, we report Root Mean Squared Error (RMSE). For recommendation di- versification (Ziegler et al. 2005), report Intra-List Similarity (ILS). For the definition of ILS and detailed hyperparameter settings, please refer to Appendix C
Results The recommendation RMSE and ILS on the two benchmarks (p- 0) are shown in Table 2 Our method performs par with state-of-the-en methods RMSE, with only lose to the best model GC-MC by 0.01 on Ciao and 0.011 on Douban. As for ILS, our method performs the best, e.g., it beats the second best model GC-MC by 0.61% Ciao and 1.25% on douban, which shows the superiority of our methods on recommendation diversification. The recommendation RMSE with respect to different noise level shown in Figure 4 As can be seen, our method performs the best with respect to different noise level (p > 0), e.g., it achieves 0.04 improvement over the second best model GC-MC on Ciao when p 1, and 0.014 4 improvement over the second best model sRGCNN on Douban, which shows the superiority when user-item interactions are noisy.
5.3
GRAPH GENERATION
Data and baselines We use three molecular graphs: MUTAG (Kriege & Mutzel 2012) contain-
ing mutagenic compounds, PTC-MR (Kriege & Mutzel 2012) containing compounds tested for careinogenicity and ZINC (Irwin al., 2012) containing druglike organic molecules, to evaluate the performance of GDN on graph generation. As our proposed autoencoder framework is not suitable for graph generation task, we test GDN on two popular variational autoencoder framework includ- ing VGAE (Kipf & Welling 2016) and Graphite (Grover et al.. 2019). Our purpose is to validate if GDN helps with the generation performance.
Setup For VGAE and Graphite, we reconstruct the graph structures using their default methods and the features using GDN. The two reconstructions share the same encoders and sample from the same latent distributions. We train for 200 iterations with a learning rate of 0.01. The output dimension of the first hidden layer is 32 and that of the second-layer is 16. For MUTAG and PTC- MR, we use all the graph samples. For ZINC, we target test datasets (See Table 6 in the Appendix for the details). We use 50% samples train set and the remaining 50% as test set.
Results Evaluating the sample quality of generative models is challenging Theis et al. 2016). In this work, we validate the generative performance with the log- likellhooho (logp( Z)), area under the receiver operating characteristics curve (AUC) and average precision (AP). As shown in Table 3 GDN can improve the generative performance of both VGAE and Graphite in general, egg. improve the AP score of Graphite by 4.9% on MUTAG the AUC score of VGAE by 19.4% on PTC-MR and log p(A) of VGAE by 5.4% on ZINC, which shows the superiority of GDN.
6 CONCLUSION
In this paper, we present a symmetric graph autoencoder framework in an unsupervised way. The proposed framework relies Graph Deconvolutional Networks (GDNs), the opposite of GCNs that recover graph signals from smoothed representations. The introduced GDN uses spectral graph convolutions with high pass filter to obtain inversed signals and then de-noises the inversed signals in wavelet domain. The effectiveness the proposed method is validated on unsupervised graph- level representation, social recommendation and graph generation tasks.

---

Under review as a conference paper at ICLR 2021
REFERENCES
Bijaya Adhikari, Yao Zhang, Naren Ramakrishnan, and B Aditya Prakash. Sub2vec: Feature learn- ing for subgraphs. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp- 170-182. Springer, 2018.
Mark Banham and Aggelos K Katsaggelos. Digital image restoration. IEEE signal processing magazine, 14(2)24-41, 1997.
Yoshua Bengio. Learning deep architectures for AI. Now Publishers Inc, 2009.
Rianne van den Berg, Thomas N Kipf, and Max Welling. Graph convolutional matrix completion. SIGKDD, 2018.
Filippo Maria Bianchi, Daniele Grattarola, and Cesare Alippi. Spectral clustering with graph neural networks for graph pooling. In Proceedings of the 37th international conference on Machine learning, pp. 2029222222222 2738. ACM, 2020.
K. M. Borgwardt and H.-P. Kriegel. Shortest-pate kernels on graphs. ICDM, pp. 74-81, 2005.
Emmanuel J Candes and Yaniv Plan. Matrix completion with noise. Proceedings of the IEEE, 98 (6):925-936, 2010.
Chih-Chung Chang and Chih-Jen Lin. Libsvm: library for support vector machines. ACM trans- actions on intelligent systems and technology (TIST), 2(3):1-27, 2011.
S Grace Chang, Bin Yu, and Martin Vetterli. Adaptive wavelet thresholding for image denoising and compression. IEEE transactions on image processing, 109: 2000.
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. Advances in neural information processing systems, pp. 3844-3852, 2016.
Chenhui Deng, Zhiqiang Zhao, Yongyu Wang. Zhiru Zhang, and Zhuo Feng. Graphzoom: multi- level spectral approach for accurate and scalable graph embedding. ICLR, 2020.
Claire Donnat, Marinka Zitnik, David Hallac, and Jure Leskovec. Learning structural node embed- dings via diffusion wavelets. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), pp. 1320-1329, 2018.
David L Donoho and Jain M Johnstone. Ideal spatial adaptation by wavelet shrinkage. biometrika, 81(3):425-4 455, 1994.
Vincent Dumoulin and Francesco Visin. A guide convolution arithmetic for deep learning. arXiv preprint arXiv:1603.0728; 2016.
Wenqi Fan, Yao Ma, Qing Li, Yuan He, Eric Zhao, Jiliang Tang, and Dawei Yin. Graph neural networks for social recommendation. In The World Wide Web Conference, pp. 417-426. ACM, 2019.
Soheil Feizi, Daniel Marbach, Muriel Médard, and Manolis Kellis. Network deconvolution as general method to distinguish direct dependencies in networks. Nature biotechnology, 31(8): 726-733, 2013.
Mário AT Figueiredo and Robert D Nowak. An em algorithm for wavelet-based image restoration. IEEE Transactions on Image Processing, 12(8):906 9166 2003.
Hongyang Gao and Shuiwang Ji. Graph u-nets. In ICML, pp.
2019.
T. Gärtner, P. Flach, and S. Wrobel. On graph kernels: Hardness results and efficient alternatives.
In Learning theory and kernel machines, pp. 129-143. Springer, 2003.
Aditya Grover, Aaron Zweig, and Stefano Ermon. Graphite: Iterative generative modeling of graphs. In The International Conference Machine Learning (ICML), pp. 2434-2444, 2019.
10

---

Under review as a conference paper at ICLR 2021
David K Hammond, Pierre Vandergheynst, and Rémi Gribonval. Wavelets graphs via spectral graph theory. Applied and Computational Harmonic Analysis, 30(2):29-1550, 2011.
Kaveh Hassani and Amir Hosein Khasahmadi. Contrastive multi-view representation learning on graphs. In Proceedings of International Conference Machine Learning, pp. 3451-3461. 2020.
John J Irwin, Teague Sterling, Michael M Mysinger, Erin S Bolstad, and Ryan G Coleman. Zine: a free tool to discover chemistry for biology. Journal of chemical information and modeling, 52 (7)11777-1768, 2012.
Mohsen Jamali and Martin Ester. A matrix factorization technique with trust propagation for rec- ommendation in social networks. In RecSys, pp. 135-142, 2010.
Thomas N Kipf and Max Welling. Variational graph auto-encoders. In Conference on Neural Information Processing Systems (NeurlPS) Workshop on Bayesian Deep Learning. 2016.
Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional net-
works. In The International Conference on Learning Representations (ICLR), 2017.
Risi Kondor and Horace Pan. The multiscale laplacian graph kernel. In Advances in Neural Infor- mation Processing Systems, pp. 2990-2998 2016.
Nils Kriege and Petra Mutzel. Subgraph matching kemelss for attributed graphs. ICML, pp. 291-298, 2012.
Deepa Kundur and Dimitrios Hatzinakos. Blind image deconvolution. IEEE signal processing magazine, 13(3):3364, 1996.
Junhyun Lee. Inyeop Lee, and Jaewoo Kang. Self-attention graph pooling. Proceedings of the 36th International Conference Machine Learning (ICML), 2019.
Jia Li, Yu Rong. Hong Cheng, Helen Meng, Wenbing Huang, and Junzhou Huang. Semi-supervised graph classification: hicrarchical graph perspective. In The World Wide Web Conference (WWW), pp. 972-999229999 982, 2019.
Jia Li, Jianwei Yu, Jiajin Li, Honglei Zhang, Kangfei Zhao, Yu Rong, Hong Cheng, and Junzhou Huang. Dirichlet graph variational autoencoder. In Neurips, 2020.
Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffusion convolutional recurrent neural net- work: Data-driven traffic forecasting. ICLR, 2018
Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3431-3440, 2015.
Federico Monti. Michael Bronstein, and Xavier Bresson. Geometric matrix completion with recur- rent multi-graph neural networks. In Neurips, pp. 3697-3707, 2017.
Annamalai Narayanan, Mahinthan Chandramohan, Lihui Chen, Yang Liu, and Santhoshkumar Sam- inathan. subgraph2vec: Learning distributed representations of rooted sub- sub-graphs from large graphs. CoRR, 000.0800.,, 2017.
Ramesh Neelamani, Hyeokho Choi, and Richard Baraniuk. Forward: Fourier- regularized deconvolution for ill-conditioned systems. IEEE Transactions on signal processing, 52(2):418- 433, 2004.
Hyeonwoo Noh, Seunghoon Hong, and Bohyung Han. Learning deconvolution network for seman- tic segmentation. In Proceedings of the IEEE international conference on computer vision, pp. 1520-1528, 2015.
N. Shervashidze, V. Vishwanathan, T. Petri, K. Mchlhom, and K. M. Borgwardt. Efficient graphlet kernels for large graph comparison. In AISTATS, pp. 488-495, 2009.
N. Shervashidze, P. Schweitzer, E. J. v. Leeuwen, K. Mchlhom, and K. M. Borgwardt. Weisfeiler- lehman graph kernels. Journal of Machine Learning Research, 12(Sep):2539-2555 22661, 2011.

---

Under review as a conference paper at ICLR 2021
David Shuman, Sunil K Narang, Pascal Frossard, Antonio Ortega, and Pierre Vandergheynst. The emerging field of signal processing on graphs: Extending high-dimensional data analysis to net- works and other irregular domains. IEEE signal processing magazine, 30())83-998, 2013.
Martin Simonovsky and Nikos Komodakis. Graphvae: Towards generation of small graphs using variational autoencoders. In International Conference on Artificial Neural Networks, pp. 412- 422. Springer, 2018.
Yun Sun, Jordan Hoffmann, Vikas Verma, and Jian Tang. Infograph: Unsupervised and semi- supervised graph-level representation learning via mutual information maximization. ICLR, 2020.
Lucas Theis, Aäron den Oord, and Matthias Bethge. A note the evaluation of generative models. ICLR, 2016.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pictro Lio, and Yoshua Bengio. Graph attention networks. ICLR, 2018.
Petar Veličkovié, William Fedus, William L. Hamilton, Pietro Liö, Yoshua Bengio, and R Devon Hjelm. Deep Graph Infomax. In International Conference on Learning Representations, 2019.
Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, Pierre-Antoine Manzagol, and Léon Bottou. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. Journal of machine learning research, 11(12), 2010.
Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. Sim- plifying graph convolutional networks. In Proceedings of the 36th International Conference on Machine Learning (ICML), pp. 6861-6871 PMLR, 2019.
Bingbing Xu, Huawei Shen, Qi Cao, Yunqi Qiu, and Xueqi Cheng. Graph wavelet neural network. The International Conference Learning Representations (ICLR), 2019a.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. networks? ICLR, 2019b.
How powerful graph neural
P. Yanardag and S. V.N. Vishwanathan. Deep graph kernels. In KDD, pp. 1365-1374, 2015.
Jingkang Yang and Santiago Segarra. Enhancing geometric deep learning via graph filter deconvo- lution. 2018 IEEE Global Conference on Signal and Information Processing (GlobalSTP), pp- 758-762. IEEE, 2018
Zhitao Ying, Jiaxuan You, Christopher Morris, Xiang Ren, Will Hamilton, and Jure Leskovec. Hi- erarchical graph representation learning with differentiable pooling. In Advances in neural infor- mation processing systems, pp. 4800-488 4810, 2018.
Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In European conference computer vision, pp. 818-833. Springer, 2014.
Matthew D Zeiler, Dilip Krishnan, Graham W Taylor, and Rob Fergus. Deconvolutional networks. In 2010 IEEE Computer Society Conference on computer vision and pattern recognition, pp. 2535.. IEEE, 2010.
Chun-Yame Yang Zhang, Junfeng Hu, Lin Yang. CL. Philip Chen, and Zhiliang Yao. Graph deconvolu- tional networks. Information Sciences, 518:330-340, 2020.
Cai-Nicolas Ziegler, Sean M McNee, Joseph A Konstan, and Georg Lausen. Improving recommen- dation lists through topic diversification. In Proceedings of the 14th international conference on World Wide Web, pp. 22-32, 2005.
12

---

Under review as a conference paper at ICLR 2021
Table 4: Statistics of the datasets used in graph classification
Datasets
IMDB-BIN
IMDE-MULTI
REDDIT-BIN
PROTEINS
DD
(No. Graphs)
1000
1500
2000
1113
1178
(No.Classes)
2
2
(Avg.Nodes)
19.8
13.0
508.5
39.1
284.3
(Avg. Edges)
193.1
65.9
497.8
72.8
715.7
Table 5: Statistics of the datasets used in social recommendation
Dataset
Users
Items
Train/Test Ratings
Rating Density
Social Connections
Social Density
Douban
3,000
3,000
123, 223,202/13,68
1.52% 0.77%
2,690
0.03%
Ciao
7,317
1,000
39,279/16,892
111,781
0.21%
A DETAILED MODEL CONFIGURATION
For the unsupervised graph-level representation experiments, use the original graph structure in the decoders the proposed graph autoencoder framework. The C parameter of LIBSVM (Chang & 10², 10*}. The depth of GNN layers the encoders
Lin 2011) is selected from {10-3 10 set to 2. The output dimension of the first layer is chosen from {64, 128, 256}. The output dimension
of the second layer is chosen from {16, 32). The number of clusters K is chosen from {16, 32). The number of epochs is chosen from [15, 20). The batch size is set to 1. We set AA 0 and Xx 1. We don't 't use Dropout as it does not improve the performance. For all methods, the embedding dimension is set to 512 and parameters of downstream classifiers are independently tuned using cross validation on training folds of data, order to have a fair comparison with previous works. The best average classification accuracy is reported for all methods.
For social recommendation, we train our model using Adam optimizer with learning rate of {0.005, 0.002]. We use the original graph structure in the decoders of the proposed graph au- toencoder framework. The output dimension of the first layer is 256. The output dimension of the second layer 128. The number of epochs is 200. We use full-batch size. We don't 't use Dropout as does not improve the performance. Intra-List Similarity (IIS) of one user defined as:
ILS,
= 2
Σ im ET
Σ
sim (m kn);
(13)
where L is the item list for the user and equal with the top-10 items in this work. sim(-.-) the similarity function and is set to cosine similarity. For the representation of items, we use the input user rating vectors.
B MORE RELATED WORKS
Unsupervised graph-level representations can be generally classified into three groups
Unsupervised graph-level representation techniques
The first group is graph kernels. Representative works
include Random Walk (Gärtner et al.) 2003). Shortest Path Kernel (Borgwardt & Kriegel 2005),
Graphlet Kernel (Shervashidze et al.) 2009). Weisfeiler-Lehman Sub-tree Kemel (Shervashidze
al.) 2011). Deep Graph Kernels (Ynardag Vishwanathan (015) and Multi-Scale Laplacian Kernel (Kondor & Pan 2016). This group of techniques works on finding informative sub-structures
and computes similarities based on these sub-structures. The second group is contrastive methods. Representative works include SUB2VEC (Adhikari et al.| (0188). GRAPH2VEC (Narayanan et al.) 2017], INFOGRAPH (Sun et al.) 2020) and MVGRL (Hassani Khasahmadi 2020). This group employs scoring function, e.g., mutual information (Veličković et al. 2019), train model that can increase the score on positive examples and decrease the score on negative samples. It thus depends the sophisticated positive/negative sampling strategy boost the performance. The third group is graph autoencoders (Kipf Welling 20166] Grover et al. 2019). Due to the lack of powerful decoders, the performance of this kind is still underestimated.
13

---

Under review as a conference paper at ICLR 2021
Table 6: Statistics of the datasets used in graph generation
Datasets
MUTAG
PTC-MR
ZINC
(No. Grahh))
188
344
5000
(ogg.Nodes)
17.9
14.3
23.1
(Avg. Edges)
19.8
14.7
49.7
Social recommendation Social recommendations assume a social network among users and these user-user interactions are helpful in boosting the recommendation performance (Jamali Ester 2010). As GNNs have been proven to be capable of learning on graph data, recently recommenda- tion methods based on GNNs have shown impressive results on recommendation accuracy. Repre- sentative works include sRGCNN (Monti et al., 2017). GC-MC (Berg al. 2018), and GraphRec (Fan et al. 2019). Specifically, GC-MC (Berg et al. 2018 models recommendations as link predic- tions using graph autoencoder. Though successful, GC-MC uses MLP decoder and fails to deal with noisy ratings (Candes & Plan 2010).
C DERIVATIVE OF INVERSE GCN
(w) A=0 XM
9e (A:) 1
(14)
X
(-1)nn!(-1) 1)a
n!
n=0
==0
where (n) denotes the n-th order derivative. Thus, (8) can be obtained as:
-1 x = Udiag
U x = U > A
(15)
n=0
m=0
n=U
14


Reviews:
[{"review_id": "zhtAN7JLQ2", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This work considers the graph deconvolution networks. It proposed a graph deconvolutional networks to reconstruct the graph signals. The concerns for this work are as below:\n\n1.\tThe contributions in this work are incremental. It seems most parts are based on the previous works. The deconvolution part follows the scheme defined in (Bianchi et al. 2020). The overall graph autoencoder also follows a very common settings by involving pooling and unpooling operations. In pooling part, two pooling methods are employed that are hard pooling and soft pooling. However, it is not clear to me how to combine these two methods since they are very different. The equation (3) shows the combination of these two methods but fails to make sense to me. I would like the authors to clarify this and provide detailed explanations for this part.\n2.\tIn both Section 4.1 inverse of GCN and Section 4.2 wavelet de-noising, the authors talked about two methods but didn’t explain why they are related to the proposed graph deconvolution networks. Also, most of the content in these two parts are following previous works like (Li et al. 2019b) and (Xu et al. 2019). I would recommend the authors to clearly point out what is the contribution and how these are related to the proposed methods.\n3.\tThe visualization section in 4.3 is confusion. Firstly, why the authors believe this is a contribution to the graph deconvolutional networks. To my understanding, there is no technical contribution for this paper. It would be better to show this in the experimental parts. Secondly, there is no description on the settings for the illustrations. I didn’t understand how the image are converted into graph and how the inverse GCN and the proposed models are used to them. Thus, the authors need to clarify this and point out the technical contributions for this part.\n4.\tThe experimental results are very limited. The four datasets used in table 1 are very small in terms of the number of graphs, which cannot provide comprehensive evaluations to the proposed results. The baseline methods are not well established. There are far better results on these datasets such as GIN. The authors claim that the unsupervised settings are used. However, it is weird to me why the unsupervised settings do not help the  overall model performances.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "QomLfEftgyu", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The main contribution of this paper is that the authors design a graph deconvolutional network that combines inverse filters in the spectral domain and de-noising layers in the wavelet domain. Further graph autoencoders are proposed based on the graph convolutional networks and the graph deconvolutional networks. Many experiments are done and many previous methods are compared to show the effectiveness of the proposed networks.\n\nMy reason for the rating:\nBefore the experiments part, I think authors clearly present the methods and related works. The results also look quite good compared to other methods. However, I have some questions for the experimental setups that they present at the end of the paper, which somehow confuses me.\n\nPros:\n1. In general, this paper is well-written and the presentation is clear and easy to understand for the method parts.\n2. The results well demonstrate the proposed network as reported. \n\nCons:\nI have some comments and concerns for this paper before I made my final decision:\n1. It would be great to give a reference to the Maclaurin series approximation for the function of a matrix near (8), or give a simple derivation. Please mention that $n$ starts from 0.\n2. For the running time part, it would be great to quantitatively show the time, when choosing a baseline. $10$ and $15$  seem approximation numbers to the reviewer.\n3. I recommend the authors explicitly state the previous methods that you have the same setups in the experiments.\n4. I find that the authors use $\\lambda_{A} = 0$ in the experiments, which means that in (6), the loss function contains only the second term. Could you please clarify this setting?\n5. The authors mention that they use best average classification accuracy for all the methods. Could you clarify this part?\n\n---------------------------\nAfter rebuttal\n\nThank the authors' response to my comments. They also provided new results (i.e., graph generation tasks) to address the Q.4 in my comments. I was about to lower my score when the first reply came as graph generation tasks are just future works in the original paper, but I changed my mind with these results. Although other reviewers may still question the novelty and contributions, I think I would stay with my score according to the comparison and good results reported in the paper. Looking forward to seeing the code someday.", "extra_scores": {}, "extra_text": {}}, {"review_id": "Cbc2Kg9t4B-", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\n\nThe authors proposed graph deconvolution layers (GDNs) and employ GDNs to learn graph embedding in a encoder-decoder framework. The authors performs inverse signal recovery in spectral domain and then conducts a de-noising step in wavelet domain to remove the amplified noise. The proposed method can outperform baseline methods on graph classification and social recommendation. However, there are some issuses should be solved.\n\nPro:\nThe proposed method is very novel. Encoder-decoder network is firsted employed on graph data for unsupervised learning. The writing and organization of this paper is good. The authors provide a clear and detailed describtion for the proposed method. For the decoding part, the authors proposed a de-noising step in wavelet domain to remove the amplified noise.  \n\nCons:\nIn this paper, the authors employ encoder-decoder framework to learn graph embedding. Why encoder-decoder framework is needed for graph data? One option is that we can use Variational Graph Auto-Encoders[1] to learn node embedding and use SUM or AVG function to obtain graph embedding. This should be added as a baseline to show that encoder-decoder framework is better. In addition, the authors should also mention how many pooling layers are used in encoder-decoder framework. And the experiments with different number of pooling layers can help understand the proposed method. \n\nOverall: I vote to accept this paper considering the importance of the task and the novely method proposed.\n\n[1].Kipf, Thomas N., and Max Welling. \"Variational graph auto-encoders.\" arXiv preprint arXiv:1611.07308 (2016).", "extra_scores": {}, "extra_text": {}}, {"review_id": "1FvCXiLffh", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Graph Autoencoders with Deconvolutional Networks\n\nThe paper proposes a graph deconvolutional network to reconstruct the original graph signal from smoothed node representations obtained by graph convolutional networks.\n\nThe proposed deconvolution incorporates a denoising component based on graph wavelet transforms.\n\nPros:\n-The idea of defining graph deconvolution operators is appealing and and may potentially lead to improvement in the performance of graph reconstruction/generation tasks.\n-The Visualization section of the paper shows an advantage of the proposed approach compared to other methods for reconstruction. It maintains information about high-frequency signal.\n\nCons:\n-The potential of a graph deconvolution operator are not fully exploited in the paper, mainly because of the considered tasks that do not require deconvolution because they are not intrinsically reconstruction tasks.\nThe paper applies the proposed approach to tasks of graph classification in Table 1 and social recommendation (matrix completion) in Table 2. While the comparison with other unsupervised learnimg methods looks favourable to the proposed approach, supervised learning methods are naturally more suited for the tasks in Table 1 and tend to perform slightly better (a comparison with supervised approaches would be appreciated).\nGraph Autoencoders are usually applied to tasks of graph generation such as molecule design, where they are one of the most suited approaches. Many works in literature face this problem. A comparison on the generation task would be interesting.\n-Considering the Ablation results, the improvement with respect to the ablation approaches seems marginal. Again, my opinion is that the considered tasks are not well suited for the proposed model.\n-Hyper-parameter selection: In Appendix A the hyper-parameter selection procedure is not sufficiently detailed. How do you choose the hyper-parameter values? You report the considered ranges but not the procedure you adopt to select them. \"parameters of downstream classifiers\", in my understanding it refers to the C parameter, or to other hyper-parameters as well?\n\nMinor:\nALATION-GCN -> ABLATION\n\n---Rebuttal--\nI acknowledge having checked the authors' response and the revised version of the manuscript, that has been improved since the first revision. Authors did not answer to my request for more details about the hyper parameter selection procedure that has been adopted. ", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The covered topic is timely and of potential impact for many application domains, such as drug design. The paper is well written and presentation is clear. The proposed approach seems to have some degree of originality. Experimental results seem to be generally good, and in the rebuttal the authors have provided further experimental support to their main claim.  There are however some issues that have not been solved by the author’s rebuttal. I think two of them are the most important and related: 

i) significance of contribution: although the authors have tried in the rebuttal to explain how the proposed approach differs from related papers, it seems that there are still doubts about the amount of innovation introduced by the paper. This issue could have been mitigated by SOTA experimental results in presence of a proper model selection, that, however does not seem to be the case here (see next point);

ii) model selection: the authors did not clearly explain the model selection procedure in the rebuttal. This is an important issue since it is often easy to get good results by picking the best run a posteriori. Unfortunately in the literature there are highly cited papers where model selection is not performed in a proper way and reviewers very often reject papers just looking at numbers without looking at how the numbers were obtained. So, I believe it is important to accept only papers where model selection is properly done and properly explained, so to allow for reproducibility of experiments. 
Decision: Reject