Deconvolutional Networks on Graph Data
Authors: Jia Li
Jiajin Li
Yang Liu
Jianwei Yu
Yueting Li
Hong Cheng
Venue: NeurIPS 2021 Poster (2021)
Keywords: deconvolutional networks
inverse problem
graph generation

Abstract: In this paper, we consider an inverse problem in graph learning domain -- "given the graph representations smoothed by Graph Convolutional Network (GCN), how can we reconstruct the input graph signal?" We propose Graph Deconvolutional Network (GDN) and motivate the design of GDN via a combination of inverse filters in spectral domain and de-noising layers in wavelet domain, as the inverse operation results in a high frequency amplifier and may amplify the noise. We demonstrate the effectiveness of the proposed method on several tasks including graph feature imputation and graph structure generation.  

Deconvolutional Networks on Graph Data
Jia Li", Jiajin LiP, Yang Liu', Jianwei Yu², Yueting Li", Hong Cheng? Hong Kong University Science and Technology The Chinese University of Hong Kong jialeeEust. hk
Abstract
In this paper, consider an inverse problem in graph learning domain "given the graph representations smoothed by Graph Convolutional Network (GCN), how reconstruct the input graph signal?" We propose Graph Deconvolutional Network (GDN) and motivate the design of GDN via a combination of inverse filters in spectral domain and de-noising layers in wavelet domain, the inverse operation results in a frequency amplifier and may amplify the noise. We demonstrate the effectiveness of the proposed method on several tasks including graph feature imputation and graph structure generation.
Introduction
Graph Convolutional Networks (GCNs) (6 20 have been widely used to transform input graph structure A and feature matrix into graph representations H, which can be taken as a mapping function XA, H. In this paper, we consider its inverse problem g (A, H) + X, i.e.,
given the graph representations smoothed by GCNs, how can we reconstruct the imput graph signal? More specifically, we initiate systematic study of Graph Deconvolutional Networks (GDNs), to reconstruct the signals smoothed by GCNs. A good GDN component could benefit many applications such as reconstruction [44] and generation [19]. For examples computer vision, several studies (45) 229] have demonstrated that the performance of autoencoders on reconstruction and generation can be further improved by encoding with Convolutional Networks and decoding with Deconvolutional Networks [46]. However, extending this symmetric autoencoder framework to graph-structure data requires graph deconvolutional operations, which remains open-ended and hasn't been well studied as opposed to the large body of solutions that have already been proposed for GCNs.
Most GCNs proposed by prior works, e.g., Cheby-GCN (6) and GCN 20 exploit spectral graph convolutions (33] and Chebyshev polynomials (15) to retain coarse-grained information and avoid explicit cigen-decompositionn of the graph Laplacian. Until recently, (39] and have noticed that GCN 220] acts as a low pass filter in spectral domain and retains smoothed representations. Inspired by prior works the domain of signal deconvolution [22] [3). we are motivated to design GDN by an inverse operation of the low pass filters embodied in GCNs. Furthermore, we observe directly reversing low pass filter results high frequency amplifier in spectral domain and may amplify the noise. In this regard, we introduce spectral-wavele GDN to decode the smoothed representations into the input graph signals. The proposed spectral-waver wavelet GDN employs spectral graph convolutions with inverse filter to obtain inversed signals and then de-noises the inversed signals wavelet domain. In addition, we apply Maclaurin series as fast approximation technique to compute both inverse filters and wavelet kernels (8)
We evaluate the effectiveness the proposed GDN with two tasks: graph feature imputation [35] and graph structure generation [19 14). For the former task, we further propose graph autoencoder
(GAE) framework that resembles the symmetric fashion of architectures (29). The proposed GAE outperforms the state-of state-of-the-arts on six benchmarks. For the latter task, our proposed GDN can
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Codfliieet
Codfidan
Coffkintt
Spectral
Spetral
Spertzal
Elgmalue
Convolution
Decorvolution
Imput graphG
H E R
Output graph
Figure I: Schematic diagram of the graph autoencoder perspective. We plot graphs in spatial domain together with their signal coefficients spectral domain. In the encoding stage, we run GCN model to derive smoothed node representations. In the decoding stage, we reconstruct the graph signal via GDN model.
enhance the generation performance of popular variational autoencoder frameworks including VGAE 19 and Graphite [14].
The contributions of this work are summarized as follows:
We propose Graph Deconvolutional Network (GDN), the opposite operation of Graph Convolutional Network (GCN) that reconstructs graph signals from smoothed node repre- sentations.
We study GDN from the perspective of inverse operations in spectral domain and identity the inverse operation results in a high frequency amplifier and may amplify the noise. To solve the noise issue, de-noising process based on graph wavelet is further proposed.
2 Preliminaries
In this work, we are given an undirected, unweighted graph G (V, is the node set and N = |V] denotes the number of nodes. The adjacency matrix A E IR NxN represents the graph structure. The feature matrix X E NNx represents the node attributes.
2.1 Convolution on graphs
The convolutional layers used to derive smoothed node representations, such that nodes that similar in topological space should be close enough in Fuclidean space. Formally, the spectral graph convolution signal x: E R is defined
Se *
(c(AN))U
(1)
where (A: are the eigenvalues of the normalized Laplacian matrix Laym DD and D are the Laplacian and degree matrices of the input graph A respectively, and U is the eigenvector matrix of Legm TAUT. In this work, focus on GCN [20], one of the popular convolution operations on graphs:
H GCN X),
where E T Nx denotes smoothed node representations. Specifically, (39 show that GCN is low pass filter in spectral domain with (A,) = - dr. i.e.
he =
(3)
where (-) is the activation function.

---

2.2 Deconvolution on graphs
The deconvolutional layers are used to recover the original graph features given smoothed node representations. Formally, the spectral graph deconvolution on the smoothed representation h E R defined as:
h = With this deconvolution operation, we can use GDNs to produce fine-grained graph signals from the encoded H.
(4)
X' = (D,,A,
(5)
where X' Y'E ER-x denotes the recovered graph features. We shall further discuss our design of GDN in Section 3
From the perspective of encoder-decoder. the convolution operation could be used as encoder and the deconvolution operation as decoder. We show a schematic diagram of this encoder-decoder perspective in Figure T
Graph Deconvolutional Network
In this section, we present our design of GDN. Motivated by prior works in signal deconvolution [16), naive deconvolutional net can be obtained using the inverse filter 9. (X:) = T-x, in spectral domain. Unfortunately. the naive deconvolutional net results in high frequency amplifier and may amplify the noise 13 noise exists.
This point is obvious for 9 (Xx) as the signal fluctuates near X;
3.1 Noise analysis
Our target is to recover the original graph features given smoothed node representations hG encoded by GCN (20). Without the noise, is naive to obtain the closed-form recovery relationship by the inverse operation 9e (A) = I-x directly, i.e.,
x Udiag(g ('dia (hh),
(6)
Here, we assume the activation function a(-) an inverse operator, e.g., Leaky ReLU, sigmoid, tanh. Actually, it is quite reasonable to consider such a prototypical model of invertible networks in the theoretical analysis. Otherwise, we need address an ill- oll-posed inverse problem instead as we automatically lose some information (e.g., null space). Hence, it is impossible to recover the ground-truth even under noiseless model, see [ for details.
Graph noise is ubiquitous in real-world applications. Following the convention in signal deconvolution [9] and image restoration [3]. let's consider the white noise model, i.c., h is the smoothed node representations mixed with graph convolution and white noise,
à =(("diag(a:(A))) +c,
where is the white Gaussian noise (i.e., ~ NO (TT)).
Next, we show that it will amplify the noise if we apply the inverse operation to the white noise model directly. That is,
Udiag(o " -
(8)
Proposition 1. Considering the independent white noise model 7 and 8 if the inversed activation function is deferentiable which satisfies do - 2 1, the variance of x! is larger than the white noised added on the generative model.
It was worthwhile mentioning that the condition regarding the inverse operator do 00-1(0) 2 is
quite reasonable. Actually, holds for many representative activation functions, including sigmoid, tanh and Leaky ReLUU. Please refer to Appendix A for details. Proposition I shows the proposed inverse operation may introduce undesirable noise into the output graph. Thus, a de-noising process is necessary thereafter.

---

Wavelet kernel g₂(A)
kernel g, (A)
First Order
0.79
Second Order
Coefficntt
Coefficent
Inverse heat kernel
-131
Second Order
Third Order
kemel
-1.3
822
0.00
875
1,00
1225
150
9.00
0.30
375
180
125
175
2.0
Eigenvallue(A)
Eigenvalue(A)
Figure 2: Low-order Maclaurin Series well approximate wavelet kernel and inverse wavelet kernel
with 1.
3.2 The proposed GDN
To ameliorate the noise issue, we propose an efficient, hybrid speetral-waveeet deconvolutional network that performs inverse signal recovery in spectral domain first, and then conducts de-noising step in wavelet domain to remove the amplified noise [28]
3.2.1 Inverse operator
Regarding the inverse operation in (6), the core is explicitly compute
We
propose to apply Maclaurin series approximation on 9c (A) = >n=g A" and get fast algorithm as below:
(
=E En=D
L Laym-
A detailed derivative of can be found in Appendix E When truncated low order approximation is
used in practice with n < 3 66 we derive a spectral filter with g. (Ax) + X + X X
Following [29], we can use another activation function and trainable parameter matrix to estimate -1, to avoid the potential non-invertible problem. Thus, the inverse operator becomes:
3
M = C(IN + Σ sym
La HWWs)
(10)
n=1
where W3 the parameter set be learned.
We contrast the difference between inverse operator in this work and other reconstruction methods including 44 and GALA 30
Inverse operator vs. GCN decoder [77
Compared with GCN decoder 47 which directly uses GCN for signal reconstruction, the proposed inverse operator demonstrates efficacy in recovering the high frequency signals of the graph, as shown in Figure 3 (b). We shall further discuss this point Section
Inverse operator vs. GALA GALA 300 can be viewed as the first-order approximation towards inverse operator in this work, with a spectral kernel of + A4. The difference is that our inverse operator can well approximate the inverse filter ge (Ax) I-x; with third-order polynomials.
3.2.2 Wavelet de-noising
Wavelet-based based methods have a strong impact on the field of de-noising, especially in image restoration when noise is amplified by inverse filters (28). In the literature, many wavelet de-noising methods have been proposed, e.g., SureShrink (9). BayesShrink (5). and they differ in how they estimate separating threshold. Our method generalizes these threshold ideas and automatically separates the noise from the useful information with a learning-based approach. Consider the input signal of wavelet system is mixed with Gaussian noise, the reasons why wavelet denoising works are: (1) graph wavelet basis holds the majority of signal energy into small number of large coefficients,

---

Spectral Confficient
Spectiaal Contiiittent
1.0
10
2.8
Ange..
Lrs Ange.
(a) original
(b) inverse operator
Spectral Confiiiiunt
Spectrral Contficient
0.0
1.0
Ange.
Les Ange.
(c) proposed GDN
(d) GCN decoder
Figure Illustration of road occupancy rates decoded by different approaches (zoom in see details). A darker color implies higher road occupancy rate. In each figure above, we plot the reconstructed signal in spatial domain together with coefficients in spectral domain. (a) the original, (b) inverse operator (RMSE = 0.11), (c) GDN/our model (RMSE = 0.07), (d) GCN decoder 47 (RMSE = 0.09).
and (2) Gaussian noise the original graph domain is again a Gaussian noise wavelet domain separate small
(with the same amplitude), [16] for details. The core thus becomes how number of large coefficients and the noise in wavelet domain. Given the latter unknown, we adopt trainable parameter matrix linearly transform the noise under zero and useful coefficients bigger than zero, where ReLU is used to eliminate the noise.
Consider set of wavelet bases V .V....... V&N), where each one V denotes signal on graph diffused away from node and is a scaling parameter (40), the wavelet bases can be written V Ug (A)UT and (-) is filter kernel. Following previous works GWNN [40] and GRAPHWAVE [8] we use the heat kernel g. (Ax) e =>>; as heat kernel admits easy calculation of inverse wavelet transform with (A,) = e In addition, we can apply Maclaurin series approximation heat kernel and neglect explicit eigen-decomposition by:
V. En
(-1)"
8 L m!
(1I)
1/-1
000 Sh -n= n..
(12)
sym
5

---

To cope with the noise threshold estimation problem, we introduce parameterized matrix in wavelet domain and eliminate these noise coefficients via a ReLU function, in the hope learning a separating threshold from the data itself. Together with the representation M raised in Section 3.2. we derive
the reconstructed signal as:
X' 12 Re MW,)W,
(13)
where W4 and Ws are trainable parameters.
We then contrast the difference between Wavelet Neural Network (WNN) in this work and other related WNNs including GWNN [40] and GRAPHWAVE (8].
Scalability An important issue in WNN is that one needs to avoid explicit eigen-decomposition to compute wavelet bases for large graphs. Both GRAPHWAVE and GWNN, though trying to avoid eigen-decomposition by exploiting Chebyshev polynomials, still rely integral operations (see Appendix D in (40)) and there is no clear way to scale up. Differently, we use Maclaurin series to approximate wavelet bases, which has explicit polynomial coefficients and can resemble the heat kernel well when n = 3. Please refer to Figure for more details.
De-noising The purpose of both GWNN and ARRAPHAVVE is to derive node presentations with localized graph convolution and flexible neighhorhood, such that downstream tasks like classification can be simplified. On the contrary, our work implements wavelet neural network in the purpose of detaching the useful information and the noise amplified by inverse operator. Due to the different purpose, our work applies the activation function in wavelet domain while GWNN in the original vertex domain.
3.3 Visualization
In Figure 3 we illustrate the difference between GDN and each component inside by visualizing
the reconstructed road occupancy rates in a traffic network. The traffic network targets District 7 of California collected from Caltrans Performance Measurement System (PeMS | We select 4438 sensor stations the node set V and collect their road average occupancy rates 5pm June 24, 2020. Following [23]. we construct an adjacency matrix A by denoting Aij = if sensor station dj and U; are adjacent on a freeway along the same direction. We reconstruct road occupancy rate of each sensor station with three decoder variants: (b) inverse operator, (c) our proposed GDN, (d) GCN decoder (47). Here the three variants share the same encoders.
As can be seen, while both GDN and GCN decoders can resemble the input signal in low frequency, GDN decoder retains more high frequency information. For inverse operator decoder, although keeping much high frequency information (mixed with noise), it drops lots of low frequencies, making the decoded signals less similar the input in Euclidean space.
4
Experiments
We validate the proposed GDN on graph feature imputation 35| 44 and graph generation [9 The former task reconstructs graph features and the latter generates graph structures.
4.1 Graph feature imputation
In our model, we consider feature imputation as feature recovery graphs. The model is trained on an incomplete version of feature matrix (training data) and further used to infer the potential missing features (test data).
Datasets We use six benchmark datasets including several domains: citation networks (Cora,
Citeseer) 322 product co- co-purchase networks (Amaphoto, Amacomp [26). social rating networks
(Douban, Ciace) P.. For citation networks and product co-purchase networks, we use the preprocessed
versions provided by (35 with uniform randomly missing rate of 10%. For Douban, we use the preprocessed dataset provided by (27). For Ciao, we use sub-matrix of 7,317 users and 1,000 items. Dataset statistics are summarized in Table in the Appendix.
http:://pent dot..a..ta...... co.vo/ /
https:: fttkps://www... ciao.

---

Table 1: RMSE test comparison of different methods on graph feature imputation
Datasets
Ciao
Douban
Cora
Citeseer
Amaphoto
Amacomp
MEAN
1.385
0.850
0.503
0.503
0,414
0.417
KNN
1.461
0.817
0.459
0.443
0.421
0.422
SVD 137
1.380
0.833
0.493
0.490
0.413
0.417
MICE
1.600
0,480
0,481
0.487
GAIN
43
1.099
0.809
0,433
0,421
0,420
0,416
GRAPE
1.042
0.742
0,430
0.419
0.399
0.402
Inverse decoder
1.115
0.812
0,447
0,431
0,409
0.407
GCN decoder(47
1.132
0.826
0,439
0.434
0.410
0,409
GALA 30
1.147
0.833
0,457
0.435
0.415
0.411
OURS
1.011
0.734
0.415
0.399
0.391
0.393
Setup Our graph autoencoder framework consists of an encoder and a decoder. Our encoder consists of two layers of Graph Convolutional Networks (GCN) [20]. Our decoder consists of one layer of proposed GDN, to produce fine-grained graphs. We use the Mean Squared Error (MSE) loss to reconstruct the features. We run 5 trials with different seeds and report the mean of the results. For detailed architecture, please refer to Appendix For detailed hyper-ppaee syper-parameter settings, please refer to Appendix D
We report Root Mean Squared Error (RMSE).
Baselines We compare with three commonly used imputation methods and three deep learning based imputation models:
MEAN, which replaces the missing feature entries with the mean values of all observed feature entries.
KNN, which first identifies the k-nearest neighbor samples and imputes the missing feature entries with the mean entries of these samples. We set k = in this work.
SVD 37
which replaces the missing feature entries based matrix completion with iterative low-rank SVD decomposition.
which is multiple regression method that infers missing entries by Markov chain Monte Carlo (MCMC) techniques.
MICE 4.
GAIN 43
which is deep learning imputation model with generative adversarial training.
GRAPE 44 which is the state-of-the-eee deep learning imputation model based on deep graph representations.
For ablation studies of the graph autoencoder framework specified in Appendix C we also include the results of decoders using GCN decoder [47). GALA [30), and inverse decoder in Section to be compared with our proposed GDN. Besides, we compare with popular matrix completion methods in Appendix B for discrete feature recovery.
Results The test RMSE the six benchmarks are shown in Table Our method performs the best on all datasets, c.g., it beats the second best model GRAPE by 0.02 on Citeseer and 0.031 on Ciao, which shows the superiority of our methods on feature imputation. The feature imputation RMSE with respect to different missing rate is shown in Figure 4 As can be seen, our method performs the best with respect to different missing rate, e.g., it achieves 0.016 improvement over the second best model GRAPE on Citeseer when the missing rate is 70%, and 0.007 improvement over the second best model GRAPE on Amacomp when the missing is 30%.
Ablation study We investigate the role of wavelet de- noising in our GDN. We let the four decoder variants share the same depth of layers and parameter size. As observed in Table [ when decoding by inverse operator, the performance is on par with that of GCN decoders [47 and outperformed by our GDN decoders performance of inverse decoder specified in Section 3.2.T 3.2.1 is always better than that of GALA [30]. which shows the superiority of our third-order approximation.
all six datasets, indicating the effectiveness of this hybrid design. The
7

---

0.550
0.550
0.525
0.525
RMSE Citeseer
0.475
0.475
0.450
0.459
RMSE
0.425
0.425
0.400
0.400
MEAN KNN
5VD
GRATE
REAM
310
GRAFE
-
G4
-0-
DunS
GAR
-0-
0515
0.350
0.1
0.3
0.5
0.7
0.350
0.1
0.3
0.5
0.7
Missing data ratio
Missing data ratio
RMSE on Amaphoto
0.48
RMSE on Amacomp
0.46
0.46
0.44
0.44
0.42
0.42
0.40
0.38
HEAN
5VD GAN
GRAPE
NCAV Kih
5VD GAN
GRAP 0LBS
0.36
65.8
-
0185
0.36
-0
0.1
0.3
0.5
0.1
0.3
0.5
0.7
Missing data ratio
Missing data ratio
Figure 4: Comparison different methods on graph feature imputation tasks. The x-axis denotes the missing ratio while the y-axis denotes RMSE on test set.
Table 2: The effect of GDN with various graph generation methods
Datasets
MUTAG
PTC-MR
ZINC
logi(A
loge(A
AUC
logp(AA))
AP
VGAE 19
-1.101
0.716
0.427
-1.366
0.566
0.433
-1.035
0.556
0,288
VGAE GDN
-1.026
0.823
0.611
-1.31
0.760
0.602
-1,006
0.858
0.611
Graphite 14
-1.100
0.732
0.447
-1.362
0.564
0.437
-1039
0.553
0.288
Gmphite
GDN
-1.024
0.818
0.608
-1347
0.773
0.613
-0.998
0.838
0.567
4.2
Graph structure generation
Data and baselines We use three molecular graphs: MUTAG [2] containing mutagenic com- pounds, PTC-MR [2T] containing compounds tested for careinogenicity and ZINC [17] containing druglike organic molecules, to evaluate the performance of GDN on graph generation. Dataset statisties are summarized in Table in the Appendix. We test GDN on two popular variational autoencoder framework including VGAE [19] and Graphite [14). Our purpose is to validate if GDN helps with the generation performance.
Setup For VGAE and Graphite, we reconstruct the graph structures using their default methods and the features using GDN. The two reconstructions share the same encoders and sample from the same latent distributions. For detailed architecture, please refer to Appendix [ We train for 200 iterations with learning rate 0.01. For MUTAG and PTC-MR, we use 50% samples as train set and the remaining 50% test set. For ZINC, we use the default train-test split.
Results Evaluating the sample quality generative models is challenging (36). this work, we validate the generative performance with the log- likeliiid p(A Z)), area under the receiver operating characteristics curve (AUC) and average precision (AP). As shown Table GDN can improve the generative performance of both VGAE and Graphite in general, e.g., improve the AP score of Graphite by 16. 1% on MUTAG the AUC score of Graphite by 20.9% on PTC-MR and log (A) of VGAE by 0.029 ZINC, which shows the superiority of GDN.
5 Related work
Deconvolutional networks The area of signal deconvolution [22 has long history in the signal processing community and is about the process of estimating the true signals given the degraded or smoothed signal characteristics (2). Later deep learning studies 46 29 consider deconvolutional

---

networks as the opposite operation for Convolutional Neural Networks (CNNs) and have mainly focused on Euclidean structures, e.g., image. For deconvolutional networks on non-Euclidean structures like graphs, the study is still sparse. [2] proposes the network deconvolution as inferring
the true network given partially observed structure. 42) formulates the deconvolution as a pre- processing step on the observed signals, in order to improve classification accuracy. [47] considers recovering graph signals from the latent representation. However, it just adopts the filter design used in GCN and sheds little insight into the internal operation of GDN.
Graph autoencoders Since the introduction of Graph Neural Networks (GNNs) 20 and autoencoders (AEs), many studies [4] have used GNNs and AEs to encode to and decode from latent representations. Although some encouraging progress has been achieved, there is still no work about graph deconvolution that can up-sample latent feature maps to restore their original resolutions. In this regard, current graph autoencoders bypass the difficulty via (1) non-parameterized decoders [99 7 (2) GCN decoders [14), (3) multi-layer perceptron (MLP) decoders [34], and (4)
Laplacian sharpening [30].
Graph feature imputation Feature imputation techniques can be generally classified into two groups: statistical approaches and deep learning models. The former group includes KNN, SVD (37..
MICE [4), in which they makes assumptions about the data distribution through parametric density function. The latter group includes GAIN (43), GRAPE [44). Recently GRAPE (44) proposes graph feature imputation that assumes network among items and the network is helpful in boosting the
feature imputation performance. A related topic is matrix completion [27 3 [T] in which usually
makes the assumption of finite and discrete feature values. Differently, both GRAPE and our method can recover both continuous and discrete feature values.
Graph structure generation Graph structure generation techniques can be classified into three
groups: one-shot [19 14 34), substructure [18] and node-by-node [25). As this work is more related to the group of one-shot, we refer the readers to [10] for the details of other groups. The group of one-shot generates the entire graph all at once. Representative works include VGAE [19]. Graphite [14] and Graph VAE [34]. Both VGAE and Graphite reconstruct graphs based on the similarities of graph structures. Differently, Graph VAE considers both the similarities of structures and node features and reconstructs node features using MLP. In this work, we show that the performance of one-shot graph generators can be improved by reconstructing node features using GDN.
Conclusion
In this paper, we present Graph Deconvolutional Network (GDN), the opposite of GCN that recovers graph signals from smoothed representations. The introduced GDN uses spectral graph convolutions with an inverse filter to obtain inversed signals and then de-noises the inversed signals in wavelet domain. The effectiveness of the proposed method is validated graph feature imputation and graph structure generation tasks.
7 Limitation
Broadly speaking, graph signal restoration is an ill-posed problem. In this regard, the prior knowledge is important for high-quality recovery. This paper only pays attention to vanilla GCN 200 (Kipf & Welling, 2017) and it is not clear whether the spectral-waveler GDN ideas can be applied to other GNNs, e.g., GAT (38). GIN [4T]. Another limitation of this work that the heat kernel in wavelet parts is not a band- pass filter, as it should be.
Broader Impact
This work initiates the restoration of graph signals smoothed by vanilla GCN (00 (Kipf & Welling, 2017). As a sequence, researchers in drug design or molecule generation may benefit from this research, since the visualization and understanding of GCN based representations is worthwhile to be further explored.

---

Acknowledgments and Disclosure of Funding
The work was supported by NSFC Grant No. U1936205 and General Research Fund from the Research Grant Council of the Hong Kong Special Administrative Region, China [Project No.: CUHK 14205618).
References
[1] Lynton Ardizzone, Jakob Kruse, Sebastian Wirkert, Daniel Rahner, Eric W Pellegrini, Ralf Klessen, Lena Maier-Hein, Carsten Rother, and Ullrich Köthe. "Analyzing inverse problems with invertible neural networks". In: ICLR (2019).
[2] Mark R Banham and Aggelos Katsaggelos. "Digital image restoration". In: IEEE signal processing magazine 14.2 (1997), pp. 24-41.
[3] Rianne den Berg, Thomas N Kipf, and Max Welling. "Graph convolutional matrix com-
pletion". In: SIGKDD (2018).
[4] Stef Buuren and Catharina Strothuis-Oudshoon "MICE: Multivariate Imputation by Chained Equations in R". In: Journal of Statistical Software 45 (Dec. 2011).
[5] S Grace Chang, Bin Yu, and Martin Vetterli. "Adaptive wavelet thresholding for image denoising and compression", In: IEEE transactions image processing 9.9 (2000), pp. 1532- 1546.
[6] Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. "Convolutional neural net- works graphs with fast localized spectral filtering". In: Advances in neural information processing systems. 2016, pp. 3844-3852.
[7] Chenhui Deng, Zhiqiang Zhao, Yongyu Wang, Zhiru Zhang, and Zhuo Feng. "Graphzoom: A multi-level spectral approach for accurate and scalable graph embedding". In: ICLR (2020).
[8] Claire Donnat, Marinka Zitnik, David Hallac, and Jure Leskovec. "Learning structural node embeddings via diffusion wavelets". In: ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD). 2018, pp- 1320-1329.
[9] David L Donoho and Jain M Johnstone. "Ideal spatial adaptation by wavelet shrinkage". In: biometrika 81.3 (1994), pp. 425-455- 455..
[10] Faezeh Faez, Yassaman Ommi, Mahdich Soleymani Baghshah, and Hamid R Rabice. "Deep Graph Generators: A Survey". In: arXiv preprint arXiv:2012. 15544 (2020).
[11] Wenqi Fan, Yao Ma, Qing Li, Yuan He, Eric Zhao, Jiliang Tang, and Dawei Yin. "Graph Neural Networks for Social Recommendation" In: The World Wide Web Conference. ACM. 2019, pp. 417-426.
[12] Soheil Feizi, Daniel Marbach, Muriel Médard, and Manolis Kellis, "Network deconvolution as a general method to distinguish direct dependencies in networks". In: Nature biotechnology
31.8 (2013), pp. 726-733.
[13] Mário AT Figueiredo and Robert D Nowak. " "An EM algorithm for wavelet-based image restoration". In: IEEE Transactions on Image Processing 12.8 (2003), pp. 906-916.
[14] Aditya Grover, Aaron Zweig, and Stefano Ermon. "Graphite: Iterative generative modeling of graphs". In: The International Conference on Machine Learning (ICML). 2019, pp. 2434-2444. [15] David K Hammond, Pierre Vandergheynst, and Rémi Gribonval. "Wavelets on graphs via spec- tral graph theory". In: Applied and Computational Harmonic Analysis 30.2 (2011), pp. 129- 150.
[16] Jeff Irion and Naoki Saito. "Efficient approximation and denoising of graph signals using the multiscale basis dictionaries". In: IEEE Transactions on Signal and Information Processing over Networks 3.3 (2016), pp. 607-616.
[17] John Irwin, Teague Sterling, Michael M Mysinger, Erin S Bolstad, and Ryan G Coleman. "ZINC: free tool to discover chemistry for biology".. In: Journal of chemical information and modeling 52.7 (2012), pp. 1757-1768.
[18] Wengong Jin, Regina Barzilay, and Tommi Jaakkola. "Junction tree variational autoencoder for molecular graph generation". In: The International Conference on Machine Learning (ICML) (2018).
10

---

[19] Thomas N Kipf and Max Welling. "Variational Graph Auto-Encoder's In: Conference on Neural Information Processing Systems (NeurIPS) Workshop on Bayesian Deep Learning. 2016.
[20] Thomas N. Kipf and Max Welling, "Semi-Supervised Classification with Graph Convolutional Networks". In: The International Conference on Learning Representations (ICLR). 2017.
[21] Nils Kriege and Petra Mutzel. "Subgraph matching kernels for attributed graphs". In: ICML.
(2012), pp. 291-298.
[22] Deepa Kundur and Dimitrios Hatzinakos. "Blind image deconvolution". In: IEEE signal processing magazine 13.3 (1996). pp- 43-64.
[23] Jia Li, Zhichao Han, Hong Cheng. Jiao Su, Pengyun Wang, Jianfeng Zhang, and Lujia Pan. "Predicting Path Failure In Time-Evolving Graphs". In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD). 2019.
[24] Jia Li, Jianwei Yu, Jiajin Li, Honglei Zhang, Kangfei Zhao, Yu Rong, Hong Cheng, and Junzhou Huang. "Dirichlet Graph Variational Autoencoder". In: Neurips. 2020.
[25] Qi Liu, Miltiadis Allamanis, Mare Brockschmidt, and Alexander L. Gaunt. "Constrained
Graph Variational Autoencoders for Molecule Design". In: The Thirty-secout Conference on Neural Information Processing Systems (2018).
[26] Julian McAuley, Christopher Targett. Qinfeng Shi, and Anton Van Den Hengel. "Image-based recommendations on styles and substitutes". In: SIGIR. 2015, pp. 43-52.
[27] Federico Monti, Michael Bronstein, and Xavier Bresson. "Geometric matrix completion with recurrent multi-graph neural networks". In: Neurips. 2017, pp. 3697-3707.
[28] Ramesh Neelamani, Hyeokho Choi, and Richard Baraniuk. "ForWaRD: woueler- regularized deconvolution for ill-conditioned systems". In: IEEE Transactions on signal processing 52.2 (2004), pp. 418-433.
[29] Hyconwoo Noh, Scunghoon Hong, and Bohyung Han. "Learning deconvolution network for semantic segmentation". In: Proceedings of the IEEE international conference on computer vision. 2015, pp. 1 1520-1528.
[30] Jiwoong Park, Minsik Lee, Hyung Jin Chang, Kyuewang Lee, and Jin Young Choi. "Sym- metric graph convolutional autoencoder for unsupervised graph representation learning". In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019, pp. 6519- 6528.
[31] Yu Rong, Wenbing Huang, Tingyang Xu, and Junzhou Huang. "DropEdge: Towards Deep Graph Convolutional Networks on Node Classification"" In: International Conference on Learn- ing Representations. 2020. URL: TRLL:https::: / net cost/foruuee httffforummmd Hirrumimkdd
[32] P. Sen, G. Namata, M. Bilgic, L. Getoor, B. Galligher, and T. Eliassi-Rad Rad. "Collective classifi- cation in network data". In: Al magazine 29.3 (2008), pp. 93-106.
[33] David I Shuman, Sunil K Narang, Pascal Frossard, Antonio Ortega, and Pierre Vandergheynst. "The emerging field of signal processing on graphs: Extending high- sigh-dimensional data analysis to networks and other irregular domains".. In: IEEE signal processing magazine 30.3 (2013), pp. 83-98.
[34] Martin Simonovsky and Nikos Komodakis. "Graphvae: Towards generation of small graphs using variational autoencoders". In: International Conference on Artificial Neural Networks. Springer. 2018, pp. 412-444244 422..
[35] Hibiki Taguchi, Xin Liu, and Tsuyoshi Murata. "Graph convolutional networks for graphs containing missing features". In: Future Generation Computer Systems 117 (2021), pp. 155- 168.
[36] Lucas Theis, Aäron van den Oord, and Matthias Bethge. "A note on the evaluation of generative models". In: ICLR (2016).
[37] Olga Troyanskaya, Mike Cantor, Gavin Sherlock, Trevor Hastie, Rob Tibshirani, David Botstein, and Russ Altman. "Missing Value Estimation Methods for DNA Microarrays". In: Bioinformatiic 17 (July 2001). pp. 520-525. 10.933. www.119//biiifffrrne [38] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. "Graph attention networks". In: ICLR (2018).
bioinformatics/1.. 6..62
[39] Felix WU, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. "Simplifying Graph Convolutional Networks". In: Proceedings of the 36th International Con- ference on Machine Learning (ICML). PMLR, 2019, pp. 6861-6871.
11

---

[40] Bingbing Xu, Huawei Shen, Qi Cao, Yunqi Qiu, and Xueqi Cheng. "Graph wavelet neural network". In: The International Conference on Learning Representations (ICLR) (2019).
[41] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. "How powerful are graph neural networks?" In: ICLR (2019).
[42] Jingkang Yang and Santiago Segarra. "Enhancing geometric deep learning via graph filter deconvolution" In: 2018 IEEE Global Conference on Signal and Information Processing
(GlobalSIP). IEEE. 2018, pp. 758-762.
[43] Jinsung Yoon, James Jordon, and Mihaela van der Schaar. "GAIN: Missing Data Imputation using Generative Adversarial Nets". In: Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018. Ed. by Jennifer G. Dy and Andreas Krause. Vol. 80. Proceedings of Machine Learning Research. PMLR, 2018, pp. 5675-5684.
[44] Jiaxuan You, Xiaobai Ma, Daisy Ding, Mykel Kochenderfer, and Jure Leskovec. "Handling Missing Data with Graph Representation Learning". In: NeurIPS (2020).
[45] Matthew Zeiler and Rob Fergus. "Visualizing and understanding convolutional networks".
In: European conference on computer vision. Springer. 2014, pp. 818-833.
[46] Matthew D Zeiler, Dilip Krishnan, Graham W Taylor, and Rob Fergus. "Deconvolutional
networks". In: 2010 IEEE Computer Society Conference on computer vision and pattern recognition. TEEE, 2010, pp. 2528-2535.
[47] Chun Yang Zhang, Junfeng Hu, Lin Yang, CL Philipp Chen, and Zhiliang Yao. "Graph decon-
volutional networks". In: Information Sciences 518 (2020), pp. 330-340.
12


Reviews:
[{"review_id": "XNmpMKqIBE5", "reviewer": "Reviewer_Vnte", "summary": "This paper derives an inverse operator, which is said to be better than other methods in ref [50] and GALA. The work also points out the inverse operation results in a high pass filter and may amplify the noise. Motivated by this observation, a de-noising layer is designed and introduced into the proposed network. The graph deconvolution uses the wavelet transforms on the graph, which is implemented based on heat kernel and the series in terms of symmetric graph Laplacian. Theoretical analysis was provided for the derivation of the deconvolutional layer.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\n* Figure 3 is not clear to observe useful information for illustrating the performance comparison for different approaches.\n \n* How to efficiently compute the series in (11) and (12), which are used in calculating the GDN convolution in (13)? If there is a truncation, what is a suitable degree?\n \n* OGB should be considered to test by the proposed GDN and compared with existing methods.\n\n* In the experiment part, there are three different tasks: imputation and graph generation. The author evaluated the graph generation results with AUC score, AP score and the negative log-likelihood of the latent variable posterior distribution. There is no standard deviation of graph generation's performance like graph classification experiment. It is hard to reproduce the model if the standard deviation is too high.\n\n* Would it be appropriate to put section 3.3 in the experiment part as an example of the model utilizing the high pass signal?\n\n* line 48, typo for 'de-nosing'\n \n* The notation for the graph is usually defined as the triplet of $V, E,w$, which represent vertex set, edge set and edge weights, respectively.\n\n* line 73, 'We show' to 'we show'", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "It is unclear to me where the low-pass and high-pass information are used in the proposed GDN model. In fact, from (10)--(13), it is difficult to tell where the multiscale property of the transform $\\Psi_s$ is employed.\n \nIt would be much more persuasive if the authors could provide a toy example that showcases the method's computational speed or test some real large datasets, such as open graph benchmarks (OGB). ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "U8jlFptFNT6", "reviewer": "Reviewer_awFr", "summary": "In this paper the authors introduce a Graph Deconvolutional Network (GDN) as a combination of inverse GCN filters (spectral domain) and denoising layers (wavelet domain). They motivate the design of their GDN by observing that a simple inverse GCN filter likely results in noise amplification.\nThe proposed architecture also has computational and scalability advantages over the previous ones in the literature.\n\nThe authors test the capabilities of the GDN on two tasks:\n1. feature imputation (reconstructing graph *features*)\n2. graph generation (reconstructing graph *structure*), where GDN is used to enhance the performance of two graph-generation techniques\n\nThey compare their proposed architecture with various baseline and state-of-the-art approaches and achieve the best performance in all of the tasks.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper tackles a new, scarcely explored problem: the definition of an efficient and effective Deconvolutional Network for Graph data.\n\nIt seems to be not the first paper to propose a definition / implementation for a GDN (see [50] Zhang et al.). The de-noising strategy is also borrowed by one of the cited works.\n\nHowever, they seem to combine the ideas to create more efficient and effective version of a GDN, moving forward the discussion on how to better face this problem.\nThey also:\n- openly and clearly point out the differences between their work and the related ones\n- provide an in-depth explanation of their design choices\n- compare the performance of all the related methods on all the tasks\n\n---\n\nThe submissions appears technically sound. All claims are supported by proofs provided in the Appendix or by experimental results.\n\nHowever, I would have found the results to be more solid if they were averaged over multiple runs (especially for the first task)\n\n---\n\nThe paper is clearly written, well organized, and provides clear and on-point explanations of the differences between the proposed method and the ones it is compared against.\n\nSome typos / confusing parts:\n- use of the word *arts* (e.g. line 24, 28) instead of *articles*\n- caption of Figure 2: *s=1* $\\rightarrow$ *s=3* (maybe)\n- line 121, wrong figure  *Figure 2 (b)* $\\rightarrow$ *Figure 3 (b)*\n- line 287,291: citation *(Kipf & Welling, 2017)* should be formatted as number\n- line 224: capitalize *The performance*\n- Confusing use of the term *inverse operator*.\n  -  at line 119 you title the paragraph \"Inverse operator vs [50]\", meaning that \"inverse operator\" is your proposed GDN (minus the de-noising part)\n  - in Figure 3/Section 3.3, \"inverse operator\" refers to [50] instead (if I understand correctly)\n  - but [50] is also called \"GCN decoder\" in Table 1\n\n  Please use consistent naming, it is really difficult to follow.\n \n---\n\nThe submission presents a well justified and well documented improvement on the GDN model, showing practical improvements over the current baselines on a selection of tasks and datasets. The paper and related additional materials can be of help to further the research on this topic.\n\n---\n\n**After Author Response**\n\nThe authors addressed all my concerns and I confirm my initial evaluation.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have clearly addressed the limitations of their method and suggested directions for further research.\n\nThey have briefly introduced possible negative societal impacts, but it might be adequate given the generality of the method introduced.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8"}, "extra_text": {}}, {"review_id": "OKXHNmmWTa", "reviewer": "Reviewer_J7HE", "summary": "This paper presents Graph Deconvolutional Networks (GDN), a kind of graph neural network designed to invert the low-pass filtering operation implemented by Graph Convolutional Networks (GCN, Kipf & Welling, 2016).\n\nGDN uses a high-pass filter to invert the effect of GCN and a learnable thresholding operation in the wavelet domain to remove any high-frequency noise.\nThe method relies on the Maclaurin series to approximate the high-pass filter (which in principle should be the exact inversion of GCN, although this is expensive) and the wavelet basis (which relies on the eigendecomposition of the graph Laplacian). \n\nThe authors conclude the paper with two main kinds of experiments: \n\n1. Graph feature imputation, in which only a fraction of the nodes in a graph is observed and the remaining must be reconstructed.\n2. Graph generation using graph variational autoencoders (Kipf & Welling, 2017) and the Graphite technique (Grover et al., 2019).", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Originality**: the contributions of the paper are novel and clear, and related works are appropriately cited. \n\n**Quality**: the paper is theoretically sound, with proofs included in the supplementary material. The derivation of the proposed methodology is clear and the authors clearly explain every choice that they have made (for example, the step at which to truncate the Maclaurin series is found experimentally).\nThe authors openly discuss the limitations of the work in a dedicated section.\n\n**Clarity**: the paper is very well written and organized, guiding the reader through the math. Some prior knowledge of graph convolution is required to follow the paper, but this can be expected for this kind of work. \nI am confident that I could reproduce the results of the paper given the description of the methodology. \n\n**Significance**: the significance of the paper is limited. \n\n- The main issue that I see is that there is no clear reason why a graph representation composed of only low and high frequencies should be more desirable than a graph representation of only low frequencies, especially for the tasks considered here. \nThe experiments mildly validate the authors' assumption but the performance improvements are not always significant w.r.t. prior art. The results can be just as well ascribed to the bigger number of parameters that the GNN has when using the GDN decoder, and it is not clear that the spectral analysis conducted to derive GDN is at all related to the performance improvements. \n    - Related to the comment above, why do the GCN decoder and inverse decoder have the same performance in the first experiment, given that they have opposite behaviors? This likely indicates that the experiment is not strongly related to the analysis conducted in the paper.\n\n- A second problem is that this method simply changes GCN from a low-pass to a band-stop filter, but there are classes of GCNs that can already implement such filtering operations, e.g., GCNs based on rational filters. \n\nOverall, while the paper and the design of GDN are interesting, I don't think that they represent a major contribution to the field of GNNs. \n\n**Comments**:\n\n- The claim that the proposed method has a complexity of O(|E|) can be misleading because the method does not use the recursive implementation of the Chebynet paper, but rather pre-computes the powers of the Laplacian. This means that the cost of GDN will quickly converge to O(N^2) and will be O(N^2) by the time that n (index of the Maclaurin series) is as big as the graph diameter. In real-world networks and small graphs (like the ones tested in the paper), this number is likely to be small and therefore the cost of computing GDN is non-negligible.\n\n- GDN is supposed to be an approximation of the exact inverse operator. However, from Fig. 3 we see that the two have qualitatively different behaviors in the low frequencies. \nWhy does GDN retrieve the low-frequency components whereas the exact inverse operator does not? Can the authors elaborate more on this aspect?\n\nI am more than happy to revise my score if the authors address my concerns.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The limitations are adequately addressed in a dedicated section and the paper is unlikely to have a negative societal impact. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "DNPcZd2u6Fy", "reviewer": "Reviewer_QTzU", "summary": "This paper proposed GDN, the opposite of GCN that recovers graph signals from smoothed representations. The introduced GDN uses spectral graph convolutions with a high pass filter to obtain inversed signals and then de-noises the inversed signals in wavelet domain. Effectiveness of GDN is validated on graph feature imputation and graph structure generation tasks.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, this work is a clearly-structured paper. I think it is about average. For the following reasons:\n\n1) Techniques:\nOverall, techniques proposed in this paper seems reasonable. \nOne problem is that there is analysis on white and Gaussian noise, analysis on other types of noise is expected to be given.\n\n2) Experiments\n\n2.1)\tThis paper lacks general tasks, such as node classification and link prediction tasks, when compared with GNNs, such as Graphsage and GAT. In highly related works, such as [1] also provides node classification results to show the effectiveness of proposed model.\n\n2.2)\tThis paper claims that proposed model is applicable to continuous and discrete feature values. However, results in Table 5 and Table 3 are all datasets with discrete values. Results on continuous feature datasets seem lacked.\n\n3) Writings:\nThe paper is clearly-structured and easy to follow, definitions of symbols are clear.\n\n[1] Handling Missing Data with Graph Representation Learning. NeurIPS, 2020.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"ethical_concerns": "No", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "12hours"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a (niche) topic that is still not well explored, i.e. the definition of an efficient and effective deconvolutional network for graph data. This is done by mainly exploiting already existing results, however introducing a significant level of novelty. During the discussion period the authors were able to clarify all the main issues raised by the reviewers.  An additional contribution to the evaluation of the merits of the paper was coming from another AC, that although recognising the technical correctness of the approach, pointed out some issues that can be summarised as follows: i) motivation of the work is weak since a clear need for GDN and a target application is not clearly identified; ii) proposed experiments do not go to the core point, both in  terms of used datasets (in Cora and Citeseer lowpass information is generally sufficient for the classification task), and more importantly - in terms of the methods GDN is compared to; in short, the experimental assessment has been adapted to search for a justification of the definition of "the opposite of GCN"; iii) since at the end of the day the main component in the proposed method is the combination of lowpass and highpass filters, essentially to formulate bandpass filters that capture sufficient information for imputation and graph generation, it would be appropriate to compare the method to other graph filter constructions: in addition to the architectures based on Chebyshev polynomials (e.g., Chebynet) mentioned by the authors, numerous highly-related works have studied the use of various filter constructions to retain and recover highpass and bandpass information, including CayleyNets (Levie et al. 2017), MixHop (Abu-El-Haija et al., 2019), interferometric graph transforms (Oyallon, 2020), GNNs with ARMA filters (Bianchi, 2021), and some extensive body of work on a variety of graph scattering architectures (for example, Gama et al. 2018; Gao et al. 2019; Zou & Lerman 2020; Ioannidis et al. 2020); these works all aim to leverage high pass, bandpass, or wavelet filters to extract information beyond the low frequencies retained by the Kipf & Welling GCN, with numerous applications considered, which would provide a more established benchmark for the proposed method; in particular, for graph generation, [Zou & Lerman (2019)](https://doi.org/10.1109/IJCNN.2019.8851705) and [Castro et al. (2020)](https://ieeexplore.ieee.org/document/9378305) both showed the efficacy of scattering networks in such tasks.
In summary, the contribution seems to be technical correct, however it is not striking due to the direct exploitation of already published results, it needs a better motivation, consequent evaluation, and presentation. Notwithstanding these issues, that should be addressed by the authors, the proposed approach could eventually be used for works in explainability, which is a very important direction of research to aim for.
Decision: Accept (Poster)