Learning from multiscale wavelet superpixels using GNN with spatially heterogeneous pooling
Authors: Maxime Bassenne
Varun Vasudevan
Lei Xing
Venue:  (2021)
Keywords: Image classification
GNN
superpixel
SLIC
wavelet

Abstract: Neural networks have become the standard for image classification tasks. On one hand, convolutional neural networks (CNNs) achieve state-of-the-art performance by learning from a regular grid representation of images. On the other hand, graph neural networks (GNNs) have shown promise in learning image classification from an embedded superpixel graph. However, in the latter, studies have been restricted to SLIC superpixels, where 1) a single target number of superpixels is arbitrarily defined for an entire dataset irrespective of differences across images and 2) the superpixels in a given image are of similar size despite intrinsic multiscale structure. In this study, we investigate learning from a new principled representation in which individual images are represented by an image-specific number of multiscale superpixels. We propose WaveMesh, a wavelet-based superpixeling algorithm, where the number and sizes of superpixels in an image are systematically computed based on the image content. We also present WavePool, a spatially heterogeneous pooling scheme tailored to WaveMesh superpixels. We study the feasibility of learning from the WaveMesh superpixel representation using SplineCNN, a state-of-the-art network for image graph classification. We show that under the same network architecture and training settings, SplineCNN with original Graclus-based pooling learns from WaveMesh superpixels on-par with SLIC superpixels. Additionally, we observe that the best performance is achieved when replacing Graclus-based pooling with WavePool while using WaveMesh superpixels.

Under review as a conference paper at ICLR 2021
LEARNING FROM MULTISCALE WAVELET SUPERPIX- ELS USING GNN WITH SPATIALLY HETEROGENEOUS POOLING
Anonymous authors Paper under double-blind review
ABSTRACT
Neural networks have become the standard for image classification tasks. On one hand, convolutional neural networks (CNNs) achieve state-of-the-arr performance by learning from a regular grid representation of images. On the other hand, graph neural networks (GNNs) have shown promise in learning image classification from an embedded superpixel graph. However, in the latter, studies have been restricted SLIC superpixels, where 1) single target number of superpixels is arbitrarily defined for an entire dataset irrespective of differences across images and 2) the superpixels in a given image are of similar size despite intrinsic multiscale structure. In this study, we investigate learning from new principled representation in which individual images are represented by an image-specific number of multiscale super- pixels. We propose WaveMesh, a wavelet-based superpixeling algorithm, where the number and sizes of superpixels in an image are systematically computed based the image content. We also present WavePool, spatially heterogeneous pooling scheme tailored to WaveMesh superpixels. We study the feasibility of learning from the WaveMesh superpixel representation using SplineCNNN state-of-the-art network for image graph classification. We show that under the same network architecture and training settings, SplineCNN with original Graclus-based pooling learns from WaveMesh superpixels on-pa with SLIC superpixels. Additionally, we observe that the best performance is achieved when replacing Graclus-bass pooling with WavePool while using WaveMesh superpixels.
INTRODUCTION
Convolutional neural networks (CNNs) achieve state-of-the-art performance on variety of image classification tasks from different domains (Tan & Le 2019: Gulshan al. [016). CNNs learn from a regular pixel-grid representation of the images. Although not all pixels provide equal amount of new information, by design the filters in the first layer of CNN operate each pixel from top-left to bottom-right in the same way. Additionally, images are typically resized to a prescribed size before feeding into CNN. In applications that use standard CNN architectures or pre-trained models new image classification dataset, the images are typically uniformly downsampled to meet the input size requirements of the architecture being used. Uniform downsampling may be suboptimal real data naturally exhibits spatial and multiscale heterogeneity. Few studies have explored the impact of input image resolution on model performance (Sabottke Spieler 2020). despite its recognized importance (Lakhani] 2020
Graph neural network (GNN) is a type of neural network that learns from graph structured data. Recent studies have shown the performance of GNNs on image graph classification tasks (Monti) ct al. 2017 Fey Fey et al., 2018 Knyazev et al. 2019 Dwivedi et al., 2020). In this task, GNN learns to classify images from embedded graphs that represent superpixels in the images. However, prior studies have been restricted to SLIC superpixels (Achanta et al. 2012). In this framework, a single target number of superpixels is arbitrarily defined for entire dataset irrespective of differences across images, and the superpixels in given image are of similar size despite intrinsic multiscale structure. Our proposed approach circumvents these limitations, shown in Figure 1

---

Under review as a conference paper at ICLR 2021
Regular pixel grid
WaveMesh
SLIC
400
100.0
Frequency (9i)
Frequency (%)
300
Frequency
200
100
7.1
0.0
0.0
0.6
0.7
0,0
1+1
2x2
4x4
Bxd
16x26
32x32
1x1
2x2
4xd
8x8
16x16
32x32
1x1
4xd
8x8
16x26
32x32
SLiC
WaveMesh
Superpixel size
Superpixel size
Superpixel size
(a)
(b)
Figure 1: (a) Average distribution of superpixel size averaged across MNIST training dataset for (left), WaveMesh (center), and SLIC (right). In each panel, an insert shows the graph representation of single sample for illustration. Size of a node in the graph proportional to the superpixel size. SLIC superpixels are not cubic yet the x-axis binning chosen to match other plots. (b) Boxplots of the # superpixels per image for CIFAR-10 training dataset.
different superpixel representation:
The objectives of our work are twofold. First, we aim to rethink the process of downsampling and/or superpixeling images by introducing multiscale superpixel representation that be considered as in between the regular grid and similar-sized superpixel representations. Secondly, we systematically study the feasibility of learning classify images from embedded graphs that represent the multiscale superpixels. In this context, the contributions of our study are as follows.
We present WaveMesh, an algorithm to superpixel (compress) images in the pixel domain. WaveMesh is based on the quadtree representation of the wavelet transform. Our sample-specife method leads to non-uniformly distributed and multiscale superpixels. The number and size of superpixels in an image systematically computed by the algorithm based on the image content. WaveMesh requires at most one tunable parameter.
We propose WavePool, a spatially heterogeneous pooling method tailored WaveMesh superpixels. WavePool preserves spatial structure leading to interpretable intermediate outputs. WavePool generalizes the classical pooling employed in CNNs, and easily integrates with existing GNNs. To evaluate the WaveMesh representation and the WavePool method for image graph elassifica- tion, we compare them with SLIC superpixels and graclus-based pooling by conducting several experiments using SplineCNN, network proposed by Fey al. (2018).
2 RELATED WORK
Superpixeling. Grouping pixels form superpixels was proposed by Ren & Malik (2033
preprocessing mechanism that preserves most the structure necessary for image segmentation. Since then many superpixeling algorithms have been proposed including deep learning based methods
(Liu et al. 2011 Li& Chen 2015 Tu et al. [2018] Giraud et al. 2018 Yang et al. 2020 Zhang et al. 2020). The SLIC algorithm proposed by Achanta et (2012) based on k-means clustering.
GNN for image graph classification. Prior studies have demonstrated the representational power and generalization ability of GNNs image graph classification tasks using SLIC superpixels. Dwivedi et al. (2020) show that message passing graph convolution networks (GCNs) outperform Weisfeiler-Lehman GNNs on MNIST and CIFAR- 10 datasets. Recognizing the importance of spatial and hierarchical structure inherent in images, Knyazev et al. (2019) model images multigraphs that represent SLIC superpixels computed different user-defined scales, and then successfully train GNNs the multigraphs. SplineCNN proposed by Fey al. (2018) is another network for learning from irregularly structured data. It builds on the work of Monti et al. (2017), but uses a spline convolution kernel instead of Gaussian mixture model kernels.
Graclus-basee pooling, Pooling is used in GNNs to coarsen the graph by aggregating nodes within specified clusters. Graclus is kernel-based multilevel graph clustering algorithm that efficiently clusters nodes large graphs without any eigenvector computation. Graclus is used in many GNNs to obtain clustering on the nodes, which is then used by the pooling operator coarsen the graph
2

---

Under review as a conference paper at ICLR 2021
(Defferrrrd et al. 2016 Monti et al. 2017 Fey al. 2018). Hereafter, refer pooling based on graclus clustering as graclus based pooling.
3 WAVEMESH: MULTISCALE WAVELET SUPERPIXELS
Superpixel meshes
Graph geseration
Input images
Output graphs
Wavnlet-hosed
quaditoc compeession
Filtered images
Figure 2: Filtering images in wavelet space generates non-uniform superpixel meshes that are then represented as embedded graphs. The leftmost images are preprocessed with the method described in section 3 with threshold value equal five times the theoretical value. Natural images are from the Pascal dataset (Everingham et al. (2100), and the medical image is from the NLST dataset.
The WaveMesh algorithm is broken down into its elementary steps below: 1) images are wavelet transformed, 2) images are filtered in wavelet space by thresholding the wavelet coefficients, and the superpixel mesh is generated from the wavelet-filter mask. The algorithm is rooted in wavelet theory's seminal work (Mallat 1989: Donoho Johnstone] 1994b). The particular way in which wavelets used in this work is inspired by their related application in the physical sciences (Schneider & Vasilyev 2010
Bassenne et al. 2017 2018).
3.1 STEP 1: WAVELET TRANSFORMATION OF THE INPUT IMAGE
Consider two-dimensional (2D) image / discretely described by its pixel values / [xo)] centered at NxN regular grid, where A is the inter-pixel spacing and (i,j) j)
locations Zo 2- 2 (iA, jA) of
1. ... 2N 1. A continuous wavelet representation of / is I(x: Ezo ( [xx])[() (x zo).
where v is the continuous pixel-space coordinate, and 6" xo) are scaling functions that form orthonormal basis of low-pass filters centered at To. with filter width A. The scaling functions have unit energy (Ø (a: - 20044 (æ: #a)) 1, where the bracket operator (y) = 1/(N A)² yy(a))d denotes the global average for general 2D continuous field y(x). In practice, when dealing with
discrete signals, 1000 [wo] cannot be computed exactly, since is only known at discrete points Zo-
Instead, it is numerically discretized and the approximation coefficients (00) [zo] are estimated as an algebraic function of /| Assuming that 6x æo) decays fast away from æ Zp. we get
(0) [x = / Addison 2017). This estimate for 700 [xo] the initialization stage of the recursive wavelet multiresolution algorithm (MRA) of Mallat (1989). which enables the computation
of wavelet coefficients at coarser scales.
The decomposition of the finest-scale low-pass filter (" 2o) in terms of narrow-band wavelet filters vi(e.)((x: 2x)) with increasingly large filter width and coarsest-scale scaling function
3

---

Under review as a conference paper at ICLR 2021
((() xs) yields the full wavelet-series expansion of I,
5
3
T(x) = Σ
((sid)
(o (xs]))( (xc xza) ) + 7 (1)
[x (2s) Tg). -5)
(1)
==1 Ro
);5) and 1 [x)] (Tar))b())) -xs)) are wavelet and approximation coefficients at scale 8 and S respectively, obtained from the orthonormality properties of the wavelet and scaling functions. In this formulation,
Here, ((x))) [xxx] (T(x))
(1, 2, 3) a wavelet directionality index, and s = (1.2.... S) is a scale exponent, with S = logg N the number of resolution levels allowed by the grid (5 for x2x32 32 images). Similarly, x, 2 (í is scale-dependent wavelet grid of (N/2*)x (N/2*) elements where the basis functions are centered, with i, j = 1, .N/2--1 1. The wavelet coefficients represent the local fluctuations of I centered at I, at scale s, while the approximation coefficient is proportional to the global mean of I. At each scale, the filter width of the wavelets is 2.A. A
In this study, the 21 orthonormal basis functions ()(5d)) (a: .z,) are products of one-dimensional (ID) Haar wavelets (Meneveau) 1991). The definition 2D wavelets as multiplicative products ID wavelets is particular choice that follows the MRA formulation (Mallat 1189), Haar wavelets have a narrow spatial support that provides a high degree of spatial localization. However, they display large spectral leakage high wavenumbers since infinite spectral and spatial resolutions cannot be simultaneously attained due to limitations imposed by the uncertainty principle (Addison (017). Different boundary conditions can be assumed for the field 7. We do not require such a choice in this study as we restrict ourselves to square images. However, the wavelet MRA framework is not limited to square inputs and can be generalized to rectangular inputs (Addison 2017 Kim al., 2018).
The definition of 2D wavelets as multiplicative products of ID wavelets is a particular choice that follows the MRA formulation described by Mallat (1989), in which, the multivariate wavelets are characterized by an isotropic scale and therefore render limited information about anisotropy in the image. A large number of alternative basis functions have been recently proposed for replacing traditional wavelets when analyzing multi-dimensional data that exhibit complex anisotropic struc- tures such as filaments and sheets. These include, but are not limited to, curvelets, contourlets, and shearlets (see Kutyniok & Labate (2012) for an extensive review on this topic).
3.2 STEP 2: FILTERING OF THE IMAGE IN WAVELET SPACE
The second step decomposes I as
1=1++1
(2)
where the filtered 1> and remainder 1g components correspond to the highest and lowest energetic wavelet modes of I, respectively. By construction, these two components are not spatially cross- correlated, as implied by the orthogonality of the wavelets and by the filtering operation described below. Note that large wavelet coefficients are associated with large fluctuations within the corre- sponding region of the scale-dependent wavelet grid xz. these being markers of underlying coherent structures. Under the assumptions that L< additive Gaussian white noise, Donoho & Johnstone (1994a) described a wavelet-based algorithm that is optimal for achieving the target decomposition 2 since it minimizes the maximum 2 esstmattion error of Iz. Is is obtained by retaining only the
wavelet coefficients y(ad))
whose absolute values satisfy
(s.d)
(a.d)
(xx) if [
(x.)) TT
(xx)
otherwise,
for all scales s, positions va and directions d. In Eqqation 3 T is a theoretical threshold defined as T = v 207 In N2,
(4)
where o2 the unknown variance of Tg. In this study, the iterative method of Azzalini et al. (2055
employed, which converges to T starting from first iteration where o7_- in Equation 4 is substituted by the variance of of the total image I. This iterative procedure does not introduce significant computational overhead, since only wavelet transform is required independently of the number of iterations. The algorithm does not introduce any hyperparameter when the theoretical threshold value is used. Note that the threshold is image-dependent, thereby ensuring that the algorithm adapts
4

---

Under review as a conference paper at ICLR 2021
the number of superpixels to each image appropriately. The above filtering operation is equivalent to applying a binary filter mask to wavelet coefficients, denoted as wavelet-filter filter mask below.
The iterative method is deemed converged when the relative variation in the estimated threshold is less than 0.1% across consecutive iterations. A maximum (10) iterations were required to obtain the results presented below. The overall computational cost is O (n,M), where n, is the number of iterations and M is the number of pixels in the image (Azzalini et al. (005). In this work, we allow for further reduction in number of superpixels by varying the threshold 7 take larger values. Figure 2 illustrates the application of this wavelet filtering method on four images, wherein for RGB images filtering applied to each channel independently. Most of the structural and edge information is preserved at all scales. However, a drawback of the method is that the superpixel boundaries are necessarily regular and axis-aligned.
3.3 STEP 3: GENERATING THE SUPERPIXEL MESH FROM THE WAVELET- MASK
(a) Quadtree of (tagged) wavelet coefficients
(b) Pruned quadtree of filtered wavelet coefficients
Wavelet scale
Wandda
Supersiuas
(c) Geverating WaveMesh superpixels from filtered waveled coeffieients
Figure 3: Illustration of the wavelet-based quadtree compression algorithm for an 8x8 image, along with the resulting adapted grid. Starting from the coarsest possible wavelet grid that contains just one superpixel, the algorithm adapts the grid by recursively splitting it. If the wavelet coefficient corresponding to region tagged (denoted by blue color), then that region is split into 2x2 superpixels.
To generate superpixels for given image, the final step is grid adaptation based on the wavelet-filter mask described in suusecction 3.2] The result is non-uniform grid of multiscale superpixels adapted around regions of the image with high variability.
Quadtree representation. The algorithm is perhaps best understood by representing the wavelet coefficients in quadtree (Finkel & Bentley 1974), tree data structure in which each node has exactly four children. A quadtree-based representation of wavelet coefficients was previously shown to be efficient data structure for wavelet-basee image compression (Banham & Sullivan 1992 Wakin et al.) 2003). Here, the height this quadtree equals the number of decomposition levels S in the wavelet transform. Each vertex given level is associated with triplet of wavelet
coefficients (((_d=))) (xx), ( ((s.d=1) (x,),
((x.d=2) (x.)). All vertices from a given level correspond wavelet coefficients across all locations at given scale. The children vertices of a root vertex are the wavelet coefficients from that region in space smaller scales. The quadtree representation of the wavelet coefficients of an 8x8 image is schematically represented in Figure 3 [aa. The number on each vertex indicates the scale, from the smallest scale s=1 associated with 2x2 pixel patches up to the largest scale s=3 associated with the entire 8x8 image. The pixel regions associated with each wavelet coefficient are delineated by solid lines in the three leftmost figures Figure 3())
Node tagging. The vertices in the tree are tagged according to the filtering algorithm described in subsection 3.2] The tagged elements of the tree denoted by blue filled color in Figure 3(a.b) correspond to those with absolute values larger than the threshold T, and therefore correspond to locations in the image with important spatial variability. In the 2D case, tagging applied at least of the wavelet coefficients of per location is larger than the threshold. Additional tagging
5

---

Under review as a conference paper at ICLR 2021
by green-filled color is applied to wavelet coefficients that smaller than the threshold T but that correspond to a spatial region with at least one tagged wavelet coefficient at a smaller scale. This corresponds to tagging all the ancestors of previously tagged vertices. This tagging procedure enforces cubic superpixels by ensuring that when there is a coherent structure at scale but not at a larger seale 8+1,, the wavelet coefficient scale 8+11 at that location also tagged, hence triggering local grid refinement at level s+11 Non-tagged vertices are pruned as shown in Figure 3/b).
Mesh generation. Starting from the coarsest possible wavelet grid a øs that contains just one superpixel, the algorithm adapts the grid by recursively splitting it as follows. If the wavelet coefficient corresponding a region tagged, then that region is split into 2x2 superpixels, which locally refines the grid. The algorithm is stopped otherwise. The same recursive loop is then applied to the refined superpixels. The final configuration of the adapted grid is obtained when none of the wavelet coefficients in any the superpixels are tagged. An example of final adapted grid is shown in Figure c).. The dashed lines correspond to the superpixel refinement due the vertex being tagged. Adapted grids from real images are shown in Figure 2 where the superpixel meshes exhibit desired level of heterogeneity with multiscale refinement around edges. For RGB images, the most restrictive mesh is employed every location and scale. In other words, tagging for the full image is applied if at least of the channels is tagged.
WAVEPOOL: SPATIALLY HETEROGENEOUS POOLING
(a) Before pooling
(b) After pooling
(c) After 2 pooling
Figure 4: Illustration of WavePool from wavelet quadtree representation. Leaf nodes x2x2 superpix- els) are recursively pooled. In the lower panel, dashed squares and lines correspond to nodes and edges in the region adjacency graph (RAG) representation of the superpixel mesh, respectively.
The proposed spatially heterogeneous pooling, WavePool, is best explained using the wavelet co- efficient quadtree representation described in subsection 3.3] One WavePool operation consists in aggregating all the leaf nodes of the wavelet quadtree. In the pixel domain, this step essentially corresponds to merging patches of x superpixels into a parent superpixel, and aggregating the node features with a choice of pooling function (mean or max typically). FFigure 4 illustrates WavePool on simple superpixel mesh and its effect on both the quadtree (Figure 4jupper panel) and region adjacency graph (Figure 4|lower panel) representation. a region adjacency graph (RAG), nodes correspond to superpixels, and edges connect neighboring superpixels. We show RAG in Figure 4 because we train GNNs to learn from embedded RAGs. RAG is not a tree and should not be confused with the wavelet coefficient quadtree.
By construction, WavePool generalizes the classical CNN pooling operation. For a regular superpixel grid as in Figure 5] WavePool exactly matches the conventional 2x2 pooling in CNN. Although more general than its CNN counterpart, WavePool is restricted to WaveMesh superpixels or more broadly any quadtree based superpixel representation (Tanimoto & Pavlidis (1975): Zhang al. (2018) unlike graclus- based pooling. However, craclus-based pooling does not converge to CNN pooling even when applied regular superpixel grids as shown in Figure
5 EXPERIMENTS AND RESULTS
DATASETS. We performed image graph classification experiments on SLIC and WaveMesh superpix- els from datasets: MNIST, Fashion-MNIST and CIFAR-10 10 (LeCun et al.) 1998) Xiao et al. 2017 Krizhevsky al. 2009). We represent superpixels by embedded region adjacency graphs (RAG),

---

Under review as a conference paper at ICLR 2021
Graclus
Regular grid
WavePool
Afier
pooling
Alter pooling
After pooling
Figure 5: Illustration of WavePool versus graclus-based pooling on regular superpixel grid.
Rabd
Rabel
Robe
Label
Label
Corvolution (Spline Cimee) + Poaling (WarePood)
Comvolution (Splin Clur) Pooling (WavePood)
Corvolation (Splinee
Pooling (WavePPod)
Flatomm
cunnected
Fully
Softmer
Figure Model architecture. WaveMesh superpixel graph of an example image (digit from MNIST dataset passing through SplineCNN with WavePool. Nodes represent superpixel centers.
where nodes correspond superpixels, and edges connect neighboring superpixels. Node embed- dings are mean intensity of superpixels, Edges in the graph are directed with pscudo-coordinates as in Fey al. (2018). For more details on the datasets, refer (subsection A.I
EXPERIMENTAL SETTINGS.
We conduct experiments on two configurations based SplineCNN
implementation available in PyTorrch Geometric (Fey & Lenssen 2019). The configurations are:
1. SplineConv (3. 3), 1,) -> Pool -> SplineConv( (3, (3, 32, 64) -> Pool -> Global mean pool -> FC(128) -> FC(1)). This configuration has 30506 parameters.
2. SplineConv(33. 3), 1, 1,32)
Pool -> SplineCoov((33. (3, 3), 64)
Pool
SplineConv(CC 3), 64, 128) -> Pool -> Global mean pool -> FC((26)) -> FC(10). This
configuration has 139178 parameters.
Through the experiments we aim to: 1) Compare how SplineCNN performs on SLIC and WaveMesh superpixels under the same network architecture and training settings; 2) Understand the effect of WavePool on learning from WaveMesh superpixels, everything else being the same. The Py Torch Geometric implementation of SplineCCNN uses Adam optimizer with initial learning rate of 0.01, which is decreased by a factor of 10 after 15 and 25 epochs. Since the goal of our experiments not tune the best model for WaveMesh superpixel representation, any hyperparameter tuning. We use the same training settings and train the network for 30 epochs on MNIST and Mashion-MNIST. and for 75 epochs CIFAR-1O. The pooling function is max for both WavePool and graclus- graclus-based pooling. All experiments are repeated times, and the mean train and test accuracy are reported along with the standard deviation. FFiuure 6 visual illustration of WaveMesh superpixel graph passing through SplineCNN network with WavePool.
conducted all experiments without
MNIST. Results on the MNIST dataset from our experiments and prior work are shown Table
We report the mean and standard deviation values for train and test accuracy, and precision. We didn't include recall since averaged one-versus-all recall and accuracy are equal for balanced datasets. Experiments 1-4 uses WaveMesh superpixels obtained using the theoretical threshold T' as described in subsection 3.2 Experiments 3 and 4 are same as and 2, but uses a network (config 2) with more parameters. Across these four experiments we first observe that SplineCNN is successful in learning to classify images from the WaveMesh representation. We also observe that the network with WavePool performs better than the one with graclus-based pooling. Experiments 7-10
7

---

Under review as a conference paper at ICLR 2021
same 1-4 but with lesser WaveMesh superpixels. These experiments done to compare with experiments 13-14 that report results from prior work on SLIC superpixels where each image has exactly 75 superpixels. To reduce the number of WaveMesh superpixels in image to about 75, increased the theoretical threshold 7' in our algorithm. From the results for experiments 7-10, we observe that the network with WavePool performs similar Experiments 5-6 and 11-12 are on SLIC superpixels that we generated using the scikit-learn package. Comparing experiments 1-4 with 5 and 6, and 7-10 with 11 and 12, observe that SplineCNN leams just as well or better from WaveMesh superpixels.
better than graclus-based pooling.
Superpixel
#Nodes
Config
Pooling
Train acc
Test ace
Precision
WaveMesh
238=500
Graclus
92.33:10.09
89.63:10.45
89.71 0.444
WaveMesh
238=:50
WavePoul
95.75:10.08
95.44:10.12
0.13
WaveMesht WaveMesh
238=50
Graclus
98.39 0.05
96 06.80=00.11
96.86=0.08
238=500
WavePool
99.6811 0.03
08.68:10.08
98.67=0.088
SLIC SLIC
241::5
Graclus
95.50+0.21
95.51=:0.29
95 05.48:00330
241::5
Graclus
98.07:10.04
97.83:10.11
WaveMesh
57::12
Graclus
93.3440.04
92.55=+0.15
92.47=0.15
WaveMesh
57::12
WavePool
96.30+0.10
93.74:100117
93.72=0016
WaveMesh
57412
Graclus
95.68+0.09
94.21:10.21
9,224=0,22
10
WaveMesit
57::112
WavePool
99.23:10004
93.84:10.48
93.93=0.44 0.40
11
SLIC
59=2
Graclus
92.34+0.11
91.18:+0.22
91.13=0.21
12
SLIC
59=2
Graclus
94.13+0.088
92.99:10..22
93.01=0.20
13
SLIC Monti et 2017
75:00
Graclus
91.11 95.22
14
SLIC (ooy
2018
75=0
Graclus
Table Results MNIST dataset. MD (%) are reported for each evaluation metric.
#
Superpixel
#Nodes
Conlig
Pooling
Train acc
'ies acc
Precision
WaveMesh
436+:129 436:::29
Graclus
80,36+0.39
63.35-t2.00 2.94
73.84::2.87
WaveMesh
WavePool
85.77::0.18
76.60-10..8
79.34:10.49
3
WaveMesh
436::129
2
Graclus
85:40=0.110
75.69:::.47
79.40:::0.8
WaveMesh
436:t:129
WavePPol
92.58:10.07
83.66:1:1.49
83.86:1:1.53
WaveMesh
Graclus
81.32:00113
76.75
0.35
77.78:40.33
WaveMesh
261:::35
WavePool
85.9110.110
81.35:1-0.69
81.99:10.48
WaveMesh
2615:35
Graclus
83.18::0.18
79.78:t0.46
80.47:#0.52
8
WaveMesh
261:1:35
2
WavePool
92.14-10.15
87.65-10.33
87.66-10.32
SLIC SLIC
259.1.
Graclus
82.91110.08
81.49.10.35
81.21:10.41
10
259.17
Graclus
86.71:10.10
85.00-10.32
84.86:10.55
11
WaveMesh
134:1:22
Graclus
80.92:1:0.16
78.85:10.099
78.87:10.06
WavePool
85.18.10.13
80.42-10.33
80.80
13
WaveMesh
134:1:22
2
Graclus
83.84:10.13
80.60:10.21
80.97:10.41
14
WaveMesh
134.1-22
WavePool
90.98.10.13
82.65:10.52
82.95:10.64
SLIC
118:14
81.46:10.16
79.59:10.38
79.18:10.34
16
SLIC
118:14
2
Graclus
84.11-1.0.15
82.12:10.27
82.08
17
SLIC Avelar et al. 2020)
S 75
83.07
Table 2: Results Fashion- MNIST. Mean (%) are reported for each evaluation metric.
FANIION-MNIST AND CIFAR-10. Experiments similar to MNIST were performed on these two datasets. Results are reported in Tables 2 and 3. For both datasets, experiments 1-4 were performed on WaveMesh superpixels obtained using the theoretical threshold 7' in our algorithm. In Table 3 experiments 7-9 report results from (Dwivedi et al. (020), where RingGNN and Gated GCN perform the worst and best. Results for MoNet are shown because SplineCNN builds on the work of MoNet. For the case of -NNSS,, experiments 1-4 with more superpixels have similar train accuracy in experiments 5-8. However, the trained model in experiments 1-4 performed poorly test data when compared to experiments 5-8. It unclear why this happened. More detailed error analysis is included in the subsection A.3
From our experiments on 3 benchmark datasets, we observe that under the same network architec- ture and training settings, Sp with original graclus-based pooling learns from WaveMesh
8

---

Under review as a conference paper at ICLR 2021
Superpixel
#Nodes
Config
Pooling
Train acc
Test acc
Precision
WaveMesh
197282
Graclus
52.63-10.36
43.36:1:0.72
47.66:10.48
WaveMesh
197
WavePuol
55.04:10.21
52.58:10.21
52.22:10..1
WaveMesh
197=82
Graclus
60.28:2-011
50:42-4:0.27
54.77=:0.24
WaveMesh
197.:82
WavePool
70.25:1:.30
36.89110.31
57.03:10.31
SLIC
215::15
Graclus
50.96:t 0.51
45.87-2:0226
47.59:10.41
Graclus
50.69:10.45
(RingGiNN)
85-150
19.30:11..12
SLIC (MoNet)
85-150
104229
65.92:1:2.52 94.55+1.00
54.66:1:0.52
(Gated GCN)
85-150
67.3140.31
Table Results on CIFAR-10. Experiments 7-9 report min and max number of nodes in column 3, and the number of parameters in the model in column 4 (Dwivedi et al. 2020).
Contgg dataset
dataset
Fashion MNIST
Confgg
dataset CIFAR-10
Moon tost accursoy (N)
WaveMesh WavePool
87.5
57.5
86
82.5
52.5
04
77.5
47.5
75.0
235-241
57-59
259-281
115-134
157-215
Config datasel - MNIST
Config
dataset Fashion-MNIST
Config
datassel CIFAR-10
100
90.0
60.0
Mean arsurcy
88
85.0
55.0
80.0
50.0
02
90
75,0
46.0
57-69
137-215
Moan number superptxols
superpxels
or superpixo's
Figure 7: Mean test accuracy versus number of superpixels for all three datasets for both network configuration. The plot compares the accuracy of WaveMesh and WavePool combination with SLIC and Graclus combination.
superpixels on-par with SLIC superpixels. Additionally, under the same settings, we observe that the best performance is achieved when replacing graclus-based pooling with WavePool while using WaveMesh superpixels. This is shown in Figiurr 7 for all three datasets. We believe this increase in per- formance is because WavePool accounts for spatial heterogeneity while aggregating nodes. Overall, we conclude that WaveMesh is reasonable sample-specific multiscale superpixeling method.
CONCLUSION
Over the last years powerful GNNs have been developed for a variety of tasks on graph structured data. Nonetheless, for image graph classification tasks, GNN studies have been restricted to graphs that model a regular grid or similar-sized SLIC superpixel representations. Looking at images through the lens of GNNs enables rethinking the process of downsampling, and offers new possibilities for image representations that explore the landscape between the regular grid and similar-sized superpixel representations. Towards this goal, we introduced WaveMesh, a superpixeling algorithm that computes spatially heterogencous superpixels of varying sizes within image. We also proposed WavePool, new pooling scheme tailored WaveMesh superpixels. We investigate the performance of both methods across three benchmark datasets. Our experiments comparing WaveMesh superpixels with SLIC superpixels and WavePool with graclus-based pooling demonstrated promising results. Multiscale spatially heterogeneous superpixels warrant further attention. As future direction, we encourage researchers to benchmark GNN models on WaveMesh superpixels and explore architectures custom to WaveMesh superpixels. In particular, envision greater interest in this direction of research from the medical machine learning community where high resolution images are ubiquitous (see subsection A.4).

---

Under review as a conference paper at ICLR 2021
REFERENCES
Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine Süsstrunk. Slic superpixels compared to state-of-the-art superpixel methods. IEEE transactions on pattern analysis and machine intelligence, 34(11):227422882, 2012.
Paul S Addison. The illustrated wavelet transform handbook: introductory theory and applications in science, engineering, medicine and finance. CRC press, 2017.
Pedro HC Avelar, Anderson R Tavares, Thiago LT da Silveira, Cláudio Jung, and Lufs C Lamb. Superpixel image classification with graph attention networks. arXiv preprint arXiv: 05544,, 2020.
Alexandre Azzalini, Marie Farge, and Kai Schneider. Nonlinear wavelet thresholding: A recursive method to determine the optimal denoising threshold. Applied and Computational Harmonic Analysis, 18(2):177-185, 2005.
Mark R Banham and Barry J Sullivan. A wavelet transform image coding technique with a quadtree structure. In [Proceedings] /CASSP-92: 1992 IEEE International Conference on Acousties, Speech. and Signal Processing, volume 4, pp. 653-656 IEEE, 1992.
Maxime Bassenne, Javier Urzay, Kai Schneider, and Parviz Moin. Extraction of coherent clusters and grid adaptation in particle-laden turbulence using wavelet filters. Phys. Rev. Fluids, 2:054301, 2017.
Maxime Bassenne, Parviz Moin, and Javier Urzay. Wavelet multiresolution analysis of particle-laden laden turbulence. Phys. Rev. Fluids, 38):008430 2018.
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks graphs with fast localized spectral filtering. In Advances in neural information processing systems, pp. 3844-3852, 2016.
David L Donoho and lain M Johnstone. Ideal spatial adaptation by wavelet shrinkage. Biometrika, 81:425-455, 1994a.
David I. Donoho and lain M Johnstone. Ideal spatial adaptation by wavelet shrinkage. biometrika, 81 (3):425 455, 1994b.
Vijay Prakash Dwivedi, Chaitanya K Joshi, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. Benchmarking graph neural networks. arXiv preprint arXiv:2003.00982. 2020.
Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International joueraaaaoaaouuraal of computer vision, 88(2): 303-338 2010.
Matthias Fey and Jan E. Lenssen. Fast graph representation learning with Py Geometric. In ICLR Workshop Representation Learning on Graphs and Manifolds, 2019.
Matthias Fey, Jan Eric Lenssen, Frank Weichert, and Heinrich Müller. Splinecnn: Fast geometric deep learning with continuous b-spline kernels. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 869-877, 2018.
Raphael A. Finkel and Jon Louis Bentley. Quad trees data structure for retrieval on composite keys. Acta informaticu, 4(1):1-9, 1974.
Rémi Giraud, Vinh-Thong Ta, and Nicolas Papadakis. Robust superpixels using color and contour features along linear path. Computer Vision and Image Understanding, 170:1-13, 2018.
Varun Gulshan, Lily Peng, Mare Coram, Martin C Stumpe, Derek Wu, Arunachalam Narayanaswamy, Subhashini Venugopalan, Kasumi Widner, Tom Madams. Jorge Cuadros, al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. Jama, 316(22):2402-244 2016.
10

---

Under review as a conference paper at ICLR 2021
J Kim, M Bassenne, CAZ Towery, PE Hamlington, AY Poludnenko, and J Urzay. Spatially localized multi-scale energy transfer in turbulent premixed combustion. Journal of Fluid Mechanics, 848: 78-1 116, 2018.
Boris Knyazev, Graham W Taylor, and Mohamed Amer. Understanding attention and generalization in graph neural networks. In Advances in Neural Information Processing Systems, pp. 2202-4212, 2019.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Gitta Kutyniok and Demetrio Labate. Shearlets: Multiscale analysis for multivariate data. Springer Science & Business Media, 2012.
Paras Lakhani. The importance of image resolution in building deep learning models for medical imaging. Radiology: Artificial Intelligence, (1))x119017] 2020.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2322 1998.
Zhengqin Li and Jiansheng Chen. Superpixel segmentation using linear spectral clustering, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1356-1363, 2015.
Ming- Yu Liu, Oncel Tuzel, Srikumar Ramalingam, and Rama Chellappa. Entropy rate superpixel segmentation. In CVPR 2011, pp. 2097-2104. IEEE, 2011.
Stephane Mallat. A theory for multiresolution signal decomposition: the wavelet representation. IEEE T. Pattern Anal., 11(7):674-693, 1989.
Charles Meneveau. Analysis of turbulence in the orthonormal wavelet representation. J. Fluid Mech., 232:469-520, 1991.
Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodola, Jan Svoboda, and Michael M Bronstein. Geometric deep learning on graphs and manifolds using mixture model cnns. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 5115-5124, 2017.
Xiaofeng Ren and Jitendra Malik. Learning a classification model for segmentation. In null, pp. 10. IEEE, 2003.
Carl Sabottke and Bradley M Spieler. The effect of image resolution on deep learning in radiography. Radiology: Artificial Intelligence, 290)se1 2020.
Kai Schneider and Oleg V Vasilyev. Wavelet methods in computational fluid dynamics. Annu. Rev. Fluid Mech., 42:473-500 503,, 2010.
Mingxing Tan and Quoe Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International Conference on Machine Learning, pp. 6105-6114, 14, 2019.
Steven Tanimoto and Theo Pavlidis. A hierarchical data structure for picture processing. Computer graphics and image processing, 4(2):104-119, 1975.
Wei-Chih Tu, Ming- Yu Liu, Varun Jampani. Deqing Sun, Shao- Yi Chien, Ming- Hsuan Yang, and Jan Kautz. Learning superpixels with segmentation-awwrre affinity loss. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 568-576, 2018.
Stéfan van der Walt, Johannes L. Schönberger, Juan Iglesias,, François Boulogne, Joshua D. Warner, Neil Yager, Emmanuelle Gouillart, Tony Yu, and the scikit-image contributors. scikit- image: image processing in Python. PeerJ, 2:e453, 6 2014. ISSN 2167-8359. doi: 10.7717/peerj. 453. httpss/tt::
www.orl.................oo.....o .. 7717 j.
Michael B Wakin, Justin K Romberg, Hyeokho Choi, and Richard G Baraniuk. Geometrie methods for wavelet-basee image compression. In Wavelets: Applications in Signal and Image Processing X, volume 5207, pp. 507-520. International Society for Optics and Photonics, 2003.
11

---

Under review as a conference paper at ICLR 2021
Xiaosong Wang. Yifan Peng, Le Lu, Zhiyong Lu, Mohammadhadi Bagheri, and Ronald M Summers. Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-superrised classi- fication and localization of common thorax diseases. In Proceedings of the IEEE conference computer vision and pattern recognition, pp- 2097-2106, 2017.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747 2017.
Fengting Yang. Qian Sun, Hailin Jin, and Zihan Zhou. Superpixel segmentation with fully convolu- tional networks. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pp. 13964-13973, 2020.
Chongyang Zhang, Guofeng Zhu, Minxin Chen, Hong Chen, and Chenjian Wu. Image segmentation based multiscale fast spectral clustering. arXiv preprint arXiv:/8/2.044 2018.
Yongxia Zhang, Qiang Guo, and Caiming Zhang. Simple and fast image superpixels generation with color and boundary probability. The Visual Computer, pp. 1-14, 2020.
12

---

Under review as a conference paper at ICLR 2021
A APPENDIX
A.1 DATASETS
MNIST: 28 grayscale images, 60k train and 10k test, 10 categories.
x8x28 grayscale images, 60k train and 10k test, 10 categories.
CIFAR-10: 32x32 color images, 50k train and 10k test, 10 categories.
SLIC superpixels were generated using the scikit-image library by setting the compactness parameter to 0.25 (van der Walt et al., 2014). A small value of compactness parameter was chosen to ensure that superpixels shapes are not all square. While computing WaveMesh superpixels MNIST and Fashion-MNIS images padded with zeros to make them x2x32. See SeglFi 8 for examples of region adjacency graphs generated from CIFAR-10 10 images using WaveMesh superpixels.
Airplane
Automobile
Ship
(
(b)
(c)
Figure 8: WaveMesh superpixels represented by region adjacency graphs (RAG). (a) Images from the CIFAR-10 dataset. (b) RAGs representing WaveMesh superpixels obtained using the theoretical threshold 7 for images shown in (a). (c) RAGs representing WaveMesh superpixels obtained using threshold equal 27'. Size of nodes in the graph are proportional the corresponding superpixel size.
13

---

Under review as a conference paper at ICLR 2021
A.2 ADDITIONAL NOTES ON STEP 2: FILTERING THE IMAGE IN WAVELET SPACE
Note that the energy-based, spatially local filter outlined in this study is fundamentally different from Fourier-based spectral filtering, in that the latter is scale-sharp filter that acts globally in pixel space and does not allow the discrimination localized. energetic structures. Additionally, unlike Fourier-based filters, the present method does not require periodicity.
A.3 ERROR ANALYSIS
Mean Superpiieds:: 238
Moan Superpiessss
Mean Superppaess: 238 Accuracy:
Moon @Sppeppnnei 5? Accuracy:
(4)
(10)
(10)
Mean #Supersivess:
Meen Superpleets: 92.99%
Mean #Superpive's 243
Meen #Superploeks 92.99%
97.83%
(6)
(12)
6
(12)
Figure 9: MNIST error analysis: The top row corresponds to experiments on WaveMesh superpixels with WavePool in config The bottom row corresponds experiments on SLIC superpixels with Graclus pooling in config 2. Left quadrant shows the confusion matrix and the right quadrant shows the error rate matrix averaged over all runs an experiment. Experiment numbers from Table are indicated below each matrix.
MNIST. Figure the confusion matrix looks good for the experiments as most images are on the main diagonal. The actual class is along the rows and predicted class is along the columns. In Figure the error rate matrix for each experiment in the right quadrant obtained by dividing each value in the confusion matrix by the number of images in the corresponding class, and by filling the main diagonal with zeros.
SLIC with Graclus. Comparing the error rate matrices of experiments 6 and 12, in both cases many digit 4 images are being misclassified as 9, and as the number superpixel reduces, many digit images are also being misclassified as 9.
WaveMesh with WavePool. Comparing the error rate matrices of experiments 6 and 12In exper- iment 4 many digit images are wrongly classified as 9. However, in experiment 10, with the number of superpixels equal one-fifth of experiment 4, many 9s are being misclassified as 4 and many 38 are being misclassified as 2.
FASIT.N-MNIST. Figure 10
from the confusion matrices, we can conclude that the model performs best in classifying images from the classes trouser, sandal, sneaker, bag, ankleboot. This is true both with WaveMesh and SLIC superpixels.
conclude that shirt is getting misclassified the most. Also, the columns for classes shirt, coat and pullover are quite bright, indicating that many images are getting misclassified into these classes.
from the emror rate matrices
14

---

Under review as a conference paper at ICLR 2021
134
Moan @Superpbetss 263 Accuracy:
Mean @Superpiess 134 Accuracy:
Accuracy: 65%
Accuracy: 82 82.65%
traser
Siuner
Touser
cor
clat
wnsker
Wester
(14)
(
(14)
Mean #Superpkeets: 259 Accuracy:
Meon @Spperpivess: 118 Accuracy:
Mean #Superptetts: 259
Mean #superpieets: 118
truser
Truser
truser
cad
cad
Senda
sandar
seeker
seeder
pretor
(10)
(16)
(10)
(16)
Figure 10: FashionMNIST error analysis: The top corresponds to experiments WaveMesh superpixels with WavePool in config 2. The bottom row corresponds to experiments on SLIC superpixels with Graclus pooling config 2. Left quadrant shows the confusion matrix and the right quadrant shows the eтrог rate matrix averaged over all runs of an experiment. Experiment numbers from Table 2] are indicated below each matrix.
Mean #Superpixels: 436
Accuracy: 83.66%
ankietrot
brouer
pullever
dress
sandal
Sneaker
beg
trouser
pullover
dress
coat
sandel
shirt
bag
ankieboot
Figure 11: Error rate matrix for experiment from Table Comparing this matrix with that of experiment 8 from Figure TO, we observe that many more images of pullover and coat are getting misclassified as shirt in this experiment when compared to experiment 8.
Overall, from the error analysis for MNIST and Fashion-MNIST we observe that misclassification patterns not very different for WaveMesh and SLIC superpixels.
15

---

Under review as a conference paper at ICLR 2021
A.4 WAVEMESH APPLIED TO MEDICAL IMAGES
+
Original Image pixels 1024x1024
WaveMesh Superpixel mesh
Filtered /cmppeesed images
# superpixels = 3843 256x256 1024x1021
Compression ratio 1:17:273
Figure 12: Left: Chest X-ray image of size 1024x 1024. Center: WaveMesh superpixel mesh. Right: Wavelet filtered chest X-ray image.
The chest X-ray image shown in Figure 12 from the NIH chest X-ray dataset (Wang et al. 2017). The image has 1024x 1024 pixels. These X-ray images are typically downsampled to 2 before using them for training CNN model. the center in Figure 12, we show the WaveMesh superpixel mesh obtained using our algorithm. It has 3843 multiscale superpixels. We note that 3843 < 256 10242. Infact, the compression ratio is 1 17: 273.
16


Reviews:
[{"review_id": "ixmuJV47EU", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Quality: The motivation of this paper is great, and the proposed WaveMesh is interesting. In general, it is a high-quality work. It will be better to supplement more experiments.\n\nClarity: The expression is clear, and especially, the figures are very exquisite, and helpful for understanding.\n\nOriginality: The proposed image-specific superpixeling algorithm WaveMesh is novel and try to circumvent the limitation of SLIC. However, the insufficient experiments affect its persuasiveness.\n\nSignificance: The WaveMesh could filter the unimportant information and focus on the important information which is a significant research. It may be applied to more image-based task, such as image segmentation. Consequently, I think this work has a certain significance, but it needs more experiments to evaluate.\n\nPros:\n1. It is a novel idea that non-uniformly downsamples the images and gets a multiscale superpixel representation.\n2. The figures are concise and precise, and they provide an intuitive understanding about the WaveMesh and WavePool.\n\nCons:\n1. The experiment is not sufficient. Take MNIST as example, this work only compares the WaveMesh and SLIC when the number of nodes is ~57 which is not convincing enough to conclude that the WaveMesh is effective. What is the acc about SLIC when the number of nodes is ~238. Besides, it seems that SLIC+SplineCNN achieves the best acc when the number of nodes is small. The experiments on Fashion-MNIST and CIFAR-10 have the same problem.\n2. WaveMesh is a superpixeling algorithm that is not coupled with classification algorithm, but all the experiments are performed on SplineCNN. What is the performance of WaveMesh with other GNN algorithms?\n3. How to evaluate the effectiveness of the image-dependent threshold T?\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "fop7RntH1M3", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper introduces a wavelet-based superpixel algorithm and a spatially heterogeneous pooling. More specifically, they introduce an algorithm to compress images in the pixel domain and it leads non-uniformly distributed and multiscale superpixels. Furthermore, they introduced a spatially heterogeneous pooling method tailored to the superpixel algorithm. Finally, they demonstrate the effectiveness of their method on MNIST, Fashion-MNIST, and CIFAR-10 datasets. \npros:\n1. The proposed method is simple and elegant. It is theoretical well-founded and easily implemented.\n2. The paper provides good initial results, showing that to some degree their method is generalizable.\ncons:\n1. In step 2 of wavemesh, the authors need to include the computational cost.\n2. In Table1, the results of the proposed method are not significant. More experiments are needed to demonstrate the gain of the proposed method.\n3. The authors should try other evaluation measurements, only acc is not enough.\n4. I believe more generalization and deeper analysis would be beneficial. \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "aVOg4c9fnfg", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper introduces a new approach to leveraging graph neural networks for image tasks. While prior work has been based on constructing graphs using a super-pixel map using methods like SLIC, that generate super-pixels of all roughly the same size, the proposed method generates a scale-adaptive partition. It also introduces a new pooling operator on this graph structure, and demonstrates that, together, these lead to improved performance when used with GNNs.\n\n- Overall, the method is novel and innovative, and well executed. The experimental evaluation is reasonably compelling (even if on relatively small datasets).\n\n- This paper adds to the body of literature exploring the use of GNNs as an alternative to CNNs for image tasks. The fact that past attempts have relied on a graph structure built at a specific scale likely handicap them when competing with CNNs, which are much better able to retain notion of space and scale across successive convolution and pooling operations.\n\n- The idea of converting a tagged/thresholded wavelet coefficients into a quad-tree structure is interesting, and indeed natural in retrospect.\n\n- I wonder though whether alternative approaches to building a multi-scale superpixel partition would perhaps not perform better. This isn't a negative for this paper (indeed, it could spur such research in the future), but perhaps something that's based on hierarchical clustering (on top of SLIC) might work better. One drawback of the wavelet approach is that the boundaries of the superpixels have to be regular and axis and aligned with the downsampling phase (have authors considered variants that are based on overcomplete wavelet transforms?).\n\n- There is an interesting effect in Fashion-MNIST where a larger number of superpixels leads to lower accuracy. This doesn't seem to have been discussed in the text.\n\n- Minor comment: For Fig 3, it would be good to have a brief legend or description in the caption itself. The caption currently points to Sec 3.3 (which is a little inconvenient when parsing the figure).\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "JkTCVP39ucb", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper present WaveMesh, a wavelet-based superpixeling algorithm and WavePool, a spatially heterogeneous pooling scheme.\n\nStrengths:\n\n\"A wavelet transform image coding technique with a quadtree structure\" (Banham and Sullivan 1992) and \"Geometric methods for wavelet-based image compression\" (Wakin et al. 2003) indicate that a wavelet-coefficient based quadtree decomposition of images is an efficient image-compression technique. This suggests that the goal of this paper is reasonable and the method can likely be motivated.\n\nWeakness:\n\nThe authors provide little theoretical justification for the use of wavelets as they are used in this paper.\n\nMoreover, the pooling method in this paper appears to be the same as that in \"Image Segmentation Based on Multiscale Fast Spectral Clustering\". In addition, the pooling method seems to be applicable to any quadtree-based method. A superpixel quadtree can be defined in many ways, and quadtrees are the basis of other superpixel methods like the \"A Hierarchical Data Structure for Picture Processing\" (Tanimoto and Pavlidis 1975), analyzed in \"Evaluation Framework of Superpixel Methods with a Global Regularity Measure.\"\n\nThere are two primary weaknesses: First, SLIC is a poor baseline and many other superpixel methods have been developed since 2010 - \"Superpixel Segmentation using Linear Spectral Clustering\" or \"Robust superpixels using color and contour features along linear path\" or even the recent \"Simple and fast image superpixels generation with color and boundary probability\".\nSecond, almost all of the gains of this method appear to come from the pooling method: In Tables 2 and 3, WaveMesh alone consistently underperforms SLIC.\n\nThird, these results appear to depend on the classification model. While the variance is low, alternative hyperparameters for the SplineCNN (or a different model) may have resulted in different performance. Given that this paper did not use any of the existing superpixel evaluation metrics, it is difficult to compare it to prior work. While of course the two papers cannot be compared directly as they are both currently under ICLR review, \"Probabilistic Numeric Convolutional Neural Networks,\" has similar performance on the 75 Superpixel MNIST dataset, using a third as many superpixels.\n\nDue to these weaknesses, this paper has neither the experimental analysis nor the theoretical grounding to be appropriate for ICLR at this time.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper received mixed reviews.  One reviewer is positive, while the remaining three reviewers are either negative or feel that the paper is below the threshold for acceptance.

The ideas presented in the paper are interesting and novel - this was acknowledged by three of the reviewers, even those who did not recommend acceptance.  The AC also recognizes the technical novelty presented.  However, as all the reviewers pointed out to varying degrees, the experimentation is problematic and the AC in agreement with this.  In particular, the heavy focus on improvement on top of SLIC makes it the applicability of the proposed approach highly limited and also not so convincing.

Recommendation for the paper is to reject and resubmit with improved experimentation.
Decision: Reject