Concentric Spherical GNN for 3D Representation Learning
Authors: James S Fox
Bo Zhao
Beatriz Gonzalez Del Rio
Siva Rajamanickam
Rampi Ramprasad
Le Song
Venue: ICLR 2022 Submitted (2022)
Keywords: spherical cnn
CNN
point cloud
graph convolution
rotation
equivariance
3D
molecular
volumetric

Abstract: Learning 3D representations of point clouds that generalize well to arbitrary orientations is a challenge of practical importance in problems ranging from computer vision to molecular modeling.
The proposed approach is based on a concentric spherical representation of 3D space, formed by nesting spatially-sampled spheres resulting from the highly regular icosahedral discretization.
We propose separate intra-sphere and inter-sphere convolutions over the resulting concentric spherical grid, which are combined into a convolutional framework for learning volumetric and rotationally equivariant representations over point clouds.
We demonstrate the effectiveness of our approach for 3D object classification, and towards resolving the electronic structure of atomistic systems.

Under review as a conference paper at ICLR 2022
CONCENTRIC SPHERICAL GNN FOR 3D REPRESENTATION LEARNING
A Anonymous authors
Paper under double-blind review
ABSTRACT
Learning 3D representations of point clouds that generalize well to arbitrary orien- tations challenge of practical importance in problems ranging from computer vision to molecular modeling. The proposed approach is based on concentric spherical representation of 3D space, formed by nesting spatially-sampled spheres resulting from the highly regular icosahedral discretization. We propose separate intra-sphere and inter-sphere convolutions over the resulting concentric spherical grid, which are combined into convolutional framework for learning volumetric and rotationally equivariant representations over point clouds. We demonstrate the effectiveness of our approach for 3D object classification, and towards resolving the electronic structure of atomistic systems.
INTRODUCTION
3D point cloud data appear in domains ranging from computer vision, geographic information systems, and molecular modeling. Learning suitable representations of point clouds for data-driven modeling is well-motivated by applications like automonous vehicles and molecular simulation. It is important and yet challenging to learn representations that generalize well to arbitrary orientations of point clouds, in an efficient and accurate manner. Spherical Convolutional Neural Networks (CNNs) were introduced in Cohen et al. 2018) and Esteves et al.) (2018) to address the challenge of rotations for spherical images, by defining rotationally equivariant convolutions harmonic-spp spaac.. Convolutions equivariant to transformations (such as translation) underpins recognizable aspects of 2D CNNs such as effective filter sharing and feature localization, key reason for their success Cohen & Welling (2016). However, Spherical CNNs operate over data projected onto the spherical manifold, which is lossy for general 3D point cloud data. It is desirable to learn features volumetrically, in order to accurately detect patterns in complex 3D shapes, or incorporate spatial relationships between atoms in describing atomic environments. We show that it is more expressive and general to operate over underlying spatial representation of concentric spheres, demonstrated experimentally.
We propose a new spatial structure consisting of multiple nested spheres, each discrctized by the icosahedral grid. This structure is used to record both the angular and radial distribution of input points, as volumetric information. The icosahedral grid produces a highly regular sampling of the sphere, which permits efficient convolutions scaling linearly with spherical resolution, explored in Jiang et al. (2019). Cohen et al.] (2019). and Defferrard et al. (2020). We then propose two types of convolution for learning over the concentrie spherical structure, by separately learning intra-sphere and inter-sphere features. We formulate intra-sphere convolution in terms of graph- based convolution over localized intra-sphere neighborhoods, and inter-sphere convolution as ID convolution over co-radial grid points. The resulting convolutions are both rotationally equivariant and scalable, each scaling (near) linearly with respect to the size of the grid. Finally, we incorporate the proposed convolutions into hierarchical and multi-resoluie architecture, CSGNN, for learning over concentric spherical feature maps, and demonstrate its applicability to point cloud data.
We first apply our approach to the problem of classifying rotated 3D objects sampled by point clouds, achieving oftte-of-the-ee performance on the task. We further apply our approach to a problem of molecular modeling, namely predicting the electronic density of states of materials. The density of states is a fundamental property of electronic structure, used in determining total energy contributions. Our approach is applied to learn localized descriptions of atomic environments, enabling more

---

Under review as a conference paper at ICLR 2022
accurate resolution of the band energy of carbon- materials compared to previous approaches. The implementation of our methods and experiments are publicly available
In summary, our primary contributions are as follows:
1. We propose a new volumetric representation as the basis for convolutional learning. This representation consists of multiple nested spheres, each discretized by the icosahedral grid. 2. We introduce a novel architecture for learning volumetric representation over concentric spheres by combining intra-sphere and inter-sphere convolutions. The proposed convolutions are rotationally equivariant, and also scale (near) linearly with grid resolution.
3. We demonstrate the applicability of our approach through experiments in 3D object classifi- cation, and resolving electronic structure of atomistic systems.
BACKGROUND AND RELATED WORK
The goal of learning representations of general 3D point cloud data has led to a diverse body of work for structured learning (Maturana & Scherer, 2015 Qi et al., 2017a by Wang al. 2019 Thomas iet al. 2019 Zhang et al. 2019b). However, the main problem shared by these methods is that they do not generalize well to general rotations of the data, which can lead catastrophic loss of performance when they are encountered. Augmenting training with rotated data helps bridge the gap somewhat, but a significant performance gap remains. The key missing piece in many earlier work their lack of rotationally equivariant model design. A model layer is equivariant to rotation if it commutes with rotation. In other words, feeding a rotated input to the model layer is same as feeding the original input to the layer and rotating its output. A rotationally invariant layer is special case of an equivariant layer where rotation does not affect the output space of the layer. Several models have been proposed which exclusively use rotationally invariant layers (Schütt et al. 2017 Gilmer et al.
2017
Schütt et al.) 2017: Chen et al. 2019 Zhang et al. 2019a Poulenard et al. 2019 Kim et al,
2020
However, using invariant layers through the entirety of model is unnecessarily restrictive, as
important information about the underlying spatial structure of the data may be lost. Our work focuses on designing rotationally equivariant layers as primary building blocks, while invariant layer(s) can be used before final output to achieve overall invariance of the model. This design has already seen extension to other structures and learning strategies relevant to point cloud representation. Thomas let al.] (2018) proposed rotation equivariant point-wise convolutions for 3D graphs, but their approach has difficulty scaling to point clouds beyond the scale of small molecules. Equivariant design has also seen extension to spherical images through Spherical CNNs (Cohen et al. 2018 Esteves et al. 2018
Tiang et al. 2019 Cohen et al. 2019 Rao et al.) 2019 Defferrard et al. 2020 Yang al. 2020).
However, they are not well-suited for direct application to point clouds. Their key limitation loss of information in constraining spatial representation from 3D domain to a 2D (spherical) manifold.
Our work overcomes this limitation by proposing convolutional learning over concentric spheres, achieving rotationally equivariant 3D feature learning with scalable convolutions.
ARCHITECTURE DESIGN
The primary goal of our proposed approach is to learn volumetric representations of 3D point clouds in rotationally equivariant and also scalable manner. To achieve this goal, we propose concentric spheres at different radii, each discretized by the icosahedral grid. The proposed construction naturally organizes 3D data by angular and radial distribution, where the resolution of each component can be controlled independently. We use the icosahedral grid as it results in a highly regular sampling of the sphere. The former permits efficient use of spatial resolution, and the later results in design of efficient and
convolutions. We propose using two separate convolutions together to learn volumetric features over concentric spheres: (1) graph-based convolution to incorporate information within spheres, and (2) radial convolutions to separately incorporate information between spheres. The proposed convolutions are extended to different spatial scales via pooling based on regular properties of the icosahedral grid, resulting in the proposed hierarchical convolutional architecture Fig. [
We explain each component of our model in more detail in subsequent sections, and pooling in A.5 of Appendix.
2

---

Under review as a conference paper at ICLR 2022
Graph
convolation
Radial convolutio
Radius
Guobal
Pool
Figure 1: Example architecture with three concentric spheres. Graph convolutions are followed by radial convolutions, at each level of spherical resolution. Radial convolution (in this example) has spatial window of three co-radial vertices, with padding applied maintain radial dimensions across convolutions. Each arrow indicates vertex neighborhood pooling and downsampling, after which convolutions proceed with new filters coarser spatial resolution. Global pooling applied obtain final feature representation.
(d)
Figure 2: Illustrative example of converting point cloud input features. (a) shows an example point cloud (black points) contained within bounding sphere. (b) The spherical volume is partitioned into concentric spheres, co-radially. (c) Each point has contribution to vertices in local neighborhood (gray circles), resulting in (d) single channel feature per vertex (gray square). (e) Co-radial vertices are further grouped, resulting smaller subset concentric spheres with multi-channel inputs. In this example, grouping results in 3 concentric spheres where vertices have 2 input channels each.
3.1 CONCENTRIC SPHERICAL DISCRETIZATION AND POINT CLOUD MAPPING
In this section we explain in detail method of volumetric discretization by concentric spheres. We further present our approach to converting arbitrary point cloud data to initial feature channels over this spatial structure.
Concentric Icosahedral Spheres. The initial icosahedron has 12 vertices forming 20 equilateral triangular faces. To increase grid resolution, each face can be sub-divided, with resolution scaling as | 10 4 + (1 is target discretization level). See Fig. for an illustration of this process. We implement concentric spheres by stacking identical icosahedral grids to form the radial dimension, shown in Fig. 3b Assuming normalization unit radius, uniformly assign concentric spheres at radii [ R2 .... IJ. Assuming single-channel feature map, the resulting grid is the matrix
H E R R where each vertex is indexed by the sphere it belongs to, and its position on the sphere. The resulting volumetric representation has several noteworthy properties. First, the icosahedral discretization results in a highly regular spatial sampling within each sphere. This allows more efficient use of spatial resolution compared to polar grids used in earlier works (Cohen al., 2018) Esteves al. 2018 You et al. 2020), which have have resolution bias towards the polar regions. Second, spatial resolution is not uniform across different spheres, as the proposed construction results in higher sampling density closer to the center. However, this non-uniformity difference is largely accounted for as function of the radius the sphere, and does not inhibit the design of efficient and rotationally equivariant convolutions.
3

---

Under review as a conference paper at ICLR 2022
Radius
Rediu
(a)
(b)
Figure 3: (a) The icosahedral grid is formed by vertices of equilateral triangles (left), which can be recursively sub-divided to form higher resolution grid (middle). This also defines natural vertex neighborhood and hierarchy for pooling and downsampling, where yellow highlighted vertices (right) are involved in pooling. (b) Two spherical grids are stacked, corresponding consecutive concentric spheres. Graph convolution involves vertices within the same sphere (convolution neighborhood highlighted orange). Conversely, radial convolution involves co-radial vertices between the two spheres (green dotted line).
Point Cloud to Concentric Spheres. We now consider the problem of describing point cloud P N3 in terms of concentric spherical feature map E RxxxV]XCC where C' number of channels. While the concentric grid representation is defined discretely, the space point positions are continuous, which we would like to capture. To do so, we summarize the contribution of points using the Gaussian radial basis function (RBF):
N Σ H( - 12)
f(x)
(1)
N is the number of data points, and the function ø = exp(->r²) is parameterized by the bandwidth y. In practice we limit computation to local neighborhood (instead of considering all points) and choose y accordingly. We refer to Fig, for illustrative example of the conversion, and Sec. A.6 for implementation details as applied in experiments.
We determine the input channels C' by grouping concentric spheres and concatenating features of co-radial vertices, also shown in Fig. 2 Suppose converting the point cloud results in a single- channel feature tensor E R x where R' is the initial number of spheres. Grouping the
features from co-radial vertices across R groups results in feature tensor H E RXXVIX $ where R is the number of spheres represented spatially, and πt' the number of spheres represented via input channels. The proposed grouping mechanism gives flexibility to significantly increase radial resolution without correspondingly increasing the size the spatial grid, which persists across convolutions. This enables balancing between computational efficiency and the representational boost having additional spheres.
3.2 CONCENTRIC SPHERICAL CONVOLUTIONS
In this section we present our implementation of rotationally equivariant intra-sphere and inter-sphere convolutions for learning volumetric features. Proof of equivariance is provided in Sec. A.I
Intra-sphere convolutions. The objective for intra-sphere convolutions is to learn localized features within each sphere. in rotationally equivariant fashion. We use graph convolutional filters for this objective, localized graph convolutions on icosahedral grids are rotationally equivariant with respect to the icosahedral rotation group. This motivates our construction of the undirected graph GLD) (V()) E() from level / icosahedron ( Vertices of the vertex set (() correspond one- to-one with vertices of (()) projected to unit sphere, and E() is the set of edges corresponding to face edges of the icosahedron. We refer Fig. (3b for an example of graph connectivity
each sphere. The resulting graph is highly regular, all vertices are degree six beyond 7(0), All edges within each sphere are also approximately equidistant (Wang & Lee 2011). We use the graph convolutional operator from Kipf & Welling (2017), but omit the degree-based normalization due the regularity of our graph construction. We introduce additional notation to define graph convolution in the context of the proposed discretization. Let E R Rx [V|CC denote a C' channel tensor of features, and Z x be learnable weights, where C and F are input and output channel sizes.
4

---

Under review as a conference paper at ICLR 2022
We to (u) denote neighbors of vertex in graph G. We also introduce subscript to indicate convolutional layer number, i E [0, R to index the radial dimension, and u E [0. [V] to index the vertices. The layer + 1 intra-sphere convolution output for vertex u of sphere is then given by Eq. 2 where a a nonlinear activation function:
(+++)
Σ H 2 (EN(u)
Inter-sphere convolutions. We introduce radial convolations to learn features between spheres. To do so, we propose applying ID convolution over co-radial vertices. In treating co-radial vertices as sequence, ID convolution is intended learn localized features across spheres while distinguishing their relative positions. Importantly, the proposed convolution is rotationally equivariant, as co-radial vertices transform identically under rotation. We refer Fig. Bb illustration. We introduce some additional notation to describe radial convolutions: let K be the size of the ID convolution kernel window. We pad inputs in the radial dimension such that the number of spheres R is maintained spatially across convolutions. Let W E xxxx be a tensor of shared parameters, where C and F input and output channel sizes. The layer radial convolution output for vertex u of sphere i is:
H(++1 o
Σ
W i+k,u
(3)
1+141
3.3 COMPLEXITY ANALYSIS
The neighborhood size of both graph and radial convolutions constant, and their filter parameters due to weight sharing. Therefore the overall complexity of both the graph convolution and radial convolution is Ó(Rx [V]). [). linear with respect to the total discretization size. This introduces an additional factor R of computational and memory cost compared to the ((V)) complexity of some spherical CNNs, corresponding to stacking multiple spherical discretizations. However, as we show in later experiments, the number of spheres can be small in practice while providing considerable performance benefit. We further provide empirical runtime analysis with respect to in Sec. A.4 Appendix.
4 EXPERIMENTS
We demonstrate the effectiveness of our approach for 3D object classification in Sec. 4.1 and applied
to resolving fundamental properties of electronic structure in materials in Sec, 4.2 We further study
the main components to the performance of our proposed approach in Sec. 4.3
4.1 POINT CLOUD CLASSIFICATION
We consider the ModelNet40 3D shape classification task, with 12308 shapes and 40 classes. We use the pre-processed point clouds from Qi et al. (2017a). and 1024 points per point cloud. In total, 9840 shapes are used for training and 2468 for testing.
Architecture and Hyperparameters. See Fig. for overview of model layers and components. Batch normalization and ReLU activation is applied after each convolution and hidden layer. A residual connection is also added between each graph convolution layer when the number of input and output channels match. The point cloud is converted to initial concentric spherical features following procedure detailed in A.6 We follow prior work in augmenting training with random translation, re-scaling, and positional jitter of input point clouds. We trained separate models when training with z-aligned vs. SO3 rotations, and refer to Sec. A.2 for hyperparameter and additional experiment details.
Results. For experimental evaluation, we consider two types rotations in training and/or testing, following convention from earlier work: 2-axis aligned rotations and arbitrary rotations (SO3). The SO3 case is most challenging, as it indicates objects can be in any rotational orientation. We compare with related work organized into three categories, based their strategy for handling
5

---

Under review as a conference paper at ICLR 2022
L
Max
128
Mex
Max
256
512
Max
512
1024
Max
1084
CraphConv
RadialConv
Vertex neighbor pooling
Concalenate
Global pooling
Fully-comected
Downsemple
Figure 4: Architecture for ModelNet40 classification. Number of output channels are shown for each layer, where applicable. convolution uses kernel size of 3 for spatial size (co-radial vertices).
initial level of discretization of icosahedral spheres. Radial
Method
Strategy
Params
z/z.
z/SO3
SO3/SO3
PointNet QQ et al.
2017 7
Augmentation
3.5M
87.5
22.9
84.9
DGCNN [aan et al
2019
Augmentation
1.8M
90.7
35.5
89.0
ShelINet (hhang al.
2019b)
Augmentation
470K
89.2
22.9
84.8
KPConv Thomas et al.
2019
Augmentation
6.1M
90.0
27.5
85.0
SPHNet (Poulenard
2019
Invariance
2.9M
86.5
85.6
87.0
RIConv Zhhng et al.
2019a
Invariance
0.7M
87.0
87.0
87.2
RI-GCN Kim et
2020
Invariance
4.4M
89.2
89.3
89.1
SFCNN Rao et al.
2019)
Equivariance
9.2M
90.8
84.2
89.6
PRIN
You et al."
2020)
Equivariance
1.7M
76.5
81.9
81.0
Ours (CSGNN)
Equivariance
4.0M
91.0
88.3
90.1
Table 1: ModelNet40 object classification overall accuracy, considering two types of rotations: z-axis aligned, and more general SO3 rotations. For example, SO3/ SO3 indicates training and testing with arbitrary rotations of input data. Strategy refers to how rotations are handled. Number of parameters is in millions.
rotations. Resulted are presented in Table [ where we ran each baseline. Our method achieves state of the art performance in z/z and SO3/SO3 settings when training and testing from same space of rotation, and achieves competitive even when generalizing to unseen type of rotations in the z/SO3 setting. Methods without rotationally equivariant design (Wang et al.) 2019; Zhang et al. 2019b Thomas al., 2019) depend primarily on augmenting training with rotations, allowing it to achieve competitive performance easier rotations, but performing very poorly when generalizing to unseen SO3 rotations. Methods like RI-GCN (Kim et al. 2020) rely on designing invariant features at input to achieve rotational invariance, and so performs well when generalizing to unseen rotations. However, their performance is not best in z/z and SO3/SO3 settings, suggesting some loss of expressiveness from enforcing strict rotational invariance throughout the model. SFCNN uses rotationally equivariant convolutions based on icosahedral discretization, similar to our work. Unlike single-sphere spatial representation, and
approach, SFCNN is based module Qi et af. (2017a) to learn point cloud projection. While competitive to our approach z/z and S03/SO3 settings, our approach generalizes much better to unseen rotations.
PointNet-like
4.2 RESOLVING ELECTRONIC STRUCTURE OF MATERIALS
Accurate molecular dynamics simulation from quantum-mechanical principles is critical to many applications, such the design of advanced materials the study of materials' properties under extreme conditions. However, accurately scaling simulations to systems beyond hundreds of atoms is problem of primary concern. The main bottleneck appears when solving the quantum mechanical questions, which provides properties describing the electronic structure, such as the electronic density of states (DOS) and the electron density. Recent MI efforts have tried overcome this issue by effectively predicting the electronic structure (output) from the atomic structure (input) (Chandrasekaran et al. 2019: Fabrizio et al.) 2019 del Rio al., 2020 Kamal et al. 2020 Ellis al. 2021). Here aim to effectively and accurately predict the DOS, which describes the energy distribution of the electrons within an atomic snapshot. The DOS a physical quantity that is invariant to rotations of the system. From the DOS, the band energy, essential component of the total energy of the system, can calculated. Due to the atomic nature of the problem, we propose learning atom-centered descriptors of local environments end-to-end, enabling data-driven and more

---

Under review as a conference paper at ICLR 2022
Graprin
G-4
(.08.))
Graphono
0.20
c oi
Figure 5: Illustrations of each type of carbon structure present in the dataset. They fall under three geometric classes: sheet-like (graphene, graphite), cylindrical (C(6,4), C(9.9), C(8,0)), and spherical (C20, C40, C60).
flexible representations compared to hand-crafted descriptors of prior work del Rio et al. (2020); ;
Ben Mahmoud al. (2020). We further introduce and make publicly available a dataset consisting of geometrically diverse structures of graphene, and show that the use of CSGNN lowers overall error
for calculating band energy, and increases the number of structures resolved
chemical accuracy.
Dataset. The datasets consists of eight different types of graphene allotropes: graphene sheet, graphite, three different fullerenes, and three single-walled nanotubes-see see Fig. 5 There are 200 atomic snapshots per structure, generated from snapshots of DFT molecular dynamics simulations run using VASP (Kresse Furthmüller 1996a b). The number of atoms per snapshot range from 20 up to 152. After alignment with respect to the vaccum energy used as global reference, the resulting DOS curve is binned into 310 windows of 0. eV each, from -30 to eV. We refer to del Rio et al. (2020) for more details on the data generation and preparation process. Using the same 80:20 split for each type of graphene structure results 1280 total files for training and 320 files for validation separate test set of size 160 is used with 20 snapshots per structure.
Problem Formulation. Each input is a snapshot of positions of carbon atoms, represented by coordinates X E Nx3 and bounding lattice vectors C E R The atoms are located inside unit cell, with periodic boundary conditions. The prediction target is the DOS, represented by a fixed dimension vector y R310 From the DOS important downstream property of interest can be computed, the band energy:
Fr
Eband
DOS(e)ede
where is the energy and Ep is the Fermi level. In practice, the integral is evaluated as a cumulative sum over discrete and bounded domain. The integration has an upper bound of Ep, a physical limit representing the highest energy of the bound electrons. Er is calculated as the energy at which the cumulative integral of the DOS curve equals the total number of electrons in the system. Since this limit a function of the DOS integral and electron number, we introduce an additional prediction target in terms of the cumulative DOS (FDOS). Including the FDOS during training enables better resolution Ep and lowers band energy error. The resulting objective function to minimize is:
L * Dos y. 9) + (1 a) LLDOS (y. ÿ)
(5)
where ý is the predicted DOS and controls the relative weighting between DOS and FDOS losses, which are both in terms of mean squared error.
Model. To model the total DOS, we use decomposed approach to predict the contribution of each atom to the overall DOS. We further assume that each atom's contribution is function of its local atomic atomic environment, an observation often exploited in electronic structure methods (Behler 2016). The closer the neighboring atoms are to the target, the stronger the effect they have on the target's properties. To account for this effect, fixed cutoff radius of angstroms used in experiments, eliminating the effect of neighbors that are further away. Our approach is then applied learn suitable descriptor each local environment for mapping to atom-wise DOS contributions, which are then summed to obtain the overall DOS. This workflow is further illustrated in Fig, 6 Each atom serves as the center for concentric spherical representation, and the atomic neighboring environment is point cloud. To convert a point cloud to concentric spherical feature map, each point assigned a contribution to its nearest vertex. The contribution is determined as the inverse of the point's distance from center, based the aforementioned neighbor effect.
All concentric spherical feature maps are combined channel-wise into a single group. resulting in single spatial sphere R 1 with C 32 channels. This eliminates effectively eliminates inter-sphere
7

---

Under review as a conference paper at ICLR 2022
CSGNN
C
D05
(5)
DOS/ateee
Energy (eV)
Carh
Figure 6: From left to right: each atom's ' local atomic environment is point cloud, and the input for our proposed model. CSGNN (shared across inputs) learns a descriptor output, which is mapped through additional dense layers and ID convolution to predict atom-wise DOS contribution. All atoms' contributions are summed obtaín total DOS.
1
Avg
Aug
128
128
256
256
AVE
256
Avg
512
512
512
310
330
Sum
GraphConw
1x1 Conv
Vertex neighbor pooling
Global vortox
poaling
connecteed
Fully
Conv1D
Atom-wsc pooling
Downsample
Figure 7: Architecture for DOS prediction. Number of output channels are shown for each layer, where applicable. L = 3 is initial icosahedral spherical resolution. Ix1 convolution is applied within channels, without spatial component. 1D convolution is applied to smooth the predicted DOS curve, represented by output energy bins.
convolutions, which instead become 1x1 convolutions in feature space. The main consideration here is computational and memory efficiency, as capturing co-radial information via input channels does require adding additional spatial dimensions. We note that this version of proposed architecture is already sufficient to obtain strong results in this problem. Fig, 7 presents the layers of the model architecture, and Sec. A.3 provides training and hyperparameter details.
Results. We present our results in Table AGNI (Botu & Ramprasad 2015 Botu et al. 2016) is
hand-crafted descriptor method for extracting rotationally invariant features atomic environments applied to this problem and dataset in del Rio et al.) (2020). SchNet (Schütt et al.) 2017) is neural message passing model for learning atom-centered features end-to-end, and is considered a strong baseline for many atomistic ML problems. Our rotationally equivariant approach achieves the lowest mean error in resolving band energy on the test set, reducing overall error by 24% relative to previous best. Our approach also demonstrates the ability of Iearned descriptors to improve over the performance hand-crafie craftd-rfeeddd descriptors for this problem. We also included a single-sphere version of model (C
for ablation, which did perform well the multi-sphere version (C 32). Since the dataset is composed of different types of Graphene geometries, we further group test error by each type of structure. Our approach achieves the lowest mean error six of eight structures. Furthermore, our approach achieves chemical accuracy on more structures-chemica accuracy (0.043 eV/atom) a widely adopted accuracy threshold computational chemistry. We achieve chemical accuracy on seven structures, compared to six for SchNet and five for AGNI, out of eight.
4.3 ABLATION STUDY
In this section further analyze some key components to the performance of our model, based on the model architecture for ModelNet40 classification task. We refer to Table all results.
Number of concentric spheres. We study the impact of concentric spheres model performance.
One option is to maintain single sphere spatially (R = 1), and group all radial information from concentric spheres into input channels (C). Inter-sphere convolution is not applicable in this case, and only intra-sphere convolution is used. While increasing in this manner effective, adding concentric spheres spatially is needed to achieve best performance,
Table 3 The number of
8

---

Under review as a conference paper at ICLR 2022
Method
Type
Parameters
Mean Error (e (eVatomm
AGNI (Botu & Ramprasad 2015)
Hand-crafted
392K
0,033
SchNet (Schütt et al. 2017)
Learndd
976K
0.035
CSGNN -, CSGNN (R 1,
Learned
1.06M
0,036
Learned
1.06M
0.025
(a) Overall error in resolving band energy for the test set. The mean absolute error per sample is normalized by number of atoms. is number of spatial concentric spheres, is number of concentric spheres input channels.
Model
Graphene
Graphite
C20
C40
C60
C(6,4)
C(9,9)
C(8.0)
AGNI
0.021
0.053
0.052
0.030
0.010
0.046
0.027
0,026
SchNet
0.042
0.045
0.065
0.030
0.022
0.022
0.033
0.019
Ours (CSGNN)
0.013
0.039
0.051
0.033
0.017
0.020
0.014
0.015
(b) Band energy mean absolute error of predictions for each type of structure. C 32 version of our model used here.
Table 2: Comparison of descriptors for resolving band energy from predicted density of states. Table 2a shows overall error over test dataset. Table 26) further categorizes energy prediction error by structure. CSGNN achieves lowest overall error, as well as lowest error in six out of eight structures.
Setting
SO3/SO3
Setting
SO3/SO3
Setting
SO3/SO3
Spheres,
Spheres, spatial
Radial convolution
channels only
kernel size
= 1,
84.1
R 4, 4,C
88.9
R 16, KRC =
86.3
1,
87.6
R 8, C
89.6
R 16, KRC
90.0
R = C,C 16
87.9
R = 16, C' 8
90.0
R 16, KRC = 5
89.9
R C,C
87.7
32, C'
90.1
Table 3: Ablation results in terms of SO3/SO3 accuracy Modelnet40 dataset. K RRC is radial convolution kernel size. C' refers to concentric spheres grouped via input channels, while refers to number of spheres represented spatially.
trainable parameters is same when increasing the spatial dimension, and so performance differences can be attributed to increased representational power via concentric spheres.
Radial convolution. We further provide justification for using the proposed radial convolution for learning features between spheres. The radial kernel size refers to the number of co-radial vertices considered in the convolution window. We set K 1, which operates over only the feature dimension and ignores the spatial dimension (analogous to Ixl convolution for 2D images). This gave significantly worse performance than K 3, confirming that non-trivial radial convolution is necessary for performance. We did not observe any benefit to increasing kernel size beyond KRC 3.
5 CONCLUSION
To address the problem of learning rotationally robust representations of point cloud data, we propose a new volumetric feature learning approach. We propose a new volumetric structure based on nested spheres formed by the icosahedral discretization. We design intra-sphere and inter-sphere convolutions for learning over the concentric spheres, combined into multi-resolution convolutional architecture. The proposed approach both rotationally equivariant and efficient, scaling linearly with grid resolution. Our method achieves state-of-the-an performance in benchmarks for classifying arbitrarily rotated 3D objects. Our method is also effective in molecular environment description, and applied towards more accurately resolving electronic structure of materials. An avenue for future work is extending our approach to variable icosahedral resolution across concentric spheres, order to balance the resolution requirements of inner vs. outer spheres.

---

Under review as a conference paper at ICLR 2022
REPRODUCIBILITY STATEMENT
To aid in the reproducibility of our work, we have taken the following measures:
We have made our model and experiment implementations publicly available at https//gghuub.coommoonyyy 10521/CSGNN This also includes access to the datasets used in experiments, from the repository.
We have also provided pre-trained versions of our models used in reporting best results, in the repository.
The repository includes instructions for setting up the environment and dependencies needed to run our implementation.
To facilitate reproducing results from training, have included relevant hyperparameters in Sec. [A.2 and AA3) of the Appendix. We have also set default parameters training scripts match those in the paper, much as possible.
REFERENCES
Jörg Behler. Perspective: Machine learning potentials for atomistic simulations. The Journal of Chemical Physics, 145(17):170011, 2016. doi: 10.1063/1.4966192. htthstts::/// http::://dooooorg/..
10.1063/11 4966192
Chiheb Ben Mahmoud, Andrea Anelli, Gábor Csányi, and Michele Ceriotti. Learning the electronic density of states in condensed matter. Physical Review B, 102(23):235130, Dec 2020. ISSN 2469-9950, 2469-9969. doi: 10.1103/hhs.Rev..102... 102.233113.
V. Botu and R. Ramprasad. Learning scheme predict atomic forces and accelerate materials simulations, Phys. Rev. B, 92:094306, Sep 2015, doi: 10.1 N0.110//PyysReeB.92..0... https://linke aps /ttp://///11..1111////eeeeeee
URL
094306
Venkatesh Botu, Rohit Batra, James Chapman, and Rampi Ramprasad. Machine learning force fields: Construction, validation, and outlook. Journal of Physical Chemistry C, 121:511--51 2016.
Anand Chandrasekaran, Deepak Kamal, Rohit Batra, Chiho Kim, Lihua Chen, and Rampi Ramprasad. Solving the electronic structure problem with machine learning, npj Computational Materials, 5 (1):1-7, 2019.
Chao Chen, Guanbin Li, Ruijia Xu, Tianshui Chen, Meng Wang, and Liang Lin. Clusternet: Deep hierarchical cluster network with rigorously rotation-invariant representation for point cloud analysis. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pp. 4994-5002, 2019.
Taco Cohen and Max Welling, Group equivariant convolutional networks. In Maria Florina Balcan and Kilian Q Weinberger (eds.), Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, pp. New York, USA, 20-22 Jun 2016. PMLR. URL http: http::::/ppooceingggs ml r. press/
New York,
cohencl6. htm
Taco Cohen, Maurice Weiler, Berkay Kicanaoglu, and Max Welling, Gauge equivariant convolutional networks and the icosahedral CNN. In International Conference on Machine Learning, pp. 1321-1330, 2019.
Taco S Cohen, Mario Geiger, Jonas Köhler, and Max Welling. Spherical CNNs. International Conference Learning Representations, 2018.
Michaël Defferrard, Martino Milani, Frédérick Gusset, and Nathanaël Perraudin. Deepsphere: graph-based spherical cnn. In International Conference on Learning Representations, 2020. URL
htttps://pperee// net net ott.://eeeee...aee.eeoooooonaaaa
10

---

Under review as a conference paper at ICLR 2022
Beatriz G. del Rio, Christopher Kuenneth, Huan Doan Tran, and Rampi Ramprasad. An efficient deep learning scheme to predict the electronic structure of materials and molecules: The example of graphene-derive allotropes. The Journal of Physical Chemistry A, (24(44))496695000 2020. doi: 10. 10,1021 http///ass.pp..////... PMID: 33138367.
URR.https://ddo /ttt:::://ooi.or//10000 wrg. TC21 /acs. jpca. 0c07458
J. A. Ellis, L. Fiedler, G. A. Popoola, N. A. Modine, J. A. Stephens, A. P. Thompson. A. Cangi, and S. Rajamanickam. Accelerating finite-temperature kohn-sham density functional theory with deep neural networks. Phys. Rev. B, 104:035120, Jul 2021. doi: 10. 10.1000/Pyss 104.035120. URL https://link aps htt.:ii.........//....oo.....
110. 2w1033//hyyyeBB..ee 104. 035120
Carlos Esteves, Christine Allen-Blanchette Ameesh Makadia, and Kostas Danilliiis Learning SO(3) equivariant representations with spherical CNNs. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 52-68, 2018.
Alberto Fabrizio, Andrea Grisañi, Benjamin Meyer, Michele Ceriotti, and Clemence Corminboeuf. Electron Density Learning of Non-Covalen Systems. Chem. Sci., 10(41))9424 9432, 2019. ISSN 20416539. doi: 10. (0.1039//9sc022666
Justin Gilmer, Samuel Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference Machine Learning Volume 70, ICML'17, pp. 1263-1272. JMLR.org. 2017.
Chiyu Max Jiang, Jingwei Huang, Karthik Kashinath, Prabhat, Philip Marcus, and Matthias Niessner. Spherical CNNs on unstructured grids. In International Conference on Learning Representations, 2019. URL httt:s::
Deepak Kamal, Anand Chandrasekaran, Rohit Batra, and Rampi Ramprasad. A charge density prediction model for hydrocarbons using deep neural networks. Machine Learning: Science and Technology, 1(2)0025033, mar 2020.
Seohyun Kim, Jae Yoo Park, and Bohyung Han. Rotation-invariant local-to-glon too-ooooglllllooo representation learning for 3d point cloud. In H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 8174-8185. Curran Associates, Inc., 2020.
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks International Conference on Learning Representations, 2017.
G. Kresse and J. Furthmüller. Efficient Iterative Schemes for Ab Initio Total-Enerya Calculations using a Plane-Wart Basis Set. Phys. Rev. B, 54(54):11169 86, 1996a.
G. Kresse and Furthmüller: Efficiency of Ab-Initio Total Energy Calculations for Metals and Semiconductors using Plane-Wav Basis Set. J. Comput. Mater. Sci., 6(6):15-50, 1996b.
Daniel Maturana and Sehastian Scherer. Voxnet: 3d convolutional neural network for real-time object recognition. In 2015 JEEE/SI International Conference on Intelligent Robots and Systems (IROS), pp. 922-928. IEEE, Sep 2015. ISBN 978-1-4799-9994-1 doi: 10.1109/ 2015. 7353481. URL http::/t/// httt:://ieeexppleee
ieee httpe///o///cooc/o///3333322
Adrien Poulenard, Marie-Julie Rakotosaona, Yann Ponty, and Maks Ovsjanikov. Effective rotation- invariant point enn with spherical harmonies kernels. In 2019 International Conference on 3D Vision (3DV), pp. 47-56. IEBE, 2019.
Charles Qi, Hao Su, Kaichun Mo, and Leonidas Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference compuler vision and patlern recognition, pp. 652-660, 2017a
Charles Ruizhongtai Qi, Li Yi, Hao Su. and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In Advances in neural information processing systems, pp. 5099-5108, 2017b.
11

---

Under review as a conference paper at ICLR 2022
Yongming Rao, Jiwen Lu, and Jie Zhou. Spherical fractal convolutional neural networks for point cloud recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 452-460, 2019.
K. Т. Schütt, P.J. Kindermans, H. E. Sauceda, S. Chmiela, A. Tkatchenko, and K.-R. Müller. Schnet: A continuous-filter convolutional neural network for modeling quantum interactions. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS' 17, pp. 992-1002, Red Hook, NY, USA, 2017. Curran Associates Inc. ISBN 9781510860964.
Kristof Schütt, Pieter-Jan Kindermans, Huziel Enoc Sauceda Felix, Stefan Chmiela, Alexandre Tkatchenko, and Klaus-Robert Müller. Schnet: continuous- convolutional neural network for modeling quantum interactions. In Advances in neural information processing systems, pp. 991-1001, 2017.
Kristof T. Schütt, Farhad Arbabzadah, Stefan Chmiela, Klaus R. Müller, and Alexandre Tkatchenko. Quantun-chemical insights from deep tensor neural networks. Nature Communications, 8(1): 13890, Apr 2017. ISSN 2041-1723. doi: 10. 1038///ommss13888
Hugues Thomas, Charles R Qi, Gean-Emmanue Deschaud, Beatriz Marcotegui, François Goulette, and Leonidas Guibas. Kpconv: Flexible and deformable convolution for point clouds. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6411-6442 2019.
Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation-and translation-equivariann neural networks for 3d point clouds. arXiv preprint arXiv: 180vv088000002/11 2018.
Ning Wang and Jin-Luen Lee. Geometric properties of the two-sphere. SIAM Journal on Scientific Computing, 33(5):2536-25 2559, 2011.
hexagonal grid on the
Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E. Sarma, Michael M. Bronstein, and Justin M. Solomon. Dynamic graph cnn for learning point clouds. ACM Trans. Graph. 38(5), October 2019. ISSN 0730-0301. doi: 10. 145/3326362. URL https:://///. //dol. org/10 1145/3326362
Qin Yang, Chenglin Li, Wenrui Dai, Junni Zou, Guo-Jun Qi, and Hongkai Xiong. Rotation equivariant graph convolutional network for spherical image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4303-4312, 2020.
Yang You, Yujing Lou, Qi Liu, Yu-Wing Tai, Weiming Wang. Lizhuang Ma, and Cewu Lu. Pointwise rotation-invariant network with adaptive sampling and 3D spherical voxel convolution. The AAAI Conference Artificial Intelligence, 2020.
Zhiyuan Zhang, Binh-Son Hua, David W Rosen, and Sai-Kit Yeung. Rotation invariant convolutions for 3d point clouds deep learning. In 2019 International Conference on 3D Vision (3DV), pp. 204-213. IEEE, 2019a.
Zhiyuan Zhang, Binh-Son Hua, and Sai-Kit Yeung. Shellnet: Efficient point cloud convolutional neural networks using concentric shells statistics. In 2019 IEEE/CVF International Confer- ence on Computer Vision (ICCV), pp. 1607-1616. IEEE, Oct 2019b. ISBN 978- 978-1-72814-803-- doi: 191100C222222009000006 URL. https: htt::://eeeeepppe /ttpss://eeexxploree eee org/ wwweeorg//oocccee documett.
901 90109967
12

---

Under review as a conference paper at ICLR 2022
A
APPENDIX
A.I EQUIVARIANCE OF INTRA-SPHIERS AND INTER-SPHERE CONVOLUTIONS
We define the feature map h: S R IR the function which maps positions the spherical volume to scalar features (without loss of generality). A rotation R on the feature map is then defined by the following operator:
(x) a(R
(6)
convolutional layer & is equivariant to the rotation group if commutes with all rotations:
(x) (Tx((A))(a)))
e
In other words, applying convolution over the rotated feature map is the same as applying rotation over the result of convolution. In this work focus equivariance the icosahedral rotation group I, a subgroup of 50(3) containing 60 discrete rotational symmetries.
We start with intra-sphere convolution implemented by graph convolution within each sphere:
Φh(x,,)) = o
Σ h(x,,)))
where æij is the position corresponding a vertex in the concentric spherical grid, indexed by radial and spherical dimension. Additionally, 0 trainable parameter, N(x;;) denotes positions of vertices in the neighborhood of the vertex at æi and nonlinearity function. Equivariance of the proposed graph convolution layer is shown as follows:
$ ([nn)((
oh(R- x1.j)
(9)
=
E h((.)),
āiik R R-
(10)
(
= [[L(()h)((a..))
(11)
The second equality follows from rotation R E I being an isometric transformation that maps the icosahedral sphere onto itself. This means that each vertex position Biy of the rotated feature map corresponds to an vertex unique position x= in the original feature map, and that vertex neighborhoods are also preserved. The final equality follows from applying Eq. and Eq. g
Next, we show that the intra-sphere convolution layer is also rotationally equivariant. Intra-sphere
convolution defined via ID convolution over co-radial vertices:
sh(x),)) = ol ((
(12)
where K is the size of the radial kernel and 3 denotes trainable parameter. We show that the convolution commutes with rotation:
& ([Rhi)(as.))) = (R(R (R)))
(13)
141
= =
h(ā h(2:+H,))
Titky = R- R-
(14)
(15)
The second equality follows trivially from the fact that co-radial vertices remain co-radial after shared rotation, thereby preserving neighborhood for convolution. Equivariance of the intra-sphere and inter-sphere convolutional layers ensures that compositions of these layers are also equivariant to rotation.
13

---

Under review as a conference paper at ICLR 2022
Concentric spheres (R)
24
32
Training (seconds/epoch)
51
83
104
150
204
Inference (millisecond//hatch)
70
75
97
119
155
Table 4: Training and inference time comparison for ModelNet40 using batch size 32, while varying the number of spatial spheres. Training time reported seconds per epoch, while inference time is milliseconds per batch.
A.2 MODELNET40 EXPERIMENT DETAILS
We specify CSGNN-z as the model trained on z-axis aligned rotations, and CSGNN-SO3 as the model trained on SO3 rotations. For concentric spheres, CSGNN-z uses 15 spheres spatially, and CSGNN-SO3 uses 20. Level 4 icosahedral discretization (2562 vertices) is used for initial spatial resolution of each sphere. Point clouds are mapped vertex features using Gaussian RBF with of threshold T' = 0.01. After vertex grouping. the number of concentric spheres as input channels 8. Each model is trained using Adam optimizer for 60 epochs and batch size 32, with early termination if learning rate falls below 1e-5. CSGNN-Z is trained with initial learning rate of 3.9e-4, and CSGNN-SO3 with initial learning rate of 2. 2e-4. For regularization, CSGNN-Z uses dropout of 0.19, and weight decay of 3.2e-7. CSGNN-SO3 uses dropout of 0.14, and weight decay of 2.7e-7. Beyond rotational augmentation in training, we apply random uniform translation the range of [-0.1, 0.1], random Gaussian noise for positional jitter with standard deviation of 0.01, and random uniform re-scaling by a factor of [0.8, 1.2] applied independently to each axis. For the z/SO3 test setting, it was beneficial additionally apply voting over 5 sampled test rotations per instance.
A.3 ELECTRONIC STRUCTURE OF MATERIALS EXPERIMENT DETAILS
We refer to the CSGNN model for density of states prediction as CSGNN-DOS. The initial spherical resolution is L, = 3 icosahedron, or 642 vertices per sphere. CSGNN-DOS is trained using Adam optimizer for 1,500 epochs, with initial learning rate of 5e-4 and early termination if learning rate falls below le-5, A batch size of 32 snapshots is used, and weight decay of le-7 is applied for regularization. Using a = 0.1 provided best performance for band energy calculation, for weighting DOS and FDOS losses. A single ID convolution layer with kernel size of 3 is applied over the atom- wise output channels, corresponding to each atom's 's predicted DOS contribution, prior to summing to obtain the total DOS.
A.4 RUNTIME ANALYSIS
We report how training and inference time scales with the addition of spatial concentric spheres to our model for the ModelNet40 task, in Table The number of input channels is fixed at C' = 8. Training time is seconds/epoch, averaged over the course of training. Inference time milliseconds/batch. averaged over spheres by a factor four, while the inference time increases by a factor of two. We also did not observe any benefit to model accuracy going beyond R 20 for the MedelNet40 task in practice. Numbers were obtained from running on a single NVIDIA V100 GPU with 16 GB memory.
over the validation set. The training time increases from single sphere to 32
A.5 INTRA- POOLING
Pooling is widely used alongside convolutional filters in CNN architectures to learn invariance to transformations of the input. The icosahedron, due to its recursive refinement by discretization level, has well-defined and natural hierarchy for pooling and coarsening. Each coarser representation also follows a uniform discretization of the sphere, which allows efficient information propagation in hierarchical fashion when combined with convolutions and pooling, To formalize the pooling
denote the feature tensor corresponding (1), the vertex set corresponding to level / discretization. Pooling is defined as H f(H : v E NN)))) where N(u) the neighborhood of vertex u E ((I) and f is a f.u
operation, we introduce the overloaded notation (()) R Rgxxv<
permutation invariant function (e.g. max operator). Pooling is followed by downsampling, where
14

---

Under review as a conference paper at ICLR 2022
Structure
Graphene
Graphite
C20
C40
C60
C(6.4)
C(9,9)
C(8,0)
Atoms
128
108
40
60
152
144
128
Table 5: Each carbon structure, and the number of atoms for each snapshot of the structure.
only vertices of the smaller vertex set V((-1) are retained. We only apply neighborhood pooling within vertices of the same sphere.
A.6 POINT CLOUD TO VERTEX FEATURES
Instead of computing the summation in Eq. with respect all points, for each data point update the features vertices in a local neighborhood. The radial basis function (RBF) á decays exponentially, and so points beyond local neighborhood have little influence (depending on choice of bandwidth 7). Furthermore, restricting to a constant size local neighborhood reduces computation cost from ((N O(N). To define the local neighborhood of data point p in this work: any given point is contained within two bounding "triangles" of the discretization (ignoring boundary conditions and degenerate cases). These correspond to the vertices 5()) = [w(0), u(t), w((() } and (((-1) = (u(++)) u(++1), ( where i indexes radial level. However, using single value for the RBF results scaling inconsistency: distances between vertices progressively shrink moving to inner spheres. Based on the assumption that RBF values should be invariant to scale, a different and corresponding RBF is defined with respect radial level. To define 7is we use the maximum pairwise distance ( hetween vertices in (s(() (5t+)))) Specifically, we set 7 -log where
d
is a lower bound target RBF value. For example, T would correspond to % = 0, or a RBF
value of 1 any distance. T E (0, is tuning parameter that enables toggling the overall sensitivity
of the RBF to distances. Based on the approximation that d is similar for any data point, d is precomputed once.
A.7 ELECTRONIC STRUCTURE DATASET
We present the number of atoms per structure in Table
15


Reviews:
[{"review_id": "h4PcqDJqhAC", "reviewer": "Reviewer_vMzd", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:  \n1. motivation: the motivation to use multiple resolution spheres with varying radius sounds valid and may help feature learning of 3D shapes, comparing to previous approaches using single unit sphere; \n2. architecture: the multiple resolution approach is widely used in image deep learning architecture, this is the first time it is extended to spherical cnn and the proposed architecture could lead to many follow-up work in both 3D point cloud processing and spherical CNNs.\n\nWeakness:  \n1. rotation-equivariant: authors claims the proposed method to be rotation equivariant and results on ModelNet40 also support this claim. This is the most important property of the proposed method. It is advisable to have a better explanation of both proposed method and related method mentioned in related work, i.e. why some methods are not rotation-equivariant. \n2. ModelNet40: results of [1] (Cohen etal. 2019) is missing in Table 1. \n3. DOS experiments: \n(1). meaning of parameters R/C: in the experiments, R=1 and C=32 are used. Does this mean 32 spheres are combined into 1?\n(2). loss functions: from A.2 and Eq.5, \\alpha=0.1 is used, does this mean loss(DOS) is less important than loss(FDOS)? If previous works using \\alpha=1.0, could the authors provides the results using the same setting? In other words, how does disabling loss(FDOS) hurt?\nI'm also keen to know if it is fair to compare to other methods using additional loss term. \n(3) Effectiveness of the methods: On DOS experiments, authors use simplified version of the architecture (R=1) and reported strong performance than baselines. Does this mean that DOS estimation is too easy for the proposed method? \n(4) Spherical CNNs baseline: Considering the baselines use in DOS experiments, nearly all SOTA spherical CNNs are not compared.\n\n\nRef: \n[1] Gauge equivariant convolutional networks and the icosahedral CNN.", "summary_of_the_paper": "The paper presents a concentric sphere representation and icosahedron-based spherical CNNs for 3D point clouds. The contributions are two folds. First, the concentric sphere representation enables learning features volumetrically. Second, two types of convolutions: intra-sphere and inter-sphere are combined towards rotation equivariant and scalable computations. ", "summary_of_the_review": "In summary, the paper is easy to read and follow. Description of the method is clear and complete. Experiments include one for 3D point cloud classification and one for DOS estimation. For the latter one, I'm not familiar with the experiment difficulty and details and thus could not evaluate the methods from the results. Overall, I appreciate the efforts of the work to propose multi-resolution version of spherical CNNs. The downside is that evaluations are limited, and some baselines are not compared. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "cnullFa9l13", "reviewer": "Reviewer_oVKM", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "PROS\n- The paper makes a very sensible contribution that extends a variant of spherical CNN to be multiscale. As such, better accuracy is expected because the finer grained details can be better captured. \n- The presentation is quite clear and I did not spot significant problems.\n- The chosen experiments are quite relevant and span two different communities. \n\nCONS\n\n- I feel like the paper presents what I would call a 'multi-resolution spherical CNN'. In other words, the only addition to the known graph based spherical CNN is the notion of multiple concentric spheres. However, the experiments seem to distinguish between Spherical CNN and the CSGNN with R=1, C=1. This confuses me a little bit. Would it be possible to discuss the differences or can we recover Spherical CNNs as a special case of the proposed approach? \n\n- 'Representation' is a fundamental aspect in 3D analysis. I would suggest the paper to be more careful or conscious with the use of the word 'volumetric'. First, why are volumetric features desirable to begin with? Second, I feel like the concentric spheres can better capture the narrow band around the point set while still leaving parts of the space unexplained. This is not necessarily a negative, as it might amount to avoiding the storage of unnecessary information. In that sense, the representation might fall somewhere between a point cloud and a dense volume - say a sweet spot maybe. I might also be wrong, but these points could be better described in the paper.\n\n- The related work does not really discriminate the proposed approach from the others so as to position this method among the state of the art. In particular, it could be good to discuss what properties/goals are not met by the prior approaches. Regarding the related works, it is also good to mention the initial attempts to develop invariant point cloud classification networks:\n* Thomas, Nathaniel, et al. \"Tensor field networks: Rotation-and translation-equivariant neural networks for 3d point clouds.\" arXiv preprint arXiv:1802.08219 (2018).\n* Zhao, Yongheng, et al. \"Quaternion equivariant capsule networks for 3d point clouds.\" European Conference on Computer Vision. Springer, Cham, 2020.\n\n- The paper seems to lack some details of the rotational invariance capabilities of the convolutions. Especially invariant/equivariant design of the radial convolutions carry importance as being the main contributions of this paper.\n\n- The paper seems to decouple two convolutions (radial and angular). Would it be possible to have a single convolution fusing information from both dimensions? (e.g. just like volumetric 3D convolutions) Could the paper explain why such a design choice is better?\n\n- The paper argues for a spherical distribution to represent the point cloud. However this introduces some discretization artifacts. So it is of wonder to see whether the benefit brought by concentric spheres outdoes the discretization error. \n\n- As shown in Fig. 2, it is possible to obtain a subset of the spheres at certain radii. As far as I understand this is somewhat a pruning mechanism. Is this only hypothetical or does it also happen with real point clouds? Could we see an example?\n\n- The spheres seem to have more structure than a simple graph. I feel like resorting to GNNs can yield some information loss. Have we ever thought about this aspect?\n\n- Can the approach estimate the pose of the object or is invariance property all we get? In case the pose can be estimated, can we see a study on this?\n\n- In Fig. 6, I notice an architectural/design difference compared to the point cloud one. In fact, this seems to have spheres anchored on local patches (an operation also applicable to point sets). Do we necessarily need this change? Can we cover both problems with similar architectures?\n\n- I feel that some kind of a theoretical analysis on the minimum number of concentric spheres required for rich feature extraction would be a nice to have. \n\nMinor Remarks:\n- such the underlying -> such that the underlying\n- challenging to invariance -> challenging to introduce invariance (maybe?)\n- 'over over' -> over\n- Are all point sets normalized to the unit ball? Can we hear some more details here?\n- What about the connections in Fig. 5? Are those used or is the discretization same as the point cloud experiment?", "summary_of_the_paper": "This paper proposes a graph convolution based multiscale spherical deep neural network. In particular, radial convolutions over concentric spheres of different radii are used in conjunction with the typical spherical convolution. A simple 5-layer architecture is presented to fuse the rich features extracted by both convolutions. Experiments on point cloud classification under arbitrary SO(3) transformations as well as predicting electronic state density of graphene allotropes, demonstrate the validity of the devised method.", "summary_of_the_review": "Simple idea, clear writing, but lacks thorough discussions and evaluations.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "_suxaLWJK2", "reviewer": "Reviewer_KVV8", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper proposes a new volumetric representation that consists of multiple nested spheres, each discretized by the icosahedral grid. It proposes to learn the 3D volumetric representation over concentric spheres by combining intra-sphere and inter-sphere convolutions. The proposed convolutions are rotationally equivariant, and also scale (near) linearly with grid resolution. It shows some experiments in 3D object classification, and resolving electronic structure of atomistic systems.\n\nThe paper overall is technical sound with some interesting results. The application in atomistic systems is interesting and seems new at least to the reviewer for the 3D point clouds.\n\nThe results in 3D object classification is not very detailed or sufficient. There are a lot of datasets I wished the paper can test. Also it doesn't reference sufficient papers in the 3D point clouds deep learning domain which is related and very relevant. e.g the KPConv paper should be referred. The shellnet paper which also uses concentric shells should be mentioned. It seems to me the authors might not be very familiar with the literature in this area. I would like to see the paper has side by side comparisons with some of these leading papers in some of the standard benchmark datasets.", "summary_of_the_paper": "This paper proposed a concentric spherical representation of 3D space, formed by nesting spatially-sampled spheres resulting from the highly regular icosahedral discretization. It utilizes separate intra-sphere and inter-sphere convolutions over the resulting concentric spherical grid, which are combined into a convolutional framework for learning volumetric and rotationally equivariant representations over point clouds.", "summary_of_the_review": "Overall the paper is sound and interesting with good results. The application in atomic system is interesting and new. However the paper doesn't do a good comparison with leading papers in this field thus it is hard to judge the effectiveness of the proposed method.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "2cVUDMZ4mY", "reviewer": "Reviewer_xhPe", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1. The author proposes a new volumetric representation as the basis for convolutional learning, which consists of multiple nested spheres discretized by the icosahedral grid.\n2. The proposed CSGNN integrates both intra-sphere and inter-sphere convolutions to learn volumetric representation over concentric spheres.\n3. The proposed CSGNN is practical in classifying arbitrarily rotated 3D objects and is also effective in molecular environment description.\n\nWeaknesses:\n1. The architecture is not novel enough, which is only based on hierarchical graph convolutions followed by radial convolutions, and the performance is restricted by the size of radius, as illustrated in Fig. 1 and the experimental results on Table 2.\n2. It is time-consuming and redundant to adopt all the points to calculate the contribution to vertices, the reviewer is curious about how to select the effective points or how to avoid the redundant information when converting the point cloud to concentric spheres. Are there any adaptive strategies other than thresholding on some scores?\n3. There are some adjustable hyper-parameters, i.e., the threshold of RBF, the number of spatial concentric spheres, the input channels, the kernel size, etc. It is not clear whether the method is robust.\n4. How to calculate the DOS and FDOS loss items, the author is expected to give explanations.\n5. Experimental results in some aspects are not state-of-the-arts, and some failure cases should also be discussed if there exists.\n", "summary_of_the_paper": "This paper addresses a challenging problem in 3D representation learning of point clouds, which aims to generalize the representations well to arbitrary orientations. The proposed CSGNN uses a convolutional framework to learn over concentric spherical feature maps, which is incorporated with both intra-sphere and inter-sphere information. The idea is interesting and reasonably well presented. In general, the proposed CSGNN is practical on spherical representation learning. However, there still remains some issues that should be addressed.", "summary_of_the_review": "This paper presents a new volumetric representation to learn 3D representations. However, some technical details should be further discussed and analyzed. Strengths and weaknesses are both discussed on the proposed CSGNN in terms of data process, method design, and experimental settings and results. Therefore, the reviewer gives rating of 6: marginally above the acceptance threshold.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper addresses the problem of learning representation of 3D point clouds and introduces an interesting approach of concentric spherical GNN with the property of rotationally equivariant. It shows some promising results on point cloud classification under SO(3) transformations and on predicting electronic state density of graphene allotropes. The reviews suggest that, while it does not suffer from any major flaws, the paper has a fairly large number of minor issues that add up to make it subpar for publication. The proposed approach have several hyperparameters, but the authors do not seem to be up front about how the parameters are selected except for stating that they use "standard tuning techniques" --- this is not a satisfactory answer and appears to be dodging the question. Many technical details and specific choices could use more thorough explanation and analysis. The distinction of the proposed approach in relation to the large body of existing literature could be more clearly spelled out. Collectively, these issues made the contribution of this paper less clear.
Decision: Reject