Efficient Long-Range Convolutions for Point Clouds
Authors: Yifan Peng
Lin Lin
Lexing Ying
Leonardo Zepeda-Nunez
Venue:  (2021)
Keywords: global convolution
point cloud
graph-cnn
NUFFT

Abstract: The efficient treatment of long-range interactions for point clouds is a challenging problem in many scientific machine learning applications. To extract global information, one usually needs a large window size, a large number of layers, and/or a large number of channels. This can often significantly increase the computational cost. In this work, we present a novel neural network layer that directly incorporates long-range information for a point cloud. This layer, dubbed the long-range convolutional (LRC)-layer, leverages the convolutional theorem coupled with the non-uniform Fourier transform. In a nutshell, the LRC-layer mollifies the point cloud to an adequately sized regular grid, computes its Fourier transform, multiplies the result by a set of trainable Fourier multipliers, computes the inverse Fourier transform, and finally interpolates the result back to the point cloud. The resulting global all-to-all convolution operation can be performed in nearly-linear time asymptotically with respect to the number of input points. The LRC-layer is a particularly powerful tool when combined with local convolution as together they offer efficient and seamless treatment of both short and long range interactions. We showcase this framework by introducing a neural network architecture that combines LRC-layers with short-range convolutional layers to accurately learn the energy and force associated with a $N$-body potential.  We also exploit the induced two-level decomposition and propose an efficient strategy to train the combined architecture with a reduced number of samples.

Under review as a conference paper at ICLR 2021
EFFICIENT LONG-RANGE CONVOLUTIONS FOR POINT CLOUDS
Anonymous authors Paper under double-blind review
ABSTRACT
The efficient treatment of long-range interactions for point clouds is challenging problem in many scientific machine learning applications. To extract global infor- mation, one usually needs a large window size, a large number of layers, and/or a large number of channels. This can often significantly increase the computational cost. this work, we present a novel neural network layer that directly incorpo- rates long-range information for point cloud. This layer, dubbed the long-range convolutional (LRC)-layer, leverages the convolutional theorem coupled with the non-uniform Fourier transform. In nutshell, the LRC-layer mollifies the point cloud to an adequately sized regular grid, computes its Fourier transform, mul- tiplies the result by set of trainable Fourier multipliers, computes the inverse Fourier transform, and finally interpolates the result back to the point cloud. The resulting global all-to-all convolution operation can be performed in nearly-linear time asymptotically with respect to the number of input points. The LRC-layer a particularly powerful tool when combined with local convolution as together they offer efficient and scamless treatment of both short and long range interactions. We showease this framework by introducing a neural network architecture that combines LRC-layers with short-range convolutional layers to accurately learn the energy and force associated with --body potential. We also exploit the induced two-level decomposition and propose efficient strategy to train the combined architecture with a reduced number of samples.
INTRODUCTION
Point-cloud representations provide detailed information objects and environments. The develop- ment of novel acquisition techniques, such as laser scanning, digital photogrammetry. light detection and ranging (LIDAR), 3D scanners, structure- from-motien (SFM), among others, has increased the interest of using point cloud representation in various applications such digital preservation, surveying, autonomous driving (Chen al. 2017). 3D gaming, robotics (Oh & Watanabe 2002). and virtual reality (Park al., 2008). In return, this new interest has fueled the development of machine learning frameworks that use point clouds as input. Historically, early methods used a preprocessing stage that extracted meticulously hand-crafted features from the point cloud, which were subsequently fed to neural network (Chen et al.) 2003) Rusu et al. [0088] Rusu et al.) 2009; AAbry et al.) 2011). or they relied on voxelization of the geometry (Savva al., 2016 Wu al. 2015 Riegler al. 2017: Maturana & Scherer 2015). The PointNet architecture Qi et al. 2017) was the first to handle raw point cloud data directly and learn features on the fly. This work has spawned several related approaches, aiming to attenuate drawbacks from the original methodology, such as PointNet++ (Qi
al. 2017). or increase the accuracy and range of application (Wang et al. 2019 Zhai et al. 2020
Li et al. 2018 Liu et al. 2019).
Even though such methods have been quite successful for machine learning problems, they rely on an assumption of locality, which may produce large errors when the underlying task at hand exhibits long-range interactions (LRIs). To capture such interactions using standard convolutional layers, one can use wider window sizes, deeper networks, and/or a large number of features, which may increase the computational cost significantly. Several approaches have been proposed to efficiently capture such interactions in tasks such semantic segmentation, of which the ideas briefly summarize below. In the multi-scall type of approaches, features are progressively processed and merged. Within this family, there exist several variants, where the underlying neural networks can

---

Under review as a conference paper at ICLR 2021
be either recursive neural networks (Ye et al. 2018), convolutional layers (Xu et al. 2019: Xu
(et al. 2018) or autoencoders (Yang et al. [018 Deng et al. 2018). Some works have proposed skip connections, following an U-net (Ronneberger et al.] 2015) type architecture (Zhou & Tuzel 2018: Qi et al. 2017), while others have focused on using a tree structure for the clustering of the points (Klokov & Lempitsky 2017 ZZng & Gevers 2018 Gadelha et al. 2018), using an reference permutohedral lattices to compute convolutions (Jampani et al. 2016)) w whose results are interpolated back the point cloud (Su et al. 2018). Although these methods have been shown to be successful in range of applications, when the task at hand presents symmetries, such as rotation, translation, and permutation invariance, there is no systematic framework to embed those symmetries into the algorithmic pipelines. Another line of work, relies on interpreting the point cloud as a graph and use spectral convolutions (Bruna et al. Defferrard et al. 2016), whose cost can scale super-linearly when dealing with LRIs.
In applications of machine learning to scientific computing, several classical multilevel matrix factorizations have been rewritten in the context of machine learning (Kondor et al. 2014). which have been adapted to handle long-range interactions in the context of end-to-end maps using voxelized geometries in (Fan et al. 2019b a Khoo & Ying. 2019: Fan & Ying/ 2019) resulting in architectures similar to U-nets (Ronneberger et al. 2013), which have been extended to point clouds in (Li et al.) 2020). Due to underlying voxelization of the geometry, it may be difficult for these networks to generalize when the resolution of the voxelization changes.
The efficient treatment of LRI for point clouds is also prominent problem in many physical applications such as molecular modeling and molecular dynamics simulation. While long-range electrostatic interactions are omnipresent, has been found that effectively short-ranged models can already describe the N-body potential and the associated force field (Behler & Parrinello 2007 Zhang et al.] [2018a b) for a wide range of physical systems. There have also been a number of recent works aiming at more general systems beyond this regime of effective short-range interactions, such as the work of Ceriotti and co-workers (Grisafi & Ceriotti 2019 Grisafi et al.; Nigam et al. 2020; Rossi et al. 2020). as well as the works of Yao et al.] 2018 Ko et al.. 2009; Hirn et al. 2017] Rupp et al. 2012: Huo & Rupp Deng et al. 2019; Bereau et al. 20188] Zhang et al. 2019)). The general strategy is to build parameterized long long-range interactions into the kernel methods or neural network models, so that the resulting model can characterize both short-range, as well as long-range electrostatic interactions. In the neural network context, the computational cost of treating the LRIs using these methods can grow superlinearly with the system size.
The idea of this work is aligned with the approaches in the molecular modeling community, which constructs a neural network layer to directly describe the LRI. In particular, we present new long- range convolutional -lRC))layer, which performs global convolutional operation in nearly-linear time with respect to number of units in the layer. By leveraging the non-uniform Fourier transform (NUFFT) (Dutt& & Rokhlin 1993, Greengard & Lee 2004 Barnett et al.) 2019) technique, the LRC-layer implements convolution with a wiset-wise multipiicttion in the frequency domain with trainable weights known as Fourier multipliers. The NUFFT is based on the regular fast Fourier transform (FFT) (Cooley & Tukey 1965) with a fast gridding algorithms, to allow for fast convolution on unstructured data. This new LRC-layer provides a new set of descriptors that can seamlessly satisfy relevant symmetries. For instance, when the kernel of the LRI is rotationally invariant, such symmetry can be directly built into the parameterization of the Fourier kernel. Such descriptors can be used in tandem with the descriptors provided by short-range convolutional layers to improve the performance of the neural network.
Efficient training of neural network with the LRC-layer for capturing the information of LRIs is another challenging problem. Short-range models can often be trained with data generated with a relatively small computational box (called the small-scale data), and they can be seamlessly deployed in large-scale systems without significantly increasing the generalization error. On the other hand, long-range models need be trained directly with data generated in a large computational box (called the large-scale data), and the generation process of such large-scale data can be very expensive. For instance, in molecular modeling, the training data is often generated with highly accurate quantum mechanical methods, of which the cost can scale steeply as O(N*), where N is the system size and a 2 3. Therefore it is desirable to minimize the number of samples with large system size. In many applications, the error of the effective short-range model is already modestly small. This motivates us to propose a two-scule training strategy as follows. We first generate many small-scal data (cheaply
2

---

Under review as a conference paper at ICLR 2021
and possibly in parallel), and train the network without the LRC-layer. Then we use small number of large-scale data, and perform training with both the short- and long-range convolutional layers.
In order to demonstrate the effectiveness of the LRC-layer and the two-scale training procedure, we apply our method to evaluate the energy and force associated with a model N N-body potential that exhibit tunable short- and long-range interactions in one, two and three dimensions. The input point cloud consists of the atomic positions, and the output data include the N-body potential, local potential, and the force (derivative of the N-body potential with respect atomic positions). In particular, the local potential and the force can be viewed as point clouds associated with the
atomic positions. The evaluation of the N-body potential is foundational component molecular modeling, and LRI plays an important role in the description of ionic systems, macroscopically polarized interfaces, electrode surfaces, and many other problems in nanosciences (French al. 20T0 Our result verifies that the computational cost of the long-range layer can be reduced from O(N)) using direct implementation, to ((N) (up to logarithmic factors) using NUFFT. Furthermore, we demonstrate that the force, i.e. the derivatives of the potential with respect to all inputs can be
evaluated with (NV) cost (up to logarithmic factors). In terms of sample efficiency, we find that for the model problem under study here, the two-scale training strategy can effectively reduce the number of large-scald samples by over an order of magnitude to reach the target accuracy. This can be particularly valuable in the context of molecular modeling, where accurate data often obtained from first principle electronic structure calculations. Such calculations are often very expensive for large scale systems, and the number of large-scale samples is thus limited.
2 LONG-RANGE CONVOLUTIONAL LAYER
Convolutional layers are perhaps the most important building-block in machine learning, due to their great success image processing and computer vision. A convolutional layer convolves the input, usually an array, with rectangular mask containing the trainable parameters. When the mask can be kept small (for example while extracting localized features), the convolution layer is highly efficient and effective. A different way for computing a convolution is to use the convolutional theorem as follows: (1) compute the Fourier transform of the input, (2) multiply with the Fourier transform of the mask, i.e.m the Fourier multiplier, and (3) inverse Fourier transform back. In this case, the trainable parameters are the DOFs of the Fourier multipliers and the Fourier transforms are computed using the fast Fourier transform (FFT). This alternative approach is particularly attractive for smooth kernels with large support (i.e., smooth long-range interactions) because the computational cost does not increase with the size of the mask. To the best of our knowledge, this direction has not been explored for LRIs and below we detail now to apply this to point clouds.
Given point cloud {ar ( C dd and scalar weights {ft)]/ we consider the problem of computing the quantity u, := Σラ=1 Φ0(x; - xz)ff at each xy. Here the function (+)-) is the kernel with a generic trainable parameter 6. At first glance the cost of this operation scales as O(N2): we need to evaluate u, for each point 24, which requires O(N) work per evaluation. By introducing generalized function f(y) Σ; Jt ō(y- x;) and defining function w(xx) = Φo(x - y) f (y)dy, one notices that u, is the value of u(x) at x = 24. The advantage of this viewpoint is that one can now invoke the connection between convolution and Fourier transform
(1) where de (k:) is trainable Fourier multiplier. This approach is suitable for point clouds since the trainable parameters are decoupled from the geometry of the point cloud. To make this approach practical, one needs to address two issues: (1) the non-uniform distribution of the point cloud and (2) how to represent the multiplier (k).
ū(k) = (k) f(k),
Non-uniform distribution of the point cloud Equation I suggests that one can compute the convolution directly using the convolution theorem, which typically relies the FFT to obtain low-complexity algorithm. Unfortunately, {a,}/ N do not form a regular grid, thus FFT can not be directly used. We overcome this difficulty by invoking the NUFF1 (utt & Rokhlin 1993), which serves as the corner-stone of our instance of the LRC-laye 2
See Appendix C.2 for further details. We point out, that one could in practice use fast summation algorithm, such as the fast multipole method (FMM) introduced by Greengard & Rokhlin 1987. evaluate 24, This would results in the same complexity if
3

---

Under review as a conference paper at ICLR 2021
Algorithm Long-range convolutional layer
Input: [ir {fi
Output: { Ji=1. {wr J where u, =ド== f00a(z; Iz).
1: Define the generalized function: f(x:) ΣØ=1 Jz8(z xz))
2: Mollify the Dirac deltas: fr(z) |== fogr (2 2z), where 9r is defined in Appendix C.2
3: Sample in regular grid: fr(x))= = 21==1 1r 0r(ee - xxz) for Ie in grid of size LeFF in each dim
4: Compute FFT: (k) = FFT (})
5: Re-scale the signal: FA:) v F F F-(k)
6: Multiply by Fourier multipliers: 6(k) = 00 (k) F())
7: Re-scale the signal: (_ - /= c ((k)
8: Compute IFFT: uー ( IFFT(U
for ze the regular grid
9: Interpolate to the point cloud: uy = u(z;) = u * (x,)
The LRC-layer is summarized in Alg. M where is chosen following Dutt & Rokhlin (993). The inputs of this layer are the point cloud [.,,]_] and the corresponding weights (fi) The outputs are u, u((;)) for i 1, ..., N. The number of elements in the underlying grid NEFt Lier is chosen such that the kernel is adequately sampled and the complexity remains low. As shown in Appendix 5 only needs a relatively small LyFT- Even though the precise number is problem- specific, given that the goal is to approximate LRIs that are supposedly smooth, it can be captured with relatively small number of Fourier modes.
The LRC-layer is composed of three steps: (1) It computes the Fourier transform from the point cloud regular grid using the NUFFT algorithm (lines 2 - in Alg. and showcased in Fig. 2 (2) It multiplies the result by a set of trainable Fourier multipliers (line 6 in Alg. T (3) It computes the inverse Fourier transform from the regular grid back the point cloud (lines 9 in Alg.
Within the LRC-layer in Alg. T the only trainable component is the parameter 0 of the Fourier multiplier (k). The remaining components, including the mollifier 9-(-) and the Cartesian grid size, are taken to be fixed. One can, principle, train them as well, but it comes with much higher cost. Among the steps of Alg. [ the sampling operator, the rescaling operator, the interpolation operator, and the Fourier transforms, are all linear and non-trainable. Therefore, derivative computations of backpropagation just go through them directly.
Alg. is presented in terms of only one single channel or feature dimension, i.e., ff E IR and u, E K. However, can be easily generalized to multiple channels, for example ʃj E RRd and u; E RRda, In this case, the Fourier multiplier (k) at each point ks is a d2 x di matrix, and all Fourier transforms are applied component-wis.
Representation of the Fourier multiplier A useful feature of the LRC-layer is that it is quite easy to impose symmetries on the Fourier multipliers. For example, if the convolution kernel de is constrained to have parity symmetry, rotational symmetry, smoothness or decay properties, these constraints can be imposed accordingly the coefficients of the Fourier multipliers (k)). When the size of the training data is limited, it often necessary to reduce the number of trainable parameters in order to regularize the kernel. For example, we may parameterize the Fourier multiplier linear combination of several predetermined functions on the Fourier grid. This is the procedure used in molecular modeling (Grisati & Ceriotti 2019 Yao et al.) 2018 Ko al., 2009), and also in our numerical examples in equation Z We also remark that the IRC-llyer described here can be applied to point clouds way similar to standard convolution layer applied images and multiple LRC-layers can be composed on top of each other.
the kernel is fixed. However, in order for the kernel to be trainable, this would require a different algorithm for each iteration, including the computation of the derivatives, thus increasing the computational cost and rendering the implementation significantly more cumbersome.
4

---

Under review as a conference paper at ICLR 2021
N
(ff.,;))
f((x)= Σ S1 zz) f+(a)) Σ
9r (ze 2z)
F(k)
" (F()))
(k( = (FT(
Figure 1:
Diagram of the NUFFT. Starting from the cloud point {x,), J. we form the mollified
function fr, sample it in regular grid, compute the Fourier transform (kk)) of the sampled function. Finally in order obtain F F(k), rescale the signal to undo the spatial convolution.
LEARNING THE N- BODY POTENTIAL
To demonstrate the effectiveness of the LRC-layer, we consider the problem of learning the energy and force associated with model N N-body potential in the context of molecular modeling. As mentioned in Section [ the potential evaluation often invokes expensive ab-initio calculations that one would like bypass for efficiency reasons.
The setup of this learning problem is follows. First, we assume access to black-box model potential, which consists of both short- and long-range interactions. However, internal parameters of the potential are inaccessible to the training architecture and algorithm. A set of training samples are generated by the model, where each sample consists of a configuration of the points {æ;} along up a deep neural network that includes (among other
with the potential and force. Second, components) the LRC-layer for addressing the long-range interaction. This network is trained with stochastic gradient type of algorithms using the collected dataset and the trained network be used for predicting the potential and forces for new point cloud configurations. These two components are described in the following two subsections in detail.
3.1 MODEL PROBLEM AND DATA GENERATION
Model We suppose that = [0, Ljd, and we denote the point cloud by x {x,), C. Q C Rd, for d 1,2, or 3. We define the total energy, the local potential and the forces acting particle by
U = Σ
(( - -))) U,(T) = Σ 2(( - xx), and -,=-0,U((0)1-- (2)
ISKJEN
igj
respectively. where the interaction kernel (b() smooth function, besides a possible singularity at the origin and decreases as |||||| + 20.
Sampling We define a snapshot as one configuration of particles, x' {n]f M together with the global energy Uk] and the forces Fl where is the index representing the number in the training/testing set. We sample the configuration of particles x randomly, with the restriction that two particles can not be closer than a predetermined value śmin in order to avoid the singularity. After an admissible configuration computed we generate the energy and forces following Appendix B This process is repeated until obtaining Neample snapshots.
For the sake of clarity, we suppose that the number of particles at each configuration is the same.

---

Under review as a conference paper at ICLR 2021
3.2 ARCHITECTURE
Our network architecture consists of separate descriptors for the short- interactions and long-range interactions, respectively. To capture the short-range interaction, we compute a local convolution using for each point only its neighboring points within a ball of predetermined radius. For the long-range interactions, we compute an all-to-all convolution using the LRC-layer introduced in Section 2 whose output is distributed to each particle and then fed to sequence of subsequent layers.
Short-range descriptor For a given particle 24, and an interaction radius R. define I₁, the interaction list of 2;, as the indices such that ||; -52z) zyyl < R. i.e., the indices of the particles that are inside ball of radius centered at 24. Thus for each particle x, we build the generalized coordinates Zij and the short-range descriptor
sijj
D
Σ fo($L,)) JET,
(3)
where fo F. > 1 function represented by a neural network specified in Appendix C.I where mar is the number of short-range features. By construction fo(s) is smooth and it satisfies fo(s) = for |s||
Long-range descriptor We feed the LRC-layer with the raw point cloud represented by { with weights {fi which for simplicity can be assumed to be equal to one here, i.c., Si = for = I....,N N. The output of the layer is a two-dimensional tensor (x;;) with vector (x.), (xx)),.
I...., and k = I..... Kehnis. Then for each zi, its corresponding slice given by the
u Kania [z:)]. is fed to a function go -Konntt + IRm,, which is repre- sented by neural network with non-linear activation functions. Here 0 is generic set of trainable parameters and mi. is the number of long-range features. The descriptor for particle 2,, which depends on all the other particles thanks to the LRC-layer, is defined by
D = go (x:). (z)), , Kernie (zz))
(4)
Short-range network When only the short-range interaction is present, the short-range descriptor for each particle fed particle-wise to a fitting network F.z : R -> TR. In this case FDr only depends on particle v; and its neighbors. Finally, the contributions from each particle are accumulated so the short-range neural network (NN) energy and forces are given by
U Σ F..
and (FAN)
Sn. U
(5)
respectively (see Fig. leff)). The derivatives are computed using Tensorflow (Abadi et al. 2015) directly. This network as shown by (Zhang et al. 2018b) is rotation, translation, and permutation invariant (Zaheer al. 2017).
We point out that this architecture can be understood as a non-linear local convolution: for each particle one applies the same function fe each of its neighbors. The result is then pooled into the descriptor D then processed locally by F.r (akin to a non-linear convolution with filter of width one), and finally pooled globally into U
Full-range network When both the short-range and long-range interactions are present, the long range descriptor and the local descriptor are combined and fed particle- wise to fitting network F + IR to produce the overall neural network (NN) energy and forces
E Σ F(D SE D(,),
and
(F -0 MNN
respectively (see Fig. rright)). Following Section 2 the long-range descriptor is translation invariant by design and can be easily made rotation invariant. Furthermore, it is well known (Zaheer et al.) 2017) that this construction is permutation invariant. Further details on the implementation of the network can be found Appendix C.3 From the structures shown in Fig. it is clear that we can recover the first architecture from the second, by zeroing some entries at the fitting network, and removing the LRC-layer.
WWe provide more detailed schematics in Fig. and Fig. in Appendix

---

Under review as a conference paper at ICLR 2021
Short-range deseriptor network
(x/
|| +ylli
Particle wise fitting network
Particle wise firting network
[x]]
[|
xylli,
Descriptor network
U
Input
U
Long-range descriptor network
LRC layer
Output
Input
Output
Figure 2: (Ieft) The short-range network architecture. (right) The full-range network architecture.
Finally, let us comment on the inference complexity of the proposed network where, for simplicity we assume that (KKehnis) (mmge) = O(m1=) = (1), and that the depth of the neural networks (1). The cost for computing U is O(N), provided that each particle has a bounded number of neighbors. The complexity for computing the forces also scales linearly in N, albeit with higher constants. The complexity of computing both U and associated forces is O(N + Ner: log NEFT).
4 NUMERICAL RESULTS
Narrpis Ei=1 || (zx 1!!1| The loss function is the mean squared error of the forces Nawegaa 2 where the i-index runs the points of each snapshot, and E runs the test samples. We also generate 100 snapshots of data to test the performance of network. This particular loss could lead to shift the potential energy by up to a global constant, which can be subsequently fixed by including the error of the energy in the loss (Zhang et al. 2018b). For the testing stage of we use the relative e
error of the forces metric, which is defined as Erei The standard training parameters are listed in Appendix C.4
Ee, e, |P F (a. 1|2 Er, 12
The experiments shown in the sequel are designed to provide a fair comparison with state-of-the-ar methods for localized interactions. They showcase that, by adding single LRC-layer, one can outperform these methods significantly.
The kernels used the experiment typically exhibit two interaction lengths: v-)) ar a/dh (+) azgi""(-)) (-), where each of ei² and 2/22 is either simple exponential kernel or Breeened-Coulome kernel (also known as the Yukawa kernel). For each of g:و and 15d9 the superscripts denote the reciprocal of the interaction length, i.e., length scale ~ 1t -1 or 122 Without loss of generality, 141 > P2. so that μl corresponds to the short-range scale and μ1 the long-range scale. We also assume that 0 S Q2 I 01 and in magnitude compared to that of the short-range interaction. In the special case of a2 O, the kemel exhibits only single scale M1 The precise definition of the kernel depends on the spatial dimension and boundary conditions, which are explained in Appendix
02 1, so that the effect of the long-range interaction can be smaller
For fixed set of kernel parameters (μ1, #2, 01, a2), consider two types of data: large- and small-scale data, generated in the domains Qig and Qge respectively (details to be defined in each experiment).
The Fourier multiplier within the LRC-layer is parameterized as
4T9
(k.)(k) |k|2 + X2
where 3 and À are trainable parameters. This is a simple parameterization, and more complex model can be used as well with minimal changes to the procedure. For all experiments shown below, two kernel channels are used and as a result there are only four trainable parameters in the LRC-layer.
The numerical results aim to show namely two properties: i) the LRC-layer is able to efficiently capture LRIs and ii) the two-scale training strategy can reduce the amount of large-scale data significantly. To demonstrate the first property, gradually increase the interaction length of the kernel. The accuracy of the short-range network with fixed interaction radius is supposed to decrease rapidly, while using the LRC-layer improves the accuracy significantly. To show the second property, we generate data with two interaction lengths and train the full-range network using the one- and
SSee Appendix C.2 and C.3 for further details.

---

Under review as a conference paper at ICLR 2021
Table 1: Relative testing error for trained screened-Coulomb type 1D models with on 1,
and varying Notic that μ2 can be arbitrary here given that Q2 0.
Pi short-range network
0.5
1.0
2.0
5.0
10.0
0.05119
0.02919
0.00597
0.00079
0.00032
full-range network 0. 0.00828
0.00602
0. 0.00336
0.00077
0.00054
10-1
a = 0.5, 0.5,0y = scale
10a
all-to-all computation
a 0.5, 5, -.5. two-scale
Normalizee wall time
LRc-layer
0 0.9.0) 02 0.1. one-scale
O(N)
OIN*)
0 0.9. = 0.1. two-saae
10a
10-2
10
109
10 10a
10
10a
102
Nsample
N
Figure 3: (left) Testing error of the trained ID model with respect to the number of snapshots using the one- and two-scale training strategies using data generated with the gcreened-Coulome potential and parameters μ1 = 5.0, P2 0.5 (right) normalized wall-time for the LRC and the direct all-to-all computation.
two-scale strategies. Finally, we also aim to demonstrate that the LRC-layer competitive against direct convolution in which the all-to-all computation is performed explicitly.
ID In the first set of experiments, the domain ? [0, N 20 and Neample 1000, where Naample is the number of snapshots and N is the total number of points in each snapshot. For the kernel, we set 02 and vary j11 to generate datasets at different interaction lengths. For each dataset we train both short-range and full-range networks using the one-scale data. The results are summarized in Table where we can observe that as the characteristic interaction length increases, the accuracy of the short-range network decreases while using the full-range network can restore the accuracy. This experiment shows that local networks are often highly accurate when the interactions are localized, but the accuracy quickly deteriorates as the interaction length increases (i.e. as μ4 decreases).
For the second set experiments we used two sets kernel parameters: one heavily biased towards localized interaction length, and another in which both interaction lengths equally weighted. For each set of kernel parameters, we generate 10, 000 small-scale snapshots using Ωgg [0, 5 and N 20, and large number of large-scale snapshots using Ω = [0, 50] and N 200 particles. The interaction radius R = 1.5, 5min = 0.05, and NEET is 501. We train the network with the one- and two-scale training strategies described in the prequel. Fig. 3 (left) depicts the advantage of using the two-scale training strategy: we obtain roughly the same accuracy at fraction of the number of large-scale training samples. We observe that when the number of large-scale training samples is sufficiently large, the resulting test accuracy is independent of the training strategy. We also observe that the training dynamies is stable with respect to different random seeds.
We compare the LRC-layer with direct all-to-all computation. We benchmark the wall time of both layers, with increasingly number of particles. To account for implementation effects we normalize the wall times in Fig. (right) and the results corroborate the complexity claims made in Section 2
2D We perform the same experiments as in the one-dimensional case. We fix Ω = [0, 15)2. N 450 and N Naample 10000. The results are summarized in Table 2 which shows that μ decreases, the full-range network outperforms the short-range one.
For the second set of experiments, R 1.5, 8min 0.05, and NEET is 31 For the small-scale data, Ω2z = 3|2 N = 18, and Neamgle = 10, 000. For the large-scale data, Ω1= = [0. 15]2 N 450. Similarly to the 1D case, train the networks with both strategies using different amounts of lcrge-scale data. The results summarized in Fig. | show that the two-scale strategy efficiently captures the long-range interactions with only small number of the long-range training samples.
8

---

Under review as a conference paper at ICLR 2021
Table 2: Relative testing error for trained screened-Coulone Coulo-h type 2D models with on 1, 02 and varying Again μ2 can be arbitrary given that = O.
j1
1.0
2.0
5.0
10.0
short-range network
0.07847
0.02332
0.00433
0.00242
full-range network
0.00785
0.00526
0.00363
0.00181
Screened Coulamb, one scale
Screened Coulomb, one scale
10
SreennedCoollmba
10-1
Screened Coufomb,
Exponential, one scale
Exponential, one mea-scalo
Exponential, two-scale
Exponential, two-scale
Error
Error
10-2
10-2
10
Naamle
Nample
Figure 4: Testing error the trained model with respect the number of snapshots using the one- and two-scale training strategies using both screened-Coulomb and exponential potentials with μ1 = 10, μ2 (left) aj = 0.9, and ag = 0.1; and (right) æд = 0.5, and a2 = 0.5.
observe that for sufficiently large-scale training samples the resulting test accuracy is identical regardless of the training strategy used. Also similar to Fig. B (left,, find that the lowest achievable test error is larger in Fig. 4 (right, with larger o:2) than that in Fig. (left, with a smaller (2)). Nonetheless, we observe that the test error of the two-scale training strategy becomes less sensitive with respect the number of training samples when a2 becomes larger, i.e. the LRI becomes more prominent.
Analogously to the 1D case,
3D The domain Ω is 3) with points in each of the 27 unit cells. The other parameters the interaction radius R = 1.0, pmir 0.1, and Neample = 1000. The Fourier domain used is of size NrFT 253. The results in Table demonstrate that full-range network is capable of maintaining good accuracy for a wide range of characteristic interactions lengths.
CONCLUSION
We have presented an efficient long-range convolutional (LRC) layer, which leverages the non- uniform fast Fourier transform (NUFFT) to reduce the cost from quadratic nearly-linear with respect to the number of degrees of freedom. We have also introduced two-scale training strategy effectively reduce the number of large-scale samples. This can be particularly important when the generation of these large-scale samples dominates the computational cost. While this paper demonstrates the effectiveness of the LRC-layer for computing the energy and force associated with a expect the LRC-layer to become useful component in designing neural
model N- N-body potential, networks for modeling real chemical and materials systems, where the LRI cannot be accurately captured using short ranged models. We also expect that the LRC-layer can be useful tool for wide range of machine learning (such as regression and classification) tasks.
Table 3: Relative testing error for trained exponential type 3D models with a 1, 022 = 0, and varying M1. Again #2 can be arbitrary given that 02
0.
7.5
10
short-range network
0.06249
0.01125
0.00175
full-range network
0.00971
0.00411
0.00151

---

Under review as a conference paper at ICLR 2021
REFERENCES
M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, Goodfellow, A. Harp. G. Irving, M. Isard, Y. Jia, R. Jozefowiez. L. Kaiser, M. Kudlur, J. Levenberg, D. Mané, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, Shlens, B. Steiner, Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viégas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng. TensorFlow: Large-scale machine learning on heterogeneous systems, 2015.
M. Aubry, U. Schlickewei, and D. Cremers. The wave kernel signature: A quantum mechanical approach to shape analysis. In 20/1 IEEE International Conference Computer Vision Workshops (ICCV), pp. 1626-1633, 2011.
A. H. Barnett, J. Magland, and L. af Klinteberg. A parallel nonuniform fast Fourier transform library based on an "exponential of semicirele" kernel. SIAM J. Sci. Comput., 41(5):C479-5504 2019.
Behler and Parrinello. Generalized netrol- representation of high-dimensional potential- energy surfaces. Phys. Rev. Lett. 98: 146401, 2007.
T. Bereau, R. A. DiStasio, A. Tkatchenko, and O. A. von Lilienfeld. Non-covalent interactions across organic and biological subsets of chemical space: Physics- based potentials parametrized from machine learning. J. Chem. Phys., 148(24):241766 2018.
Bruna, W. Zaremba, A. Szlam, and Y. LeCun. Spectral networks and locally connected networks on graphs. arXiv:1312.6200
D. Chen, X. Tian, Y. Shen, and O. Ming. On visual similarity based 3D model retrieval. Computer Graphics Forum, 22(3):223-232, 2003.
X. Chen, H. Ma, Wan, B. Li, and Xia. Multi-view 3D object detection network for autonomous driving. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp- 6526-6534, 2017.
J. W. Cooley and J. W. Tukey, An algorithm for the machine calculation of complex Fourier series. Math. Comput., 19(90))277331, 1965.
M. Defferrard, X. Bresson, and Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in Neural Information Processing Systems 29, pp. 3844 2016.
H. Deng, T. Birdal, and S. Ilie. PPF-FoldNet: Unsupervised learning of rotation invariant 3D local descriptors. In Proceedings of the European Conference on Computer Vision (ECCV), September 2018.
Z. Deng, C. Chen, X.G. Li, and S. P. Ong. An electrostatic spectral neighbor analysis potential for lithium nitride. NPJ Comput. Mater, 5, 2019.
A. Dutt and V. Rokhlin. Fast fourier transforms for nonequispaced data. SIAM Sci. Comput., 14(6): 1393,, 1993.
Y Fan and L. Ying. Solving optical tomography with deep learning. arXiv:1910.04755 2019.
Y. Fan, J. Feliu-Fabà, L. Lin, L. Ying, and L. Zepeda-Náñez. A multiscale neural network based hierarchical nested bases. Res. Math. Sci., 6(2):21, Mar 2019a
Y. Fan, L. Lin. L. Ying, and L. Zepeda-Núñez A multiscale neural network based on hierarchical matrices. Multiscale Model. & Sim., 17(4): 189-1213. 1213, 2019b.
Roger H. French, Adrian Parsegian, Rudolf Podgornik, Rick F. Rajter, Anand Jagota, Jian Luo, Dilip Asthagiri, Manoj K. Chaudbury, Yet Ming Chiang, Steve Granick, Sergei Kalinin, Mchran Kardar, Roland Kjellander, David C. Langreth, Jennifer Lewis, Steve Lustig, David Wesolowski, John S. Wettlaufer, Wai Yim Ching, Mike Finnis. Frank Houlihan, O. Anatole Von Lilienfeld, Carel Jan Van Oss. and Thomas Zemb. Long range interactions in nanoscale science. Reviews of Modern Physics, 82(2): 1897-14487,, 2010. ISSN 00346861. doi: 10. Mod/ReewddPhys.8..1158
10

---

Under review as a conference paper at ICLR 2021
M. Gadelha, R. Wang, and S. Maji. Multiresolution tree networks for 3d point cloud processing. In Proceedings of the European Conference on Computer Vision (ECCV), September 2018.
L. Greengard and J. Lee. Accelerating the nonuniform fast fourier transform. SIAM Review, 46(3): 443-454, 2004.
L. Greengard and V. Rokhlin. A fast algorithm for particle simulations. J. Comput. Phys., 73:325-348, 1987.
A. Grisaf and M. Ceriotti. Incorporating long-range physics in atomic-scale machine learning. J. Chem. Phys., 151(20)204400, 2019.
A. Grisafi, J. Nigam, and M. Ceriotti. Multi-scale approach for the prediction of atomic scale properties. arXiv:2008. 12122.
M. Him, S. Mallat, and N. Poilvert. Wavelet scattering regression of quantum chemical energies. Multiscale Model Simul., 15(2):827-663, 2017.
H. Huo and M. Rupp. Unified representation of molecules and crystals for machine learning. 17XY:1770400439.
V. Jampani. M. Kiefel, and P. V. Gehler. Learning sparse high dimensional filters: Image filtering. dense CRFs and bilateral neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
Y. Khoo and L. Ying. SwitchNet: A neural network model for forward and inverse scattering problems. SIAM J. Sci. Comput., 416)) 2019.
D. Kingma and J. Ba. Adam: a method for stochastic optimization. In Proceedings of the International Conference on Learning Representations (ICLR), May 2015.
R. Klokov and V. Lempitsky. Escape from cells: Deep Kd-networks for the recognition of 3D point cloud models. In 2017 IEEE International Conference on Computer Vision (ICCV), pp. 863-872. 2017.
T. W. Ko, J. A. Finkler, S. Goedecker, and J. Behler. A fourth-generation high-dimensional neural net- work potential with accurate electrostatics including non-local charge transfer. arXiv:2009.0644 2009.
R. Kondor, N. Teneva, and Garg. Multiresolution matrix factorization. volume 32 Proceedings of Machine Learning Research, pp. 1620-1622 2014.
Y. Li, R. Bu, M. Sun, W. Wu, X. Di, and B. Chen. PoinCNN:: Convolution on x-transformed points. In Advances in Neural Information Processing Systems 31, pp. 820-830. 2018.
Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, and A. Anandkumar. Multipole graph neural operator for parametric partial differential equations. urXiv:2006.0955: 2020.
Y. Liu, B. Fan, Xiang, and Pan. Relation-shape convolutional neural network for point cloud analysis. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8887-8896, 2019.
D. Maturana and S. Scherer. Voxnet: A 3D convolutional neural network for real-time object recognition. In 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 922 922-928,12 2015.
Nigam, S. Pozdnyakov, and M. Ceriotti. Recursive evaluation and iterative contraction of N-body equivariant features. J. Chem. Phys., 153(12):121101, 2020.
Y.J. Oh and Y. Watanabe, Development of small robot for home floor cleaning. In Proceedings of the 41st st SICE Annual Conference, volume 5, p.pp.222 3222-3222 2002.

---

Under review as a conference paper at ICLR 2021
Y. Park, V. Lepetit, and W. Woo. Multiple 3D object tracking for augmented reality. In Proceedings of the 7th IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 117-120, 2008.
C. Qi, H. Su, K. Mo, and L. Guibas. Pointnet: Deep learning on point sets for 3D classification and segmentation. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 77-85, 2017.
C. R. Qi, L. Yi, H. Su, and L. J. Guibas. Pointnet++: Deep hierarchical feature learning on point sets in metric space. In Advances in Neural Information Processing Systems 30, pp. 1099-51088 2017.
G. Riegler, A. O. Ulusoo, and A. Geiger. Octnet: Learning deep 3D representations at high resolutions. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 6620 6629, 2017.
O. Ronneberger, P. Fischer, and T. Brox. U-net:
Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assiste Intervention - MICCAI 2015, pp. 234-241, Cham, 2015. Springer International Publishing.
K. Rossi, V. Jurásková, R. Wischert, L. Garel, C. Corminboeuf, and M. Ceriotti. Simulating solvation and acidity in complex mixtures with first-principles accuracy: the case of CH3SO3H H and H₂O₂ in phenol. J. Chem. Theory Comput., 16(8):5139-511 2020.
M. Rupp, A. Tkatchenko, K. Müller, and OA. Von Lilienfeld. Fast and accurate modeling of molecular atomization energies with machine learning. Phys. Rev. Lett., 108(5)005830,, 2012.
R. B. Rusu, N. Blodow, Z. C. Marton, and M. Beetz. Aligning point cloud views using persistent feature histograms. In 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 311333884999 2008.
B.B. Rusu, N. Blodow, and M. Beetz. Fast point feature histograms (FPFH) for 3D registration. In Proceedings of the 2009 IEEE International Conference on Robotics and Automation, pp. 1848-1853 2009.
M. Savva, F. Yu, H. Su, A. Kanezaki, T. Furuya, R. Ohbuchi, 7. Zhou, R. Yu, S. Bai, X. Bai, M. Aono, A. Tatsuma, S. Thermos, A. Axenopoulos, G. Th. Papadopoulos, Daras, X. Deng, 7. Lian, B. Li, H. Johan, Y. Lu, and S. Mk. Large-scale 3D shape retrieval from shapenet core55. In Eurographics Workshop on 3D Object Retrieval, 2016.
H. Su, V. Jampani, D. Sun, S. Maji, E. Kalogerakis, M. Yang, and J. Kautz. SPLATNet: Sparse lattice networks for point cloud processing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2530-2539, 2018
L. Wang, Y. Huang, Y. Hou, S. Zhang, and J. Shan. Graph attention convolution for point cloud semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
Z. Wu, S. Song, A. Khosla, F. Yu. L. Zhang, X.Tang, and J. Xiao. 3D shapenets: A deep representation for volumetric shapes. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1912-1920, 2015.
M. Xu, W. Dai, Y. Shen, and H. Xiong. MSGCNN: Multi-scale graph convolutional neural network for point cloud segmentation. In 2019 IEEE Fifth International Conference Multimedia Big Data (BigMM), pp. 118-127,200 2019.
Y. Xu, T. Fan, M. Xu, L. Zeng. and Y. Qiao. SpideCNN: Deep learning on point sets with param- eterized convolutional filters. In Proceedings of the European Conference on Computer Vision (ECCV), September 2018.
Y. Yang, C. Feng, Y. Shen, and D. Tian. FoldingNet: Point cloud auto-encoder via deep grid deformation. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 206-215, 2018.
12

---

Under review as a conference paper at ICLR 2021
K. Yao, J. E. Herr, D. W. Toth, R. Mckintyre, and J. Parkhill. The tensorMol-OO. model chemistry: neural network augmented with long-range physics. Chem. Sci., 9:2261-2269, 2018.
X. Ye, J. Li, H. Huang, L. Du, and X. Zhang. 3D recurrent neural networks with context fusion for point cloud semantic segmentation. In Proceedings of the European Conference Computer Vision (ECCV), September 2018.
M. Zaheer, S. Kottur, S. Ravanbakhsh, B. Poczos, R. R. Salakhutdinov, and A. J. Smola. Deep sets. pp. 3391-3401. 2017.
W. Zeng and T. Gevers. 3DContextNet: K-d tree guided hierarchical learning of point clouds using local and global contextual cues. In Proceedings of the European Conference Computer Vision (ECCV) Workshops, September 2018.
Zhai, Zhang, and L. Yao. Multi-scale dynamic graph convolution network for point clouds classification. IEEE Access, 8:65591-65598, 2020.
L. Zhang, J. Han, H. Wang, R. Car, and W. E. Deep potential molecular dynamics: A scalable model with the accuracy of quantum mechanics. Phys. Rev. Lett.,
Apr 2018a.
L. Zhang, J. Han, Ha. Wang, W. Saidi, R. Car, and W. End-to-end symmetry preserving inter- atomic potential energy model for finite and extended systems. In Advances in Neural Information Processing Systems 31, pp. 4441-4451, 2018b.
L. Zhang, M. Chen, X. Wu, H. Wang, W. E, and R. Car. Deep neural network for the dielectric response of insulators. arXiv: 11344, 2019.
Y. Zhou and Tuzel. Voxelnet: End-to-En learning for point cloud based 3D object detection. In 2018 IEFF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4490-4499, 2018.
A NOTATION
A table of notations is summarized in Table
B DATA GENERATION
We provide further details about the data generation process and how the parameter dictates the characteristic interaction length.
Exponential kernel: Suppose Ω be the torus [0, Ljd and that x = {xs) C Q C IR for d = 1,2,
or 3. The exponential kernel is defined as
(r v) = -ll-~vll
(8)
where || -
is the Euclidean norm over the torus. Following Section 3.1 define the total energy and the potential as
N
N and U,(x) = Σ
U=
Σ e
c
-
(9)
ijj
i
respectively. The forces are given by
N
F5 = -
-1|1===051||
(10)
Ir Tj
Screened-Coulome Coullmb kernel: In 3D, the screened-Coulone potential with free space boundary condi- tion is given by
(x x - 3) =
4T wl e -|| wll
(1I)
13

---

Under review as a conference paper at ICLR 2021
Table 4: Symbols introduced in the current paper with their corresponding meaning. Notation
Symbol Data
Meaning
Spatial dimension of the problem
Q = 0, L c gd
Computational Domain
{ar)==
Point cloud
N
Number of points in the point cloud
N Nsample
Number of snapshots for training
Interaction kernel
1
Inverse characteristic interaction length Potential
U
F
Forces exerted over the j-th particle
Networks
D
Descriptor associated to I;
F
Fitting Network
Generic trainable parameters
fo go
Trainable function inside the descriptor
Trainable function inside the fitting network
R
Interaction radius
LRC-layer
9r
Mollifier of the Dirac deltas defined in equation 20 Broadening factor the mollifier
FFT, IFFT
Fast Fourier transform and its inverse
do
Kernel with trainable parameters 0
Fourier transform of the kernel
LeF:
Number of Fourier modes per dimension
NEET
L
Total number of Fourier modes
Over the torus 4J,, the kernel v y) is the Green's 's function (z, v) defined via
AG (z. y) 100-~12-0 Gx, y) = (x),
(12)
with the periodic boundary condition. In order compute the cally, a spectral method is used: in particular,
potential numeri-
ciky
dh - =
= G G" (x.
+ n Xe (k))
(13)
||-||2
where F-1 stands for the inverse Fourier transform and (k))) is smoothing factor, usually Gaussian, to numerically avoid the Gibbs phenomenon. Similar to the exponential case, the parameter controls the localization of the potential. addition, the derivatives are taken numerically the Fourier
domain.
Visualization: To visualize the relation between y and the characteristic interaction length in 1D, consider given particle, c.g., 2100 and compute the force contribution from the other particles. Fig 5 shows that force contribution is extremely small outside a small interaction region for μr = 5.0 while the interaction region for μ 0.5 is much larger.
C DETAILS OF ARCHITECTURE AND TRAINING
C.1 SHORT-RANGE DESCRIPTOR
Here we specify the structure of D' introduced Section 3.2 interaction radius R, define the interaction list I, of .r the set of indices such that ||æ;-æyl R. assume that
For a given particle w; and an
where || stands for the distance over the torus [0, L]d To simplify the discussion, there exists a maximal number of neighbors Nmaxwetgi for each T;. We stack the neighbors in tensor whose dimensions constant across different particles. This value is chosen be sufficiently
14

---

Under review as a conference paper at ICLR 2021
μ=
μ
1.0
4-
0.5
0-
0.0
-0.5
-4
25
50
75
100
125
150
175
200
0
25
50
75
100
125
150
175
200
particles
particles
Figure 5: The force contribution particle 2100 from other particles. Results are shown for two different characteristic interaction lengths.
Nmaxneight we pad the tensor with dummy values. large to cover the number of elements in the interaction list. If the cardinality of I4 is less than
In the 1D case the generalized coordinates defined as
Sijj = ||: - xyll
and
(14)
Fyy
for j Ig We introduce two fully- conleceodde neural networks fo.: R + Rm, where each consists of five layers with the number of units doubling at each layer and ranging from 32. The activation function after each layer is tanh and the initialization follows Glorot normal distribution.
For particle z; the short-range descriptor is defined as the concatenation of
D
Σ fer Pi and JEZ,
D2
D
Σ Sor (fig) JEI:
(15)
where fijj 8t. the normalized copies of rij and 8i.j with mean equals to one. The mean and standard deviation are estimated by using small number of snapshots. We multiply the network's output fo by fr. (which is zero if j is dummy particle). This procedure enforces a zero output for particles not in the interaction list. The construction satisfies the design requirement mentioned Section 3.2
and standard deviation
In the short-range network, one concatenates the two descriptor above and feeds them particle-wizg wise the short-range fitting network. The fitting network For 7 + R is residual neural network (ResNet) with six layers, each with 32 units. The activation function and initialization strategy are the same as the ones for the short-range descriptors. Fig. shows the detailed architecture of the short-range network.
N F(D 8c = Σ F(Di. D2,az)
N
UNN
(16)
In 2D and 3D, there is slight difference of generalized coordinates: we compute
T3
8i. I
and
Tú ||,, zzyl'
(17)
where 82, is a vector now. The local descriptors are defined in the following forms:
Di
Σ (s,
and
D D2
Σ (frg)))
(18)
JEI:
JEZ:
C.2 NUFFT
In this section we provide further details for the NUFFT implementation. Suppose that the input of the NUFFT is given by {a:)aN Rdd where each point has given associated weight fú. The first
15

---

Under review as a conference paper at ICLR 2021
[x x||
U
Short-range descriptor nelworkl
Σ
UNNA
[|x x/10,,,
Jel
Generate
Particle-wise residual fitting network
(x,}
NN
Input
coordinates
[1/12, x/BB,
Short-range descriptor network2
Output
Figure The structure of short-range network for case.
step to construct the weighted train of Dirac deltas as
N f(x) = 15 xz))
(19)
#=1
We point out that in some of the experiments f, simply equals to 1. One then defines a periodic Gaussian convolution kernel
9r (x) Σ -111--LL1///rr
(20)
LE2d
where L is the length of the interval and determines the size of mollification. In practice good
choice is r 12 I. )² (Dutt & Rokhlin, 1993), where Leet is the number of points in each
2m
dimension and NEPT = Lg We define
J f(v)g(( y)dy Σ h0 x.z).
(+(1) [(0))=(g(()))
(21)
With the Fourier transform defined
1 F+(k) La JoLce fx( 12 -20kkiia
(22)
for k E 2d, we compute its discrete counterpart
(k)
Σ
fr (Lm/ Lovz) 12nkmm/ Teax
(23)
NETT
me(0, 1]d
Σ
Σ for (Lm/ Leer xz)) -22kkmm/m
(24)
N NETT
me(0,
1=
This operation can be done in ONNer log(NNer) steps, independently of the number of inputs. Once this computed, one can compute the Fourier transform of f at each frequency point by
F(k)
4/2
c 2 F.())
(25)
Once the Fourier transform of the Dirac delta train is ready, we multiply it by the Fourier multiplier (k)), which is the Fourier transform of Φz
i(k) F(k)
(26)
16

---

Under review as a conference paper at ICLR 2021
In the next sage, needs compute the inverse transform, and evaluate into the target points {2;). First we deconvolve the signal
d/2
0_r(k) (k)
ell ((k))
(27)
and compute the inverse Fourier transform
u (-+(x)=
E
(k)
(28)
Next, we interpolate to the point cloud
u (xz) = u-, 97 (Iz) =
La JOL11
u--(0)gr (x, - x) dx
(29)
E
u
9r (xz Lmm LovT)
(30)
NEFT mE (0
Even though in the current implementation all the parameters of the NUFFT are fixed, they in principle be trained along with the rest of the networks. This training, done naively increases significantly the computational cost. How perform this operation efficiently is direction of future research.
Derivatives For the computation of the forces in equation 5 one needs to compute the derivatives of the total energy U with respect to the inputs, in hearly-linear time. The main obstacle is how to compute the derivatives of the LRC-layer with respect to the point-cloud efficiently. To simplify the notation, we only discuss the case that d 1, but the argument can be seamlessly extended to the case when
Recall that = Ej ===1 Φ0(0; -z))js then the Jacobian of the vector with respect to the inputs is given by
(Vu).. = Du: {
-
if fx0 TR), if
(31)
dx, Ensifolle
As it will be explained in the sequel, for the computation of the forces in equation 5 one needs to compute the application of the Jacobian of u to a vector. For fixed vector v E R the product (Vu)- e can be written component-wise as
((Vu) v); - Σ
W5b 2z)) + vs Σ S1bb(1; xz))
jt1
jtt
N
N
12 xz,) + 24 Σ
foo xz)) -x))
3=1
where have added fu,f,d' (0) in the last equation and then distributed it within both sums. Let us define the following two long-range convolutions
N
N
W::
Σ 15, mz)) and
Ps = Σ
fobla kz))
(32)
1=1
each of which can be performed in O(N + NEFT log Ngrr) steps using the NUFFT algorithm combined with the convolution theorem. In this case the derivative can be computed numerically in the Fourier domain to very high accuracy. Now one leverage the expression above rewrite (Vu) v as
(Vu)) v); w; caPi,
(33)
which can then be computed in nearly-linear time. The same is also true for v (Vu).
C.3 LONG-RANGE DESCRIPTOR
As mentioned before, the output of the LRC-layer is given by {u(x;;) For each particle we feed the output u(x;;) the long-range descriptor network he R Rmas, whose structure is the same
17

---

Under review as a conference paper at ICLR 2021
as the local descriptor fo mentioned in appendix C.1 except that the activation function is taken to be
ReLU. The long-range descriptor, defined as
Di. go (u((r))
(34)
for the particle 2; concatenated with the corresponding short-range descriptor (which it is itself the concatenation of two short-range descriptors) and fed together to the total fitting network F 1 - R. The results are then added together to obtain the total energy
U
F(D Di.)
(35)
is clear that the energy can be evaluated in nearly-linear complexity.
In what follows show that the force computation is also of nearly-linear. For simplicity we focus on the one-dimensional network and assume that Kahnls = 1, ((ma)) = ((m O(1) and that the depth of the neural networks is O(1). As defined in the prequel the forces are given by F U Which can be written component wise as
N ar Di [O,F(Di,
Fj 0z, U
Σ
Der + 32F(D)) D Di,)gb(,,)0,
(36)
or in a more compact fashion as
pNN = (Var Dar + Uir Vu).
(37)
Here Usr, and Uir are vectors defined component-wise as (Ear); = FiF(Di OgF(D' Di, 196 (u:). In addition (Da:)., = 0z, Drr and Vu is defined above.
5.1 Di.), and (01r); =
The first term in the right-hand side is easy to compute, given that D3= is sparse: the i. entry is non-zero only if the particle 2; is in the interaction list of zj- Given that the cardinality of the interaction list is bounded, Dar has (NN) non-zero entries in which each entry requires O(1) work, thus the first term in the right-hand side of equation 37 can be computed in (NN). At first glance the complexity of second term seems to be much higher. However, as discussed above. by using equation 33 we can apply the matrix (or its transpose) to a vector in O(N + NNrT log NrrT) time and the computation of vector 01r requires ((1) work per entry, thus resulting in a complexity of O(N + NEFT log NET1) for computing the second term in equation 37 Finally, adding both contributions together results in an overall O(N + Ngrr log NerT) complexity for the forces.
To summarize, both the computation of the energy and the forces can be performed (NN) time.
C.4 TRAINING
We use the Adam optimizer (Kingma & Ba 2015) along with exponential scheduler. The learning rate with the initial learning rate taken to be 0.001 and, for every 10 epochs, it decreases by a factor of 0.95. In order to balance the computational time and the accuracy, multi-stage training is adopted, where at each stage we modify the batch-size and the number of epochs. In particular, four stages are used: start using a batch size of 8 snapshots and train the network 200 epochs and then each stage we double both the size of the batch size and the number of epochs. In the two-scale training strategy, the same training parameters defined above are used for each stage.
C.5 DEPENDENCY ON NEFT
We measure the impact of Nrer on the approximation error, using couple of examples in the one- and two-dimensional settings.
For the mie-dimensional case, we test a screened-Coulome type potential with parameters μ1 = 5.0. Hgg = 0.5, 01 = 0.5, 0g = 0.5, and Neample = 1000. The domain Ω is [0, 50] and N = 200. We run the one-scale training procedure with varying Ner: (the number of Fourier multipliers), starting from Nger = 63 and doubling them until NErr = 501. T shows that the errors are relatively insensitive to the value of Nerz. The accuracy achieved by the architecture without the LRC-layer (denoted as None in Tables 5 is added order to demonstrate that the architecture is indeed capturing the I.Rss.
18

---

Under review as a conference paper at ICLR 2021
Σ U
Short-range descriptor network1
Generate
[ 2
jel
Particle-wise residual fitting network
{x
U
Input
coordinates
[1/||z; xy 1111lys
Short-range descriptor network2
Output
Long-range descriptor network
LRC layer
Dr
(x,z
NUFFT
UNNS
Figure 7: The structure full-range network.
Table 5: Error with respect to Nerz in the 1D case
NEFT
None
63
125
251
501
Relative testing error 0.06143
0.00536
0.00546
0.00545
0.00539
For the two-dimensional case, screened-Coulone type potential is tested with μq = 10.0, μ2 1.0, on 0.9, Q2 0.1. Here Ω 5]2. N 50 and sample = 1000. Starting with N = 21 we steadily increase its value and repeat the same training procedure. The results are summarized in Table Table/where one observes the same trend as in the one-dimensional case.
Table 6: Error with respect to Nerr in the 2D case
NEET
None
21
31
452
63
Relative testing error 0.01872
0.00202
0.00168
0.00153
0,00177
In addition, we recall that the Fourier multipliers are parametrized following
4 | + X2
(kk)
(38)
where and are two trainable parameters with À providing measure of the decay in space. Therefore, Nyrr only determines the number of Fourier modes and not the parameters of the ansatz. As long as the Fourier kernel is properly sampled, the method able to compute the correct characteristic interaction length.
One can observe this phenomenon in the experiment above, in which we extract the terminal value after training of the parameters Xι and X that correspond to the two channels in the LRC-layer, as summarized Table ? We observe that the value of X2 is very close to that #2, which is responsible for the LRIs even for small values of Nerr-
19

---

Under review as a conference paper at ICLR 2021
Table Values of parameters X and A after training with respect Nrer.
1D) case
2D case
NETT
63
125
251
501
21a
31
45
632
A
2.697
3.689
4.181
4.599
3.608
3.664
3.096
2.955
22
0.522
0.525
0.517
0.519
0.926
1.039
1.088
1.082
20


Reviews:
[{"review_id": "zm2EgYTfCcp", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes an efficient long-range convolution method for point clouds by using the non-uniform Fourier transform. The long-range convolutional (LRC)-layer mollifies the point cloud to an adequately sized regular grid, computes its Fourier transform, multiplies the results by a set of trainable Fourier multipliers, computes the inverse Fourier transform, and finally interpolates the result back to the point cloud. The method is demonstrated to be effective by a N-body problem.\n\nOverall, the paper is clearly written with high quality. The originality of the paper seems to be not very strong since it directly adapts the NUFFT to this work. Besides that, there are several concerns about this paper that need to be addressed:\n\n1. How to choose the grid size L_{FFT} in the Fourier Space? How does this parameter affect the results?\n\n2. The global pooling layers in DNN can also capture the long-range information to some extent, and are also very efficient. How does the LRC compare with the global pooling layers?\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "KthYYqQls1a", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Edit: I was unaware that papers could be submitted to arXiv simultaneously, I am sorry for that. Here is my (very late) review.\nI made it before reading other reviewers' reviews.\n\nAn efficient method for fitting long-range style interactions in point clouds is presented.\nIt makes use of NUFFT rather than convoluting a long-ranged (thus system-size and expensive) kernel directly in real space.\nAlong with this architecture, a method for training it efficiently is presented. This 2 steps strategy consists in training the short-range part of the kernels well with a lot of (supposedly inexpensive) short range data, and fitting the long-range kernels with less data in a second time.\n\nOverall, the paper is clearly written and clearly exposes the methods used.\nThe results are interesting for applications but do not seem ground-breaking (although I am not an expert of point clouds -specialized networks).\nIn terms of experimental results, I think a couple of points would deserved to be answered (see below).\n\nIn conclusion, I think the paper is marginally above acceptance level.\n\n\nIn terms of results, the paper clearly shows that NUFFT scales essentially like O(N) (with N the points number) whereas naive direct space convolution scales as O(N^2).\nHowever, when I read the algorithm (page 3), my main curiosity is : how well does the algorithm deals with large systems (large domain \\Omega), and the competing parameters seems to be the resolution of the function g_\\tau (which plays the role of mollified dirac) versus the system size \\Omega or L. Concretely, I expect that for too large \\tau/L (presented in appendix, Equation 20), the precision of the long range kernels will be poor and error will be large; and in the opposite case of very small \\tau/L, precision will be good but compute time will increase (I guess it would increase as FFT does, in O(L/\\tau log(L/\\tau))).\nProbably there is a regime where direct convolution (which does not need the approximation introduced by g_\\tau, as far as I understood), is better then the NUFFT approach introduced here. I would guess that in the large system size (large \\Omega=L^d) and small particles number N limit (i.e. the low density limit), the direct approach does well ?\nI think such a discussion (and possibly a couple of experiments) would improve the paper a lot, showing the limits of the method and letting the reader understand the reasons for its strengths (which are very real, I do believe it !)\n\n\nA bit more of commenting on the experiments' results would be appreciated. For instance, it seems that the 2-scale training strategy is especially efficient (not needing many samples) when the LRIs are sufficiently strong (figure 3, right). This is probably an effect of the \"screening\" of LRI by short-range Interactions when alpha_1 is tooo strong compared to alpha_2, making it harder to learn about LRIs.\nAlso, the fact that all curves essentially collapse beyond a given N_sample could receive a comment.\nA couple of remarks like that about the strengths and limits of the approach would be nice.\n\n\nAside from these 2 main comments, I have minor remarks of presentation:\n- figure 2 left and figure 3 both sides: poor choice of colors/linestyles. Curves come in pairs, and this should be suggested in the choice of display, using similar colors for pairs, and different linetyles in different pairs. It would improve readability (also, think of the color blind people!)\n- fig2 , right: the O(N), O(N^2) scalings are un-readable. make them parallel to the plots you do and simple black dashed to improve clarity.\n\nIn the tables, although here results are \"trivial\" (always the same line has the smallest error), you could use the convention of putting in bold the better numbers.\n\nTable 1, you say \"mu_2 can be arbitrary here\". What you mean is that it needs not be defined, because alpha_2=0 ?\nI found this sentence confusing (maybe it's just me).\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "K6oGTndTwG", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper concentrated on exploring how to efficiently extract the interaction information from the point clouds.  A key point lies in utilizing the non-uniform Fourier transform, rather than the regular Fourier transform. However, there exist some issues that need to be solved. \n\n+ves: \n+ The exploration of long-range interactions for point clouds is interesting.\n\n+ The paper is well written. The related work makes a clear description of many fields about point cloud.\n\nConcerns: \n1. In the introduction part, the authors describe many tasks that rely on point-cloud presentation. However, the authors ignore pointing out the existing issues. The authors should clearly present them.\n\n2. In the algorithmic section, the authors claim that \"NUFFT serves as the corner-stone of the LRC-layer\". So \"NUFFT\" is the unique solution? Whether or not some operations can replace FFT? \n\n3. Point-cloud is indeed important for many tasks as described in the introduction part, but the authors just explore the effects of the proposed algorithm in a \"synthetic\" experiment. The experimental results are not convincing for readers, the authors should conduct more real-world tasks to verify the effectiveness of the proposed method.\n\n4. The presentation of this work needs to improve, if possible, the authors should provide an intuitive schematic diagram to present the procedure of proposing this idea.\n\n5. In the experimental section, the author should replot Figure2-3 to ensure clear enough for a better read.  \n\n#########################################################################\n\nMinor Comments:  \n\n(1)  “N_sample” and \"N\" maybe exist the inclusion relation，it will be better to replace one of them with the other form;\n\n(2)  The formula system is a little vague, if possible, the authors can simplify them to clearly describe.\n ", "extra_scores": {}, "extra_text": {}}, {"review_id": "0WQeS4qeZCk", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper is clearly written, and presents an approach to efficiently utilize long range convolutions through a nonuniform FFT in for coulomb particle configurations. \n\nPros: \n- Presents a long range convolution layer, with an efficient implementation so that a neural network model can benefit from both short and long range interactions among data points. \n\n- To model pairwise relations between points of the N-body potential, two short range descriptor networks are utilized, where the input relations are constructed both repulsively and attractively. \n\nCons:\nExperimentally, the new long-short range NN model is validated only for computing the energy and forces for a N-body model. There are no experiments with real-life point cloud data.\n\nIn the proposed algorithm, a sampling in a regular grid is required. A fixed size Cartesian grid is utilized in the experiments, therefore, the effect of the gridding resolution is not analyzed.\n\nThe quantitative experimental results in Tables 1, 2, and 3 dwell on varying the screening coefficient mu, hence investigates how the test error changes for point configurations with varying screening parameters, which signifies the amount of diffusion of the potential. It can be seen that the test error for the proposed approach falls as the screening gets higher, that is when we have more localized interactions between the particles. What could be the significance of this finding for a practical application is not discussed, therefore not made clear. \n\nThe authors state that the proposed method could be a useful tool in a wide range of Machine Learning tasks. However, in the paper, only estimation of potential energy for Coulomb particle configurations is demonstrated as an application. I am curious to see how the presented approach could find use in learning tasks of real point cloud configurations.\n\nThe paper presents a way to incorporate long range convolutions in neural networks, which could be beneficial. However, experimental evaluation is not satisfactory, as immediate implications in point cloud analysis is not obvious. ", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This work proposes an efficient method for modelling long-range connections in point-cloud data. Reviewers found the paper to be generally well-written. On the less positive side, reviewers felt that the novelty of the work was marginal, and that the experimentation, limited to synthetic data in one domain, was too limited. These concerns remain after the discussion phase. In addition, the authors stated during the discussion that "Our goal is indeed to develop an efficient strategy to model LRIs in real chemical and materials systems”, which conflicts with the presentation of the work as motivated by more general point cloud modelling problems. Given these weaknesses, the final decision was to reject.
Decision: Reject