AdvectiveNet: An Eulerian-Lagrangian Fluidic Reservoir for Point Cloud Processing     
Authors: Xingzhe He
Helen Lu Cao
Bo Zhu
Venue:  (2020)
Keywords: Point Cloud Processing
Physical Reservoir Learning
Eulerian-Lagrangian Method
PIC/FLIP

Abstract: This paper presents a novel physics-inspired deep learning approach for point cloud processing motivated by the natural flow phenomena in fluid mechanics.  Our learning architecture jointly defines data in an Eulerian world space, using a static background grid, and a Lagrangian material space, using moving particles. By introducing this Eulerian-Lagrangian representation, we are able to naturally evolve and accumulate particle features using flow velocities generated from a generalized, high-dimensional force field.  We demonstrate the efficacy of this system by solving various point cloud classification and segmentation problems with state-of-the-art performance. The entire geometric reservoir and data flow mimic the pipeline of the classic PIC/FLIP scheme in modeling natural flow, bridging the disciplines of geometric machine learning and physical simulation.

ADVECTIVENET: AN EULERIAN-LAGRANGIAN FLUIDIC RESERVOIR FOR POINT CLOUD PROCESSING
Xingzhe He Dartmouth College Rutgers University"
Helen Lu Cao Dartmouth College f
Bo Zhu
Dartmouth College
ABSTRACT
This paper presents a novel physics-inspired deep learning approach for point cloud processing motivated by the natural flow phenomena in fluid mechanies. Our learning architecture jointly defines data in an Eulerian world space, using a static background grid, and a Lagrangian material space, using moving particles. representation, we are able to naturally
By introducing this Fulerian-Lagrangee evolve and accumulate partiele features using flow velocities generated from a generalized, high-dimensional force field. We demonstrate the efficacy of this system by solving various point cloud classification and segmentation problems with state-of-the-art performance. The entire geometric reservoir and data flow mimics the pipeline of the classic PIC/FLIP scheme modeling natural flow, bridging the disciplines of geometric machine learning and physical simulation.
INTRODUCTION
The fundamental mechanism of deep learning is to uncover complex feature structures from large data sets using a hi- erarchical model composed of simple layers. These data structures, such as a uniform grid (Lecun et al. 1998). an unstructured graph (Kipf & Welling, 2016).
features
flow particle
hierar- chical point set (Qi et al. 621 2017). function as ge- ometric reservoirs yield intricate underpinning patterns by evolving the massive input data in a nigh-dimensional parameter space. On another front, computational physics researchers have been mastering the art of inventing geo- metric data structures and simulation algorithms to model complex physical systems (Gibou et al. 2019). Lagrangian structures, which track the motion moving local frame such as particle system (Monaghan) 1992), and Eulerian structures, which describe the evolution in a fixed world frame such Cartersian grid (Fedkiw et al.) 2001), are the two mainstream approaches. Various differential oper- ators have been devised top of these data structures model complex fluid or solid systems.
Lagrangian advection
Eulerian velocity
We build advective net- work to create a fluidic reservoir with hybrid Eulerian- Lagganginn represen- tations for point cloud processing.
Figure 1:
Pioneered by E 2017 and popularized by many others, e.g., (Long al. 2018 Chen al. 2018 Ruthotto & Haber 2018), treating the data flow as the evolution of a dynamic system is connecting machine learning and physics simulation. As (2017) notes, there exists mathematical equivalence between the forward data propagation on neural network and the temporal evolution of a dynamic system. Accordingly, the training process of neural network amounts finding the optimal control forces exerted on a dynamic system to minimize specific energy form.
Point cloud processing of particular interest under this perspective. The two main challenges: to build effective convolution stencils and to evolve learned nonlinear features (Qi et al. 2016a
Xingzhe..he rutgers.edu
L.elen.L.Cao. 22 dartmouth.edu
1 "bo zhu @ dartmouth.edi

---

map conceptually to the challenges of devising world- frame differential operators and tracking material-space continuum deformations when simulating PDE-driven dynamic system in computational physics. We envision that the key to solving these challenges lies in the adaption of the most suited geometric data structures to synergistically handle the Bulerian and Lagrangian aspects of the problem. In partieular, it is essential to devise data structures and computational paradigms that can accommodate global fast convolutions, and at the same time track the non-linear feature evolution.
Atzmon et al. 2018 Wang et al., 2019),
The key motivation of this work originates from physical computing that tackles its various frame- dependent and semporally-evolver computational challenges by creating the most natural and effec- tive geometric toolsets under the two different viewpoints. We are specifically interested in uncov- ering the intrinsic connections between a point cloud learning problem and a computational fluid dynamic (CFD) problem. We observe that the two problems share an important common thread regarding their computational model, which both evolve Lagrangian particles in an Eulerian space guided by the first principle of energy minimization. Such observations shed new insight into the 3D point cloud processing and further opens the door for marrying the state-of-the-art CFD techniques to tackle the challenges emerging in point cloud learning.
To this end, this paper conducts a preliminary exploration to establish an galeerin-LLgraageee flu- idic reservoir that accommodates the learning process of point clouds. The key idea of the proposed method is to solve the point cloud learning problem as a flow advection problem jointly defined in Eulerian world space and Lagrangian material space. The defining characteristic distinguishing our method from others is that the spatial interactions among the Lagrangian particles can evolve temporally via advection in learned flow field, like their fluidic counterpart in a physical circum- stance. This inherently takes advantage of the fundamental flow phenomena in evolving and sepa- rating Lagrangian features non- non-linearly (see Figure D]. In particular, we draw the idea of Lagrangian advection on an Eulerian reservoir from both the Paricle-In-Cee (PIC) method (Evans & Harlow) 1957] and the Phudd-Implicit-Parteee Particle (FLIP) method (Brackbill et al. 1987), which are wholly rec- ognized as "PIC/FLIP" in modeling large-scale flow phenomena in both computational fluids, solids, and even visual effects. We demonstrate the result of this synergy by building a learning pipeline with straightforward implementation and matching the state-of-the-art with this framework.
The key contributions of our work include:
An advective scheme to mimic the natural flow convection process for feature separation;
A fluid-inspired learning paradigm with effective particle-grid transfer schemes;
A fully Eulerian- Lagrangian approach to process point clouds, with the inherent advantages in creating Eulerian differential stencils and tracking Lagrangian evolution;
A simple and efficient physical reservoir learning algorithm.
2 RELATED WORKS
This section briefly reviews the recent related work on point cloud processing. According to data structures used for building the convolution stencil, the methods can be categorized Lagrangian (using particles only), Eulerian (using a background grid), and hybrid (using both). We also review the physical reservoir methods that embed network training into physical simulation process.
Lagrangian Lagrangian methods build convolution operators on the basis of local points. Exam- ples include PointNet (Qi et al. 2016a). which conducts max pooling to combat any disorganized points, PointNet++ (Qi et al.; 2017). which leverages farthest point sampling to group particles, and a set of work (Wang et al., 2019; Xu al. 2018 et al. 201 a Tiang al.) 2018) based on k-nearest neighbors. Beyond the mesh-free approaches, researchers also seek to build effective point-based stencils by establishing local connectivities among points. Most significantly, geometric deep learning (Bruna et al.) 2013 Bronstein et al.) 2016) builds convolution operators mesh to uncover the intrinsic features of objects' geometry. In particular, we want to highlight the work dynamic graph CNN (Wang et al., 2201)). which builds directed graphs in extempora- neous fashion in feature space to guide the point neighbor search process, which shares similarities with our approach.
top of
2

---

pertile poution katare spflele
G:
P:
ATrutuns
Figure 2: Workflow overview: a) The feature vector for each particle is initialized by a 1 con- volution; b) Particles are embedded in an Fulerian grid; e) Features are interpolated from particles to the grid, denoted as I d) 3D convolution is applied on the grid to calculate the generalized forces and grid features; e) A velocity field is generated on the background grid; f) Particles advect in the Eulerian space using the interpolated velocities; grid features are interpolated to particles, denoted as Ig. and appended to its feature vector; g) Particles aggregate. The workflow consists of one loop to update the particle positions and features iteratively with temporal evolution. Finally, the Lagrangian features are fed into a fully-connected network for classification and segmentation.
Eulerian Hulerian approaches leverage background discretizations to perform computation. The most successful Fulerian method is the CNN (Lecun et al. 1998), which builds the convolution op- erator on a 2D uniform grid. This Eulerian representation can be used to process 3D data by using multiple views (Su et al. 2015; Qi al.. 20166 Feng et al. 2018) and extended 3D volumetric grids (Maturana & Scherer 2015; Qi et al. 2016b) Z. Wu 2015). Grid resolution is the main per- formance bottleneck for 3D CNN methods. Adaptive data structures such as Octree (Riegler al.,
2016 Wang et al., 2017; 2018). Kd-tree (Klokov & Lempitsky (017), and multi-level 3D CNN GGadai et al. | (0188) were invented to alleviate the problem. Another example of Eulerian structures Spherical CNN (Cohen et al., 2018) that projects 3D shapes onto a spherical coordinate system to define equivalent rotation convolution. In addition to these voxel- -bsedd diffusive representations, shapes can also be described as a sharp interface modeled as an implicit level set function (Hu al. 2017| Park et al., 2019] Mescheder et al. 2019). For each point the space, the level set function acts binary classifier checking whether the point is inside the shape not.
Hybrid There have been recent attempts to transfer data between Lagrangian and Eulerian rep- resentations for efficient convolution implementation. These data transfer methods can be one-way
[Wang et al. [017 Klokov & Lempitsky 2017 Tchapmi et al. 2017 Le & Duan 2018). in which
case the data is mapped from points to grid cells permanently, or two-way (Su et al.) [2018] Atzmon
let al. 2018 Liu et al. 2019; Groueix et al. 2018), in which case data is pushed forward from parti- cle to grid for convolution and pushed backward from grid to particle for evolution. Auto-encoders
on point clouds (Fan et al., 2016; Achlioptas et al. 2017 Yang al., 2017: Yu et al., 2018 Zhao
(et al. (219) can be also regarded as a hybrid approach, where encoded data is Eulerian and decoded mention the physical reservoir computing techniques that focus on the leverage of the temporal, physical evolution to solve learning problems, e.g., see (Jaeger 200I) and (Maass et al.) (2002). Physical reservoir computing demonstrating successes
data is Lagrangian. In addition,
in various applications (Jalalvand et al. 2015 Jaeger 2002 Hauser et al. 2012 Lukoševičius &
Jaeger 2009 Tanaka al. 2019).
ALGORITHM
PIC/FLIP overview Before deseribing the details of our method, we begin with briefly surveying the background of the PIC/FLIP method. PIC/FLIP uses a hybrid grid-particle representation to describe fluid evolution. The particles are used for tracking materials, and the grid is used for dis- cretizing space. Properties such as mass, density, and velocity carried particles. Each simula- tion step consists of four substeps: particle-to- -rii transfer IŞ, grid force calculation (Projection), grid-to-particle transfer E and moving particles (Advection). In the 1 step, the properties on each particle are interpolated onto a background grid. In the Projection step, calculations such adding body forces and enforcing incompressibility are conducted on the background grid. After this, the velocities on grid nodes interpolated back onto particles, i.c., IC. Finally, particles move their new positions for the next time step using the updated velocities (Advection). As summarized above, the key philosophy of PIC/FLIP to carry all features on particles and perform all differential
3

---

calculations the grid. The background grid functions computational paradigm that can be established extemporaneously when needed. Data will transfer from particle to grid and then back to particle to finish simulation loop.
Our proposed approach follows the same design philosophy as PIC/FLIP by storing the learned features on particles and conducting differential calculations the grid. The Lagrangian features will evolve with the particles moving in an Eulerian space and interact with local grid nodes. As shown in Figure 2 the learning pipeline mimics the PIC/FLIP simulation loop in the sense that Lagrangian particles are advected passively in an Eulerian space guided by a learned velocity field.
Initialization We initialize particle system P and a background grid G as the Lagrangian and Eulerian representations respectively for processing point clouds. We use the subscript p to refer to particle indices and i to refer to the grid nodes. For the Lagrangian . portion, the particle system has n particles, with each particle Pp carrying its position Fp E R velocity Up R3, mass my E R, and feature vector fp R (k= 64 initially). The particle velocity is zero at the beginning. The particle mass mp = will keep constant over the entire evolution. To initialize the feature vector fp. we first put the particles in a grid with size N For each cell, we calculate 1) the center of mass of all the particles in the cell, and 2) the normalized vector pointing from each particle the this mass center. For each particle, we concatenate these two vectors to the initial feature vector. This process is repeated for N 2, 4, 6, 8, 10, 12. The resulting feature vector with the length of 6 x are fed into multi-layer perceptron (MLP) to generate the feature vector fp.
For the Eulerian part, we start with a 3D uniform grid to represent the bounding box of the particles. The resolution of the grid is N (N = 16 for most of our cases). At the beginning, the particle system and its bounding box are normalized to the space [-1, 1]s. Each grid node G, of G stores data interpolated from the particles.
Particle-grid transfer Both the interpolation from grid to particle and particle to grid are executed using tri-linear interpolation, which common scheme for property transfer simulation and learning code.
Generalized grid forces With the feature vectors transferred from particles to grid nodes, we devise a 3D CNN on the grid to calculate a generalized force field based on the Eulerian features. The network consists of three convolution layers, with each layer as combination of 3D convolution, batch norm, and ReLU, The input of the network a vector field F NxN composed of the feature vectors on all grid nodes, with & as the feature vector size (64 by default) and j as the iteration index in the evolution loop (see Figure 2b. The output is convoluted vector field F(x))xNxNXX with the same size as F.
We use F. for two purposes: 1) To interpolate F. from the grid back onto particles and append it to the current feature vector in order to enrich its feature description; 2) To feed F. into another single- layer network generate the new Eulerian velocity field V for the particle advection. Specifically, this V is interpolated back onto particles in the same way as the feature interpolation to update the particle positions for the next iteration (see Advection for details).
Advection The essence of an advection process is solve the advection equation with the La- grangian form De/ Di = 0 or the Eulerian form dv 0v/Ot ( + v. Vo = 0. The advection equation de- scribes the passive evolution of particle properties within a flow field. With the learned grid velocity field in hand, we will update the particle velocity following the conventional scheme of PIC/FLIP. Specifically, the new velocity is first updated by interpolating the Fulerian velocity to particles (the PIC step):
v PIC 16(00)))
Then, we interpolate the difference between the new and the old Eulerian velocity:
PLIP
a+1 of (0 (vg+1 18 (o))),
(2)
and then add them to the particle with weight (=0.5 in default.):
"
n+1 a PIC + (1 a) PLLiP
n+1
(3)
4

---

With the updated velocity each particle from the 1 interpolation, the particle's position for the next time step can be updated using a standard time integration scheme (explicit Euler in our implementation);
Pp
p + up At.
(4)
Boundary conditions We apply soft boundary constraints by adding an penalty term in the objective function avoid particles moving outside the grid:
Φb 1 Σ m0 |
where up represents the pth particle in the whole batch and is the number of particles in the whole batch. We penalize on all the particles that run outside the grid.
We also design the gather penalty and the diffusion objectives enhance the particle diffusion and clustering effects during evolution (specifically for the segmentation application):
0
=
Σ
Σ
mar - ||e - Gmll)
(6)
m
dd ΣΣ
||e
Fipll
P
where and are the centers of particles of label I and m and zip is the pth particle with label I.
4 NETWORK ARCHITECTURE
The global architecture of network is shown in Figure 3 with the position of each point. After an initialization step ending with a two-layer MLP (64,64), each point carries feature vector of length 64. These features are fed into the advection module to exchange information with neighbors. The generated features have two uses: to generate the velocity for each particle, and be used along with the new advected particle position collect information from neighbors. This process repeats for a few times to accumulate features in the feature space and aggregate particles in the physical space.
Our model starts from a point cloud
Advection module The data flow inside the advection module starts with particles, passes through layers of grids, then sinks back to particles. This module takes the position and the feature vector as input. The feature vectors are first fed into MLP reduce its dimensions to 32, which putational time and prevents over-fitting. Then, we apply three layers of convolution that are each combination of 3D convolution, batch norm, and ReLU, with hidden-layer size (32,16,32) on the grid, to obtain a high-dimensional, generalized force field on the grid. Afterwards, velocity field is generated from this force field by another two-layer network. The veloeity field is then interpolated back to particles for Lagrangian advection. Additionally, to generate the output feature vector, the input and output features (with 32-dimension each) are concatenated together and appended to the original feature vector. The output of the advection module is set of particles with new positions and new features that are ready to process for the next iteration as in Figure) 2
com-
5 EXPERIMENTS
We conducted three parts of experiments, including the ablation tests and the applications for classi- fication and segmentation. We implemented the system in PyToorch (see the submitted source code) and conducted all the tests on single RTX 2080 Ti GPU. In the ablation tests, we evaluated the functions of the advection module, temporal resolution, grid resolution, and the functions of the PIC/FLIP scheme on ModelNet10 (Z. Wu 2015) and ShapeNer (Yi et al.) 2016). For classification, we tested our network on Ned4 and its subset ModelNetIO. We used the class prediction accuracy as metric. For segmentation, S3DIS data set (Armeni et al., 2016). We used mean Intersection over Union (mloU) to evaluate our method and compare with other benchmarks.
tested our network on ShapeNet YY et al. (016) and
5

---

Inmrnnaaiinn dasuitattoo
edpisooo outtuct NOTER
inpue poinnts
MLP(1024)
x 1000
mix pooling
30224
MLPI512, 256,
Advecttin
generation
Module
repoat
MIP(255, 128,
replace
[oxx3 3]
PIC/FUP
[
velocit
input points [ox3]
(16,
(p ((en)) ((px)) MG4
(o (1-22)))
P2G
B3, 16, 321
G2P
Interpaattoon
Interpobrion
Advection Module
Figure 3: Network architectures: The top diagram demonstrates the global architecture of our net- work with detailed information for tensor dimensionality and modular connectivity. The blue box is for particle states and the orange box indicates grid states. The dotted green box the module gen- erating the initial Lagrangian features. The dotted red box is for the functional module of advection (see the bottom diagram). The states are connected with multi-layer perceptrons (black arrows in the diagram). Each MLP has number of hidden layers with different number of neurons (spec- ified by the numbers within the parentheses). The bottom figure shows the details of the advection module updating the particle features by transferring data the grid and concatenating particles. Meanwhile, updates the particle positions with the generalized Hulerian forces calculated on the grid.
5.1 ABLATION EXPERIMENTS
Advection We turn off the advection module verify the its effectiveness for the final perfor- mance. We conducted the comparison on the ShapeNet data set (ii et al. 2016). The mloU reached 86.2% with the advection module in comparison to 85.3% without it, necessitating the role of the advection step.
Temporal resolution (Physical Intuition) The evolu- tion of a dynamic system can discretized on the tem- poral axis by the numerical integration with a number of steps. Given a fixed total time, the number timesteps is in an inverse ratio to the length of each step. For typ- ical explicit scheme (e.g., explicit Euler), small timestep leads to numerically secure result at the expense of performing more time integrations; while large timestep, although efficient, might explode out of the stable region.
Table 1: Temporal resolution
#
Acc 93.2 9S.2
948 94.7 95.1 95.1 95.2 95.
(Numerical Tests) Motivated by this numerical intuition, in- vestigated the effects of temporal resolution on our learning prob- lem. Specifically, garding both the learning accuracy and the evolved shape by sub- dividing the numerical integration into 0-8 steps (0 means in- tegration). The test was performed on ModelNet10. As shown in Table ! and Figure the learning accuracy stabilizes around 95% as the number of integration increases, with 2 steps and 4 steps as the maximum (95.4%) and minimum (94.7)), indicating a minor effect from the temporal resolution on learning accuracy. For the shape convergence, we demonstrated that different temporal resolutions converge to very similar final equilibrium states, despite of the different time step sizes. As shown Figure 5 the point- cloud model of an airplane is advected with different velocity fields generated on different temporal resolutions. The final shapes with timestep 2, 3, and all exhibit the same geometric feature separa- tions and topological relations. This result evidences our conjecture that all the temporal resolutions we used are within the stable region, motivating us to pick a larger time step size (total time/3 for most of our cases) for efficiency.
tested the performance of the network
Figure Temporal accuracy

---

time steps
time stops
time stegs
Figure 5: Visualization the advection of airplane is shown with time steps of and 6. Note that we rotate the point cloud and normalize the velocity field for visualization purposes.
Figure 6: Visualization segmentation. Examples different categories are depicted, consisting of initial shape, intermediary grouping, and final part prediction.
Table 2: Spatial resolution
Spatial resolution (Physical Intuition) For a typ-
8
16a
32
ical particle-grid simulation in CFD, the resolution of the grid and the number of particles are corre-
ModelNetl0 Ace (1024 pats)
94.4
95.4
95.1
ModelNet10 pats per cell
6.8
ShapeNet mloU (2048
85.4
86.1
86.2
lated. Making sure that each grid cell should contain
ShapeNel puis per cell
21.4
5.2
enough number of particles (c.g., 1-2 particles per
cell), ensures information exchange between these two discretizations accurate. Empirically, an overly refined grid will lead to inaccurate Eulerian convolution due to the large bulk of empty cells, while an overly coarse grid will dampen the motion of particles due artificial viscosity (e.g., see Evans & Harlow (1957); Brackbill et al. (197)), which makes the number of particles per cell ppc a key hyperparameter.
(Numerical Validation) We validate this grid-particle design art from scientific computing by testing our network with different grid resolutions. As shown Table we tested the grid resolution of 83, 163, and 32 two datasets with 1024 and 2048 particles separately. We observed that a 163 grid fits the 1024 dataset best and a 323 grid fits the 2048 dataset best. By calculating the average ppc for each case, we made preliminary conclusion that the optimal ppc is around 1.5-1.8. This also implies an optimal grid resolution for point-set with N particles to be (ppc N)1/3.
PIC/FLIP (Physical Intuition) Temporal smoothness is key for developing dynamic system to achieve its equi- librium state. PIC/FLIP obtains such smoothness by av- eraging weighted velocities between two adjacent time steps. (Numerical Validation) To highlight the role of this averaging, compared the accuracy between PIC/FLIP and PIC only (no temporal averaging) on ModelNet10. We can see from Figure 7 that the model with PIC/FLIP quickly stabilizes to high accuracy, outperforming the model with PIC only.
Accuracy
Time Steps Figure 7: PIC/FLIP VS PIC

---

5.2 APPLICATIONS
Table 3: Classification on ModelNet.
Classification We tested our network on ModelNet40 (Z. Wu, 2013) and ModelNet10 for classification. We
Method
Ingui
ModelNet 10 ModelNesd0
SO-Net
2048 pnts
94.1
90.9
use grid resolution 16 to train both the networks. As
PCNN
1024
94,9
92.3
PointNet
1024 pats
89.2
shown in Table B our result outperforms the state-of- art on ModeINet1O, noticeably surpassing those using grids. On ModelNet40, our result rivals DGCNN (92.8% v.s. 92.9%). But our parameter number is significantly smaller than DGCNN (1M v.s. 21M.)
PointGrid
1024
92.0
DGCNN
1024 pnts
92,9
PoinICNN 1024 pars
PointNel++ pnls,
91.9
SpiderCNN pats, nons
92.4
O-CNN
nors
91.0
86.5
VoxNet
grid (32")
92.0
83.0
Kd-Net kd-tree
94.0
91.8
Segmentation We tested algorithm for object part segmentation on ShapeNet (Yi et al.] 2016). We used grid resolution and 323 for training and testing. We
FPNN grid
87.5
MRCNN multi-level vox
91.3
86.2
Ours (16') 1024 pnis
95.4
92.8
showed the state-of-art performance of our approach in Table 4 Since the category of each input
object is known beforehand, we trained separate models for each category. Note that we only com- pared with point-based methods that had similar input (points or/and normals) as ours. It can be seen that we outperform all the state-of-art art with less parameters (1.1M, time steps) Some exam- ples animating the segmentation process can be seen in Figure 6
Table 4: Segmentation results on ShapeNet.
Method
input
mloU
acro bag
cap
car
chair
car phone
guitar
knife lamp laptop motor mug pistol rocket
board
table
PointNet
2k pots
83.7
78.7
82.5
74.9
89.6
73.0
91.3
85.9
80.8
95.3
65.2
93.0
81.2
57.9
72.8
80.6
PCNN
2k puts
85.1
82.4
85
79.5
90.8
86.0
95.7
73.2
83.3
51.0
75.0
81.8
Kd-Net
4k pats
82.3
80.1
74.6
74.3
70.3
88.6
73.5
90.2
87.2
81.0
94.9
57.4
86.7
78.1
51.8
69.9
80.3
DGCNN
2k pots
85.1
84.2
83,7
84.4
77.1
90.9
78.5
91.5
87.3
96.0
67.0
93.3
82.6
59.7
75.5
82.0
PoinCCNN
2k
86.1
84.1
86.4
86.0
80.8
90.6
79.7
92.3
88.4
85.3
96.1
77.2
95.2
84.2
64.2
80.0
82,9
PointNet++ pats, nors 85.1
82.4
79.0
87.7
77.3
90.
71.8
91.0
85.
83.7
95.3
71.6
94,1
813
58.7
76.4
82.6
SO-Net
pats, 84.9
77.3
94.8
83.0
SpiderCNN pnis, nors 85.3
83.5
81.0
87.2
77.5
90.7
76.8
91.1
87.3
83.3
95.8
70,2
93.5
82.7
59.7
75.8
82.8
SPLATTNe pats, img 85.4
84.3
80.3
90.7
87.1
96.3
75.6
81.8
64.0
75.5
81.8
Ours (32") 2k puts 86.2 83.8
96.4
83.4
6 DISCUSSION AND CONCLUSION
This paper presents a new perspective in treating the point cloud learning problem as dynamic advection problem using& a learned background velocity field. The key technical contribution of the proposed approach is to jointly define the point cloud learning problem as a flow advection problem in a world space using a static background grid and the local space using moving particles. Compared with the previous hybrid grid-point learning methods, e.g. two-way coupled particle-gee
schemes (Su et al. 2018 Atzmon et al. 2018 Liu et al., 2019), our approach solves the learning problem from dynamic system perspective which accumulates features in a flow field learned temporally. The coupled Lagraian-Lagranggian data structure in conjunction with its accommodated interpolation schemes provide an effective solution to tackle the challenges regarding both stencil construction and feature evolution by leveraging a numerical infrastructure that is matured in the scientific computing community. On another hand, our approach can be thought of as exploration in creating a new physical reservoir motivated by continuum mechanics in order to find alternative solutions for the conventional point cloud processing networks. Thanks to the cin-dimensional physical space and the large time step our network allows, our learning accuracy rivals the state-of- the-art deep networks such PointCNN (Li et al., 2018h) and DGCNN (Wang et al., 2019) while using significantly fewer network parameters (4% to 25% in our comparisons). Our future plan is scale the algorithm larger data and handle more complex point clouds with sparse and adaptive grid structures.
7 ACKNOWLEDGEMMEN
This project is support in part by Dartmouth Neukom Institute CompX Faculty Grant, Burke Re- search Initiation Award, and NSF MRI 1919647. Helen Lu Cao is supported by the Dartmouth Women in Science Project (WISP) and Undergraduate Advising and Research Program (UGAR).
8

---

REFERENCES
Panos Achlioptas, Olga Diamanti, Ioannis Mitliagkas, and Leonidas J. Guibas. Representation learn- ing and adversarial generation of 3d point clouds. CoRR, abs/1707.02330 2017.
I. Armeni, A. Sax, A. R. Zamir, and S. Savarese. Joint 2D-3D- D0-3D-Semmattic Data for Indoor Scene Understanding. ArXiv e-prints, February 2017.
Iro Armeni, Ozan Sener, Amir R. Zamir, Helen Jiang, Ioannis Brilakis. Martin Fischer, and Silvio Savarese. 3d semantic parsing of large-scale indoor spaces. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
Matan Atzmon, Haggai Maron, and Yaron Lipman. Point convolutional neural networks by exten- sion operators. CoRR, abs/1803 111111100091111 10091, 2018.
U. Brackbill, D. B. Kothe, and H. M. Ruppel (eds.). FLIP sosiidImplicit-Partiekee low- dissipation, particle-in-ced method for fluid flow, April 1987.
Michael M. Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. Geo- 2016.
metric deep learning: going beyond euclidean data. CoRR,
Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. Spectral networks and locally connected networks on graphs. CoRR, abs/1312.6203, 2013.
Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural ordinary dif- ferential equations. Advances in Neural Information Processing Systems, 2018.
Taco S. Cohen, Mario Geiger, Jonas Köhler, and Max Welling. Spherical cnns. abs/1801 101,, 2018.
CoRR,
Weinan E. A proposal machine learning via dynamical systems. Communications in Mathematics and Statistics, 5(1):1-11, Mar 2017. ISSN 2194-671X. doi: 10:10077/400344007--0000--
M.W. Evans and F.H. Harlow. The particle-in-cell method for hydrodynamic calculations. 6 1957.
Haoqiang Fan, Hao Su, and Leonidas J. Guibas. A point generation network for 3d object reconstruction from a single image. CoRR, abs/ 10120066.,,,,, 2016.
Ronald Fedkiw, Jos Stam, and Henrik Wann Jensen. Visual simulation of smoke. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH '01, pp. 15-22, New York, NY, USA, 2001. ACM. ISBN 1-58113-374-X, doi: 10. 10.1145/383259. 383260.
Yifan Feng, Zizhao Zhang, Xibin Zhao, Rongrong Ji, and Yue Gao. Gvenn: Group-view convolu- tional neural networks for 3d shape recognition. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
Sambit Ghadai, Xian Ycow Lee, Aditya Balu, Soumik Sarkar, and Adarsh Krishnamurthy, Multi- resolution 3d convolutional neural networks for object recognition. CoRR, aass/800. 12554, 2018. URL ptttp://arxivv org/ abs /805 12254
Frederic Gibou, David Hyde, and Ron Fedkiw. Sharp interface approaches and deep learning tech- niques for multiphase flows. Journal of Computational Physics, 380:442-463, 2019.
Thibault Groueix, Matthew Fisher, Vladimir G. Kim, Bryan Russell, and Mathieu Aubry. AtlasNet: A Papicr-Mäché Approach to Learning 3D Surface Generation. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2018.
Helmut Hauser, Auke Ijspeert, Rudolf M. Füchslin, Rolf Pfeifer, and Wolfgang Maass. The role of feedback in morphological computation with compliant bodies. Biological Cybernetics, 106 (10):595-613, Nov 2012. ISSN 1432-0770. doi: 10.10077440442--0---55666
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. arXiv preprint 15Xiv: 2015.

---

wttp://.rr..........orreeee..e.e Ping Hu, Bing Shuai, Jun Liu, and Gang Wang. Deep level sets for salient object detection. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 540-549, 2017.
Herbert Jaeger. The" echo state" approach to analysing and training recurrent neural networks-with an erratum note'. BoRn, Germany: German National Research Center for Information Technology GMD Technical Report, 148, 01 2001.
Herbert Jaeger. Adaptive nonlinear system identification with echo state networks. In NIPS, 2002.
Azarakhsh Jalalvand, Glenn Wallendael, and Rik Van de Walle. Real-time reservoir computing network-based systems for detection tasks on visual contents. pp. 146-151, 06 2015. doi: 10. (109/CICSSN.2015..3
Mingyang Jiang, Yiran Wu, and Cewu Lu. Pointsift: sift-like network module for 3d point cloud semantic segmentation. CoRR, htss/8807.00652. 2018.
Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional net- works. CoRR, 0bbs/1609.02990 2016.
Roman Klokov and Victor S. Lempitsky. Escape from cells: Deep kd- kd-networks for the recognition of 3d point cloud models. CoRR, abs/1704.01 222, 2017.
Truc Le and Ye Duan. Pointgrid: A deep network for 3d shape understanding. In The IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR), June 2018.
Yann Lecun, Leon Bottou, Y Bengio, and Patrick Haffner. Gradient-basee learning applied to doc- ument recognition. Proceedings of the IEEE, 86:2278 2324, 12 1998. doi: 10. 19965.7669.
Jiaxin Li, Ben M. Chen, and Gim Hee Lee. So-net: Self-organizing network for point cloud analysis. CoRR, abs/l 803.04249. 2018a.
Yangyan Li, Rui Bu, Mingchao Sun, Wei Wu, Xinhan Di, and Baoquan Chen. Pointenn: Convo- lution x-transformed points. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa- Bianchi, and R. Garnett (eds.), Advances Neural Information Processing Systems 31, pp. 820- 830. Curran Associates, Inc., 2018b.
Zhijian Liu, Haotian Tang, Yujun Lin, and Song Han. Point-voxel CNN for efficient 3d deep learn- ing. CoRR, abs/1 atts/190..0.. 00739933 2019.
Zichao Long, Yiping Lu, Xianzhong Ma, and Bin Dong. PDE-net: Learning PDEs from data. In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Leurning Reseurch, pp. 3208-3216, Stockholmsmässan, Stockholm Sweden, 10-15 Jul 2018. PMLR.
Ilya Loshchilov and Frank Hutter. Fixing weight decay regularization in adam. CoRR, abs/1711.05101, 2017. UtLhtttp://aaee /ttxpp:/aarxx..ooo org/ htse//bs//111 05101
Mantas Lukoševičius and Herbert Jaeger. Reservoir computing approaches recurrent neural network training. Computer Science Review, 3(3):127 149, 2009. ISSN 1574-0137. doi:
Wolfgang Maass, Thomas Natschläger, and Henry Markram. Real-time computing without stable states: A new framework for neural computation based perturbations. Neural Comput., 14 (11):2531-255 2560, November 2002. ISSN 0899-7667. doi: 10. 10.11200897660277407755.
D. Maturana and S. Scherer. VoxNet: A 3D Convolutional Neural Network for Real-Time Object Recognition. In IROS, 2015.
Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3d reconstruction in function space. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2019.
Joe J Monaghan. Smoothed particle hydrodynamics. Anmual review of astronomy and astrophysics, 30(1):543-574, 1992.
10

---

Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In The IEEE Conference Computer Vision and Pattern Recognition (CVPR), June 2019.
Gabriel Pereyra, George Tucker, Jan Chorowski, Eukasz Kaiser, and Geoffrey Hinton. Regularizing neural networks by penalizing confident output distributions. arXiv preprint arXiv: arXiv:/701.00 1701.06548, 2017.
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. arXiv preprint arXiv: 00093, 2016a.
Charles Ruizhongtai Qi, Hao Su, Matthias Nießner, Angela Dai, Mengyuan Yan, and Leonidas J. Guibas. Volumetric and multi-view cnns for object classification 3d data. 2016 IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR), pp. 5648-5656, 2016b.
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J. Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. CoRR, abs/1706.02413, 2017.
Gemot Riegler, Ali Osman Ulusoy, and Andreas Geiger. Octnet: Learning deep 3d representations at high resolutions. CoRR, htts/1611.05009 2016.
Lars Ruthotto and Eldad Haber. Deep neural networks motivated by partial differential equations. CoRR, abs/1804.045 2018.
Hang Su, Subhransu Maji, Evangelos Kalogerakis, and Erik G. Meilled Multi-view convo- lutional neural networks for 3d shape recognition. In Proc. ICCV, 2015.
Hang Su, Varun Jampani, Deqing Sun, Subhransu Maji, Evangelos Kalogerakis, Ming-Hsuan Yang, and Jan Kautz. Splatnet: Sparse lattice networks for point cloud processing. CoRR, hts//882.08275. 2018
Gouhei Tanaka, Toshiyuki Yamane, Jean Benoit Héroux, Ryosho Nakane, Naoki Kanazawa, Seiji Takeda, Hidetoshi Numata, Daiju Nakano, and Akira Hirose. Recent advances in physical reser- voir computing: A review. Neural Networks, 115:100 - 123, 2019, ISSN 0893-6080, doi: https://doi.or//10.. 1016/.0111...... new.et/..u
Lyne P. Tchapmi, Christopher B. Choy, Iro Armeni, JunYonng Gwak, and Silvio Savarese. Segeloud: Semantic segmentation of 3d point clouds. CoRR, abs/1710.07563, 2017.
Peng-Shun Wang, Yang Liu, Yu-Xiao Guo, Chun-Yu Sun, and Xin Tong. O-CNN: Octree-based Convolutional Neural Networks for 3D Shape Analysis. ACM Transactions Graphics (SIG- GRAPH), 36(4), 2017.
Peng-Shuai Wang, Chun-Ya Yu Sun, Yang Liu, and Xin Tong. Adaptive o-emn: A patch-based deep representation of 3d shapes. ACM Trans. Graph., 37(6)211:1-217:11, December 2018. ISSN 0730-0301. doi: 10.1 10.11422222277.32550000
Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E. Sarma, Michael M. Bronstein, and Justin M. Solomon. Dynamic graph cnn for learning on point clouds. ACM Transactions on Graphics (TOG), 2019.
Yifan Xu, Tianqi Fan, Mingye Xu, Long Zeng, and Yu Qiao. Spiderenn: Deep learning on point sets with parameterized convolutional filters. CoRR, abs/1803. 1152, 2018.
Yaoqing Yang, Chen Feng, Yiru Shen, and Dong Tian. Foldingnet: Interpretable unsupervised learning on 3d point clouds. CORR, abs/1712.07262, 2017.
Li Yi, Vladimir G. Kim, Duygu Ceylan, I-Chao Shen, Mengyan Yan, Hao Su, Cewu Lu, Qixing Huang, Alla Sheffer, and Leonidas Guibas. A scalable active framework for region annotation in 3d shape collections. SIGGRAPH Asia, 2016.
Lequan Yu, Xianzhi Li, Chi ChigWing Fu, Daniel Cohen-Or, and Pheng- Heng. Pu-net: Point cloud upsampling network. CoRR, abs/1801 801 .06761,, 2018.

---

A. Khosla F. Yu L. Zhang X. Tang J. Xiao Z. Wu, S. Song. 3d shapenets: A deep representation for volumetric shapes. In Computer Vision and Pattern Recognition, 2015.
Yongheng Zhao, Tolga Birdal, Haowen Deng, and Federico Tombari. 3d point capsule networks. In Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
12

---

A PERFORMANCE ON S3DIS
In this part, we further discuss our algorithm and its performance on the large-scale S3DIS dataset (Armeni al. 2017). Unlike ModelNet and ShapeNet, the S3DIS consists of colored point clouds collected from real world. We train on the area 1,2,3,4,6 and test on the area 5. We make some modifications on our network structure better fir this dataset.
1. We set the number of time steps 4 instead of 2.
2. To allow large number of time steps (deeper networks), we replace our original MLP with the ResNet blocks (He et al. 2015).
3. We use more MLPs to encode the initial features on each point.
4. We scale the point cloud into the space [-0.9, 0.9]a because the data contains many points on the border plane, such as ceiling and floor.
The performance on the S3DIS in the table
Table Segmentation results on S3DIS.
Method
mloU ceiling floor wall beam column window door table chair sofa
bookcase
board clutter
PointNet 41.09 88,80 97.33 69.80 0,05
3.92
46.26
10.76 58,93 52.61 5.85
40.28
26.38 33.22
SPGraph
58.04 89.35 96.87 78.12 0.00
42.81
48.93
61.58 84.66 75,41 69.84
52.60
2.10 52.22
SegCloud
48.92 90.06 96.05 69.86 0.00
18.37
38.35
23.12 70.40 75.89 40.88
58.42
12.96 41.60
PCCN
58.27
92.26 96.20 75.89 0.27
5.98
69.49
63.45
66.87 65.63 47.28
68.91
59.10
46.22
PointCNN
57.26
92.31 98.24 79.41 0.00
17.60
22.77
62.09
74.39 80.59 31.67
66.67
62.05
56.74
Ours
54.37 92.41 98.28 74.88 0.68
18.74
45.36
48.70
74.11 78.50 35.60
57.02
38.84 43.73
From the table we see that the result obtained by AdvectiveNet is comparable the states of the art (with the highest mloU in ceiling, floor, and beam), though it is less impressive to the performance on ModelNet and ShapeNet. We observe that, in the S3DIS, the relative positions of different parts are more flexible and less structured compared to ModelNet and ShapeNet. For example, ShapeNet, the wings of the airplanes are always on the two sides of the fuselages. Hence, we would interpret our performance on ShapeNet thanks to the ability in detecting intrinsic structures underlying the relative positions of the parts. The tendency to focus on relative positions also explains why our algorithm outperforms the states of the art on detecting ceiling and floor (they are always on the two sides of the rooms).
B IMPLEMENTATION DETAILS
We follow the data augmentation methods in et et al. 2018b). We use dropout ratio 0.3 on the last fully connected layer before class score prediction. The decay rate for batch normalization starts with 0.5 and gradually decreased to 0.01. We use adamw optimizer (Loshchilov & Hutter 2017) with initial learning rate 0.001, weight decay rate 0.005, momentum 0.9 and batch size 32. The learning rate is multiplied by 0.8 every 20 epochs. We train the model for 200 epochs. We use the label smoothing techique (Pereyra al. 2017) with confidence 0.8. We use the grid size 16 and 32 for classification and segmentation, respectively.
13


Reviews:
[{"review_id": "ryxSuiIjFB", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper presents a method for point-based learning that is inspired by a hybrid Eulerian-Lagrangian fluid simulation method. The work first explains how the simulation algorithm is mapped to the learning problem: MLPs are employed to learn sets of particle based features which are mapped to a Eulerian grid. A second MLP infers a particle based velocity, which is likewise mapped to the grid and used to advect the grid quantities. This is repeated for a certain number of steps to obtain final positions. The \"warped\" features are then projected back onto the particles to solve, e.g., a classification task. In contrast to a typical flow solver, the motion can be divergent, i.e., not necessarily conserves volume.\n\nThe paper presents a brief ablation study for number of iterated steps, grid size and point count, before presentation two comparisons with existing baselines.\n\nOverall, I found the idea to employ FLIP for Lagrangian learning tasks novel and very interesting. Unfortunately, the paper (as mentioned in the text) only contains only a somewhat preliminary study. The method does not yield clear gains over previous work, but rather a similar performance for classification and segmentation of ShapeNet and S3DIS data is shown. Given the fairly complicated construction, I think it would be important to actually show improvements at least for specific learning tasks. Several of the deformations shown in figure 5 and 6 are also not really intuitive\n\nAlso, on second sight, I don't fully understand the motivation for employing and learning a grid based deformation. The grids seem to inherently limit the spatial extent of the point clouds, and the features that can be resolved. Features smaller than a grid cell will essentially \"stick together\", and can't be separated. It's also not obvious how to choose parameters such as the number of time steps. Intuitively, I'd expect the method to \"converge\" to a position for a larger number of steps.\n\nTo conclude, the direction this paper takes is certaily new and interesting, but the preliminary results in combination with the complexity and limitations introduced by the grid-based representation make me hesitant to recommend accepting this paper in its current form. (The nine pages also contribute to this assessment.)\n", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "SJghlklntH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper is about using classical PIC/FLIP scheme in Computational Fluid Dynamics for solving the learning problem of 3D object detection and segmentation. In general, there are extrinsic CNNs like the Vox net etc. which look for global features which the authors refer to as Eulerian formulation of the data representation, and there are intrinsic CNNs like the GCN(graph convolutions), Point nets etc. which look for localized neighborhood information which the authors refer to as Lagrangian formulation. The authors acknowledge that hybridizing the extrinsic CNNs and intrinsic CNNs is not new and several works are cited. The key contribution is to look at this problem from the perspective of PIC/FLIP scheme which has been used in CFD for decades. \n\nThe idea is very nice, well describes and quite novel in my opinion. I really liked the adoption of classical CFD approaches in learning. This provides a very interesting perspective to 3D deep learning. \n\nHowever, the papers struggles to demonstrate why the 3D deep learning community would adopt this approach. The results are not that conclusive. The algorithm works (understood well from the ablation study). However, the performance of the proposed approach is at best comparable to some of the state-of-the-art methods such as PointCNN or SE-Net. The authors need to clarify what potential advantages could there be other than accuracy (if any) such that the community uses the proposed method. \n\nAlso, the grids used in the study are too low to make any conclusive remarks on what happens when dealing with higher resolutions of grid. The authors themselves acknowledge the limitation of not being able to go higher in resolution of grid. Interestingly, such limitations of CFD has recently motivated the community to explore deep learning based fast and agile surrogates for computationally tractable approaches.  \n\nSome of the new works in 3D object recognition and segmentation such as  Deep SDF(https://arxiv.org/abs/1901.05103), AtlasNet(https://arxiv.org/abs/1802.05384),  Deep Level Sets (https://arxiv.org/abs/1901.06802), occupancy networks (https://arxiv.org/pdf/1812.03828v1.pdf), http://openaccess.thecvf.com/content_cvpr_2018/html/Yu_PU-Net_Point_Cloud_CVPR_2018_paper.html, Adaptive O-CNN (https://dl.acm.org/citation.cfm?id=3275050), https://arxiv.org/abs/1805.12254, 3D Point Capsule Networks, http://t.cvlibs.net/publications/Niemeyer2019ICCV.pdf etc. can be compared with or at least contrasted in the related works.\n\nIn summary, I really liked the algorithmic idea, but skeptical about its practical relevance from the results.", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "HJgxfRhEqS", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper addresses the task of learning with point clouds for semantic labeling (classification and segmentation). The authors propose a novel point-based architecture based on viewing the learning process as an advection in the 3D space. This formulation aims at an explicit connection the two formulations for learning with point clouds, the first being focused on points (Lagrangian formulation), the second on the regular spatial grid not necessarily coinciding with points (Eulerian formulation). While the connection between the two formulation is known in the literature, the paper does a good overview of the relevant work and highlights the interplay between the two treatments for learning, which is valuable to the reader. The proposed view of learning with point clouds is, as far as I know, novel.\n\nWith the proposed learnable operations, the authors are able to efficiently learn the functions defined in 3D space, such as the semantic class labels. The operations include transferring the features between the grid and the point cloud, advection, and interpolation, all implemented in a unified learnable model. \n\nThe architecture is evaluated on classification and segmentation tasks with common datasets, where it performs on par with existing methods. While the experimental evaluation does not indicate that the proposed method is a new state-of-the-art, it convincingly validates that the proposed method is capable of learning powerful enough representations. \n\nI believe the paper should be accepted for publication, as (1) the proposed method is generally novel while it bases on solid and well-known foundations, (2) the experimental validation is sufficient to demonstrate the capabilities of the approach. ", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper treats the task of point cloud learning as a dynamic advection problem in conjunction with a learned background velocity field.  The resulting system, which bridges geometric machine learning and physical simulation, achieves promising performance on various classification and segmentation problems.  Although the initial scores were mixed, all reviewers converged to acceptance after the rebuttal period.  For example, a better network architecture, along with an improved interpolation stencil and initialization, lead to better performance (now rivaling the state-of-the-art) as compared to the original submission.  This helps to mitigate an initial reviewer concern in terms of competitiveness with existing methods like PointCNN or SE-Net.  Likewise, interesting new experiments such as PIC vs. FLIP were included.
Decision: Accept (Poster)