Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: The paper addresses the task of learning with point clouds for semantic labeling (classification and segmentation). The authors propose a novel point-based architecture based on viewing the learning process as an advection in the 3D space. This formulation aims at an explicit connection the two formulations for learning with point clouds, the first being focused on points (Lagrangian formulation), the second on the regular spatial grid not necessarily coinciding with points (Eulerian formulation). While the connection between the two formulation is known in the literature, the paper does a good overview of the relevant work and highlights the interplay between the two treatments for learning, which is valuable to the reader. The proposed view of learning with point clouds is, as far as I know, novel.

With the proposed learnable operations, the authors are able to efficiently learn the functions defined in 3D space, such as the semantic class labels. The operations include transferring the features between the grid and the point cloud, advection, and interpolation, all implemented in a unified learnable model. 

The architecture is evaluated on classification and segmentation tasks with common datasets, where it performs on par with existing methods. While the experimental evaluation does not indicate that the proposed method is a new state-of-the-art, it convincingly validates that the proposed method is capable of learning powerful enough representations. 

I believe the paper should be accepted for publication, as (1) the proposed method is generally novel while it bases on solid and well-known foundations, (2) the experimental validation is sufficient to demonstrate the capabilities of the approach. 
review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: The paper is about using classical PIC/FLIP scheme in Computational Fluid Dynamics for solving the learning problem of 3D object detection and segmentation. In general, there are extrinsic CNNs like the Vox net etc. which look for global features which the authors refer to as Eulerian formulation of the data representation, and there are intrinsic CNNs like the GCN(graph convolutions), Point nets etc. which look for localized neighborhood information which the authors refer to as Lagrangian formulation. The authors acknowledge that hybridizing the extrinsic CNNs and intrinsic CNNs is not new and several works are cited. The key contribution is to look at this problem from the perspective of PIC/FLIP scheme which has been used in CFD for decades. 

The idea is very nice, well describes and quite novel in my opinion. I really liked the adoption of classical CFD approaches in learning. This provides a very interesting perspective to 3D deep learning. 

However, the papers struggles to demonstrate why the 3D deep learning community would adopt this approach. The results are not that conclusive. The algorithm works (understood well from the ablation study). However, the performance of the proposed approach is at best comparable to some of the state-of-the-art methods such as PointCNN or SE-Net. The authors need to clarify what potential advantages could there be other than accuracy (if any) such that the community uses the proposed method. 

Also, the grids used in the study are too low to make any conclusive remarks on what happens when dealing with higher resolutions of grid. The authors themselves acknowledge the limitation of not being able to go higher in resolution of grid. Interestingly, such limitations of CFD has recently motivated the community to explore deep learning based fast and agile surrogates for computationally tractable approaches.  

Some of the new works in 3D object recognition and segmentation such as  Deep SDF(https://arxiv.org/abs/1901.05103), AtlasNet(https://arxiv.org/abs/1802.05384),  Deep Level Sets (https://arxiv.org/abs/1901.06802), occupancy networks (https://arxiv.org/pdf/1812.03828v1.pdf), http://openaccess.thecvf.com/content_cvpr_2018/html/Yu_PU-Net_Point_Cloud_CVPR_2018_paper.html, Adaptive O-CNN (https://dl.acm.org/citation.cfm?id=3275050), https://arxiv.org/abs/1805.12254, 3D Point Capsule Networks, http://t.cvlibs.net/publications/Niemeyer2019ICCV.pdf etc. can be compared with or at least contrasted in the related works.

In summary, I really liked the algorithmic idea, but skeptical about its practical relevance from the results.
review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: The paper presents a method for point-based learning that is inspired by a hybrid Eulerian-Lagrangian fluid simulation method. The work first explains how the simulation algorithm is mapped to the learning problem: MLPs are employed to learn sets of particle based features which are mapped to a Eulerian grid. A second MLP infers a particle based velocity, which is likewise mapped to the grid and used to advect the grid quantities. This is repeated for a certain number of steps to obtain final positions. The "warped" features are then projected back onto the particles to solve, e.g., a classification task. In contrast to a typical flow solver, the motion can be divergent, i.e., not necessarily conserves volume.

The paper presents a brief ablation study for number of iterated steps, grid size and point count, before presentation two comparisons with existing baselines.

Overall, I found the idea to employ FLIP for Lagrangian learning tasks novel and very interesting. Unfortunately, the paper (as mentioned in the text) only contains only a somewhat preliminary study. The method does not yield clear gains over previous work, but rather a similar performance for classification and segmentation of ShapeNet and S3DIS data is shown. Given the fairly complicated construction, I think it would be important to actually show improvements at least for specific learning tasks. Several of the deformations shown in figure 5 and 6 are also not really intuitive

Also, on second sight, I don't fully understand the motivation for employing and learning a grid based deformation. The grids seem to inherently limit the spatial extent of the point clouds, and the features that can be resolved. Features smaller than a grid cell will essentially "stick together", and can't be separated. It's also not obvious how to choose parameters such as the number of time steps. Intuitively, I'd expect the method to "converge" to a position for a larger number of steps.

To conclude, the direction this paper takes is certaily new and interesting, but the preliminary results in combination with the complexity and limitations introduced by the grid-based representation make me hesitant to recommend accepting this paper in its current form. (The nine pages also contribute to this assessment.)

review_title: Official Blind Review #2

Meta-review: This paper treats the task of point cloud learning as a dynamic advection problem in conjunction with a learned background velocity field.  The resulting system, which bridges geometric machine learning and physical simulation, achieves promising performance on various classification and segmentation problems.  Although the initial scores were mixed, all reviewers converged to acceptance after the rebuttal period.  For example, a better network architecture, along with an improved interpolation stencil and initialization, lead to better performance (now rivaling the state-of-the-art) as compared to the original submission.  This helps to mitigate an initial reviewer concern in terms of competitiveness with existing methods like PointCNN or SE-Net.  Likewise, interesting new experiments such as PIC vs. FLIP were included.
Decision: Accept (Poster)