Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework
Authors: Xu Ma
Can Qin
Haoxuan You
Haoxi Ran
Yun Fu
Venue: ICLR 2022 Poster (2022)
Keywords: point cloud representation
local relation
mlp

Abstract: Point cloud analysis is challenging due to irregularity and unordered data structure. To capture the 3D geometries, prior works mainly rely on exploring sophisticated local geometric extractors, using convolution, graph, or attention mechanisms. These methods, however, incur unfavorable latency during inference and the performance saturates over the past few years. In this paper, we present an ovel perspective on this task. We find detailed local geometrical informationprobably is not the key to point cloud analysis – we introduce a pure residual MLP network, called PointMLP, which integrates no local geometrical extractors but still performs very competitively. Equipped with a proposed lightweight geometric-affine module to stabilize the training, PointMLP delivers the new state-of-the-art on multiple datasets. On the real-world ScanObjectNN dataset, our method even surpasses the prior best method by 3.3% accuracy. We emphasize PointMLP achieves this strong performance without any sophisticated operations, hence leading to a prominent inference speed. Compared to most recent CurveNet, PointMLP trains 2× faster, tests 7× faster, and is more accurate on ModelNet40 benchmark. We hope our PointMLP may help the community towards a better understanding of point cloud analysis. The code is available at https://github.com/ma-xu/pointMLP-pytorch.

Published as a conference paper at ICLR 2022
RETHINKING NETWORK DESIGN AND LOCAL GEOM- ETRY IN POINT CLOUD: A SIMPLE RESIDUAL MLP FRAMEWORK
Xu Ma', Can Qin', Haoxuan You', Haoxi Ran', Yun Fu!
Northeesterr University, Boston, MA, USA
CColumbia University, New York, NY, USA
(ma.xul, gin. ca, ran.h/ wa..eeenootteeeeeeeeeeee. www.hh/oortheateee.eeeeeee (haoxuanyou, ranhaoox eegmmml..coom yunfuêece
www.eeeeeeeu.duu neu.edu
ABSTRACT
Point cloud analysis challenging due to irregularity and unordered data struc- ture. To capture the 3D) geometries, prior works mainly rely on exploring sophis- ticated local geometric extractors using convolution, graph, or attention mecha- nisms. These methods, however, incur unfavorable latency during inference, and the performance saturates over the past few years. In this paper, we present a novel perspective this task. We notice that detailed local geometrical infor- mation probably is not the key to point cloud analysis we introduce a pure residual MLP network, called PointMLP, which integrates no "sophisticated" lo- cal geometrical extractors but still performs very competitively. Equipped with a proposed lightweight geometric affine module, PointMLP delivers the new state- of-the-art on multiple datasets. On the real-world ScanObjectNN dataset, our method even surpasses the prior best method by 3.3% accuracy. We emphasize that PointMLP achieves this strong performance without any sophisticated oper- ations, hence leading to a superior inference speed. Compared to most recent CurveNet, PointMLP trains 2 faster, tests 7x faster, and is more accurate on ModelNet40 benchmark. We hope our PointMLP may help the community to- wards better understanding of point cloud analysis. The code is available at https::/github..oom http///maaxuupppuuie wwtpppoiinM-PPttttottt
INTRODUCTION
Lately, point cloud analysis has emerged as a popular topic in 3D understanding, attracting attention from academia and industry (Qi et al., 2017a; Shi et al., 2019; Xu et al., 2020). Different from 2D images represented by regular dense pixels, point clouds are composed unordered and irregular sets of points P Nx making it infeasible to apply image processing methods point cloud analysis directly. Meanwhile, the nature of sparseness and the presence of noises further restrict the performance. In the past few years, endowing with neural networks, point cloud analysis has seen a great improvement in various applications, including 3D shape classification (Qi et al., 2017a), semantic segmentation (Hu al., 2020) and object detection (Shi & Rajkumar, 2020), etc.
Recent efforts have shown promising results for point cloud analysis by exploring local geometric in- formation, using convolution (Li et al., 2021a), graph (Li et al., 202 or attention mechanism (Guo et al., 2021) (see Section 2 for details). These methods, despite their gratifying results, have mainly relied on the premise that an elaborate local extractor is essential for point cloud analysis, leading to the competition for careful designs that explore fine local geometric properties. Nevertheless, sophisticated extractors are not without drawbacks. On the one hand, due to prohibitive computa- tions and the overhead of memory access, these sophisticated extractors hamper the efficiency of applications in natural scenes. As an example, until now, most 3D point cloud applications are still based the simple PointNet and PointNet++) the noxel-based methods (Liu et al., 2021; Li et al., 202 lb; Zhang et al., 2021). However, applications that employ the aforementioned advanced methods are in literature. On the other hand, the booming sophisticated extractors saturate the

---

Published as a conference paper at ICLR 2022
performance since they already describe the local geometric properties well. A more complicated design is no longer to improve the performance further. These phenomena suggest that we may need to stop the race of local feature extraction designing, rethinking the necessity of claborate local feature extractors and further revisiting the succinet design philosophy in point cloud analysis.
In this paper, aim at the ambitious goal of build- ing a deep network for point cloud analysis using only residual feed-forward MLPs, without any del- icate local feature explorations. By doing so, we eschew the prohibitive computations and continued memory access caused by the sophisticated local geometric extractors and enjoy the advantage of ef- ficiency from the highly-optimized MLPs. To fur- ther improve the performance and generalization ability, We introduce a lightweight local geometric affine module that adaptively transforms the point feature in a local region. We term our new network architecture as PointMLP. In the sense of MLP- based design philosophy, our PointMLP is similar to PointNet and PointNet++ (Qi et al., 201 70;b). However, our model more generic and exhibits promising performance. Different from the models with sophisticated local geometric extractors (e.g.. DeepGCNs (Li et al., 2019), RSCNN (Liu et al.,
MLi
94.3
CurveNet
PointMLP- elite
Overall accuracy
(2021)
GDANet (2021)
GBNet (2021)
93.8
93.3
KPConv (2019)
92.8
PointConv (2019)
92.3
0
30
60
90
120
150
180
Inference speed (samples/second)
Figure 1: Accuracy-speed tradeoff on Model- Net40. Our PointMLP performs best. Please refer to Section 4 for details.
2019b), RPNet (Ran et al., 2021).), our PointMLP is conceptually simpler and achieves results on par or even better than these state-of-the-art methods (see Figure 1). Keep in mind that we did not challenge the advantages of these local geometric extractors and we acknowledge their contribu- tions; however, more succinet framework should be studied considering both the efficiency and accuracy. In Table I, we systemically compare our PointMLP with some representative methods.
Even though the design philosophy is simple, PointMLP (as well as the elite version) exhibits su- perior performance on 3D point cloud analysis. Specifically, we achieve the state-of-the-ar classi- fication performance, 94.5% %, on the ModelNet40 benchmark, and we outperform related works by 3.3% % accuracy on the real-world ScanObjectNN dataset, with a significantly higher inference speed.
2 RELATED WORK
Point cloud analysis. There are mainly two streams to process point cloud. Since the point cloud data structure is irregular and unordered, some works consider projecting the original point clouds to intermediate voxels (Maturana & Scherer, 2015; Shi et al., 2020) images (You et al., 2018; Li et al., 2020), translating the challenging 3D task into a well-explored 2D image problem. In this regime, point clouds understanding is largely boosted and enjoys the fast processing speed from 2D images or voxels. Albeit efficient, information loss caused by projection degrades the representational quality of details for point clouds (Yang et al., 2019). To this end, some methods are proposed to process the original point cloud sets directly. PointNet (Qi et al., 2017a) is a pioneering work that directly consumes unordered point sets as inputs using shared MLPs. Based on PointNet, PointNet++ (Qi et al., 2017b) further introduced a hierarchical feature learning paradigm to capture the local geometric structures recursively. Owing to the local point representation (and multi-scale information), PointNet++ exhibits promising results and has been the comerstone of modern point cloud methods (Wang et al., 2019; Fan et al., 2021; Xu et al., 2021a). Our PointMLP also follows the design philosophy of PointNet+ but explores a simpler yet much deeper network architecture.
Local geometry exploration. As PointNet++ built the generic point cloud analysis network framework, the recent research focus is shifted to how to generate better regional points repre- sentation. Predominantly, the explorations of local points representation can be divided into three categories: convolution-, graph-, and attention- based methods. One of the most distinguished convolution-based methods is PointConv (Wu et al., 2019). By approximating continuous weight and density functions in convolutional filters using an MLP, PointConv is able to extend the dynamic filter to new convolution operation. Also, PAConv (Xu et al., 2021a) constructs the convolution
2

---

Published as a conference paper at ICLR 2022
kernel by dynamically assembling basic weight matrices stored in a weight bank. Without modifying network configurations, PAConv can be seamlessly integrated into classical -bsPdd pipelines. Unlike convolution-basee methods, Graph- based methods investigate mutually correlated relation- ships among points with graph. In Wang et al. (2019), an EdgeConv is proposed to generate edge features that describe the relationships between point and its neighbors. By doing so, local graph is built, and the point relationships are well preserved. In 3D-GCN (Lin et al., 2021), authors aim at deriving deformable 3D kernels using a 3D Graph Convolution Network. Closely related to graph- based methods, the attention- based methods exhibit excellent ability on relationship exploration as well, like PCT (Guo et al., 2021) and Point Transformer (Zhao et al., 2021; Engel al., 2020). With the development of local geometry exploration, the performances on various tasks appear to this paper, we show-
be saturated. Continuing on this track would bring minimal improvements. case that even without the carefully designed operations for local geometry exploration, pure deep hierarchical MLP architecture is able to exhibit gratifying performances and even better results.
Deep network architecture for point cloud. Interestingly, the development of point cloud analysis is closely related to the evolution of the image process- ing network. In the early era, works in the image processing field simply stack several learning layers to probe the per- formance limitations (Krizhevsky et al., 2012; Simonyan & Zisserman, 2015; Dong et al., 2014). Then, the great success of deep learning was signif- icantly promoted by deep neural ar- chitectures like ResNet (He et al., 2016), which brings profound impact to various research fields. Recently, attention- based models, including atten-
Table I: Systematic comparison among some representa- tive methods. "Deep" indicates that a model is expand- able along depth. "Opt." stands for the principal operator.
Method
hierarchy
locality
deep
opt.
PointNet
MLP
PointNet++
MLP
DGCNN
X
/
X
GCN
DeepGCNs
V
GCN
PointConv
Conv.
Point Trans.
V
Atten.
PointMIP
/
MLP
tion blocks (Wang al., 2018) and Transformer architectures (Dosovitskiy al., 2021), further flesh out the community. Most recently, the succinet deep MLP architectures have attracted a lot of atten- tion due to their efficiency and generality. Point cloud analysis follows the same develop history as well, from MLP--asee PointNet (Qi et al., 2017a), deep hierarchical PointNet++ (Qi et al., 2017b), convolution//rrppp/rreaaiio based methods (Wu et al., 2019; Wang et al., 2019; Ran et al., 2021), to state-of-the-an -hrtra Transformer-basedd models (Guo et al., 2021; Zhao et al., 2021). In this paper, we abandon sophisticated details and present a simple yet effective deep residual MLP network for point cloud analysis. Instead of following the tendency in the vision community deliberately, we in pursuit of an inherently simple and empirically powerful architecture for point cloud analysis.
DEEP RESIDUAL MLP FOR POINT CLOUD
We propose to learn the point cloud representation by a simple feed-forward residual MLP network (named PointMLP), which hierarchically aggregates the local features extracted by MLPs, and aban- dons the use of delicate local geometric extractors. To further improve the robustness and improve the performance, we also introduce a lightweight geometric affine module to transform the local points to a normal distribution. The detailed framework of our method is illustrated in Figure 2.
3.1 REVISITING POINT-BASED METHODS
The design of baset-based methods for point cloud analysis dates back to the PointNet and Point- Net++ papers (Qi et al., 2017a;b), if not carlier. The motivation behind this direction is to directly consume point clouds from the beginning and avoid unnecessary rendering processes.
Given a set of points P = {p.lt = 1..... .N E R Nx3, where indicates the number of points in (z, y, 22 Cartesian space, baset methods aims to directly learn the underlying representation ff of P using neural networks. One of the most pioneering works is PointNet++. which learss hierarchical features by stacking multiple learning stages. In each stage, N., points re-sampled by
3

---

Published as a conference paper at ICLR 2022
N
ResP Block
ResP Block
ResP Block
ResP Block
ResP Black
Geometric Affine
pre
pos
a B
Repeat
MLP
BN
ReUU
MLP
Product
Summation
Geometric Affine Module
Residual Point (ResP) Block
Figure 2: Overview of one stage in PointMLP. Given an input point cloud, PointMLP progressively extracts local features using residual point MLP blocks. In each stage, we first transform the local points using geometric affine module, then they are extracted before and after the aggregation operation, respectively. PointMLP progressively enlarges the receptive field and models complete point cloud geometric information by repeating multiple stages.
the farthest point sampling (FPS) algorithm where indexes the stage and K neighbors are employed for each sampled point and aggregated by max-pooling to capture local structures. Conceptually, the kernel operation PointNet++ can be formulated as:
9i A ( (fr,) = 1,. K),
(1)
where (.) means aggregation function (max-pooling in PointNet++), (-) denotes the local fea- ture extraction function (MLP in PointNet++), and Ji.j is the j-th neighbor point feature of i-th sampled point. By doing so, PointNet++ able to effectively capture local geometric information and progressively enlarge the receptive fields by repeating the operation.
In the sense of network architecture design, PointNet++ exhibits a universal pipeline for point cloud analysis. Following this pipeline, some plug-and-play methods have been proposed, mainly focus- ing on the local feature extractor Φ (-) (Xu et al., 2021a; Liu et al., 2019b; Thomas et al., 2019; Zhao et al., 2021). Generally, these local feature extractors thoroughly explore the local geometric information using convolution, graph, or self- attention mechanisms. In RSCNN (Liu et al., 2019b), extractor mainly achieved by exploring point relations as follow:
(fi.)) = MLP ([I - zillg Fzgj 24, Ti.j, x;]) fi E (L,..,,K),
where [-] is the concatenation operation and MLP is a small network composed of a Fully-comnected (FC) layer, Batch Normalization layer, and activation function. Unlike RSCNN, Point Transformer introduces the self-atten atffnattention mechanism into point cloud analysis and considers the similarities be- tween pair-wise points local region. To this end, re-formulates extractor as:
+ (fi) = - (ft)) + @ (a (fr. + +));
(3)
where 7: s2, and a are linear mapping function, "0" is a Hadamard product, and p soft- max normalization. In particular, Point Transformer introduces relative position encoding, = 0 (I; x, where the relative position encoded by two FC layers with a ReLU non- linearity layer, into both attention weights and features. The lightweight positional encoder largely improves the performance of Point Transformer.
While these methods easily take the advantage of detailed local geometric information and usu- ally exhibit promising results, two issues limit their development. First, with the introduction of delicate extractors, the computational complexity is largely increased, leading to prohibitive infer-
4

---

Published as a conference paper at ICLR 2022
ence latency !. For example, the FLOPs of Equation 3 in Point Transformer would be 14Kd², ignoring the summation and subtraction operations. Compared with the conventional FC layer that enjoys 2f FLOPs, it increases the computations by times. Notice that the memory access cost is not considered yet. Second, with the development local feature extractors, the performance gain has started to saturate popular benchmarks. Moreover, empirical analysis in Liu et al. (2020) veals that most sophisticated local extractors make surprisingly similar contributions to the network performance under the same network input. Both limitations encourage us develop new method that circumvents the employment of sophisticated local extractors, and provides gratifying results.
3.2 FRAMEWORK OF POINTMLP
In order to get rid of the restrictions mentioned above, we present a simple yet effective MLP-baaed network for point cloud analysis that no sophisticated or heavy operations are introduced. The key operation of our PointMLP can be formulated as:
= pas (A (f
11...........
(4)
where Pree (-) and & pos (-) are residual point MLP blocks: the shared pre (.) is designed to learn shared weights from a local region while the Ppoe (-) leveraged to extract deep aggregated fea- tures. In detail, the mapping function can be written as a series of homogeneous residual MLP blocks, MLP (x) + v, in which MLP is combined by FC, normalization and activation layers (re- peated two times). Following Qi et al. (2017a), max-pooling operation. Equation 4 describes one stage of of PointMLP. For hierarchical and deep network, we recursively repeat the operation by stages. Albeit the framework of PointMLP is succinct, it exhibits some prominent merits. 1) Since PointMLP only leverages MLPs, it is naturally invariant to permutation, which perfectly fits the characteristic of point cloud. 2) By incorporat- ing residual connections, PointMLP can be easily extended to dozens layers, resulting deep feature representations. 3) In addition, since there is no sophisticated extractors included and the main op- eration is only highly optimized feed-forward MLPs, even we introduce more layers, our PointMLP still performs efficiently. Unless explicitly stated, the networks in our experiments use four stages, and two residual blocks in both pre (.) and + pas (-). We employ k-nearest neighbors algorithm (kNN) to select the neighbors and set the number K to 24.
consider the aggregation function A as
3.3 GEOMETRIC AFFINE MODULE
While it may be easy to simply increase the depth by considering more stages stacking more blocks in @ pre and pass we notice that simple deep MI.P structure will decrease the accuracy and stability, making the model less robust. This is perhaps caused by the sparse and irregular geometric structures in local regions. Diverse geometric structures among different local regions may require different extractors but shared residual MLPs struggle achieving this. We flesh out this intuition and develop a lightweight geometric affine module to tackle this problem. Let {ftr)j= E Rkxd be the grouped local neighbors of E Rdd containing k points, and each neighbor point fas is a d-dimensional vector. We transform the local neighbor points by the following formulation:
[fa] f σ 0+C
(fr)) =
+ B,
(fu, f.)
(5)
kXnX d
where Idd and E Rd are learnable parameters, @ indicates Hadamard production, and c 1e 1e-5 is a small number for numerical stability (loffe & Szegedy, 2015; Wu & He, 2018; Dixon & Massey Jr, 1951). Note that is a scalar describes the feature deviation across all local groups and channels. By doing so, we transform the local points to a normal distribution while maintaining original geometric properties.
3.4 COMPUTATIONAL COMPLEXITY AND ELITE VERSION
Although the FC layer is highly optimized by mainstream deep learning framework, the theoretical number of parameters and computational complexity still high. To further improve the efficiency,
We emphasize that the model complexity could not be simply revealed by FLOPs or parameters, other metries like memory access cost (MAC) and the degree of parallelism also significantly affect the speed (Ma al., 2018; Zhang et al., 2020). However, these important metries are always ignored in point clouds analysis.
5

---

Published as a conference paper at ICLR 2022
Table 2: Classification results ModelNer40 dataset. With only 1K points, method achieves state-of-the-art results on both class mean accuracy (mAcc) and overall accuracy (OA) metrics. We also report the speed of some open-soureed methods by samples/second tested one Tesla V100- pcie GPU and four cores AMD EPYC 7351 @2.10226000HH0 2.60GHZ CPU. For KPConv, we take the results from the original paper. The best is marked in bold and second best blue.
Train speed
Test
Method
Inputs
mAcc(%)
OA(%)
Param.
speed
PointNet (Qi ct al., 2017a)
1k P
86.0
89.2
PointNet++ (Qi ct al., 201 7b)
1k P P+N
90.7
1.41M
223.8
308.5
PointNet++ (Qi al., 2017b)
91.9
1.41M
PoinCCNN (Li ct al., 2018b)
Ik
88.1
92.5
PointConv (Wu al., 2019)
P+N
92.5
18.6M
10.2
KPConV (Thomas et al., 2019)
7k
92.9
15.2M
31.0*
80.0
DGCNN (Wang et al., 2019)
1k P 1k P
90.2
92.9
RS-CNN (Liu et al,, 2019h)
92.9
DensePoint (L.iu ct al., 2019a)
1k
93.2
PointASNL (Yan al., 2020)
92.9
PosPool (Liu et al., 2020)
5k P Ik P
93.2
Point Trans. (Engel et al., 2020)
92.8
GBNet (Qiu ct al., 2021b)
1k P
91.0
93.8
8.39M
16.3
112
GDANet (Xu al., 2021b)
93.8
0.93M
26.3
14.0
PA-DGC (Xu et al., 2021a)
1k
93.9
MLMSPT (Han et al., 2021)
1kP
92.9
PCT (Guo et al., 2021)
1k
93.2
Point Trans. (Zhao al., 2021)
Ik
90.6
93.7
CurveNet (Xiang al., 2021)
94.2
2.04M
20.8
15.0
PointMI.P P w/o vot.
1k P P
91.3
94.1
12.6M
47.1
112
PointMI.P .P w/ vot.
94.5
12.6M
47.1
112
PointMLP-elic w/o vot.
90.9
93.6
0.68M
116
176
PointMLP-elit w/ vot.
Ik 90.7
94.0
0.68M
116
176
we introduce lightweight version of PointMLP named as PointMLP-elie with less than 0.7M parameters and prominent inference speed (176 sumples/second ModelNe140 benchmurk).
Inspired by He al. (2016); Hu al. (2018), we present bottleneck structure for the mapping function & pre and 4 pos. We opt to reduce the channel number of the intermediate FC layer by factor of r and increase the channel number as the original feature map. This strategy is opposite to the design in Vaswani et al. (2017); Touvron et al. (2021) which increases the intermediate feature dimensions. Empirically, we do not observe significant performance drop. This method reduce the parameters of residual MLP blocks from 2d to 2 dd By default, set elite. Besides, we also slightly adjust the network architecture, reducing both the MLP blocks and embedding dimension number (see appendix for details). Inspired by Xie et al. (2017), investigated grouped FC operation in the network that divides one FC layer into g groups of sub- FC layers, like group convolution layer. However, empirically found that this strategy would largely hamper the performance. As result, we did not consider it in our implementation.
in PointMLP.
also
4 EXPERIMENTS
In this section, comprehensively evaluate PointMLP several benchmarks. Detailed ablation studies demonstrate the effectiveness of PointMLP with both quantitative and qualitative analysis.
4.1 SHAPE CLASSIFICATION ON MODELNET40
We first evaluate PointMLP on the ModelNet40 (Wu et al., 2015) benchmark, which contains 9,843 training and 2,468 testing meshed CAD models belonging 40 categories. Following the standard practice in the community, we report the glass-average accuracy (mAcc) and overall accuracy (OA) on the testing set. We train all models for 300 epochs using SGD optimizer.

---

Published as a conference paper at ICLR 2022
Ovenall accuracy (OA)
Overall acquracy (OA)
Overall accurazy (OA)
70
70
60
60
60
20-
30
50
A
w/o Alline
wio A line
w/b AfTin
25
50
75
100
175
200
50
75
100
150
175
200
25
50
75
100
175
200
T Training spoch
Triining epoch
Training spoch
PointMLP 24-Layers
(b) PointMLP 40-Layers
(c) PointMLP 56-Layers
Figure 3: Four results (mean + std) of PointMLP with/without geometric affine module ScanObjectNN test set. We zoom in on the details of PointMLP40 to show the stability difference.
Experimental results are presented in Table 2. Among these methods, our PointMI.P clearly outper- forms state-of trate-of-the-art method CurveNet by 0.3% (94.5% 94.2%) overall accuracy with only Ik points. Note that this improvement could be considered as a promising achievement since the results on ModelNet40 recent methods have been saturated around 94% for long time. Even without the voting strategy (Liu et al., 2019b), our PointMLP still performs on par or even better than other methods that tested with voting strategy.
Despite having better accuracy, our method is much faster than the methods with sophisticated cal geometric extractors. We compare PointMLP to several apen-sourced methods and report the parameters, classification accuracy, training, and testing speed. As we stated previously, a key intu- ition behind this experiment is that model complexity can not directly reflect efficiency. For exam- plc, CurveNet is lightweight and delivers a strong result, whereas the inference cost is prohibitive (15 samples/second). On the contrary, our PointMLP presents a high inference speed (112 sam- ples/second). To further reduce the model size and speed up the inference, we present a lightweight PomtMLP-eeitee which significantly reduces the number of parameters to 0.68M, while maintaining high-perfin hesh-performacc 90.9% mAcc and 94.0% OA ModelNet40. With PointMLP-elite we further speed up the inference 176 samples/second.
4.2 SHAPE CLASSIFICATION ON SCAN
While ModelNet40 is the de-facto canonical benchmark for point cloud analysis, it may not meet the requirement of modern meth- ods due to its synthetic nature and the fast development of point cloud analysis. To this end, also conduct experiments on the ScanObjectNN benchmark (Uy et al., 2019).
Table 3: Classification results on ScanObjectNN dataset.
We examine all methods the challenging variant (B__S..RS). For our pointMLP and PointMLL-elite, we train and test for four runs and report mean std results.
Method
mAcc(%)
OA(%)
3DmFV
PointNet (Qi ct al., 2017a)
63.4
68.2
SpiderCNN (Xu et al., 2018)
69.8
73.7
PointNet++ (Qi et al., 2017b)
75.4
77.9
DGCNN (Wang et al., 2019)
73.6
78.1
ScanObjectNN is recently leased point cloud benchmark that contains 15,000 objects that are categorized into 15 classes with 2,902 unique object in- stances in the real world. Due to the existence of background, noise, and occlusions, this bench- mark poses significant challenges to existing point cloud analysis methods. We consider the hard- est perturbed variant (PB. (PB_T50RRS) in experiments. We train model using SGD optimizer for 200 epochs with a batch size of 32. For better illustration, we train and test our method for four runs and report the mean + standard deviation in Table 3.
PointCNN (Li 20188)
75.1
78.5
BGA- AGA-DGGNN (Uy et al., 2019)
75.7
79.7
BGA-PN++ (Uy al., 2019)
77.5
80.2
DRNet (Qiu al., 2021a)
78.0
80.3
GBNet (Qiu al., 2021b)
77.8
Simple (Goyal et al., 2021)
80.5+L0.3
PRANet (Cheng et al., 2021)
79.1
82.1
MVTN (Hamdi et al., 2021)
82.8
PoinMMIP (ours)
83.9+-0.5
0..4 81.8-+0.8
PointMI.P-elite (ours)
83.8-+0..6

---

Published as a conference paper at ICLR 2022
Classification accuracy of pointMLP on ScanObjectNN test set using 24, 40, and 56 layers, respectively.
Table 4:
Table 5: Component ablation studies ScanObjectNN test set.
pre
pos
Affine
mAcc(())
OA(%)
Depth
mAcc(%)
OA(%)
x
/
/
80.8+0..4
82.8+0.00
layers
83,4100.4
84.8.10.5
/
x
/
83.3++0.3
84.7++0.2
40 layers
83,9:10.5
85,4+0.3
/
79.1+1..7
81.5++1.4
56 layers
83.2+0.2
85.0+0.1
/
v
83.9+
85,4++0.3
Empirically, our PointMLP surpasses all methods by a significant improvement on both class mean accuracy (mAcc) and the overall accuracy (OA). For example, outperform PRANet by 4.8% mAcc and 3.3% OA. Even compared with the heavy multi-view projection method MVTN (12 views), our PointMLP still performs much better (85.39% 82.8%). Notice that achieve this by fewer training epochs and did not consider the voling strategy. Moreover, we notice that our method achieves the smallest gap between class mean accuracy and overall accuracy. This phenomenon indicates that PointMLP did not bias to particular category, showing decent robustness.
4.3 ABLATION STUDIES
Network depth has been exploited in many tasks but is rare in point cloud anal- Network Depth.
ysis. We first investigate the performance of PointMLP with different depths in Table 4. We vary network depth by setting the number of homogeneous residual MLP blocks to 1, 2. and re- spectively, resulting in 24, 40, and 56-layers PointMLP variants. Detailed depth formulation can be found in Appendix D. At first glance, we notice that simply increasing the depth would not ways bring better performance; model gets stable with more layers introduced, as demonstrated by the decreasing standard devi- ation. When the depth is set 40, achieve the best tradeoff' between accuracy and stability (85.4% mean accuracy and 0.3 standard deviations). Remarkably, PointMLP consistently achieves gratifying results that outperform recent methods, regardless the depth.
appropriate depth would be good solution. Additionally, the
Geometric Affine Module. Other work provides sophisticated local geometric extractors to ex- plore geometric structures. Instead, our PointMLP discards these burdensome modules and intro- duces a lightweight geometric affine module. Figure presents the results of PointMLP with/without the geometric affine module. By integrating the module, systematically improve the performance of PointMLP by about 3% for all variants. The reasons for this large improvement are two-fold. First, the geometric affine module maps local input features normal distribution, which eases the training of PointMI.P. IP. Second, the geometric affine module implicitly encodes the local geometrical information by the channel-wise distance local centroid and variance, remedying the deficiency of geometric information. Besides the gratifying improvements, the geometric affine module also largely boosts the stability PointMLP, suggesting better robustness.
Component ablation study. Table 5 reports the results on ScanOb- jectNN of removing each individual component in PoinMMP. Consistent with Figure 3, geometric affine module plays an important role in PointMLP, improv- ing the base architecture by 3.9%. Remarkably, even without this module, which is an unfair setting for PointMLP, our base network stills achieves
Fedingg
(a) PointMLP w/o Res.
(b) PointMLP
Figure 4: Loss landscape along two rand directions. By introduc- ing residual connection, we ease the optimization of PointMLPP and achieve flat landscape like simple shallow network intuitively.
8

---

Published as a conference paper at ICLR 2022
Table 6: Part segmentation results the ShapeNerPart dataset. Empirically, method is much faster than the best method KPConv, and presents a competitive performance.
mloll
mila..
cap
guitur
knife
lamp
laptup
trkke
mug
pistal
board
PointNer
82.5
12.8
PointNet-
87.7
71.8
91.0
83.7
95.3
81.3
Kd Net
$2.3
80.1
74.3
70.3
$3.6
73.5
90.2
57.2
$1.0
91.9
57.4
86.7
78.1
51.8
69.9
80.3
SO-Net
$4.9
82.8
77.8
88.0
77.3
90.6
73.5
90.7
83.9
$2.8
94.8
69.1
94.2
90.9
53.1
72.9
83.0
PUNN
818
55.1
K2.4
50.1
85.5
79.5
90.8
73.2
91.3
86.0
$5.0
95.7
73.2
94.8
83.3
51.0
75.0
81.8
DGIONN
82.3
85.3
84.0
83.4
86.7
77.8
90.6
74.7
91.2
87.5
82.8
95.7
66.3
94.9
81.1
63.5
74.5
82.6
P2Sequence
55.2
82.6
87.5
77.3
90.8
77.1
9
83.9
95.7
70,8
96.5
79.3
58.1
75.2
82.8
PuinCCNN
PoinASNL
RS-CNN
88.8
91.6
96.0
73.7
60.5
77.7
SynSpec
$2.0
$4.7
81.6
$1.7
81,9
75.2
90.2
74.9
93.0
86.1
$4.7
95.6
66.7
92.7
81.6
60.5
82.9
82.1
SPLAINet
83.7
35.4
K3-2
84.3
89.1
80.3
90.7
75.5
92.1
87.1
83.9
96.3
75.6
95.8
83.8
64.0
75.5
81.8
SpideeCNN
82.4
53.3
K3.5
$1.
87.2
71.5
90.7
76.8
91.1
87.3
83.3
95.8
70.2
93.5
82.7
59.7
75.8
82.8
KPConv
85.1
56.4
84.6
16.3
87.2
81.1
91.1
77.8
92.6
88.4
82.7
96.2
78.1
95.8
85.4
69.0
82.0
83.6
PA-DGC
84.6
56.1
84.3
90.4
79.7
80.8
92.0
88.7
82.2
95.9
73,9
96.7
84.7
65.9
81,4
84,0
PoinMLP
81.5 1.4% OA, outperforming most related methods (see Table 3). Removing Ppre function (MLPs before aggregator A), the performance drops 2.6% overall accuracy. Combining all these components together, we achieve the best result 85.4% OA. See Appendix C for more ablations.
Loss landscape. We depict the 3D loss landscape (Li al., 2018a) in Figure Simply increas- ing the network depth may not achieve a better representation and even hamper the results. When removing the residual connection PointMLP, the loss landscape turns sharp, and performance plummets to 88.1 1 % (6% drop) on ModelNet40. With residual connection, we greatly ease the opti- mization course of PointMLP and make possible to train a deep network.
4.4 PART SEGMENTATION
also be gen- eralized to other 3D point cloud tasks. We next test PointMLP for 3D shape part segmentation task on the ShapeNetPart benchmark (Yi et al., 2016). The ShapeNetPart dataset con- sists of 16,881 shapes with 16 classes belonging to 50 parts labels in total. In each class, the number of parts is between 2 and 6. We follow the set- tings from Qi et al. (2017b) that ran- domly select 2048 points as input for a fair comparison. We compare our methods with several recent works, including SyncSpecCNN (Yi et al., 2017), SPLATNet (Su et al., 2018), etc. We also visualize the segmentation ground truths and predictions in Figure 5. Intuitively, the predictions of our PointMLP are close to the ground truth. Best viewed in color.
Our PointMLP
Figure 5: Part segmentation results ShapeNetPart. Top line is ground truth and bottom line is our prediction.
5 CONCLUSION
In this paper, we propose simple yet powerful architecture named PointMLP for point cloud anal- ysis. The key insight behind PointMIPP that sophisticated local geometric extractor may not be crucial for performance. We begin with representing local points with simple residual MLPs as they are permutation-invarte module to boost the performance. To improve efficiency further, we also introduce a lightweight counterpart, dubbed as PointMLP-elite. Experimental results have shown that PointMLP outper- forms related work different benchmarks beyond simplicity and efficiency. We hope this novel idea will inspire the community to rethink the network design and local geometry in point cloud.
and straightforward. Then we introduce lightweight geometric affine

---

Published as a conference paper at ICLR 2022
REFERENCES
Silin Cheng, Xiwu Chen, Xinwei He, Zhe Liu, and Xiang Bai. Pra-net: Point relation-awan network for 3d point cloud analysis. IEEE Transactions on Image Processing, 30:4436 4448, 2021.
Wilfrid J Dixon and Frank
Massey Jr. Introduction to statistical analysis. 1951.
Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. Learning deep convolutional network for image super-resolution. In ECCV, pp- 184-199. Springer, 2014.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, al. An image is worth 16x 16 words: Transformers for image recognition at scale. ICLR, 2021.
Nico Engel, Vasileios Belagiannis, and Klaus Dietmayer. Point transformer. arXiv preprint arXiv:2011.00931, 2020.
Siqi Fan, Qiulei Dong, Fenghua Zhu, Yisheng Lv, Peijun Ye, and Fei- Yue Wang. Sef-net: Learning spatial contextual features for large-scale point cloud segmentation. In CVPR, pp. 14504-1455 2021.
Ankit Goyal, Hei Law, Bowei Liu, Alejandro Newell, and Jia Deng. Revisiting point cloud shape classification with simple and effective baseline. ICML, 2021.
Meng-Hao Guo, Jun- niong Cai, Zheng-Ning Liu, Tai-Jiang Mu, Ralph R Martin, and Shi-Min Hu. Pct: Point cloud transformer. Computational Visual Media, 7(2):187-199 2021.
Abdullah Hamdi, Silvio Giancola, and Bernard Ghanem. Mytn: Multi-view transformation network for 3d shape recognition. ICCV, 2021.
Xian-Feng Han, Yu-Jia Kuang, and Guo-Qiang Xiao. Point cloud learning with transformer. arXiv preprint arXhv:2104.13633 2021.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, pp. 770-778, 2016.
Jie Hu, Li Shen, and Gang Sun. geueezz-and-exxitalie networks. In CVPR, pp. 7132-7141, 2018.
Qingyong Hu, Bo Yang, Linhai Xie, Stefano Rosa, Yulan Guo, Zhihua Wang, Niki Trigoni, and Andrew Markham. Randla-net: Efficient semantic segmentation of large-scale point clouds. In CVPR, pp. 11108-11117, 2020.
Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In /CML, pp. 448-456. PMLR, 2015.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton. Imagenet classification with deep convo- lutional neural networks. In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger (eds.), NeurIPS, volume 25. Curran Associates, Inc., 2012.
Guohao Li, Matthias Muller, Ali Thabet, and Bernard Ghanem. Deepgens: Can gens go deep as cnns? ICCV, pp. 9267-9276 2019.
Guohao Li, Matthias Müller, Guocheng Qian, Itzel Carolina Delgadillo Perez, Abdulellah Abual- shour, Ali Kassem Thabet, and Bernard Ghanem. Deepgens: Making gens go deep as cnns. TPAMI, 2021a.
Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss land- scape of neural nets. NeurIPS, pp. 6391-6401, 2018a.
Lei Li. Siyu Zhu, Hongbo Fu, Ping Tan, and Chiew- Tai. End-to-end learning local multi-view descriptors for 3d point clouds. In CVPR, pp. 1919-1928, 2020.
Yangyan Li. Rui Bu, Mingchao Sun, Wei Wu, Xinhan Di, and Baoquan Chen. Pointenn: Convolu- tion on x-transformed points. NeurlPS, 31:820-830. 2018b.
10

---

Published as a conference paper at ICLR 2022
Zhichao Li, Feng Wang, and Naiyan Wang. Lidar r-cnn: An efficient and universal 3d object detec- tor. In CVPR, pp. 7555, 2021b.
Zhi-Hao Lin, Sheng Yu Huang, and Yu-Chiang Frank Wang. Learning of 3d graph convolution networks for point cloud analysis. TPAMI, 2021.
Yongcheng Liu, Bin Fan, Gaofeng Meng. Jiwen Lu, Shiming Xiang, and Chunhong Pan. Dense- point: Learning densely contextual representation for efficient point cloud processing. In ICCV, pp. 10.5522955444 2019a.
Yongcheng Liu, Bin Fan, Shiming Xiang, and Chunhong Pan. Relation-shape convolutional neural network for point cloud analysis. CVPR. pp. 8895-8904, 2019b.
Ze Liu, Han Hu, Yue Cao, Zheng Zhang, and Xin Tong. A closer look at local aggregation operators in point cloud analysis. In ECCV, pp. 226-342. Springer, 2020.
Ze Liu, Zheng Zhang, Yue Cao, Han Hu, and Xin Tong. Group-free 3d object detection via trans- formers. arXiv preprint arXiv:2104.00678, 2021.
Ilya Loshchilov and Frank Hutter. 2017.
Sgdr: Stochastic gradient descent with warm restarts. ICLR,
Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun. Shufflenet v2: Practical guidelines for efficient cnn architecture design. In ECCV, pp. 116-131, 2018
Daniel Maturana and Sebastian Scherer. Voxnet: A 3d convolutional neural network for real-time object recognition. In IROS, pp. 922-928. IEEE, 2015.
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning point sets for 3d classification and segmentation. In CVPR, pp. 652-660, 2017a
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet+++: Deep hierarchical feature learning on point sets in a metric space. NeurlPS, 30, 2017b.
Shi Qiu, Saeed Anwar, and Nick Barnes. Dense-resolution network for point cloud classification and segmentation. In WACV, pp. 3813-3822, 2021a.
Shi Qiu, Saeed Anwar, and Nick Barnes. Geometric back pack-proecttion network for point cloud classi- fication. IEEE Transactions on Multimedia, 2021b.
Haoxi Ran, Wei Zhuo, Jun Liu, and Li Lu. Learning inner-group relations on point clouds. In ICCV, pp. 15477-15487, 2021.
Shaoshuai Shi, Xiaogang Wang, and Hongsheng Li. Pointrenn: 3d object proposal generation and detection from point cloud. In CVPR, pp. 770-779, 2019.
Shaoshuai Shi, Chaoxu Guo, Li Jiang, Zhe Wang, Jianping Shi, Xiaogang Wang. and Hongsheng Li. Pv-renn: Point-voxel feature set abstraction for 3d object detection. In CVPR, pp. 10529-10538, 2020.
Weijing Shi and Raj Rajkumar. Point-gnn: Graph neural network for 3d object detection in a point cloud. In CVPR, pp. 1711-1719, 2020.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In Yoshua Bengio and Yann LeCun (eds.), ICLR, 2015. URL http:///e/ //arivv. org/abs/ /1099 1556.
Hang Su, Varun Jampani, Deqing Sun, Subhransu Maji, Evangelos Kalogerakis, Ming-Hsuan Yang, and Jan Kautz. Splatnet: Sparse lattice networks for point cloud processing. In CVPR. pp. 2530- 2539, 2018.
Hugues Thomas, Charles R Qi, -mmmnnell Deschaud, Beatriz Marcotegui, François Goulette, and Leonidas J Guibas. Kpconv: Flexible and deformable convolution for point clouds. In ICCV, pp. 641 1-620,, 2019.
11

---

Published as a conference paper at ICLR 2022
Hugo Touvron, Piotr Bojanowski, Mathilde Caron, Matthieu Cord, Alaaeldin El-Nouby, Edouard Grave, Armand Joulin, Gabriel Synnaeve, Jakob Verbeek, and Hervé Jégou. Resmlp: Feedforward networks for image classification with data-efficient training. arXiv preprint arXiv:2105.03404, 2021.
Mikaela Angelina Uy, Quang- Hieu Pham, Binh-Son Hua, Thanh Nguyen, and Sai-Kit Yeung. Revis- iting point cloud classification; A new benchmark dataset and classification model on real-world data. In ICCV, pp. 1588-15977 2019.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, pp.
2017.
Xiaolong Wang, Ross Girshick, Abhinav Gupta, and Kaiming He. Non-local neural networks. In CVPR, pp. 7794-7803, 2018.
Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael Bronstein, and Justin M Solomon. Dynamic graph enn for learning point clouds. Acm Transactions On Graphics (tog), 38(5): 1-12, 2019.
Wenxuan Wu, Zhongang Qi, and Li Fuxin. Pointconv: Deep convolutional networks on 3d point clouds. CVPR, pp. 9621-9630, 2019.
Yuxin Wu and Kaiming He. Group normalization. In Proceedings of the European conference on computer vision (ECCV). pp. 3-19, 2018.
Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In CVPR, pp. 1912-1920,
2015.
Tiange Xiang, Chaoyi Zhang, Yang Song, Jianhui Yu, and Weidong Cai. Walk in the cloud: Learning curves for point clouds shape analysis. In ICCV, October 2021.
Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual trans- formations for deep neural networks. In CVPR. pp. 1492-1500 2017.
Chenfeng Xu, Bichen Wu, Zining Wang, Wei Zhan, Peter Vajda, Kurt Keutzer, and Masayoshi Tomizuka. Squeezesegv3: Spatially-adaptive convolution for efficient point-cloud segmentation. In ECCV, pp. 1-19. Springer, 2020.
Mutian Xu, Runyu Ding, Hengshuang Zhao, and Xiaojuan Qi. Paconv: Position adaptive convolu- tion with dynamic kernel assembling on point clouds. In CVPR, pp. 3173-3182, 2021a.
Mutian Xu, Junhao Zhang, Zhipeng Zhou, Mingye Xu, Xiaojuan Qi. and Yu Qiao. Learning geometry-disentanglee representation for complementary understanding of 3d object point cloud. In AAAI, volume 35, pp. 3056-30000 3056, 2021b.
Yifan Xu, Tianqi Fan, Mingye Xu, Long Zeng, and Yu Qiao. Spidercan: Deep learning on point sets with parameterized convolutional filters. In ECCV, pp. 87-102, 2018.
Xu Yan, Chaoda Zheng, Zhen Li, Sheng Wang, and Shuguang Cui. Pointasnl: Robust point clouds processing using nonlocal neural networks with adaptive sampling. In CVPR, pp. 5589-5598, 2020.
Zetong Yang, Yanan Sun, Shu Liu, Xiaoyong Shen, and Jiaya Jia. Std: Sparse-to-dense 3d object detector for point cloud. In ICCV, pp. 1951-1960, 2019.
Li Yi, Vladimir G Kim, Duygu Ceylan, I-Chao Shen, Mengyan Yan, Hao Su, Cewu Lu, Qixing Huang, Alla Sheffer, and Leonidas Guibas. A scalable active framework for region annotation in 3d shape collections. ACM Transactions on Graphics (ToG), 35(6):1-12, 2016.
Li Yi. Hao Su, Xingwen Guo, and Leonidas Guibas. Synespecenn Synchronized spectral enn for
3d shape segmentation. In CVPR, pp. 2282-2290, 2017.
12

---

Published as a conference paper at ICLR 2022
Haoxuan You, Yifan Feng, Rongrong Ji, and Yue Gao. Pvnet: A joint convolutional network of point cloud and viewi-view for shape recognition. In ACM MM, pp. 1310-131,, 2018.
Hang Zhang, Chongruo Wu, Zhongyue Zhang, Yi Zhu, Haibin Lin, Zhi Zhang, Yue Sun, Tong He, Jonas Mueller, R Manmatha, al. Resnest: Split-attention networks. arXiv preprint arXiv:2004.08955, 2020.
Zaiwei Zhang, Rohit Girdhar, Armand Joulin, and Ishan Misra. Self-supervised pretraining of 3d features on any point-cloud. arXiv preprint arXiv:2101.0263 2021.
Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip Torr, and Vladlen Koltun, Point transformer. In ICCV, 2021.
13

---

Published as a conference paper at ICLR 2022
A POINTMLP DETAIL
We detail the architecture of PointMLP in Figure (as well as PointMLP-ell in Figure for a bet- ter understanding. Compared with PointMLP, the elite version mainly adjusts three configurations: 1) it reduces the number of residual point (Resp) MIP blocks; 2) reduces the embedding dimen- sion from 64 to 32, hence the overall model overhead is significantly alleviated; 3) by introducing bottleneck structure, PointMLP further reduces the parameters by four times.
For part segmentation task, we the framework presented in PointNet (Qi et al., 2017a) and replace the backbone to our PointMLP. With the only modification, we improve the performance from 85.1 to 86.1 Instance mloU.
Stage
Stage
Stage
Stager
classi fior
FC BN Red U (55)
Resp MLP
Resp MLP
Embeddinng
Nggregation
Dr Drogout 10.5%
Output
RCBN
[64,1024]
FC
[022.3]
[128,512]
(61.24.12]
[256,256]
(,26,22266]
[5121288]
[25624.128
Figure 6: Detail architecture PointMLP for classification.
Suage
Siage
Stage
Stage
classifier
FC SN RoLU t Se
M Bottleneck Resp MLP In clite
MLP
pas Reep MLP
cless mumi
Emiheddig
Aggregatten
Shoos (L.
Drapped (0.5)
FC TN Rel
RC
[120,256
(4426.256)
(class_num)
(266.220
1228244128
61266
(56,24/44
[104432
512,2333
Figure 7: Detail architecture of PointML.Peliite for classification.
B DETAIL EXPERIMENTAL SETTING
B.1 MODELNET40 AND SCANOBJECTNN
Our implementations are based on PyTorch. For ModelNet40, we train models for 300 epochs on one Tesla V100 GPU with batch size of 32. All our models are trained using synchronous SGD with a Nesterov momentum of 0.9 and weight decay of 0.0002. The learning rate is set to 0.1 initially. We use the cosine annealing scheduler (Loshchilov Hutter, 2017) to adjust the learning rate. For each sample, randomly select 1024 points and consider the same augmentation strategy as Qi et al. (2017b). The setting for ScanObjectNN similar to ModelNet40, except we train all models for only 200 epochs.
For the reported speed Table 2, we test the open-source source code on a Tesla V100-peic GPU, All the source codes used are listed? in the footnote.
"ll tested methods are listed bellow
PointNet++: http://juhhbb.mmmiiaggettggngnut
Porch
CurveNet:
GBNet: http:///giibbb.oom///ihnnnn44449
GDANet: http://gttubb.omm/muiiann/DDDDD
PointConv: http://lthhbb.oo////aaaasssssiit
KPConw: htwp://hhh..co///ggeee//////////
14

---

Published as a conference paper at ICLR 2022
B.2 SHAPENETPART
Our setting for part segmentation task is following PointNet (Qi et al., 2017a). We randomly sample 2048 points for each sample and re-scale the input in range of [0.67, 1.5). Note that did not test the result using a multi-scale testing strategy, which could further improve the performance, but is realizable real-world applications. Hence, we only report the single-scale results. Even the comparison is unfair, we still achieve competitive performance.
C MORE DETAILED ABLATION STUDIES
Skip connection. Figure shows the loss landscapes of our PointMLP with and without skip connec- tions. We also consider adding skip connections to PointNet++ to validate the effectiveness of skip connections. Due to the structure of PointNet++, only two skip connections could be added without modifying the original architecture PointNet++. By adding the skip connections, we achieve a classification accuracy of 92.7% ModelNet40 in our re-implementation.
Pre-MLP block vs. Pos-MLP block. we also modified the configuration of our PointMI.P and re- trained the model to investigate the importance of Pre-MLP and Pos-MLP blocks. In our origi- nal implementation, we set the pre-MLP block list to [2, 2, 2, 2] and the pos-MLP blocks list to [2, 2, 2). Here, we remove the pos-MLP blocks and change the pre-MLP blocks to [4, 4, 4, to match the block number. The 3-layer classifier can be considered as the MLP at the end of the last stage. We trained the models two times and got average OA of 84.13% (83.87% and 84.39%), which lower than vanilla PointMLP 85.4%, and even the result in Table 5 second-row 84.7%. PointMLP, and simply adding more pre-MLP
This result indicates that Pos-MLP does benefit blocks does not help. We acknowledge that the effect of pos-MLP is not as strong as other com- ponents and believe that a detailed fine-tuning of the configurations would deliver an even better performance-fffcienee balance.
Geometric Affine Module Applications. Geometric affine module plays an essential role our PointLPP, exhibiting promising performance improvements. While this module can be consid- ered as a plug-and-pll and-play method, the overlap with some local geometric extractors in other methods may limit its application. Here we integrate the module to two popular methods, PointNet++ and DGCNN, for illustration and experiment on the ModelNet40 benchmark. By integrating the geomet- ric affine module, improve the performance of PointNet++ to 93.3%. achieving an improvement of 1.4%. However, when integrating the module to DGCNN, we get a performance of 92.8%, which is slightly lower than the original results (92.9%). Note that both results tested without voting.
D POINTMLP DEPTH
Here format the detailed formulation of layer number in PointMLP. For the sake of clarity, ignore Batch Normalization layers and activation functions. Let Pre; and Pos; indicate the repeating number of the & pre block (which includes 3 layers) and pos block (which includes layers) in i-th stage, respectively. Note that we have one layer in feature embedding in the beginning, one layer for channel number matching in each stage, and three layers in the classifier. Hence, the total number of learnable layers L would be
L L-1+
Σ (1+2xPre) 2 x Pos;) + 3.
==1
As result, the depth configuration of our network (24, 40, and 56) can be summarized as:
Depth
[Pre Prez, Pres, Prej]
[Pos], Pos2, Pos3, Posi]
24
1,1,
1,1,1,11
40
2, 2,
2,2,2,2
56
3,3,
3,3,
15


Reviews:
[{"review_id": "pZmOWixXXlc", "reviewer": "Reviewer_7Y1Q", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The proposed PointMLP achieves better accuracy while being simple and faster. This is not surprising. Empirically for point cloud processing I also found a lot of 'sophisticated' networks can be easily outbeat by much simple networks with tuned #layers and #channels. \n\nMy biggest concern for this paper is the lack of novelty and depth of this paper. It simply present the model and results, without telling why this simple model can outperform other delicately designed networks. This is more like a technical report.\n\nSeveral claims made me confused:\n1. 'even without the carefully designed operations for local geometry exploration': geometric affine module still uses local geometry. Did you compare PointMLP w/o geometric affine module with other models? I did not see any evidence that networks without local geometry exploration work better.\n2. in part segmentation task, 'our method can achieve the best accuracy-speed tradeoff'. I did not see any speed comparison.", "summary_of_the_paper": "This paper proposed an alternative point cloud feature extractor, named PointMLP. PointMLP is composed of residual MLPs and geometric affine modules. Classification results on ModelNet40 dataset show the proposed methods achieves slightly better accuracy while using much smaller number of parameters and faster runtime. The proposed method achieves better accuracy in classification results on ScanObjectNN dataset and is on par with state-of-the-art methods for 3D shape part segmentation task on the ShapeNetPart benchmark.", "summary_of_the_review": "My biggest concern for this paper is the lack of novelty and depth of this paper. I still give a positive review because I got sick of papers that boast about their fancy design but in reality cannot outperform a simple network. \n\n\n==========================================\n\nBased on other reviewers' comments and the authors' responses, I would like to keep my original score.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "QkaQz39twbv", "reviewer": "Reviewer_vGXQ", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "*Strengths*\n- The paper is very well-written and easy to read.\n- The architecture of PointMLP is simple for implementation. Since PointMLP is mainly based on pure residual MLPs, we can take advantage of the efficiency from the highly-optimized MLPs when training PointMLP.\n- PointMLP outperforms previous SOTA models on ModelNet40 and ScanObjectNN.\n\n*Weakness*\n- As PointMLP is a refinement of PointNet++, I do not think the novelty of the idea is high.\n- The performance of PointMLP is comparable with CurveNet on ModelNet40. In my opinion, the improvement by 0.3% is not significant.\n- The results of the PointMLP in part segmentation task on ShapeNetPart are worse than the old architecture KPConv.\n\nSome more comments and questions:\n- Page 4, line 3: the local feature extraction of PointNet++ is formulated as\n$$g_i = \\mathcal{A}( \\{ \\Phi(f_{i,j})|j=1,...,K \\} ).$$\n- Similarly, in equation (4), the local feature extractor of PointMLP must be\n$$g_i = \\Phi_{pos}(\\mathcal{A}( \\{ \\Phi_{pre}(f_{i,j})|j=1,...,K \\} )).$$\n- Why the small $\\epsilon$ in equation (5) can stabilize the numerical computation?\n- The authors claim that PointMLP is much faster than KPConv in part segmentation tasks. But I do not find anywhere in the paper nor the tables that support this claim.\n- I am not sure if the performance of recent models based on sophisticated local geometric extractors is saturated or not as more and more efficient and accurate models of these kinds are still being investigated. Furthermore, as we see in the experiments, PointMLP does not outperform the previous SOTA model with significant margin (on ModelNet40 and ShapeNetPart). Therefore, stronger reasons are needed to support the claim that \"detailed local geometrical information probably is not the key to point cloud analysis\".", "summary_of_the_paper": "The authors present a refinement of PointNet++, called PointMLP, in which the MLPs using in the local feature extractions in PointNet++ are replaced by residual MLPs. Together with a lightweight geometric affine module to stabilize the training, PointMLP outperforms previous models in the classification tasks on ModelNet40 and ScanObjectNN.", "summary_of_the_review": "Since the idea of PointMLP is quiet simple and it can be considered as a refinement of PointNet++, I do not think that the novelty is very high, although the experiments show some interesting results.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "1zRUHGssZCD", "reviewer": "Reviewer_8anY", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The strength of this paper is the outperforming results and fast inference speed of the proposed method compared to most of the previous methods in various tasks, especially for a challenging task like classification with ScanObjectNN data. Also, the architecture seems very easy to implement from a PointNet++ code.\n\nBut, I also have several questions that I hope to be addressed by the authors in the rebuttal:\n\n- What is the \"voting\" in the experiments? (E.g., w/o vot and w/ vot in Table 2)? I couldn't figure out what this voting meant.\n- In Table 2, It seems the accuracy difference between CurveNet (Xiang et al., 2021) and the proposed method is marginal, while CurveNet is much lighter and faster. Also, the results of CurveNet seem to be without voting, so I guess there might be a chance for CurveNet to improve the accuracy if it uses the voting. What would be the other advantages of the proposed method compared to CurveNet? Would it be easier to implement? What happens if the proposed method is compared to CurveNet in the task of ScanObjectNN classification (Table 3) and ShapeNet classification (Table 6)?\n- Why is the proposed method lighter and faster than PointNet++? Is it because of the residual connections? What happens if only the residual connections are used in PointNet++?\n- It would also be great to see in Table 5 what happens when only one of the components in the columns on the left is used.\n- Wouldn't it be possible to apply the geometric affine transformation (Equation 5) to any other architectures (compared in Table 1, for example)? It seems like this is the core module improving the performance.\n\nThe followings are also comments/questions about correction/clarification:\n- What are the exact meanings of 24-Layers, 40-Layers, and 56-Layers in Figure 3 and Table 4? Is the number of MLP layers in \"each\" stage of the architecture shown in the appendix?\n- Is having the post-MLP (\\Phi_{pos}) the same as having a deeper pre-MLP in the intermediate stages, and adding an MLP at the end of the last stage? The effect of this post-MLP also seems marginal when seeing Table 5.\n- In Equation 5, shouldn't the (f_{i,j} - f_i)^2 be a squared norm (\\| f_{i,j} - f_i \\|^2) so that the \\sigma becomes a scalar? The text says that f_{i,j} and f_i are d-dimensional vectors.\n- In Section 3.2. our PointMLP still performances efficiently -> our PointMLP still performs efficiently.\n- In Equation 3, the symbol with a circle and a dot is not introduced in the text.\n", "summary_of_the_paper": "This paper introduces a lightweight and fast neural network architecture processing 3D point cloud. While the idea is quite simple, the proposed architecture outperforms or matches the performance of the previous architectures in terms of both accuracy and also inference time with various tasks including ModelNet40 classification, ScanObjectNN classification, and ShapeNetPart segmentation, as shown in the experimental results. The main idea is to make two modifications in the PointNet++ architecture, 1) adding MLP layers after each of the point feature aggregation (max-pooling) steps, and 2) applying a geometric affine transformation to the features processed with an MLP in each small PointNet module. It's interesting to see that these small modifications (especially the second) make dramatic changes in the results.", "summary_of_the_review": "I think this work introduces an interesting finding that small modifications in PointNet++ make dramatic changes in the results. But, more analyses would be needed to clearly see how each component in the proposed method (post-MLP, geometric affine, residual connections) affects the accuracy and inference time. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a new architecture for point cloud processing, with good empirical results. All reviewers recommended accept. AC does not see a reason to overturn the consensus.
Decision: Accept (Poster)