PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies
Authors: Guocheng Qian
Yuchen Li
Houwen Peng
Jinjie Mai
Hasan Abed Al Kader Hammoud
Mohamed Elhoseiny
Bernard Ghanem
Venue: NeurIPS 2022 Accept (2022)
Keywords: Point Cloud
Training Strategy
Model Scaling
3D
Geometric Deep Learning

Abstract: PointNet++ is one of the most influential neural architectures for point cloud understanding. Although the accuracy of PointNet++ has been largely surpassed by recent networks such as PointMLP and Point Transformer, we find that a large portion of the performance gain is due to improved training strategies, i.e. data augmentation and optimization techniques, and increased model sizes rather than architectural innovations. Thus, the full potential of PointNet++ has yet to be explored. In this work, we revisit the classical PointNet++ through a systematic study of model training and scaling strategies, and offer two major contributions. First, we propose a set of improved training strategies that significantly improve PointNet++ performance. For example, we show that, without any change in architecture, the overall accuracy (OA) of PointNet++ on ScanObjectNN object classification can be raised from 77.9% to 86.1%, even outperforming state-of-the-art PointMLP. Second, we introduce an inverted residual bottleneck design and separable MLPs into PointNet++ to enable efficient and effective model scaling and propose PointNeXt, the next version of PointNets. PointNeXt can be flexibly scaled up and outperforms state-of-the-art methods on both 3D classification and segmentation tasks. For classification, PointNeXt reaches an overall accuracy of 87.7 on ScanObjectNN, surpassing PointMLP by 2.3%, while being 10x faster in inference. For semantic segmentation, PointNeXt establishes a new state-of-the-art performance with 74.9% mean IoU on S3DIS (6-fold cross-validation), being superior to the recent Point Transformer. The code and models are available at https://github.com/guochengqian/pointnext.

PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies
Guocheng Qian'*, Yuchen Li Houwen Pengat,
Jinjie Mai', Hasan Abed Al Kader Hammoud' Mohamed Elhoseiny's Bernard Ghanem
KKin Abdullah University of Science and Technology (KAUST), "Microsoft Research
Abstract
PointNet++ is one of the most influential neural architectures for point cloud understanding. Although the accuracy of PointNet++ has been largely surpassed by recent networks such as PointMLP and Point Transformer, we find that a large portion of the performance gain is due to improved training strategies, i.e. data augmentation and optimization techniques, and increased model sizes rather than architectural innovations. Thus. the full potential of PointNet++ has yet to be explored. In this work, we revisit the classical PointNet+++ through systematic study of model training and scaling strategies, and offer two major contributions. First, we propose set of improved training strategies that significantly improve PointNet++ performance. For example, we show that, without any change architecture, the overall accuracy (OA) of PointNet++ on ScanObjectNN object classification can be raised from 77.9% to 86.1%, even outperforming state-of-the- art PointMLP. Second, we introduce inverted residual bottleneck design and separable MLPs into PointNet++ to enable efficient and effective model scaling and propose PointNeXt, the next version of PointNets. PointNeXt can be flexibly scaled up and outperforms state-of-th the-hhharr art methods on both 3D classification and segmentation tasks. For classification, PointNeXt reaches overall accuracy of 87.7% on ScanObjectNN, surpassing PointMI_P P by 2.3%. while being 10x faster in inference. For semantic segmentation, PointNeXt establishes art performance with 74.9% mean IoU on S3DIS (6-fold cross-validation) being superior to the recent Point Transformer. The code and models are available https:/ htt///gtttttt tig..u..... com wwwpchhhnggiaapoiinttane
state-of-the-
Introduction
Recent advances in 3D data acquisition have led surge in interest for point cloud understanding. With the rise of PointNet [27] and PointNet++ [28], processing point clouds in their unstructured for- mat using deep CNNs become possible. Subsequent "PointNets", many point-based networks introduced with the majority focusing on developing new and sophisticated modules to extract local structures, e.g. the pseudo-grid convolution in KPConv [41] and the self-attention layer in Point Trans- former [54]. These newly proposed methods outperform PointNet++ by a large margin in a variety of tasks, leaving the impression that the PointNet++ architecture is too simple to learn complex point cloud representations. In this work, we revisit PointNet++, the classical and widely used network, and find that its full potential has yet be explored, mainly due two factors that were present at the time of PointNet++: (1) superior training strategies and (2) effective model scaling strategies.
Through comprehensive empirical study various benchmarks, e.g., ScanObjecNN [42] for object classification and S3DIS [1] for semantic segmentation, we discover that training strategies, i.e., data augmentation and optimization techniques, play important role in the network's performance. In fact, large part of the performance gain of ofate-of-the-art (SOTA) methods [44, 41, 54) over Point- Net++ [28] is due to improved training strategies that are, unfortunately, less publicized compared to
"Equal contribution. CCorrepponding authors.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

ScanObjectNN
77.9
83.7
85.4 86.1
87.7
S3DIS
$4.5
66.8
73.5 74.9
PointNet++
Optimization Techniques Model Scaling
PointMLP
Data Augmentation
Point Transformer
Figure 1: Effects of training strategies and model scaling on PointNet++ [28]. We show that improved training strategies (data augmentation and optimization techniques) and model scaling can significantly boost PointNet++ performance. The average overall accuracy and mloU (6-fold (ross-validation) are reported on ScanObjectNN [42] and S3DIS [1].
architectural changes. For example, randomly dropping colors during training can unexpectedly boost the testing performance of PointNet++ by 5.9% mean IoU (mloU) on S3DIS [1], as demonstrated in Tab. 5. In addition, adopting label smoothing [37] can improve the overall accuracy (OA) on SeanObjectNN [42] by 1.3%. These findings inspire us to revisit PointNet++ and equip it with new advanced training strategies that are widely used today. Surprisingly, shown in Fig. 1, utilizing the improved training strategies alone improves the OA of PointNet++ by 8.2% on ScanObjectNN (from 77.9% 86.1%), 1.)1. establishing new SOTA without introducing any changes to the architecture (refer to Sec. 4,4.1 for details). For the S3DIS segmentation benchmark, the mloU evaluated in all areas by increase by 13.6% (from 54.5% 68. 1.). outperforming many modern 6-fold cross-validation
architectures that are subsequent to PointNet++, such as PointCNN [21] and DeepGCN [20].
Moreover, we observe that the current prevailing models [19, 41, 54] for point cloud analysis have ployed many more parameters than the original PointNets [27, 28]. Effectively expanding +ointNet++ from its original small scale to larger scale is topic worth studying because larger models gen- erally expected to enable richer representations and perform better [2, 18, 53]. However, we find that the naive way of using more building blocks increasing the channel size in PointNet++ only leads to an overhead in latency and no significant improvement in accuracy (see Sec. 4,4.2). For effective and efficient model scaling, we introduce residual connections [12], inverted bottleneck design [34], and separable MLPs [30] into PointNet++. The modernized architecture is named PointNeXt, the next version of PointNets. PointNeXt
be scaled up flexibly and outperforms SOTA on various benchmarks. As demonstrated in Fig. 1, PointNeXt improves the original PointNet++ by 20.4% mloU (from 54.5% 74.9%) on S3DIS [1] 6-fold and achieves 9.8% OA gains on ScanObjecNN [42], surpassing SOTA Point Transformer [54] and PointMLP [26]. We summarize our contributions next:
We present the first systematic study of training strategies in the point cloud domain and show that PointNer++ strikes back (+8.2% OA on ScanObjectNN and +13.6% mloU on S3DIS) by simply adopting improved training strategies alone. The improved training strategies general and can be easily applied to improve other methods [27, 44, 26).
We propose PointNeXt, the next version of PointNets. PointNeXt scalable and surpasses SOTA on all tasks studied, including object classification [42, 47], semantic segmentation [1, 5], and part segmentation [51], while being faster than SOTA in inference.
Preliminary: A Review of PointNet++
Our PointNeXt is built upon PointNet++ [28], which uses U-Net [33] like architecture with encoder and a decoder, as visualized in Figure 2. The encoder part hierarchically abstracts features of point clouds using number of set abstraction (SA) blocks, while the decoder gradually interpolates the abstracted features by the same number of feature propagation blocks. The SA block consists of subsampling layer to downsample the incoming points, a grouping layer to query neighbors for each point, a set of shared multilayer perceptrons (MLPs) to extract features, and a reduction layer aggregate features within the neighbors. The combination of the grouping layer, MLPs, and the reduction layer is formulated as:
RRALDEN the (Ix):p) -
(I)

---

where R. is the reduction layer (e.g. max- max-pooling) that aggregates features for point i from its neighbors denoted as {j (i,j) N.. J- P X x are the input coordinates, the input features, and the features of neighbor in the nth layer of the network, respectively. he denotes the shared MLPs that take the concatenation of x) and the relative coordinates (D) pt) as input. Note that, since PointNet++ with single-scale grouping that uses one SA block per stage is the default architecture used in the original paper [28], we refer
PointNet++ throughout and use it as our baseline.
3 Methodology: From PointNet++ to PointNeXt
In this section, we present how modernize the classical architecture PointNet++ [28] into PointNeXt, the next version of PointNet++ with SOTA performance. Our exploration mainly focuses on two aspects: (1) training modernization improve data augmentation and optimization techniques, and (2) architectural modernization to probe receptive field scaling and model scaling, Both aspects have important impact on the model's performance, but were under-explored by previous studies.
3.1 Training Modernization: PointNet++ Strikes Back
We conduct systematic study to quantify the effect of each data augmentation and optimization technique used by modern point cloud networks [44, 41, 54] and propose a set of improved training strategies. The potential of PointNet++ can be unveiled by adopting our proposed training strategies.
3.1.1 Data Augmentation
Data augmentation is one of the most important strategies to boost the performance of neural network; thus we start our modernization from there. The original PointNet++ used simple combi- nations of data augmentations from random rotation, scaling, translation, and jittering for various benchmarks [28]. Recent methods adopt stronger augmentations than those used in PointNet++. For example, KPConv [41] randomly drops colors during training, Point-BERS [52] uses a common point resampling strategy to randomly sample 1, 024 points from the original point cloud for data scaling, while RandLA-Ner [14] and Point Transformer [54] load the entire scene as input in segmentation tasks. In this paper, quantify the effect of each data augmentation through an additive study.
We start our study with PointNet++ [28] as the baseline, which trained with the original data augmentations and optimization techniques. We remove each data augmentation to check whether it necessary or not. We add back the useful augmentations but remove the unnecessary ones. We then systematically study all the data augmentations used in the representative works [44, 41, 30, 54, 26, 52], including data scaling such as point resampling [52] and loading the entire scene as input [14], random rotation, random scaling, translation to shift point clouds, jittering to add independent noise to each point, height appending [41] (i.e., appending the measurement of each point along the gravity direction of objects additional input features), color auto-contrast to automatically adjust color contrast [54], and color drop that randomly replaces colors with zero values. We verify the effectiveness of data augmentation incrementally and only keep the augmentations that give better validation accuracy. At the end of this study, we provide a collection of data augmentations for each task that allow for the highest boost in the model's 's performance. Sec. 4.4.1 presents and analyzes in detail the uncovered findings.
3.1.2 Optimization Techniques
Optimization techniques including loss functions, optimizers, learning rate schedulers. and hyperpa- rameters are also vital to the performance of a neural network. PointNet++ uses the same optimization techniques throughout its experiments: CrossEntropy loss, Adam optimizer [15], exponential learning rate decay (Step Decay), and the same hyperparmeters. Owing to the development of machine learning theory, modern neural networks can be trained with theoretically better optimizers (e.g. AdamW [25] vs. Adam [15]) and more advanced loss functions (CrossEntropy with label smoothing [37]). Similarly to our study on data augmentations, we also quantify the effect of each modern optimization technique on PointNet+++ We first perform sequential hyperparameter search for the learning rate and weight decay. We then conduct an additive study on label smoothing, optimizer, and learning rate scheduler. We discover set of improved optimization techniques that further
3

---

Feat.
Set Abstraction
Fest. prtionation
MLP
ResL
Set bsrractionn
Fost. Propaggtion
huvResMLP
Set Astracction
invResMLP
Set Abstraction
In-RosMLP
Propucation
[N:32]
[N,32]
[N
NN4664
[N/16,128]
[N/642256]
[N/64,256]
[N11,1225]
[N/256.512]
The differences proposed PointNeXI and the classical PointNett* are highlighted borders.
Subsamplc
Grouping
MLPs
Reduction
Grouping
MLP (256)
Reductioo
MLP MLP (1024)
MLP (25) (256)
Intespolate
concat
Figure 2: PointNeXt architecture. PointNeXt shares the same Set Abstraction and Feature Propaga- tion blocks as PointNet++ [28]. while adding an additional MLP layer at the beginning and scaling the architecture with the proposed Inverted Residual MLP (InvResMLP) blocks.
boost performance by decent margin. In general, CrossEntropy with label smoothing, Adam W, and Cosine Decay can decently optimize models various tasks. See Sec, 4.4.1 for detailed findings.
3.2
Architecture Modernization: Small Modifications - Big Improvements
In this subsection, we modernize PointNet++ [28] into the proposed PointNeXt. The modernization consists of two aspects: (1) receptive field scaling and (2) model scaling.
3.2.1 Receptive Field Scaling
The receptive field significant factor in the design space of neural network [36, 6]. There are at least two ways to scale the receptive field in point cloud processing: (1) adopting a larger radius to query the neighborhood, and (2) adopting a hierarchical architecture. Since the hierarchical architecture has been adopted in the original PointNet++, we mainly study (1) in this subsection. Note that the radius of PointNet++ is set to initial value r that doubles when the point cloud downsampled. We study a different initial value in each benchmark and discover that the radius is dataset-specific and can have significant influence on performance. This is elaborated Sec. 4.4.2.
Furthermore, we find that the relative coordinates A, = p'} p in Eq. (I) make network optimization harder, leading decrease in performance. Thus, we propose relative position normalization (Ap normalization) to divide relative position by the neighborhood query radius:
(2) Without normalization, values of relative positions (A, P p¹) considerably small (less than the radius), requiring the network to learn a larger weight to apply on Ap. This makes the optimization non-trivial, especially since weight decay is used to reduce the weights of the network and thus tends to ignore the effects of relative position. The proposed normalization alleviates this issue by rescaling and in the meantime reduces the variance of Ap among different stages.
x = READEN (he (Ix); (p', - PP)//11)}
3.2.2 Model Scaling
PointNet++ is a relatively small network, where the encoder consists of only stages in the clas- sification architecture and 4 stages for segmentation. Each stage consists of only 1 SA block, and each block contains layers MLP. The model sizes of PointNet++ for both classification and segmentation are less than 2M, which is much smaller compared to modern networks that typically use more than 10M parameters [41, 26, 30). Interestingly, we find that neither appending more SA blocks using more channels leads to a noticeable improvement in accuracy, while causing significant drop in throughput (refer to Sec. 4.4.2), mainly due to vanishing gradient and overfitting. Therefore, in this subsection, study how scale up PointNet++ in an effective and efficient way.
We propose an Inverted Residual MLP (InvResMI.P. block to be appended after the first SA block, per stage, for effective and efficient model scaling. InvResMLP is built on the SA block and is

---

illustrated at the bottom middle of Fig. 2. There are three differences between InvResMLP and SA. (1) A residual connection between the input and the output is added to alleviate the vanishing gradient problem [12], especially when the network goes deeper. (2) Separable MLPs are introduced to reduce computation and reinforce pointwise feature extraction. While all layers of MLPs in the original SA block are computed on the neighborhood features, InvResMLP separates the MLPs into single layer computed on the neighborhood features (between the grouping and reduction layers) and two layers for point features (after reduction), as inspired by MobileNet [13] and ASSANet [30]. (3) The inverted bottleneck design [34] is leveraged to expand the output channels of the second MLP by 4 times to enrich feature extraction. Appending InvResMLP blocks is proven to significantly improve performance compared to the appending of the original SA blocks (see Sec. 4.4.2).
In addition to InvResMLP, we present three changes in the macro architecture. design of PointNet++ encoder for classification and segmentation, i.e., scaling the number of SA blocks for classification from 2 to 4 while keeping the original number (4 blocks) for segmentation at each stage. (2) We utilize a symmetric decoder in which its channel size is changed to match the encoder. (3) We add stem MLP, an additional MLP layer inserted at the beginning of the architecture, to map the input point cloud to higher dimension.
(1) We unify the
In summary, we present PointNeXt, the next version of PointNets [27, 50], modified from PointNet++ by incorporating the proposed InvResMLP and the aforementioned macro-aree aacro-architecturll changes. The architecture of PointNeXt is illustrated in Fig. 2. We denote the channel size of the stem MLP as C' and the number of InvResMLP blocks as B. A larger C' leads to an increase in the width of the network (i.e., width scaling), while larger B leads to an increase in the depth of the network (i.e., depth scaling). Note that when = 0 only one SA block and no InvResML. blocks are used at each stage. The number of MLP layers in the SA block is set to 2, and a residual connection is added inside each SA block. When B= * O, InvResMLP blocks are appended after the original SA block. The number of MLP layers in the SA block in this case is set to 1 to save computation cost. The configuration of our PointNeXt family is summarized as follows:
PointNext-S: C C= 32, B =
PointNeXt-L: C =32, 2= (2,4,,,,
PointNeXt-B: C = 32, B = 11,)
PointNcXt-XL: C = 64, B (3,6,3.3
4
Experiments
We evaluate PointNeXt five standard benchmarks: S3DIS [1] and ScanNet [5] for semantic segmentation, ScanObjecıNN [42] and ModelNe140 [47] for object classification, and ShapeNetPart [3] for object part segmentation.
Experimental Setups. We train PointNeXt using CrossEntropy loss with label smoothing [37], AdamW optimizer [25], an initial Iearning rate Ir 0.001, weight decay 10 with Cosine Decay, and a batch size of 32, with a 32G V100 GPU, for all tasks, unless otherwise specified. The best model on the validation set is selected for testing. For S3DIS segmentation, point clouds are voxel downsam- pled with a voxel size of 0.04m following common practice [41, 30, 54). PointNeXt is trained with an initial Ir 0.01, for 100 epochs (training set is repeated by 30 times), using fixed number of points (24, 000) per batch with a batch size of as input. During training. the input points are obtained by querying the nearest neighbors of a random point in each iteration. Following Point Transformer [54], we evaluate PointNeXt using the entire soxxe-ddwnnsmplee. scene as input. For ScanNet scene segmentation, we follow the Stratified Transformer [16] and train PointNeXt with multi-step learning rate decay and decay at [70,90] epochs with a decay rate of 0.1 without label smoothing. The voxel size is set to 0.02m and input number of points in training set 64, 000. We train the model for 100 epochs (training set is repeated for 6 times) with a batch size of per GPU with 8 GPUs. For ScanOb- jectNN classification, PointNeXt is trained with a weight decay of 0.05 for 250 epochs. Following Point-BERT [52], the number of input points is set to 1, 024, where the points are randomly sampled during training and uniformly sampled during testing (denoted as point resampled augmentation). For ModelNet40 classification, PointNeXt is trained similarly as ScanObjectNN but for 600 epochs. For ShapeNetPart part segmentation, we train PointNeXt using a batch size of per GPU with GPUs, and Poly FocalLoss [17] as criterion, for 400 epochs. Following PointNet+++ 2,048 randomly sampled points with normals are used as input for training and testing. The details of data augmentations used in S3DIS, ScanNet, ScanObjectNN, ModelNet40 and ShapeNetPart are detailed in Sec. 4.4.1.
5

---

Table 1: 3D semantic segmentation in S3DIS (evaluation by 6-Fold or in Area 5) and ScanNet V2. For PointNeXt in S3DIS Area 5, the average results without voting in three random runs are reported. The improvements of PointNeXt over the original performance reported by PointNet++ [28] are highlighted in green color. PointNet++ (ours) denotes PointNet++ trained using our improved data augmentation and optmization techniques. Methods are in chronological order.
Method
mlnU
S3DIS 6-Fold
OA
mloL (8)
S3DIS Area-5
OA
Val mloU
ScanNet V2
Test mlaU (%)
Params.
FL.PPs Throughput
(%)
(%)
(%)
(%)
M
(ins/sec.)
PointNet [22]
47.6
78.5
41.1
PoiniCNN [21]
65.4
88.1
57.3
85,9
45.8
0.6
DGCNN [44]
56.1
84.1
47.9
83,6
1.3
DeepCCN [20]
60.0
85.9
KPCone [41]
70.6
67.1
69.2
68.6
15.0
RandLA-Net [14]
20.0
88.0
64.5
1.3
5,8
159
BAAF-Net
72.2
88-9
65.4
88.9
5.0
Transformer (54)
73.5
90.2
90.8
70.6
7.8
CHI. [89]
75.1
89.6
69.4
90.6
70.5
18.6
PointNett+ [28]
54.5
81.
53.5
83.0
53.5
55.7
1.0
7.2
186
PointNets (ouns)
68.14-188)
87.6
63.2+04.9.77 +9.7)
37.5+02.-445)
57.2:433)
PointNeXt-S (ours)
68.0.-13.5)
63.4-0..8 +9.9)
37.9:10.3:449)
64.5:-11.))
0.8
3.6
227
PointNeXt-IB (ours)
71.5:+12.0)
-1.8
67.3++.2:4433.
89.4+0011444
68.4;-149)
3.8
8.9
158
PointNeXt-L (ours)
73.90
89.8 8:30
69.0::0:5 (15.5)
90.0:0.1::700
69.4:1159)
PointNeXt-XL (ours)
74.9 (-30.4)
90.3 (19.3)
70.5+ -17.0)
90.6:201::744
71.5
-1.2
41.6
84.8
For all experiments except ShapeNetPart segmentation, we do not conduct any voting [22]², since more standard to compare the performance without using any ensemble methods suggested by Simple [8]. However, we found that the performance in ShapeNetPart of nearly all models is quite close to each other, where it is hard to achieve state-of-the-e art IoUs without voting. We also provide model parameters (Params.) and inference throughput (instances per second) for comparison. The throughput of all methods is measured using 128 1024 (batch size 128, number of points 1024) as input in ScanObjectNN and ModelNet40 and 64 x 2048 in Shape In S3DIS, 16 x 15, 000 points are used measure throughput following [30], since some methods [44, 19] could not process the whole scene due to memory constraints. The throughput of all methods is measured using an NVIDIA Tesla V100 32GB GPU and a 32 core Intel Xeon @ 2.80GHz CPU.
4.1 3D Semantic Segmentation in S3DIS and ScanNet
S3DIS [1] (Stanford Large-Scale 3D Indoor Spaces) is a challenging benchmark composed of 6 large-scale indoor areas, 271 rooms, and 13 semantic categories in total. The standard 6-fold cross-validation results in S3DIS are reported in Tab. Note that the official PointNet++ [28] did not conduct experiments in S3DIS. Here, we use the results reported by PointCNN [21] for comparison. Our PointNeXt-S, the smallest variant, outperforms PointNet++ by 13.5% 6.4%, and 10.2% in terms of mean IoU (mloU), overall accuracy (OA), and mean accuracy (mAcc), respectively, while being faster terms of throughput. The increased speed is due to the reduced number of layers in the SA block for PointNeXt-S (see Sec. 3.2.2). With the proposed model scaling, the performance of PointNeXt can be gradually boosted, For example, PointNeXt-L outperforms SOTA Point Transformer [54] by 0.4% in mloU while being 3x faster. Note that Point Transformer utilizes most of the improved training strategies ours. XXL,, the extra large variant, achieves mloU/OA/mAAc of 74.9%/90.33% 3/8.0%%%%//..0.0. 3.0%, while running faster than Point Transformer. As limitation, our PointNeXt-XL. consists of more parameters and more computationally expensive terms of FLOPs, mainly due to channel expansion (x4) in the inverted bottleneck and doubled initial channel size (C = 64). We also provide the results of XointNeXt in S3DIS area 5 in the Tab. with tsd in three random runs, where PointNeXt achieves similar improvements as the 6-fold experiments.
ScanNet [5], another well-known large-scale segmentation dataset, contains 3D indoor scenes of various rooms with 20 semantic categories. We follow the public training, validation, and test splits, with 1201, 312 and 100 scans, respectively. For PointNet+++ we use the results reported from the Stratified Transformer [16] for comparison. As shown in Tab. 1, we improve PointNet++ from 53.5% mlou to 57.2% mloU in the validation set by adopting the improved training strategies (detailed in supplementary material). PointNeXt-S further gains 11.0 in val mloU over the original PointNet++ mostly due to the use of a smaller radius (0. - 0.03m) and relative position normalization. The performance in ScanNet improves steadily with the increase in model sizes. Our largest variant,
'The voting strategy combines results by using randomly augmented points as input enhance performance.

---

Table 2: 3D object classification in ScanObjectNN and ModelNet40. Averaged results three random runs using 1024 points imput without normals and without voting are reported.
ScanObjectNN (PB. (PB_T50 RS)
ModelNeet40
Params.
FLOPs
Throughput (ins./sec.)
Method
(%)
(%8
(%)
PointNet [27]
68.2
63.4
89.2
86.2
3.5
0.9
4212
PoinCCNN [21]
78.5
75.1
92.2
88.
0.6
44
DGCNN [44]
78.1
73.6
92.9
90.2
1.8
4.8
402
DeepGCN [19]
93.6
90.9
2.2
3.9
263
KPConv [41]
92.9
14.3
ASSANct-L [30]
92.9
118.4
153
Simple View [8]
80.510.3
93.020.4
90.520.8
MVTN [11]
82.8
93.5
92.2
3.5
1.8
236
Point Cloud Transformer [10]
93.2
CurveNet [48]
93.8
2,0
22
PointMT.P [26]
83.9+-1.5
94.1
13.2
191
PoiniNet++ [28]
77.9
75.4
91.9
1.5
1872
PointNet++ (ours)
(6-
84.2++0 9 +888)
92.8:00
89.9=20.8
1.5
1.7
1872
PointNeXt-SS (ours)
(7:7--0.44--.88)
65.8+-0.6:
93.2.0.1:413))
90.8+00.2
1.4
2040
PointNeXt-Xi outperforms PointNet++ by 18.0% mloU validation and achieves 71.2% mloU in testing, beating the recent methods Point Transformer [54] and CBL [39].
4.2 3D Object Classification in ScanObjectNN and ModelNet40
ScanObjectNN [42] contains about 15, 000 real scanned objects that are categorized into 15 classes with 2, 902 unique object instances. Due occlusions and noise, ScanObjectNN poses significant challenges to existing point cloud analysis methods. Following PointMLP [26]. we experiment on PB T50 RS, the hardest and most commonly used variant of ScanObjectNN. As reported in Tab. 2, the proposed PointNeXt-S surpasses existing methods by non-trivial margins in terms of both OA and mAcc, while using much fewer model parameters and running much faster. Built upon PointNet++ [28], PointNeXt achieves significant improvements over the originally reported performance PointNet++,, i.e. +9.8% OA and +10.4% mACC. This demonstrates the efficacy the proposed training and model scaling strategies. PointNeXt also outperforms SOTA PointMLP [26] (i.e. +2.3% OA, +1.9% mACC), while running 10x faster. This shows that PointNeXt simple, yet effective, and efficient baseline. Note that we did not experiment with upscaled variants of PointNeXt on this benchmark, since we found that the performance had saturated using PointNeXt-S mostly due to the limited scale of the dataset.
commonly used 3D object classification dataset, which has 40 object categories, each of which contains 100 unique CAD models. However, recent works [11, 26, 32] show increasing interest in the real-world scanned dataset ScanObejectNN compared to this synthesized dataset. Following this trend, we mainly benchmarked PointNeXt in ScanObjectNN, Here, also provide our results ModelNet40. Tab. 2 shows that advanced training strategies improve PointNet++ from 91.9% OA to 92.8% OA without any architecture change. PointNeXI-S (C 32) outperforms the original reported PointNet++ by 1.3% OA, while being faster. Note that PointNeXt-S with a larger width C = 64 can achieve higher overall accuracy (94.0%).
ModelNer40 [47]
4.3 3D Object Part Segmentation in ShapeNetPart
ShapeNetPart [51] is widely-used dataset for object-level part segmenta- tion. It consists of 16, 880 models from 16 different shape categories, 2-6 parts for each category, and 50 part labels in total. As shown in Tab. 3, our Poi with default width (C 32) obtains a performance comparable
Table 3: Part segmentation in ShapeNetPart.
Method
mlol.
mIoU
Params.
Throughpur
PaintNet
83.7
80.4
3.6
4.9
1184
DGCNN [44]
85.2
82.3
1.3
12.4
147
KPCom [41]
86.4
85.1
44
CurveNei [48]
86.8
97
ASSANet-L
86.1
6410
Point Transformer [54)
86.6
83.7
7.8
297
PointMLPP M..PPPPP [26]
86.
84.6
270
Stratihedformer [16]
86.6
85.1
398
PointNete+ [28]
85.1
81.9
1.0
708
PointNeXt-S
86.7:10.0:++..
04.4::0.2::-.55
1.0
4.5
782
PointNeXt-S (C=64)
86.91.0.1:
3.7
17.8
331
PointNeXt-S (C=160)
87.0 97.0--0.1::-110
8..2 +
22.5
110.2
76

---

to that of the SOTA CurveNet [48] and outperforms a large number of representative networks, such as KPConv [41] and ASSANet [30] in terms of both instance mean IoU (ins. mloU) and throughput. Due to the small scale of ShapeNetPart, the model would overfit after being depth scaled. However, we find by increasing the width from 32 to 64 instead, PointNeXt can outperform CurveNet, while being over 4x faster. It is also worth highlighting that PointNeXt with an even larger width (C' 160) reaches 87.0% Ins. mloU, whereas the performance of point- methods has saturated below this value for years. We highlight that we used voting only in ShapeNetPart by averaging the results of randomly scaled input point clouds, with scaling factors equal to [0.8,1.2]. Without voting, notice performance drop around 0.5 instance mloU.
4.4 Ablation and Analysis
Tab. 4 and Tab. 5 present additive studies for the proposed training and scaling strategies in ScanObjectNN [42] and S3DIS [1], respectively. We adopt the original PointNet++ the baseline. In ScanObjectNN, PointNet++ was trained by [42] with CrossEntropy loss, Adam optimizer, learning rate le-3, weight decay of le-4, step decay of 0.7 for every 20 epochs, and batch size of 16, for 250 epochs, while using random rotation and jittering as data augmentations. The official PointNet++ did not conduct experiments in S3DIS dataset. We refer to the widely used reimplementation [50], where PointNet++ was trained with the same settings as ScanObjeetNN except that only random rotation was used augmentation. Note that for all experiments, we train all our models for 250 epochs in ScanObjectNN and for 100 epochs in S3DIS.
4.4.1 Training Strategies
Data augmentation is the first aspect that we study to modernize PointNet++. We draw four conclusions based on observations in Tab. 4 and 5. (1) Data scaling improves performance for both classification and segmentation tasks. For example, point resampling is shown to boost the performance by 2.5% OA in ScanObjectNN, Taking the entire scene as input instead of using the block or sphere subsampled input as done in PointNet++ [28] and other previous works [41, 20, 30] improves the segmentation result by 1.1% mloU, (2) Height appending improves performance, especially for object classification. Height appending makes the network aware of the actual size of the objects, thus leading to increase in accuracy (+1.1% OA). (3) Color drop is strong augmentation that significantly improves the performance of tasks where colors are available. Adopting color drop alone adds 5.9% mloU in S3DIS area 5. We hypothesize that color drop forces the network to focus more on the geometric relationships between points, which in turn improves performance. (4) Larger models favor stronger data augmentation. Whereas random rotation drops the performance PointNet++ by 0.3% mloU in S3DIS (2nd row in Tab. 5 data augmentation part), it is shown to be beneficial for larger-scale models (e.g. raises 1.5% mloU PointNeXt-B), Another example in ScanObjectNN shows that the removal of random jittering also adds 1.1 % OA. In general,
Table 4: Additive study of sequentially applying train- ing and scaling strategies for classification on ScanOb- jectNN. We use light green, purple, yellow, and pink background colors to denote data augmentation, op- timization techniques, receptive field scaling, and model scaling, respectively.
Table 5: Additive study of sequentially apply- ing training and scaling strategies for segmen- tation on S3DIS area 5. +/- denote adopt-
ing/removing the strategy.
Improvements
mloU (%)
A
PointNet++
51.5
Improvements
OA (%)
A
+ Entire scene as input
52.6 52.6 0.5
+1.1
52.9
+0.3
PointNet++
77.9
Height appending
0.4
+0.5
Point resampling
81.4 0.6
+3,5
+ Color drop
0.7
+5.9
Jittering
82.5 0.4
+1.1
+ Color auto-contrast
6..0
+1.7
+ Height appending
83.6
+1.1
Ir 0.001 0.01
0.5
+0.5
+ Lahee Smoothing
+1.
+0.4
Random scaling
8.2
+0.1
+ Adam AdamW
0.6
+0.6
Label Smoothing
0.5
+1.3
Adamm - SGD
0.5
-3.1
Adam AdamW
0.1
+0.6
Step Decay Cosine Decay
63.2 014
+0.7
AdamW - SGD
84.8 +.8
-0.8
+ Normalize A
63
+0.4
+ Step Decay + Cosine Decay
86.1 0.7
+0.5
+ Scale down (PointNeXi-S)
63.4 018
-0.2
+ Scale up (PointNeXt-B)
65.8 0.5
+2.4
Radius 0.2 - 0.15
86.4 + 0.3
+0.3
Rotation
67.3
+1.5
Normalizing Ap (Eqn. (2))
86.7 + 0.3
+0.3
Scale up (PointNeXt-L)
69.0 0.5
+1.7
Scale up (PointNeXi-S)
87.7 1 0.4
+1.0
+ Scale up (PointNeXt-XL)
70.5 1 0.3
+1.5
8

---

with the improved data augmentations, the OA of PointNet++ in ScanObjectNN and the mloU in S3DIS area are increased by 5.8% and 9.5%, respectively.
Optimization techniques involve loss functions, optimizers, learning rate schedulers, and hyperpa- rameters. As shown in Tab. 4 and 5, Label Smoothing, AdamW [25] optimizer, and Cosine Decay consistently boost performance in both classification and segmentation tasks. This reveals that the more developed optimization methods such as label smoothing and AdamW are generally good for optimizing a neural network. Compared to Step Decay, Cosine Decay is also easier to tune (usually only the iniiill and minimum learning rates are required) and can achieve performance similar to Step Decay. Regarding hyperparameters, using a learning rate greater than that used in PointNet++ improves the segmentation performance in S3DIS.
In general, our training strategies consisted of stronger data augmentation and modern optimization techniques can increase the performance of PointNet+ from 77.9% 86.1% OA in ScanObjectNN dataset, impressively surpassing SOTA PointMLP by 0.7%. The mloUs in S3DIS area 5 and S3DIS 6-fold (illustrated in Fig. are boosted by 11.7 and 13.6 absolute percentage points, respectively. Our observations imply that significant portion of the performance gap between classical +ointNet+ and SOTA due to the training strategies.
Generalize to other networks. Although the training strategies proposed for +ointNet++ [28], find that they can be applied to other methods such as PointNet [27], DGCNN [44], and PointMLPP [26], and also improve their performance. Such generalizability is validated in ScanObjectNN [42]. As shown in Tab. 6, the OA of the representative methods can all be improved when equipped with our training strategies.
Table 6: The generalizability of im- proved training strategies. OA on ScanObjectNN networks trained with improved training strategies is reported.
Method
ours
PointNet [27]
74.4 + 0.9
+6.2
DGCNN [44]
86.0 0.5
PointMLP [26]
87.1 + 0.7
+1.7
4.4.2 Model Scaling
Receptive field scaling includes both radius scaling and normalizing A, defined in Eqn. (2), which are also validated in Tab, 4 and 5. The radius is dataset specific, while down-scaling the radius from 0.2 to 0.15 improves 0.3% OA in ScanObjectNN, keeping the radius the same 0.1 achieves the best performance in S3DIS. Regarding normalizing Ap. it improves the performance in ScanObjeetNN and S3DIS by 0.3 OA and 0.4 mloU, respectively. Furthermore, in Tab. Ap has a larger impact (2.3 mloU in S3DIS dataset) on the bigger model PointNext-XL.
show that normalizing
Model scaling scales PointNet++ + by the proposed InvResMLP and some macro- aanro-arcciitcturall changes (see Sec. 3.2.2). Tab. 4, we show that PointNeXt-S using the stem MLP, the symmetric decoder, and the residual connection in the SA block improves 1.0% OA in ScanObjectNN. Performance in the large-scale S3DIS dataset can be further unveiled (from 63.8% to 70.5% mloU) by up- scaling PointNeXt-S using more blocks of the proposed InvResMLP, as demonstrated in Tab. 5. Furthermore, in Tab. 7,we ablate each component of the proposed InvResMLP block and different stage ratios in S3DIS area using the perf-perf model PointNeXt-XL as the baseline. As observed, each architectural change indeed con-
Table 7: Ablate architectural changes on S3DIS. and denote removing from baseline and throughput.
Ablate
mloU
A
TP
baseline (PointNeXt-XI.)
70.5 0.3
normalizing Åp
68.2 0.7
-2.3
45
tributes to increased performance. Among
residual connection
64.0 +.00+.. 1.0
-6.5
45
all changes, the residual connection is the
MIP
0.4
-0.4
46
most essential, without which the mIoU
Separable MLPs
6.8
-3.9
will drop from 70.5% only 64.0%. The separable MLPs increase 3.9% mloU while speeding up the network 3 times. Removing the inverted bottleneck from the baseline leads to drop of 1.5% mloU with less than 1% gain in speed. Adding more blocks inside each stage after removing inverted bottleneck can improve its performance to 69.7 0.3 but is still lower than the baseline. Another possibility is to use
Inverted bottleneck
6.4
-1.5
48
Inverted bottleneck
69.7
-0.8
43
stage ratio - (1:1:1:1)
6.68
-0.7
52
stage ratio - (2:1:1:1)
6.4
-1.1
41
stage ratio - (1:1:2:1)
6.6
-0.6
47
stage ratio - (1:1:1:2)
6.4
-1.0
48
stage ratio + (1:3:1:1)
7..1
-0.4
39
naive width scaling
59.4
-11.1
43
naive depth scaling
6.5
-7.1
53
naive compound scaling
62.3
-8.2
24

---

bottleneck design to shrink the channel size by 4 times in the middle of the module, and expand the network width or depth to achieve the same speed as the baseline. However, the best performance of bottleneck design only achieves 1.4% less mloU compared to inverted bottleneck. Tab. 7 also shows the performance of naive width scaling that increases the width of PointNet++ from 32 to 256 to match the throughput of PointNeXt-XI., naive depth scaling to append more SA blocks in PointNet++ to obtain the same number of blocks of PointNext-Xi XL whose B = (3, 6, 3, 3), and naive compound scaling to double the width of the naive depth scaled model to the same width as Poi (C' = 64). Our proposed model scaling strategy achieves much higher performance than these naive scaling strategies, while being much faster.
5 Related Work
Point-based methods process point clouds directly using their unstructured format compared to voxel-based methods [9, 4] and multi view-based methods [35, 11, 8]. PointNet [27]. the pioneering work of point-based methods, proposes to model the permutation invariance of points with shared MLPs by restricting feature extraction to be pointwise. PointNet++ [28] is presented to improve PointNet by capturing local geometric structures. Currently, most point- based methods focus on the design of local modules. [44, 43, 29] rely on graph neural networks. [49, 21, 41, 40] project point clouds onto pscudo grids to allow for regular convolutions. [46, 22, 23] adaptively aggregate neigh- borhood features through weights determined by the local structure. In addition, very recent methods leverage Transformer-lise
networks [54, 16] extract local information through self-attention Our work does not follow this trend in local module design. In contrast, we shift our attention to another important but largely under-explored aspect, L..., the training and scaling strategies.
Training strategies are studied recently in [2, 45, 24] on image classification. In the point cloud domain, SimpleView [8] is the first work to show that training strategies have a large impact on the performance of a neural network. However, Simple simply adopts the same training strategies as DGCNN [44]. On the contrary, we conducted a systematic study quantify the effect of each data augmentation and optimization technique, and propose a set of improved training strategies that boost the performance of PointNet++ [28] and other representative works [27, 44, 26].
Model scaling can significantly improve the performance of network, as shown in pioneering works in various domains [38, 53, 20]. Compared to PointNet++ [28] that uses parameters less than 2M, most current prevailing networks consist of parameters greater than 10 M, such as KPConv [41] (15M) and PointMLP [26] (13M). In our work, we explore model scaling strategies that can scale up PointNet++ in an effective and efficient manner. We offer practical suggestions on scaling technologies that improve performance, namely using residual connections and an inverted bottleneck design, while maintaining throughput by using separable MLPs.
Conclusion and Discussion
In this paper, we demonstrate that with improved training and scaling strategies, the performance of PointNet++ can be increased to exceed the current state of the art. More specifically, we quantify the effect of each data augmentation and optimization technique that are widely used today, and propose set of improved training strategies. These strategies can be easily applied to boost the performance of PointNet++ and other representative works. We also introduce the Inverted Residual MLP block into PointNet++ to develop PointNeXt. We demonstrate that PointNeXt has superior performance and scalability over PointNet++ on various benchmarks while maintaining high throughput. This work aims to guide researchers toward paying more attention to the effects of training and scaling strategies and motivate future work in this direction.
Limitation. Even though PointNeXt-XL is one of the largest models among all representative point-based networks, its number of parameters (44M) is still below that of small networks in image classification such as ConNeXt-S [24] (50M) and ViT-B [7] (87M), and is far from their large variants, including Con (350M) and ViT-L (305M). We do not push the model size further, mainly due to the smaller-scale nature of point cloud datasets. Moreover, our work is limited to existing modules since the focus is not on introducing new architectural changes.
Acknowledgement The authors would like to thank the reviewers of NeurlPS' 22 for their constructive suggestions. This work was supported by the KAUST Office of Sponsored Research through the Visual Computing Center (VCC) funding, as well as, the SDAIA-KAUST Center of Excellence in Data Science and Artificial Intelligence SDAIA-KAUSH AI).
10

---

References
[1] Iro Armeni, Ozan Sener, Amir R Zamir, Helen Jiang, loannis Brilakis, Martin Fischer, and Silvio Savarese. semantic parsing of large-scale indoor spaces. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1534-1543, 2016.
[2] Irwan Bello, William Fedus, Xianzhi Du, Ekin Dogus Cubuk, Aravind Srinivas, Tsung-Yr Yi Lin, Jonathon Shlens, and Barret Zoph. Revisiting resnets: Improved training and scaling strategies. Advances in Neural Information Processing Systems (NeurIPS), 34, 2021.
[3] Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song. Hao Su, et al. Shapenet: An information-rich 3d model repository. arXiv preprint /512 2015.
[4] Christopher Choy, Jun Young Gwak, and Silvio Savarese. 4d spatio-temporal convnets: Minkowski convolutional neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 3075-3084, 2019.
[5] Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias NieBner. ScanNet: Richly- annotated 3D reconstructions of indoor scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
[6] Xiaohan Ding, Xiangyu Zhang, Yizhuang Zhou, Jungong Han. Guiguang Ding, and Jian Sun. Scaling up your kernels 31x31: Revisiting large kernel design in cans. In Proceedings the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), 2022.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x 16 words: Transformers for image recognition at scale. In International Conference on Learning Representations (ICLR), 2021.
[8] Ankit Goyal, Hei Law, Bowei Liu, Alejandro Newell, and Jia Deng. Revisiting point cloud shape classification with a simple and effective bascline. In Proceedings of the International Conference on Machine Learning (ICML), pages 3809-3820. PMLR, 2021.
[9] Benjamin Graham, Martin Engelcke, and Laurens Van Der Maaten. 3d semantic segmentation with submanifold sparse convolutional networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9224-92222222 9232, 2018.
[10] Meng-Hor Hao Guo, Jun Lun-Xiong Cai, Zheng-Ning Liu, Tai- Jiang Mu, Ralph R Martin, and Shi-Min Hu. Pet: Point cloud transformer. Computational Visual Media, 7(2):187-199 2021.
[11] Abdullah Hamdi, Silvio Giancola, and Bernard Ghanem. Mvtn: Multi-view transformation network for 3d shape recognition. In Proceedings of the IEEE/CVF International Conference Computer Vision (ICCV), pages 1-11, 2021.
[12] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 770-778, 2016.
[13] Andrew Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv: 1e0.. 04861, 2017.
[14] Qingyong Hu, Bo Yang, Linhai Xie, Stefano Rosa, Yulan Guo, Zhihua Wang. Niki Trigoni, and Andrew Markham. Randla-net: Efficient semantic segmentation of large-scale point clouds. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition (CVPR), pages 1118--1117, 2020.
[15] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR), 2015.
[16] Xin Lai, Jianhui Liu, Li Jiang, Liwei Wang, Hengshuang Zhao, Shu Liu, Xiaojuan Qi, and Jiaya Jia. Stratified transformer for point cloud segmentation. In Proceedings of the IEFF/CVF Conference on Computer Vision and Puttern Recognition (CVPR), 2022.
[17] Zhaoqi Leng. Mingxing Tan, Chenxi Liu, Ekin Dogus Cubuk, Xiaojie Shi, Shuyang Cheng, and Drago Anguelov. Polyloss: A polynomial expansion perspective of classification loss functions. In International Conference on Learning Representations (ICLR), 2022.

---

[18] Guohao Li, Matthias Müller, Bernard Ghanem, and Vladlen Koltun. Training graph neural networks with 1000 layers. In Proceedings the International Conference on Machine Learning (ICML), volume 139. pages 6437-6449, PMLR, 2021.
[19] Guohao Li, Matthias Müller, Guocheng Qian, Itzel C. Delgadillo, Abdulellah Abualshour, Ali K. Thabet, and Bernard Ghanem. Deepgens: Making gens go as deep as cnns. IEEE transactions on pattern analysis and machine intelligence (T-PAMI), PP, 2021.
[20] Guohao Li, Matthias Muller, Ali Thabet, and Bernard Ghanem, Deepgens: Can gens go deep as cnns? In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 9267-9276, 2019.
[21] Yangyan Li, Rui Bu, Mingehao Sun, Wei Wu, Xinhan Di, and Baoquan Chen. Pointenn: Convolution on X' r--transformed points. Advances in Newral Information Processing Systems (NeurIPS), 31, 2018.
[22] Yongeheng Liu, Bin Fan, Shiming Xiang, and Chunhong Pan. Relation-shape convolutional neural network for point cloud analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8887-8896, 2019.
[23] Ze Liu, Han Hu, Yue Cao, Zheng Zhang, and Xin Tong. A closer look local aggregation operators in point cloud analysis. In Proceedings of the European Conference on Computer Vision (ECCV), pages Springer. 2020.
[24] Zhuang Liu, Hanzi Mao, Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xic. A convnet for the 2020s. In Proceedings of the IEÉE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), 2022.
[25] Tlya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations (ICLR), 2019.
[26] Xu Ma, Can Qin, Haoxuan You, Haoxi Ran, and Yun Fu. Rethinking network design and local geometry in point cloud: A simple residual MLP framework. In International Conference on Learning Representations (ICLR), 2022.
[27] Charles Ruizhongtai Qi, Hao Su, Kaichun Mo, and Leonidas J. Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
[28] Charles Ruizhongtai Qi, Yi, Hao Su, and Lconidas Guibas. Pointnet++: Deep hicrarchical feature lcarning on point sets in metric space. In Advances in Neural Information Processing Systems (NeurIPS), 2017.
[29] Guocheng Qian, Abdulellah Abualshour, Guohao Li, Ali Thabet, and Bernard Ghanem. Pu-gen: Point cloud upsampling using graph convolutional networks. Tn Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11683-11661 1662,, June 2021.
[30] Guocheng Qian, Hasan Hammoud, Guohao Li, Ali Thabet, and Bernard Ghanem. Assanet: An anisotropic separable set abstraction for efficient point cloud representation learning. volume 34, 2021.
[31] Shi Qiu, Saeed Anwar, and Nick Barnes, Semantic segmentation for real point cloud scenes via bilateral augmentation and adaptive fusion. In Proceedings of the IEEF//VF Conference Computer Vision and Pattern Recognition (CVPR), pages 1757-1767, 2021.
[32] Haoxi Ran, Jun Liu, and Chengjic Wang. Surface representation for point clouds. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
[33] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer assisted intervention (MICCAT). 2015.
[34] Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Biang-Chhel Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4510-4520, 2018
[35] Hang Su, Subhransu Maji, Evangelos Kalogerakis, and Erik G. Learned-Mllee Multi-view convolutional neural networks for 3d shape recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 2015.
12

---

[36] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott E. Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), 2015.
[37] Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zhigniew Wojna. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patiern Recognition (CVPR), 2016.
[38] Mingxing Tan and Quoe V. Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In Proceedings of the International Conference on Machine Learning (ICML), volume 97, pages 6105-61 14, PMLR, 2019.
[39] Liyao Tang. Yibing Zhan, Zhe Chen, Baosheng Yu, and Dacheng Tao. Contrastive boundary learning for point cloud segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
[40] Maxim Tatarchenko, Jaesik Park. V. Koltun, and Qian- Vian-YiZe Zhou. Tangent convolutions for dense prediction in 3d. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pages 3887-3896, 2018.
[41] Hugues Thomas, Charles R Qi, (lan--mmaanre Deschaud, Beatriz Marcotegui, François Goulette, and Leonidas J Guibas. Kpconv: Flexihle and deformable convolution for point clouds. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019.
[42] Mikaela Angelina Uy, Quang-Hieu Pham, Binh-Son Hua, Due Thanh Nguyen, and Sai-Kit Yeung. Revisit- ing point cloud classification; A new benchmark dataset and classification model on real-world data. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019.
[43] Lei Wang. Yuchun Huang, Yaolin Hou, Shenman Zhang, and Jie Shan. Graph attention convolution for point cloud semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
[44] Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay Sarma, Michael M. Bronstein, Justin M. Solomon. Dynamic graph can for learning on point clouds. ACM Transactions on Graphics (TOG), 2019.
[45] Ross Wightman, Hugo Touvron, and Hervé Jégou. Resnet strikes back: An improved training procedure in timm. arXiv preprint arXiv:21 10.00477, 2021.
[46] Wenxuan Wu, Zhongang Qi, and Li Fuxin. Pointconv: Deep convolutional networks on 3d point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
[47] Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. shapenets: deep representation for volumetric shapes. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
[48] Tiange Xiang, Chaoyi Zhang, Yang Song, Jianhui Yu, and Weidong Cai. Walk in the cloud: Learning curves for point clouds shape analysis. In Proceedings the IEEE/CVF International Conference on Computer Vision (ICCV), pages 915-924, 2021.
[49] Yifan Xu, Tianqi Fan, Mingye Xu, L. Zeng, and Yu Qiao. Spiderenn: Deep learning on point sets with parameterized convolutional filters. In Proceedings of the European Conference on Computer Vision (ECCV), 2018
[50] Xu Yan. Pointnet/pointuett*+ pytorch, 2019.
[51] Li Yi, Vladimir Kim, Duygu Ceylan, Shen, Mengyan Yan, Hao Su, ARCewu Lu, Qixing Huang, Alla Sheffer, Leonidas Guibas, al. A scalable active framework for region annotation in 3d shape collections. ACM Transactions on Graphics (TOG), 35(6)2210, 2016.
[52] Xumin Yu, Lulu Tang, Yongming Rao, Tiejun Huang, Jie 7hou, and Jiwen Lu. Point-bert: Pre-training 3d point cloud transformers with masked point modeling. In Proceedings of the IEFF/CVF Conference on Computer Vision and Puttern Recognition (CVPR), 2022.
[53] Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. Scaling vision transformers. In Proceedings of the IEFE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 12104-12113, 2022.
[54] Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip HS Torr and Vladien Koltun. Point transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 2021.
13


Reviews:
[{"review_id": "n_w2z7l38F", "reviewer": "Reviewer_8bUj", "summary": "The paper aims to improve PointNet++ by slightly modifying PointNet++, increasing model scale and employing a few training strategies. Comprehensive experiments on S3DIS for semantic segmentation, ScanObjectNN for real-world object classification, and ShapeNetPart for object part segmentation. ", "questions": "1. The novelty seems a bit limited. The receptive field scaling (normalization) looks interesting and technically sound. However, according to the ablation study, the improvement is not that significant.  \n\n2. In Table 4 and Table 5, it cloud be better to include SGD as a reference. \n", "limitations": "No significant limitations. ", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "1. Although the novelty seems limited, the paper may have a positive effect on the community. Some existing works (even actually also based on PointNet++) just follow the trend and apply popular techniques to point clouds without very convincing reasons. The paper may cool down such behaviours. \n\n2. The paper looks solid. The proposed method achieves several SOTAs on multiple tasks and datasets. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "jFk3dYVtln-", "reviewer": "Reviewer_jBW5", "summary": "This paper explores a variety of tricks to improve the performance of PointNet++ for point cloud classification and segmentation. They first show that by incorporating modern training strategies and data augmentations, the vanilla version of PointNet++ can be improved significantly. Then, to scale the PointNet++ for better performance, they propose the InvResMLP that adopts residual connection and separate MLPs in Set Abstraction layer to support scalable PointNet++, where the relative coordinates are also normalized to ease the model optimization. The experiments demonstrate that the proposed improved version of PointNet++ (PointNeXt) achieves promising performance on 3D semantic segmentation (S3DIS), 3D object classification (ScanObjectNN) and 3D part segmentation (ShapeNetPart). ", "questions": "How about the performance of PointNeXt in larger datasets like ScanNet? Can it be extended to larger outdoor scenes like SemanticKITTI?", "limitations": "The limitations have been well discussed in the paper. ", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n* The paper conducts extensive experiments to explore various data augmentation and training strategies, which greatly improve the performance of vanilla PointNet++.\n* Although simple, the proposed normalization of relative coordinate is reasonable for easing the optimization.   \n* The experiments show that the proposed InvResMLP can enable a larger model to improve the model performance. \n\nWeaknesses:\n*  Although the reviewer agrees that the authors have systematically explored the potential of vanilla PointNet++, the technical contribution of this paper is still not enough to be published at NeurIPS: see below comments.  \n* The explored data augmentation and training strategies are well-known tricks for training a point cloud model, and it is also well-known that vanilla PointNet++ can achieve much better performance than the original officially reported performance. \n* The proposed InvResMLPis is also straightforward by adding a residual connection (to solve vanishing gradient) and moving more MLPs after max-pooling (to reduce calculations). \n* The PointNet++ is hard to scalable to larger point cloud scenes, such as some outdoor scenes, and also the performance of PointNet++ series is inferior to sparse-voxel-based networks (like in larger dataset ScanNet). This paper does not show that an improved version of PointNet++ can still be the next-generation framework for point cloud processing in such a larger dataset.    \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "2GYnhDYK8a", "reviewer": "Reviewer_fSmm", "summary": "In this paper, authors work on the point cloud understanding task and point out that we may need to revisit the training startegies and scaling methods in thi field. By detailed analysis and experiments, authors claimed that a large improvement in this field is introduced by these two aspects, rather than the network architecture designs in recent years. Furthermore, authors also introduce micro designs to improvement the PointNet++, and achieve state-of-the-art performance in three benchmarks. \n\nI like this paper, revisting the training strategies and micro designs instead of proposing something \"novel\" and new. I will give an accpet. ", "questions": "1. In L45， authors claimed that \"For the S3DIS segmentation benchmark, can increase by 13.6% \". Is this simply achieved by training strategies (i.e., data augmentation and optimization techniques) and without any modifications to network structure?\n\n2. I wonder why the authors consider the InvBlock design? What if we change it to a bottleneck structure in ResNet? \n\n3. From Tab.4 and Tab. 5, we can see that normalizing the relative positions by Eq. (2) could only improve the performance by 0.3\\% for both ScanObjectNN and S3DIS. Compared to other modifications, it could not be considered a \"significant influence (L138)\".\n\n4. In L142, the authors claimed: \"without normalization, ... larger weight...\" I am interested in the distributions of position encoding layer weights. Also, if the authors cloud explain L143, \"This makes ..., ... weight decay...\" What results would weight decay cause?", "limitations": "Authors addressed the limitations and  potential negative societal impact of their work.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. I like the motivation in this paper. A revisiting in point cloud understanding is necessary and can help others greatly.\n2. The detailed experiments and analysis make the experimental results solid. \n3. The presentation and organization are good. \n\nIn general, I like this paper and recommend accept.\n\nWeaknesses:\n1. In L138, the authors claimed that \"the radius is dataset-specific ...\" This phenomenon is not a finding. We can always expect detailed hyper-parameters can cause varying performances on different datasets. A general fixed setting is appreciated.\n2. InvResMLP present nothing new. the Inv-Res-Block has been proposed by MobileNetV2 and ConvNeXt already. \n3. For the experiments, I acknowledge authors repeated all experiments three times. However, a critical issue is that all three datasets are small. To prove the main idea's effectiveness, I suggest authors conduct experiments on large datasets, like ScanNet for segmentation and SUN RGB-D for object detection. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "1q2oN6W1SD8", "reviewer": "Reviewer_LDZj", "summary": "As the title suggests, the paper revisits and studies PointNet++, a popular neural network for point cloud representation learning. Based on the study, the paper discovers two results: first, it proposes a set of training strategies to boost the performance on PointNet++ so that it achieves SOTA results on various benchmarks; and second, it shows how PointNet++ can be modified and scaled to further boost performance.", "questions": "Several questions have been asked in the Weakness sections. Overall the paper has merits but there are parts where the paper is weak. I will update my review based on the rebuttal.", "limitations": "NA", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The paper achieves impressive performance on various benchmarks (87.7% on ScanObjectNN, 74.9% mean IOU on S3IS, 87.2 IOU on ShapeNetPart)\n- The paper conducts a systematic study showing the improvement in performance because of various factors (Table 4 and Table 5). In my opinion, such findings are useful.\n- The paper adopts ways to improve the PoinNet++  architecture. This could be useful for future research as they can use a more improved and modern version of PointNet++.\n- The paper shows how the findings about training strategies are generalizable to network architectures in Table 6, however, this result should be made more rigorous as discussed in the weaknesses section.\n\nWeaknesses:\n- The study showing that the findings about training strategies are generalizable to other networks is very useful (as discussed in the strengths) but limited. Specifically, only two other networks have been evaluated. I would suggest evaluating at least one simple model like SimpleView and a recent model like PointMLP.\n- It is unclear why information about FLOPs has been omitted in Table 2 and Table 3. It would be important for readers to know the strengths and limitations of various methods. Similarly, information about Parameter Count is only omitted in Table 3 for the ShapeNetPart dataset. It is unclear why specific parameters are omitted for different datasets in the main paper.\n- In Table 2, PointNet, DGCNN, and additional models (SimpleView, PointMLP) with improved training strategy should be added, so that readers can quantify the difference in performance purely as a result of network architecture and also compare models across parameter count, throughput and FLOPs.\n- Potential misrepresentation of prior work: The paper says “However, SimpleView simply adopts the same training strategies as DGCNN. In contrast, we conduct a systematic study to quantify the effect of each data augmentation and optimization technique”. It seems like a misrepresentation of prior work as SimpleView does compare models across different augmentation and optimization schemes, and finds that the ones used in DGCNN are better. The paper should provide more information about what is meant in this comparison.\n- Information about scaling is confusing. From my understanding, PointNeXt-S is the smaller valiant (L182-L185), however, in Table 4 PointNeXt-S is mentioned alongside Scale-up. Similarly, scaling rules for ShapeNetPart are not consistent (Table 3). Does the usual scaled-up model not perform well on this dataset? If so, this should be discussed.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept