R ETHINKING N ETWORK D ESIGN AND L OCAL G EOMETRY IN P OINT C LOUD : A S IMPLE R ESIDUAL MLP F
RAMEWORK
A BSTRACT
Point cloud analysis is challenging due to irregularity and unordered data structure.
To capture the 3D geometries, prior works mainly rely on exploring sophisticated local geometric extractors using convolution, graph, or attention mechanisms.
These methods, however, incur unfavorable latency during inference, and the performance saturates over the past few years.
In this paper, we present a novel perspective on this task.
We notice that detailed local geometrical information probably is not the key to point cloud analysis – we introduce a pure residual MLP network, called PointMLP, which integrates no “sophisticated” local geometrical extractors but still performs very competitively.
Equipped with a proposed lightweight geometric afﬁne module, PointMLP delivers the new stateof-the-art on multiple datasets.
On the real-world ScanObjectNN dataset, our method even surpasses the prior best method by 3.3 % accuracy.
We emphasize that PointMLP achieves this strong performance without any sophisticated operations, hence leading to a superior inference speed.
Compared to most recent CurveNet, PointMLP trains 2 × faster, tests 7 × faster , and is more accurate on
ModelNet40 benchmark.
We hope our PointMLP may help the community towards a better understanding of point cloud analysis.
The code is available at https:/ github.com ma-xu/pointMLP-pytorch.
I NTRODUCTION
Lately, point cloud analysis has emerged as a popular topic in 3D understanding, attracting attention from academia and industry (Qi et al., 2017a; Shi et al., 2019; Xu et al., 2020).
Different from 2D images represented by regular dense pixels, point clouds are composed of unordered and irregular sets of points P ∈ R N × 3 , making it infeasible to apply image processing methods to point cloud analysis directly.
Meanwhile, the nature of sparseness and the presence of noises further restrict the performance.
In the past few years, endowing with neural networks, point cloud analysis has seen a great improvement in various applications, including 3D shape classiﬁcation (Qi et al., 2017a), semantic segmentation (Hu et al., 2020) and object detection (Shi & Rajkumar, 2020), etc.
Recent efforts have shown promising results for point cloud analysis by exploring local geometric information, using convolution (Li et al., 2021a), graph (Li et al., 2021a), or attention mechanism (Guo et al., 2021)
(see Section 2 for details).
These methods, despite their gratifying results, have mainly relied on the premise that an elaborate local extractor is essential for point cloud analysis, leading to the competition for careful designs that explore ﬁne local geometric properties.
Nevertheless, sophisticated extractors are not without drawbacks.
On the one hand, due to prohibitive computations and the overhead of memory access, these sophisticated extractors hamper the efﬁciency of applications in natural scenes.
As an example, until now, most 3D point cloud applications are still based on the simple PointNet ( and PointNet++) or the voxel-based methods (Liu et al., 2021;
Li et al., 2021b; Zhang et al., 2021).
However, applications that employ the aforementioned advanced methods are rare in literature.
On the other hand, the booming sophisticated extractors saturate the performance since they already describe the local geometric properties well. A more complicated design is no longer to improve the performance further.
These phenomena suggest that we may need to stop the race of local feature extraction designing, rethinking the necessity of elaborate local feature extractors and further revisiting the succinct design philosophy in point cloud analysis. [Figure] [Table] [Figure]
[Figure]
Net40.
Our PointMLP performs best.
Please refer to Section 4 for details.
In this paper, we aim at the ambitious goal of building a deep network for point cloud analysis using only residual feed-forward MLPs, without any delicate local feature explorations.
By doing so, we eschew the prohibitive computations and continued memory access caused by the sophisticated local geometric extractors and enjoy the advantage of efﬁciency from the highly-optimized MLPs.
To further improve the performance and generalization ability, We introduce a lightweight local geometric afﬁne module that adaptively transforms the point feature in a local region.
We term our new network architecture as PointMLP.
In the sense of MLPbased design philosophy, our PointMLP is similar to PointNet and PointNet++ (Qi et al., 2017a;b).
However, our model is more generic and exhibits promising performance.
Different from the models with sophisticated local geometric extractors (e.g.,
DeepGCNs (Li et al., 2019), RSCNN (Liu et al., 2019b), RPNet (Ran et al., 2021).), our PointMLP is conceptually simpler and achieves results on par or even better than these state-of-the-art methods (see Figure 1).
Keep in mind that we did not challenge the advantages of these local geometric extractors and we acknowledge their contributions; however, a more succinct framework should be studied considering both the efﬁciency and accuracy.
In Table 1, we systemically compare our PointMLP with some representative methods.
Even though the design philosophy is simple, PointMLP (as well as the elite version) exhibits superior performance on 3D point cloud analysis.
Speciﬁcally, we achieve the state-of-the-art classiﬁcation performance, 94.5% , on the ModelNet40 benchmark, and we outperform related works by 3.3% accuracy on the real-world ScanObjectNN dataset, with a signiﬁcantly higher inference speed.
R ELATED W ORK
Point cloud analysis.
There are mainly two streams to process point cloud.
Since the point cloud data structure is irregular and unordered, some works consider projecting the original point clouds to intermediate voxels (Maturana & Scherer, 2015; Shi et al., 2020) or images (You et al., 2018;
Li et al., 2020), translating the challenging 3D task into a well-explored 2D image problem.
In this regime, point clouds understanding is largely boosted and enjoys the fast processing speed from 2D images or voxels.
Albeit efﬁcient, information loss caused by projection degrades the representational quality of details for point clouds (Yang et al., 2019).
To this end, some methods are proposed to process the original point cloud sets directly.
PointNet (Qi et al., 2017a) is a pioneering work that directly consumes unordered point sets as inputs using shared MLPs.
Based on PointNet,
PointNet++ (Qi et al., 2017b) further introduced a hierarchical feature learning paradigm to capture the local geometric structures recursively.
Owing to the local point representation (and multi-scale information), PointNet++ exhibits promising results and has been the cornerstone of modern point cloud methods (Wang et al., 2019; Fan et al., 2021; Xu et al., 2021a).
Our PointMLP also follows the design philosophy of PointNet++ but explores a simpler yet much deeper network architecture.
Local geometry exploration.
As PointNet++ built the generic point cloud analysis network framework, the recent research focus is shifted to how to generate better regional points representation.
Predominantly, the explorations of local points representation can be divided into three categories: convolution-, graph-, and attention-based methods.
One of the most distinguished convolution-based methods is PointConv (Wu et al., 2019).
By approximating continuous weight and density functions in convolutional ﬁlters using an MLP, PointConv is able to extend the dynamic ﬁlter to a new convolution operation.
Also, PAConv (Xu et al., 2021a) constructs the convolution kernel by dynamically assembling basic weight matrices stored in a weight bank.
Without modifying network conﬁgurations, PAConv can be seamlessly integrated into classical MLP-based pipelines.
Unlike convolution-based methods, Graph-based methods investigate mutually correlated relationships among points with a graph.
In Wang et al.
(2019), an EdgeConv is proposed to generate edge features that describe the relationships between a point and its neighbors.
By doing so, a local graph is built, and the point relationships are well preserved.
In 3D-GCN (Lin et al., 2021), authors aim at deriving deformable 3D kernels using a 3D Graph Convolution Network.
Closely related to graphbased methods, the attention-based methods exhibit excellent ability on relationship exploration as well, like PCT (Guo et al., 2021) and Point Transformer (Zhao et al., 2021; Engel et al., 2020).
With the development of local geometry exploration, the performances on various tasks appear to be saturated. Continuing on this track would bring minimal improvements.
In this paper, we showcase that even without the carefully designed operations for local geometry exploration, a pure deep hierarchical MLP architecture is able to exhibit gratifying performances and even better results.
[Table]
(cid:51) (cid:51)
MLP
Deep network architecture for point cloud.
Interestingly, the development of point cloud analysis is closely related to the evolution of the image processing network.
In the early era, works in the image processing ﬁeld simply stack several learning layers to probe the performance limitations (Krizhevsky et al., 2012; Simonyan & Zisserman, 2015;
Dong et al., 2014).
Then, the great success of deep learning was significantly promoted by deep neural architectures like ResNet (He et al., 2016), which brings a profound impact to various research ﬁelds.
Recently, attention-based models, including attention blocks (Wang et al., 2018) and Transformer architectures (Dosovitskiy et al., 2021), further ﬂesh out the community.
Most recently, the succinct deep MLP architectures have attracted a lot of attention due to their efﬁciency and generality.
Point cloud analysis follows the same develop history as well, from MLP-based PointNet (Qi et al., 2017a), deep hierarchical PointNet++ (Qi et al., 2017b), convolution-/graph-/relation- based methods (Wu et al., 2019; Wang et al., 2019; Ran et al., 2021), to state-of-the-art Transformer-based models (Guo et al., 2021; Zhao et al., 2021).
In this paper, we abandon sophisticated details and present a simple yet effective deep residual MLP network for point cloud analysis.
Instead of following the tendency in the vision community deliberately, we are in pursuit of an inherently simple and empirically powerful architecture for point cloud analysis.
D EEP R ESIDUAL MLP FOR P OINT C LOUD
We propose to learn the point cloud representation by a simple feed-forward residual MLP network (named PointMLP), which hierarchically aggregates the local features extracted by MLPs, and abandons the use of delicate local geometric extractors.
To further improve the robustness and improve the performance, we also introduce a lightweight geometric afﬁne module to transform the local points to a normal distribution.
The detailed framework of our method is illustrated in Figure 2.
3. R EVISITING P OINT - BASED M ETHODS
The design of point-based methods for point cloud analysis dates back to the PointNet and PointNet++ papers (Qi et al., 2017a;b), if not earlier.
The motivation behind this direction is to directly consume point clouds from the beginning and avoid unnecessary rendering processes.
Given a set of points P = { p i | i = 1 , · · · , N } ∈ R N × 3 , where N indicates the number of points in a ( x, y, z ) Cartesian space, point-based methods aims to directly learn the underlying representation f of P using neural networks.
One of the most pioneering works is PointNet++, which learns hierarchical features by stacking multiple learning stages.
In each stage, N s points are re-sampled by
Geometric
Affine
R e s P
B l o c k
R e s P
B l o c k
R e s P
B l o c k
R e s P
B l o c k
R e s P
B l o c k
R e s P
B l o c k
R
Geometric Affine Module
R e L U M L P B N M L P
B N
R e L U
Residual Point (ResP) Block
R Repeat
Subtraction
Product
Had.
Product
Summation the farthest point sampling (FPS) algorithm where s indexes the stage and K neighbors are employed for each sampled point and aggregated by max-pooling to capture local structures.
Conceptually, the kernel operation of PointNet++ can be formulated as:
[Equation] where A ( · ) means aggregation function (max-pooling in PointNet++), Φ ( · ) denotes the local feature extraction function (MLP in PointNet++), and f i,j is the j -th neighbor point feature of i -th sampled point.
By doing so, PointNet++ is able to effectively capture local geometric information and progressively enlarge the receptive ﬁelds by repeating the operation.
In the sense of network architecture design, PointNet++ exhibits a universal pipeline for point cloud analysis.
Following this pipeline, some plug-and-play methods have been proposed, mainly focusing on the local feature extractor Φ ( · ) (Xu et al., 2021a;
Liu et al., 2019b; Thomas et al., 2019;
Zhao et al., 2021).
Generally, these local feature extractors thoroughly explore the local geometric information using convolution, graph, or self-attention mechanisms.
In RSCNN (Liu et al., 2019b), the extractor is mainly achieved by exploring point relations as follow:
[Equation]
where [ · ] is the concatenation operation and MLP is a small network composed of a Fully-connected (FC) layer, Batch Normalization layer, and activation function.
Unlike RSCNN, Point Transformer introduces the self-attention mechanism into point cloud analysis and considers the similarities between pair-wise points in a local region.
To this end, it re-formulates the extractor as:
[Equation]
where γ, ϕ, ψ and α are linear mapping function, “ (cid:12) ” is a Hadamard product, and ρ is a softmax normalization.
In particular, Point Transformer introduces a relative position encoding, δ = θ ( x i − x i,j ) , where the relative position is encoded by two FC layers with a ReLU nonlinearity layer, into both attention weights and features.
The lightweight positional encoder largely improves the performance of Point Transformer.
While these methods can easily take the advantage of detailed local geometric information and usually exhibit promising results, two issues limit their development.
First, with the introduction of delicate extractors, the computational complexity is largely increased, leading to prohibitive infer- ence latency 1 .
For example, the FLOPs of Equation 3 in Point Transformer would be 14 Kd 2 , ignoring the summation and subtraction operations.
Compared with the conventional FC layer that enjoys 2 Kd 2 FLOPs, it increases the computations by times.
Notice that the memory access cost is not considered yet.
Second, with the development of local feature extractors, the performance gain has started to saturate on popular benchmarks.
Moreover, empirical analysis in Liu et al.
(2020) reveals that most sophisticated local extractors make surprisingly similar contributions to the network performance under the same network input.
Both limitations encourage us to develop a new method that circumvents the employment of sophisticated local extractors, and provides gratifying results.
3. F RAMEWORK OF P OINT MLP
In order to get rid of the restrictions mentioned above, we present a simple yet effective MLP-based network for point cloud analysis that no sophisticated or heavy operations are introduced.
The key operation of our PointMLP can be formulated as:
[Equation]
where Φ pre ( · ) and Φ pos ( · ) are residual point MLP blocks: the shared Φ pre ( · ) is designed to learn shared weights from a local region while the Φ pos ( · ) is leveraged to extract deep aggregated features.
In detail, the mapping function can be written as a series of homogeneous residual MLP blocks, MLP ( x ) + x , in which MLP is combined by FC, normalization and activation layers (repeated two times).
Following Qi et al.
(2017a), we consider the aggregation function A ( · ) as max-pooling operation.
Equation 4 describes one stage of of PointMLP.
For a hierarchical and deep network, we recursively repeat the operation by s stages.
Albeit the framework of PointMLP is succinct, it exhibits some prominent merits.
1) Since PointMLP only leverages MLPs, it is naturally invariant to permutation, which perfectly ﬁts the characteristic of point cloud.
By incorporating residual connections, PointMLP can be easily extended to dozens layers, resulting deep feature representations.
In addition, since there is no sophisticated extractors included and the main operation is only highly optimized feed-forward MLPs, even we introduce more layers, our PointMLP still performs efﬁciently.
Unless explicitly stated, the networks in our experiments use four stages, and two residual blocks in both Φ pre ( · ) and Φ pos ( · ) .
We employ k-nearest neighbors algorithm (kNN) to select the neighbors and set the number K to 24.
3. G EOMETRIC A FFINE M ODULE
While it may be easy to simply increase the depth by considering more stages or stacking more blocks in Φ pre and Φ pos , we notice that a simple deep MLP structure will decrease the accuracy and stability, making the model less robust.
This is perhaps caused by the sparse and irregular geometric structures in local regions.
Diverse geometric structures among different local regions may require different extractors but shared residual MLPs struggle at achieving this.
We ﬂesh out this intuition and develop a lightweight geometric afﬁne module to tackle this problem.
Let { f i,j } j =1 , ··· ,k ∈
R k × d be the grouped local neighbors of f i ∈ R d containing k points, and each neighbor point f i,j is a d -dimensional vector.
We transform the local neighbor points by the following formulation:
[Equation]
where α ∈ R d and β ∈ R
d are learnable parameters, (cid:12) indicates Hadamard production, and (cid:15)
= 1 e − 5 is a small number for numerical stability (Ioffe & Szegedy, 2015; Wu & He, 2018; Dixon & Massey Jr, 1951).
Note that σ is a scalar describes the feature deviation across all local groups and channels.
By doing so, we transform the local points to a normal distribution while maintaining original geometric properties.
3. C OMPUTATIONAL C OMPLEXITY AND E LITE V ERSION
Although the FC layer is highly optimized by mainstream deep learning framework, the theoretical number of parameters and computational complexity are still high.
To further improve the efﬁciency, [Table] we introduce a lightweight version of PointMLP named as pointMLP-elite , with less than 0.7M parameters and prominent inference speed (176 samples/second on ModelNet40 benchmark) .
Inspired by He et al.
; Hu et al.
[Table] we present a bottleneck structure for the mapping function Φ pre and Φ pos .
We opt to reduce the channel number of the intermediate FC layer by a factor of r and increase the channel number as the original feature map.
This strategy is opposite to the design in Vaswani et al.
(2017); Touvron et al.
(2021) which increases the intermediate feature dimensions.
Empirically, we do not observe a signiﬁcant performance drop.
This method reduce the parameters of residual MLP blocks from 2 d 2 to 2 r d 2 .
By default, we set r to 4 in PointMLPelite.
Besides, we also slightly adjust the network architecture, reducing both the MLP blocks and embedding dimension number (see appendix for details).
Inspired by Xie et al.
(2017), we also investigated a grouped FC operation in the network that divides one FC layer into g groups of subFC layers, like group convolution layer.
However, we empirically found that this strategy would largely hamper the performance.
As a result, we did not consider it in our implementation.
E XPERIMENTS
In this section, we comprehensively evaluate PointMLP on several benchmarks.
Detailed ablation studies demonstrate the effectiveness of PointMLP with both quantitative and qualitative analysis.
4. S HAPE CLASSIFICATION ON M ODEL N ET
We ﬁrst evaluate PointMLP on the ModelNet40 (Wu et al., 2015) benchmark, which contains 9, training and 2,468 testing meshed CAD models belonging to 40 categories.
Following the standard practice in the community, we report the class-average accuracy (mAcc) and overall accuracy (OA) on the testing set.
We train all models for 300 epochs using SGD optimizer.
[Figure]
Experimental results are presented in Table 2.
Among these methods, our PointMLP clearly outperforms state-of-the-art method CurveNet by 0.3% (94.5% vs. 94.2%) overall accuracy with only 1k points.
Note that this improvement could be considered as a promising achievement since the results on ModelNet40 recent methods have been saturated around 94% for a long time.
Even without the voting strategy (Liu et al., 2019b), our PointMLP still performs on par or even better than other methods that are tested with voting strategy.
Despite having better accuracy, our method is much faster than the methods with sophisticated local geometric extractors.
We compare PointMLP to several open-sourced methods and report the parameters, classiﬁcation accuracy, training, and testing speed.
As we stated previously, a key intuition behind this experiment is that model complexity can not directly reﬂect efﬁciency.
For example, CurveNet is lightweight and delivers a strong result, whereas the inference cost is prohibitive (15 samples/second) .
On the contrary, our PointMLP presents a high inference speed (112 samples/second) .
To further reduce the model size and speed up the inference, we present a lightweight PointMLP-elite, which signiﬁcantly reduces the number of parameters to 0.68M, while maintaining high-performance 90.9% mAcc and 94.0% OA on ModelNet40.
With PointMLP-elite, we further speed up the inference to 176 samples/second .
4. S HAPE C LASSIFICATION ON S CAN O BJECT NN
We examine all methods on the most challenging variant (PB T50 RS).
For our pointMLP and PointMLP-elite, we train and test for four runs and report mean ± std results.
[Table]
While ModelNet40 is the de-facto canonical benchmark for point cloud analysis, it may not meet the requirement of modern methods due to its synthetic nature and the fast development of point cloud analysis.
To this end, we also conduct experiments on the ScanObjectNN benchmark (Uy et al., 2019).
ScanObjectNN is a recently released point cloud benchmark that contains 15,000 objects that are categorized into 15 classes with 2,902 unique object instances in the real world.
Due to the existence of background, noise, and occlusions, this benchmark poses signiﬁcant challenges to existing point cloud analysis methods.
We consider the hardest perturbed variant (PB T50 RS) in our experiments.
We train our model using an SGD optimizer for 200 epochs with a batch size of 32.
For a better illustration, we train and test our method for four runs and report the mean ± standard deviation in Table 3. [Table] [Table]
Empirically [Table] our PointMLP surpasses all methods by a signiﬁcant improvement on both class mean accuracy (mAcc) and the overall accuracy (OA) .
For example, we outperform PRANet by 4.8% mAcc and 3.3% OA.
Even compared with the heavy multi-view projection method MVTN ( views), our PointMLP still performs much better (85.39% 82.8% ).
Notice that we achieve this by fewer training epochs and did not consider the voting strategy.
Moreover, we notice that our method achieves the smallest gap between class mean accuracy and overall accuracy.
This phenomenon indicates that PointMLP did not bias to a particular category, showing decent robustness.
4. A BLATION STUDIES
Network Depth.
Network depth has been exploited in many tasks but is rare in point cloud analysis.
We ﬁrst investigate the performance of PointMLP with different depths in Table 4.
We vary the network depth by setting the number of homogeneous residual MLP blocks to 1, 2, and 3, respectively, resulting in 24, 40, and 56-layers PointMLP variants.
Detailed depth formulation can be found in Appendix D. At ﬁrst glance, we notice that simply increasing the depth would not always bring better performance; an appropriate depth would be a good solution.
Additionally, the model gets stable with more layers introduced, as demonstrated by the decreasing standard deviation.
When the depth is set to 40, we achieve the best tradeoff between accuracy and stability (85.4% mean accuracy and 0.3 standard deviations).
Remarkably, PointMLP consistently achieves gratifying results that outperform recent methods, regardless of the depth.
Geometric Afﬁne Module.
Other work provides sophisticated local geometric extractors to explore geometric structures.
Instead, our PointMLP discards these burdensome modules and introduces a lightweight geometric afﬁne module.
Figure 3 presents the results of PointMLP with/without the geometric afﬁne module.
By integrating the module, we systematically improve the performance of PointMLP by about 3% for all variants.
The reasons for this large improvement are two-fold.
First, the geometric afﬁne module maps local input features to a normal distribution, which eases the training of PointMLP.
Second, the geometric afﬁne module implicitly encodes the local geometrical information by the channel-wise distance to local centroid and variance, remedying the deﬁciency of geometric information.
Besides the gratifying improvements, the geometric afﬁne module also largely boosts the stability of PointMLP, suggesting better robustness.
[Figure]
[Equation] [Figure] [Equation] [Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
Component ablation
study.
Table 5 reports the results on ScanObjectNN of removing each individual component in
PointMLP.
Consistent with
Figure 3, geometric afﬁne module plays an important role in PointMLP, improving the base architecture by 3.9%.
Remarkably, even without this module, which is an unfair setting for PointMLP, our base network stills achieves [Table] 81.4 84.0 PointMLP 84.
5 ± 1 .
4% OA, outperforming most related methods (see Table 3).
Removing Φ pre function (MLPs before aggregator A ), the performance drops 2.6% overall accuracy.
Combining all these components together, we achieve the best result 85.4% OA.
See Appendix C for more ablations.
Loss landscape.
We depict the 3D loss landscape (Li et al., 2018a) in Figure 4.
Simply increasing the network depth may not achieve a better representation and even hamper the results.
When removing the residual connection in PointMLP, the loss landscape turns sharp, and the performance plummets to 88.1% (6% drop) on ModelNet40.
With residual connection, we greatly ease the optimization course of PointMLP and make it possible to train a deep network.
4. P ART SEGMENTATION
Our PointMLP can also be generalized to other 3D point cloud tasks.
We next test PointMLP for 3D shape part segmentation task on the ShapeNetPart benchmark (Yi et al., 2016).
The shapeNetPart dataset consists of 16,881 shapes with 16 classes belonging to 50 parts labels in total.
In each class, the number of parts is between 2 and 6.
We follow the settings from Qi et al.
(2017b) that randomly select 2048 points as input for a fair comparison.
We compare our methods with several recent works, including SyncSpecCNN (Yi et al., 2017), SPLATNet (Su et al., 2018), etc.
We also visualize the segmentation ground truths and predictions in Figure 5.
Intuitively, the predictions of our PointMLP are close to the ground truth.
Best viewed in color.
C ONCLUSION
In this paper, we propose a simple yet powerful architecture named PointMLP for point cloud analysis.
The key insight behind PointMLP is that a sophisticated local geometric extractor may not be crucial for performance.
We begin with representing local points with simple residual MLPs as they are permutation-invariant and straightforward.
Then we introduce a lightweight geometric afﬁne module to boost the performance.
To improve efﬁciency further, we also introduce a lightweight counterpart, dubbed as PointMLP-elite.
Experimental results have shown that PointMLP outperforms related work on different benchmarks beyond simplicity and efﬁciency.
We hope this novel idea will inspire the community to rethink the network design and local geometry in point cloud.
P OINT MLP D ETAIL
We detail the architecture of PointMLP in Figure 6 (as well as PointMLP-elite in Figure 7) for a better understanding.
Compared with PointMLP, the elite version mainly adjusts three conﬁgurations: 1) it reduces the number of residual point (Resp) MLP blocks; 2) it reduces the embedding dimension from 64 to 32, hence the overall model overhead is signiﬁcantly alleviated; 3) by introducing a bottleneck structure, PointMLP further reduces the parameters by four times.
For part segmentation task, we use the framework presented in PointNet (Qi et al., 2017a) and replace the backbone to our PointMLP. With the only modiﬁcation, we improve the performance from 85.1 to 86.1 Instance mIoU.
[Figure]
I npu t
[Figure]
m
[Figure]
e dd i ng
[Figure]
gg
[Figure]
g
[Figure] F C [Figure] N
R e L U (
[Figure]
D r opou t ( 0 .
[Figure]
F C B N
R e L U (
[Figure]
D r opou t ( 0 . 5 )
F C ( c l a ss
[Figure]
nu
[Figure]
[Figure]
t
[Figure]
B D ETAIL E XPERIMENTAL S ETTING B. M ODEL N ET 40 AND S CAN O BJECT NN
Our implementations are based on PyTorch.
For ModelNet40, we train models for 300 epochs on one Tesla V100 GPU with a batch size of 32.
All our models are trained using synchronous SGD with a Nesterov momentum of 0.9 and a weight decay of 0.0002.
The learning rate is set to 0.
initially.
We use the cosine annealing scheduler (Loshchilov & Hutter, 2017) to adjust the learning rate.
For each sample, we randomly select 1024 points and consider the same augmentation strategy as Qi et al.
(2017b).
The setting for ScanObjectNN is similar to ModelNet40, except we train all models for only 200 epochs.
For the reported speed in Table 2, we test the open-source code on a Tesla V100-pcie GPU.
All the source codes we used are listed 2 in the footnote.
B. S HAPE N ET P ART
Our setting for part segmentation task is following PointNet (Qi et al., 2017a).
We randomly sample 2048 points for each sample and re-scale the input in a range of [0 . 67 , 1 . 5] .
Note that we did not test the result using a multi-scale testing strategy, which could further improve the performance, but is not realizable in real-world applications.
Hence, we only report the single-scale results.
Even the comparison is unfair, we still achieve competitive performance.
C M ORE D ETAILED A BLATION S TUDIES
Skip connection.
Figure 4 shows the loss landscapes of our PointMLP with and without skip connections.
We also consider adding skip connections to PointNet++ to validate the effectiveness of skip connections.
Due to the structure of PointNet++, only two skip connections could be added without modifying the original architecture of PointNet++.
By adding the skip connections, we achieve a classiﬁcation accuracy of 92.7% on ModelNet40 in our re-implementation.
Pre-MLP block vs. Pos-MLP block.
we also modiﬁed the conﬁguration of our PointMLP and retrained the model to investigate the importance of Pre-MLP and Pos-MLP blocks.
In our original implementation, we set the pre-MLP block list to [2, 2, 2, 2] and the pos-MLP blocks list to [2 , 2 , 2 , 2] .
Here, we remove the pos-MLP blocks and change the pre-MLP blocks to [4 , 4 , 4 , 4] to match the block number.
The 3-layer classiﬁer can be considered as the MLP at the end of the last stage.
We trained the models two times and got an average OA of 84.13% (83.87% and 84.39%), which is lower than vanilla PointMLP 85.4%, and even the result in Table 5 second-row 84.7%.
This result indicates that pos-MLP does beneﬁt our PointMLP, and simply adding more pre-MLP blocks does not help.
We acknowledge that the effect of pos-MLP is not as strong as other components and believe that a detailed ﬁne-tuning of the conﬁgurations would deliver an even better performance-efﬁciency balance.
Geometric Afﬁne Module Applications.
Geometric afﬁne module plays an essential role in our
PointMLP, exhibiting promising performance improvements.
While this module can be considered as a plug-and-play method, the overlap with some local geometric extractors in other methods may limit its application.
Here we integrate the module to two popular methods, PointNet++ and
DGCNN, for illustration and experiment on the ModelNet40 benchmark.
By integrating the geometric afﬁne module, we improve the performance of PointNet++ to 93.3%, achieving an improvement of 1.4%.
However, when integrating the module to DGCNN, we get a performance of 92.8%, which is slightly lower than the original results (92.9%).
Note that both results are tested without voting.
D P OINT MLP D EPTH
Here we format the detailed formulation of layer number in our PointMLP.
For the sake of clarity, we ignore Batch Normalization layers and activation functions.
Let Pre i and Pos i indicate the repeating number of the Φ pre block (which includes 3 layers) and Φ pos block (which includes 2 layers) in i -th stage, respectively.
Note that we have one layer in feature embedding in the beginning, one layer for channel number matching in each stage, and three layers in the classiﬁer.
Hence, the total number of learnable layers L would be
[Equation]
As a result, the depth conﬁguration of our network (24, 40, and 56) can be summarized as:
Depth [Pre 1 , Pre 2 , Pre 3 , Pre 4 ] [Pos 1 , Pos 2 , Pos 3 , Pos 4 ] [1 , 1 , 1 , 1] [1 , 1 , 1 , 1] [2 , 2 , 2 , 2] [2 , 2 , 2 , 2] [3 , 3 , 3 , 3] [3 , 3 , 3 , 3]