PointMamba: A Simple State Space Model for Point Cloud Analysis
Authors: Dingkang Liang
Xin Zhou
Wei Xu
Xingkui Zhu
Zhikang Zou
Xiaoqing Ye
Xiao Tan
Xiang Bai
Venue: NeurIPS 2024 poster (2024)
Keywords: Point cloud analysis; State space model; Mamba; PointMamba
Primary area: deep_learning_architectures

Abstract: Transformers have become one of the foundational architectures in point cloud analysis tasks due to their excellent global modeling ability. However, the attention mechanism has quadratic complexity, making the design of a linear complexity method with global modeling appealing. In this paper, we propose PointMamba, transferring the success of Mamba, a recent representative state space model (SSM), from NLP to point cloud analysis tasks. Unlike traditional Transformers, PointMamba employs a linear complexity algorithm, presenting global modeling capacity while significantly reducing computational costs. Specifically, our method leverages space-filling curves for effective point tokenization and adopts an extremely simple, non-hierarchical Mamba encoder as the backbone. Comprehensive evaluations demonstrate that PointMamba achieves superior performance across multiple datasets while significantly reducing GPU memory usage and FLOPs. This work underscores the potential of SSMs in 3D vision-related tasks and presents a simple yet effective Mamba-based baseline for future research. The code is available at https://github.com/LMD0311/PointMamba.

PointMamba: A Simple State Space Model for Point Cloud Analysis
Dingkang Liang",, Xin Zhou " Wei Xu', Xingkui Zhu', Zhikang Zou?,
Xiaoqing Ye², Xiao Tan?, Xiang Bai
Huazhong University of Science Technology, Baidu Inc. {dkliang, xzhou03, #hait. wwt.:/husstuuuu..... edu.cn
ORGONEY
78-770-25
180.92
(ECCV 23)
163.81
41
w log scale
14.81
15.97
54.2
PointGT (NeurIPS 23)
30.2
Point-MAE
PointMamba (ouns)
Poin&Mamba (Ours)
0.4913
0.25
MadeNoee
256
512
1024
2048
4096 8192 16384
32768
Siage Nar
The length of point tokens
(8) Performance comparison
(b) Inference speed comparison
80
OOME
750
696.02
GPU memory (G)
60
50
Point-MAR
50.52
Poins-MAE
40
PointMamba (ours)
FLOP:
450
PointMumbu (ours)
24.9
5.2
30
20
150
132.88
10
2.33 3.21
256
512
1024
2048
4096 8192 16384 32768 65536
256
512
1024
2048
4096 8192 16384 32768
The length of point tokens
The longth of point tokens
(c) GPU usage compurison
(d) FL.PSs comparison
Figure 1: Comprehensive comparisons between our PointMamba and its Transformer-based counter- parts [33, 6, 11]. (a) Without bells and whistles, our PointMamba achieve better performance than the representative Transformer- bassed methods the various point cloud analysis datasets. (b)-(d) The Transformer presents quadratic complexity. while our PointMamba has linear complexity. For example, with the length of point tokens increasing, we significantly reduce GPU memory usage and FLOPss and have a faster inference speed compared to the most convincing Transformer-based method, i.e., PointMAE [33].
Abstract
Transformers have become one of the foundational architectures in point cloud analysis tasks due to their excellent global modeling ability. However, the attention mechanism has quadratic complexity, making the design of linear complexity method with global modeling appealing. In this paper, we propose PointMamba 2s transferring the success of Mamba, recent representative state space model (SSM), from NL.P to point cloud analysis tasks. Unlike traditional Transformers, PointMamba employs a linear complexity algorithm, presenting global model- ing capacity while significantly reducing computational costs. Specifically, our
'Equal contribution. Corresponding author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

method leverages space-filling curves for effective point tokenization and adopts an extremely simple, non non-nionoane hierarchical Mamba encoder as the backbone. Compre- hensive evaluations demonstrate that PointMamba achieves superior performance across multiple datasets while significantly reducing GPU memory usage and FLOPS. This work underscores the potential of SSMs in 3D vision-related tasks and presents a simple yet effective Mamba- based baseline for future research. The code is available at https: // //g github. com LMD031 1/oonttaaa.i..iaaaaa...
1 Introduction
Point cloud analysis is one of the fundamental tasks in computer vision and has a wide range of real-world applications [47, 55, 8]. including roboties, autonomous driving, and augmented reality. It is a challenging task due to the intrinsic irregularity and sparsity of point clouds. To address the issues, there has been rapid progress in deep bearning-based methods [38, 33, 41, 44], consistently pushing the performance to the new record.
Recently, Transformers have achieved remarkable progress in point cloud analysis. The key to the Transformer is the attention mechanism, which can effectively capture the relationship of a set of points. By integrating self-supervised learning paradigms with fine-tuning for downstream tasks, these Transformer-basee methods have achieved superior performance [60, 33, 42]. However, the complexity of attention mechanisms quadratic, bringing significant computational cost, which is not friendly to low-resource devices. Thus, this naturally raises a question: how to design a simple, elegant method that operates with linear complexity, thereby retaining the benefits of global modeling for point cloud analysis?
We note the recent advance of the State Space Models (SSMs). As pioneer, the Structured State Space Sequence Model [17] (S4) has emerged as a promising class of architectures for sequence modeling thanks to its strong representation ability and linear-time complexity (achieved by elimi- nating the need to store the complete context). Another pioneer, Mamba [14], adopts time-varying parameters to the SSM based on S4, proposing an efficient hardware-aware algorithm enable highly efficient training and inference with dynamic modeling. Recently, few concurrent methods [71, 35] successfully transfer the DD-sequence Mamba from NLP to 2D vision tasks (c.g., image classification and segmentation), achieving similar or surpass the Transformer counterpart [12] while significantly reducing memory usage. However, regarding the more complex, unstructured data, e.g., 3D point cloud, the effectiveness of Mamba remains unclear. The lack of early exploration of Mamba's 's poten- tial for point cloud-related tasks hinders further development of its capabilities across the diverse range of applications in this domain.
Inspired by this, this paper aims to unlock the potential of SSM in point cloud analysis tasks. discussing whether it can be a viable alternative to Transformers in this domain. Through a series of pilot experiments, we find that directly using the pionecring SSM. Mamba [14], can not achieve ideal performance. We argue that the main inherent limitation comes from the unidirectional modeling employed by the default Mamba, as the context obtained by compressing the historical hidden state instead of through the interaction between each element. In contrast, the self-attention of the Transformer is invariant to the permutation of the input elements. Given the dimensionl nature (e.g., unstructured and disordered) of point clouds, using a single scanning process often struggles to concurrently capture dependency information across various directions, which makes it difficult to construct global modeling for the RNN-like modes (e.g., Mamba).
Therefore, we introduce a simple yet effective Point Cloud State Space Model (denoted as Point- Mamba % with global modeling and linear complexity. Specifically, to enable Mamba to capture the point cloud structure causally, we first use a point tokenizer to generate two types of point tokens via point scanning strategy, employing two space- mime-filling curves to scan key points from different directions. As a result, the unstructured 3D point clouds can be transformed into a regular sequence. The first type of token has local modeling capabilities through sequential encoding, with the latest token holding global sequence information. Consequently, the second type of token can achieve global modeling by containing global information that comes from the first type. Besides, we propose an extremely simple order indicator to maintain the distinct spatial characteristics of different scanning when training, preserving the integrity of the spatial information.
2

---

To make the model as simple as possible, PointMamba only employs plain and non hierarchical Mamba as the backbone to extract features for given serialized point tokens without bells and whistles. We demonstrate that PointMamba is very flexible in the pre-training paradigm, where we customize an MAF-like pertaining strategy to provide a good prior, which chooses random serialization strategy from pre-defined serialization bank to perform mask modeling. It facilitates the model to exact the general local relationships from different scanning perspectives, better matching the requirement of indirection modeling of mamba.
Despite no elaborate or complex designs in the model, our PointMamba achieves superior performance on various point cloud analysis datasets (Fig. I(a)). Besides the superior performance, thanks to the linear complexity of Mamba, we show the surprisingly low computational cost', as shown in Fig. l(b)-(c). These notable results underscore the potential of SSM in 3D resated tasks.
In conclusion, the contributions of this paper are twofold. 1) We introduce the first state space model for point cloud analysis, named PointMamba & which features global modeling with linear complexity. Despite the absence of elaborate or complex structural designs, PointMamba demon- strates its potential as an optional model for 3D vision applications. 2) Our PointMamba exhibits impressive capabilities, including structural simplicity (e.g., vanilla Mamba), low computational cost, and knowledge transferability (e.g., support for self-supervised learning).
2 Related work
2.1 Point Cloud Transformers
Vision Transformer (VIT) [12] has become one of the mainstream architectures in point cloud analysis tasks due to its excellent global modeling ability. Specifically, Point-BERT [60] and Point-MAE [38] introduce a standard Transformer architecture for self-supervised learning and is applicable to different downstream tasks. Several works further introduce GPT scheme [6], multi- scale [64, 61], and multi-modal [11, 42, 43] to guide 3D representation learning. On the other hand, some researchers [20, 68, 51] focus on modifying the Transformers for point clouds. The PCT [20] conducts global attention directly on the point cloud. Point Transformer [68] applies vector attention [67] to perform local attention between each point and its adjacent points. The later Point Transformer series [56, 55] further extends the performance and efficiency of the Transformer for different tasks. OctFormer [51] leverages sorted shuffled keys of octrees to partition point clouds and significantly improve efficiency and effectiveness.
Standard Transformers can be smoothly integrated into autoencoders using an encoder-decoder design [25], which makes this structure ideal for pre-training and leads to significant performance improvements in downstream point cloud analysis tasks [60, 38, 6, 62, 63]. However, the attention mechanism has a time complexity of O(n2d), where n represents the length of the input token sequence and d represents the dimension of the Transformer. This implies that as the input sequence grows, the operational efficiency of the Transformer is significantly constrained.
In this work, we focus on designing a simple State Space Model (SSM) for point cloud analysis without attention while maintaining the global modeling advantages of the Transformer.
2.2 State Space Models
Linear state space equations [15, 18), combined with deep learning, offer a compelling approach for modeling sequential data, presenting an alternative CNNs or Transformers. The Structured State Space Sequence Model [17] (S4) leverages linear state space for contextualization and shows strong performance on various sequence modeling tasks, especially with lengthy sequences. To alleviate computational burden, HTTYH [19], DSS [22], and S4D [16] propose employing diagonal matrix within S4, maintaining performance without excessive computational costs. The
S5 [48] proposes parallel scan and the MIMO SSM, enabling the state space model to be efficiently utilized and widely implemented. Recently, Mamba [14] introduced the selective SSM mechanism, breakthrough achieving linear-time inference and effective training using hardware-awe algorithm, garnering considerable attention. In the vision domain, Vision Mamba [35] compresses the visual
Note that we removed the tokenizer of both Moin and PointMamba, directly fed with predefined sequence, to better illustrate the structural efficiency.
3

---

representation through bidirectional state space models. VMamba [71] introduces the Cross-Scan Module, enabling ID selective scanning in 2D images with global receptive fields. Besides, the great potential of Mamba motivates series of work across diverse domains, including graph [50, 3], medical segmentation [36, 34], video understanding [30, 7] and generative models [29.31].
To the best of our knowledge, there are limited works that study SSMs for point cloud analysis. In this work, we delve into the potential of Mamba in point cloud analysis and propose PointMamba, which achieves superior performance and significantly reduces computational costs.
3 Preliminaries
State Space Model. Drawing inspiration from control theory, the State Space Model (SSM) represents a continuous system that maps a state to through an implicit latent state h& E R. To integrate
SSMs into deep models, S4 [17] defines the system with four parameters (A, B, C, and sampling step size A). The sequence-to-sequens transformation is defined as:
h4 A + BIt.
Vv
Ch+ + Dr..
(1)
where C' RIXN is project parameter, and D E IRINN represents residual connection. The parameters A, B are defined using the zero-order hold (ZOH) discretization rule:
A E R NxN exp( (A),
BERNXI= (AA) (exp( (A) - 1) AB. (2)
However, the parameter (A, B, C, A) are fixed across all time steps due to the Linear Time-Invarian (LTT) property of SSMs, which limits their capacity to handle varied input sequences.
Recently, Selective SSM (S6) considers parameters B, C, A functions of the input, effectively transforming the SSM into time-variant model. Our PointMamba adopts hardware-awarr imple- mentation [14] of S6, showing linear complexity and strong sequence modeling capability.
Space-filling curve. Space-filling curves are paths that traverse every point within higher- dimensional discrete space while maintaining spatial proximity to certain degree. Mathematically. they can be defined as a bijective function & Z Za for point clouds. Our PointMamba focuses on the Hilbert space- space-filling curve [27] and its transposed variant (called Trans-Hilbert which are recognized for effectively preserving locality, ensuring that data points close in Z space remain close after transformation to ZZ. We note that some methods [55, 51] utilize space-filling curves to partition the point cloud for capturing spatial contexts, whereas our work mainly focuses on sequences and combine with Mamba to implement
both of
transferring the point clouds to serialization-base global modeling. The motivation and objective are different.
PointMamba
This paper aims design simple yet solid Mamba-based Point cloud analysis method. The pipeline of our method is shown in Fig. 2. Starting with an input point cloud, we first sample the key points via Farthest Point Sampling (FPS). Then, simple space-scanning strategy is applied to reorganize these points, resulting in serialized key points. Under a KNN and lightweight PointNet [40], we obtain the serialized point tokens. Finally, the entire sequence is subsequently processed by a plain, non-hierarchical encoder structure composed of several stacked Mamba blocks. Besides, to provide good prior for PointMamba, we propose serialization-base mask modeling paradigm, which randomly chooses a space-filling curve for serialization and mask, as shown in Fig. 4.
4.1 The structure of PointMamba
In this section, we introduce the structure of our PointMamba. The goal of this paper is to provide simple yet solid Mamba baseline for point cloud analysis tasks and explore the potential plain and non-hierarchical Mamba. Thus, in the spirit of Occam's 's razor, we make the structure as simple as possible without any complex or elaborate design.
Point scanning strategy. Building on the pioneer works [60, 38), we first utilize the Farthest Point Sampling (FPS) to select the key points. Specifically, given an input point cloud M where M is the number of points, the FPS is applied to sample key points from the original point cloud

---

KNN
Order indicator
Scale
Shint
Lineary
Hilboit
Tokan anbedding laya
Selective
Vanilla Mamba
Tisk hend
SSM
FPS
block
Traus- Htibber
DW
Libcar
Linest
FPS: Fambest Sampling
Scale
Shift
LN
Order indicator
Mambba block
The pipeline of our PointMamba. It is simple and elegant, without bells and whistles. We first utilize Farthest Point Sampling (FPS) to select the key points. Then, we propose to utilize two types of fpalcefilling curves, including Hilbert and Trans-Hilbert, to generate the serialized key points. Based on these. the KNN is used to form point patches, which will be fed to the token embedding layer to generate the serialized point tokens. To indicate the tokens generated from which ppace-filling curve, the order indicator is proposed. The encoder is extremely simple, consisting of Nx plain and hh-hhaerrchical Mamba blocks.
Figure 2:
P. resulting in p E RRxx3, In general, the order of the sampling key points is random, without specific order. This is not significant problem for the previous Transformer-based methods, the Transformer is invarian when processing sequence data: in the self-attention mechanism, each element at a given position can interact with all other elements in the sequence through attention weights. However, for the selective state space model, i.c., Mamba, we argue that it is hard to model the unstructured point clouds due to the unidirectional modeling. Thus, we propose leverage the ppace-filling curves to transform the unstructured point clouds into a regular sequence. Specifically, we choose two representative space-filling curves to scan the key points: the Hilbert curve [27] and its transposed variant, denoted as Trans-Hilbert Compared with the random sequence, space-filling curves like the Hilbert curve can preserve spatial locality well, i.e., along the scanned 1D serialized point sequence, adjacent key points often have geometrically close positions in 3D space. We argue that this property ensures that the spatial relationships between points are largely maintained, which crucial for accurate feature representation and analysis in point cloud data. As complementary, the Trans-Hilber performs similarly but scans from different clues, which can provide diverse perspectives spatial locality. By applying Hilbert and Trans-Hilbert to the key points, obtain two different point serializations, Pk. and ps', which will be used to construct point tokens.
Point tokenizer.. After obtaining the two serialized key points Pk. and Plv, we then utilize the KNN algorithm to select nearest neighbors for each key point, forming token patches T, E R=n and Th. E [ with patch size k. To aggregate local information, points within each patch are normalized by subtracting the key point to obtain relative coordinates. We map the unbiased local patches to feature space using a lightweight PointNer [40] (point embedding layer), obtaining serialized point tokens E E = and Ed' Ra where the former is the Hilbert-based and the latter is Trans-Hilbert-baze
Order indicator. Directly fed the two type serialized point tokens E E B and E xx into Mamba encoder might cause confusion as E and E actually share the same center but with different order. Maintaining the distinct characteristics these different scanning strategies important for preserving the integrity of the spatial information. Thus, we propose an extremely simple order indicator to indicate the scanning strategy used. Specifically, the proposed order indicator performs the linear transformation to transfer features into different latent spaces. The formulation can be written as follows:
b = E 2 Yh BR, = E Yh + Bw,
where Yh/Tw/ RC and Br1B C refer to the scale and shift factors, respectively. @ is the Hadamard product and implemented by the broadcast mechanism. We then concat Z and z resulting in Zo E 2
5

---

Mamba encoder. After obtaining the token Zo. we will feed it into the encoder, containing N Mamba block, to extract the feature. Specifically. for each Mamba block, layer normalization (LN),, Selective SSM, depth-wise convolution [10] (DW), and residual connections are employed. A standard Mamba layer is shown in Fig. 2, and the output can be summarized as follows:
- = LL (Z)-1).
Z (DW (Linear (Z(-1))
Z (Linear (Z(-)),
Z Linear (SelectiveSSM (Zi) Z(") + Z
Zi E R22xx is the output of the 1-th block, and o indicates SiLU activation [26]. The SelectiveSSM is the key to the Mamha block, with a detailed de- scription in Sec. 3. To better understand why the proposed PointMamba has global modeling capac- ity, we provide an intuitive visualization. As shown in Fig. 3, after modeling the first group of point to- kens (i.e., Hilbert-based), the accumulated global information can improve the serialization process for the next set of tokens (i.e., Houns--illbrt-baseeee This mechanism ensures that each serialized point in the Trans-Hilher sequence informed by the entire history of the previously processed Hilbert sequence, thereby enabling a more contextually rich and globally aware modeling process. More discussions can be found in Appendix A.2.
Modeling direction
Global information
Serialized point tokens z from Hilbert
Serialized point tokens z from Truns-Hilbert
Figure 3: An intuitive illustration of global mod- cling from PointMamba.
In our study, we show that even very simple Mamba block without specific designs, our Point- Mamba can surpass the various Transformer-based point cloud analysis methods.
4.2 The serialization-based mask modeling
One intriguing characteristic of Transformers-buue based methods [33, 60, 6] their improved perfor- mance using the pre- traininiin scheme, especially mask modeling [25]. In this paper, considering the unidirectional modeling of Mamba, we customize a simple yet effective serialization-batt mask modeling paradigm, shown in Fig. 4.
Specifically, after obtaining the key points, randomly choose Hilbert or Trans-Hilbert curve to implement serialization in each iteration, resulting in serialization-basse key points, i.c., Ph. and Pic, are obtained from Hilbert and Trans-Hilbert, respectively. Such a scheme allows the model to exact the local relationships from different scanning clues. Then, the KNN and the token embedding layer are used generate the point tokens. To discriminate the point tokens serialized from which space-filling curves, we apply the order indicator to the point tokens, where different serialized point tokens have different order indicators, which are similar to the mentioned Eq. Next, we randomly mask the serialization- based point tokens with a high ratio of 60%. Then, an asymmetric autoencoder, consisting of several vanilla Mamba blocks, is employed to extract the point feature, and the final layer of the autoencoder utilizes a simple prediction head for reconstruction. To reconstruct masked point patches coordinate space, we employ a linear head to project the masked token to the shape of the masked input points. The Chamfer Distance [13] is then used as the reconstruction loss to recover the coordinates of the points in each masked point patch.
We demonstrate that with such simple serialization-basee mask modeling paradigm, Point- Mamba can easily achieve superior performance.
5 Experiments
5.1
Implementation details
SSM is new to 3D point cloud analysis, with no existing works detailing the specific implementation. To handle different resolutions of the input point cloud, we divide them into different numbers of patches with linear scaling (e.g., .M- = 1024 input points are divided into = 64 point patches), with each patch containing 32 points determined by the KNN algorithm. The PointMamba encoder has N 12 vanilla Mamba blocks, each Mamba block featuring C' = 384 hidden dimensions. For the pre-training, we utilize Seapee [5] as the dataset, following previous methods [60, 38, 6]. In addition, we utilize 4 x Mamba blocks the decoder reconstruct the masked point clouds.

---

KNN
Loss
Pred
Order
Trant- Hilherr
www.olingg boper
Vanilla Marnba decoder
FPS
Choose
Hiibert
Vanilla Mamba encoder
Point
Point center
Serialization
Autoencoder pro-training
Figure 4: The details of our proposed serialization- mask modeling. During the pre-training, we randomly choose one space- space-filling curve to generate the serialized point tokens for mask modeling. and different serialized point tokens have different order indicators.
Table Object classification on the ScanObjectNN dataset [49]. We evaluate PointMamba on three variants, with PB-T50-RS being the most challenging. Overall accuracy (%) is reported. Param. denotes the number tunable parameters during training. indicates that using simple rotational augmentation [11] []] for training.
Methods
Backbone
(M)
OBI-BG OBI-ONLY
PB-TSU-RS
Supervised Learning Only
PointNet [40]
CVPR 17
3.5
0.5
73.3
79.2
68.0
PointNet++ [41]
NeurIPS
PoinCCNN [32]
NeurIPS 18
0.6
86.1
85.5
78.5
DGCNN [52]
TOG 19
1.8
82.8
86.2
78.1
PRANet
TIP 21
81.0
MVTN [23]
ICCV 21
11.2
43.7
82.8
PointNeXt [44]
NeurIPS
PoinMLP [37]
ICLR 22
13.2
31.4
85.4
RepSurf-U [45]
CVPR
ADS
JCCV 23
87.5
Training from pre-training (Single-Modol)
Point-BEKT [60]
CVPR
Transformer 22.1
87.43
58.12
83.07
MaskPoint [33]
CVPR
Transformer
22.1
89.30
88.10
84.30
Point-MAF. [38]
BCCV
Transformer
22.1
4.8
90.02
85.
Point-M2AE [64]
NeurIPS
Transformer
15.3
86.43
PointDir [69]
CVPR 24
Transformer
93.29
91.91
87.61
[63]
Poln--MAE-DAPTI
CVPR 24
Transformer
1.1
5.0
90.88
90.19
85.08
Point-MAE? [38]
ECCV 22
Transforner
22.1
4.8
92.77
91.22
89.04
PointGPT-S [6]
NeurIPS
Transfomer
29.2
5.7
93.39
92.43
89.17
PointMamba (ours)
Mamba
12.3
94.32
92.60
Training from pre pre-training (Cruss-oodal)
ACT' [1]]
Joint-MAF [2]]
ICA1 23
Transformer
90.94
88.86
86.07
12P-MAB [65]
CVPR 23
Transformer
15.3
94.15
91.57
90.11
RECox [42]
ICML 23
Transformer
43.6
95.18
93.29
90.63
5.2
Compared with Transformer-based counterparts
This paper aims unlock the potential of Mamba point cloud tasks, discussing whether can viable alternative to Transformers. Thus, in the following experiments, we mainly compare with the state-of-the-art vanilla Transformer-based point cloud analysis methods.
Real-world object classification on ScanObjectNN. ScanObjectNN [49] is a challenging 3D dataset comprising about 15,000 objects across 15 categories, scanned from real-world indoor scenes with cluttered complexity backgrounds. As shown in Tab. 1, we conduct experiments on three versions ScanObjectNN (i.e., OBJ-BG, OBJ-ONLY, and PB-T50-RS), each with increasing complexity. When compared with the most convincing Transformer-bast based method, i.e.. Point- MAE [33], PointMamba surpasses it by 1.55%, 1.38% and 0.27% on OBJ-BG, OBJ-ONLY, and PB-T50-RS respectively while using less computational costs. Besides, we also outperform the SOTA PointGPT-S [6] by 0.93%, 0. 0.17%, 141 across three variants on a comparable scale setting. Note that method follows Occam's 's Razor, without auxiliary tasks like generation during fine-tuning

---

Table 2: Classification ModelNet40 [57]. Overall accuracy (%) is reported. The results obtained from 1024 points without voting.
Table 3: Few-shot learning on ModelNet40 [57]. Overall accuracy (he standard deviation (%) without voting is reported.
Methods
Param. (M)
FLOPs
(A6
5-way
10-way
Methods
Supervised Leaniing Only
10-shot
20-sbot
10-shot
20-shot
Pointeet [40)
3.5
0.5
89.2
Supervised Learning Only
PointNets [41]
90.7
PoimCNN [53]
0.6
92.2
PointNet [40]
52.0=3.8 57.82499 46.624.33
35.214.8
DGCNN [39]
PointNet CrossPoint
90,911.9 93.5.14,4 84.6.4.77
90.2.2.2
PointeeX [44]
1.4
1.6
92.9
DGCNN [52]
31.6=2288 40,6
19.9
16.9
DGCNN-CrossPoimt [1]
92.5=3.0 94,942,1 83.645.3
87.944.22
OctFormer [53]
3.98
31,3
92.7
with Self-supervised
pre--i-rii pre-training
with Self supertised pre tralninng
Point-BERT [60]
94,6=3,1 96,3+2,77 91.0=554 92.745.11
Point BERT [60]
22.1
2.3
92.7
MaskPoint [33]
95.0=3.7 97.21177 91.4+400 93,4+355
MaskPoist [33]
22.1
92.6
Point-MAE [38]
96,3=2.5 97,8+18
92.6#4.1
95.0#3.0
Point-MAE (38]
(iin-M2AF[(6]
22.1
12.8
4.6
03.2
93.4
Point-M2AB [64]
96.8118 98.3114
92.34.55
95.0130
PoinGGPP: [6]
20.2
2.9
93.3
PointGPT
96.8.2.0
98.611.1
92.614.6
95.21344
ACT [1]]
22.1
2.4
93.6
ACT [1]]
96.8=2.3
98.0=-1.4
93.3--400
95.61-88
PountMambaa (ours)
12.3
1,5
93.6
PaintMamba (ours)
96.9-2.0
99.0==111
93.0-444
95.6433.
used in PointGPT [6]. Furthermore. compared to cross-modal learning methods [11, 42] that use additional training data (cross-modal information) or teacher models, which is not fair comparison, our PointMamba still maintains highly competitive. We mainly want to introduce a new Mamba- based point cloud analysis methods paradigm. Although using some complex designs can bring improvement, they might be heuristics. More importantly, these heuristic designs will decrease the objectivity of the evaluation of our method.
Synthetic object classification on ModelNet40. ModelNer40 [57] is a pristine 3D CAD dataset consisting of 12,311 clean samples across 40 categories. As shown in Tab.
report the over- all accuracy without adopting the voting strategy. The proposed PointMamba achieves the best results compared with various self-supervised Transformer- basedd methods [33, 60, 6]. In particular, PointMamba surpasses Point-MAE [33] and PointGPT-S [6] by 0.4% and 0.3% respectively. It worth noting that the single-modal-learmed PointMamba achieves comparable results with the cross-modke modal-basee ACT [11] while significantly reducing parameters and FLOPSs about 44% and 38%, respectively. Additionally, PointMamba demonstrates competitive performance against elaborately designed Transformer models like OctFormer [51].
Few-shot learning. We further conduct few- shot experiments on ModelNer40 [57] to demon- strate our few-shot transfer ability. Consistent with prior studies [60], we utilize the "n-way, m-shot" setup, where E (5, 10) denotes the category count and m E {10, 20} represents the samples per category. Following standard proce- dure, we carry out 10 separate experiments for each setting and reported mean accuracy along with the standard deviation. As indicated in Tab. 3, our PointMamba shows competitive re- sults with limited data, e.g., +1.0% mean ac- curacy compared to the cross-modal method ACT [11] on the 5-way 20-shot split.
Table 4: Part segmentation on the ShapeNet- Part [58]. The mloU for all classes (Cls.) and for all instances (Inst.) are reported.
Methods
Cls. mloU (%) Inst. mloU (%)
Supervised Learing Only
PointNet [40]
80.39
83.7
PointNett+ [41]
81.85
85.1
DGCNN [52]
82.33
85.2
APES [54]
83.67
85.8
with Self-super pre-truining
Transformer [60]
83.4
85.1
OcCo [60]
83.4
85.1
MaskPoint [33]
84.6
86.0
Part segmentation on ShapeNetPart. Part seg- mentation on ShapeNetPart [58] is challenging task that aims predict more detailed label for each point within a sample. As shown in Tab, 4, we report mean loU (mloU) for all classes (Cls.)
Point-BERT [60]
84.1
85.6
Point-MAF [38]
84.2
86.1
PointGPT-S [6]
84.1
86.2
ACT [11]
84.7
86.1
PointMamba (ours)
84.4
86.2
and all instances (Inst.). Our PointMamba model demonstrates highly competitive performance compared to the Transformer-based counterparts [33, 6, 60]. These impressive results further prove
the potential of SSM in the point cloud analysis tasks.
8

---

Lincar
Lineur
Linear
Lineur
Selective
Attention
MLP
SSM
DWConv
DWConv
DWCD
DWCoRY
Linear
Lincar
Linear
Lincar
Lineur
Linear
Linear
Lincar
(a) w/ Identity
(b) w/ Attention
(c) w/ MLP
(d) w/ Selective SSM (ours)
Figure 5: Different variant of PointMamba. (a) Directly removing the SSM part. (b) Replacing SSM
with attention. (c) Replacing SSM with MLP. (d) Ours PointMamba with Selective SSM.
Table 5: The effect of each component.
Table 6: The effect of different scanning curves.
Hilbert Trans Hilbert
Order indicator
OBJ-BG OBJ-ONLY
Scanning
OBJ-BG
OBJ-ONLY
Random
92.60
90.18
Random
92.60
90.18
92.94
91.05
Hilben and Trans-Hilbert
94.32
92.60
91.74
7-order and Trans- Trans-7-zorder
93.24
90.36
93.80
91.91
Hilber and Z-order
93.29
90.88
94.32
Hrin--ilbert and Trans-Z-order
93.29
91.91
Table 7: The effect of Selective SSM.
Table 8: The effect of Order indicator.
Setting
Param.
OBJ-BG
OBJ-ONLY
Setting
OBJ-BG
OBJ-ONLY
w/ Identity
11.4
93.80
91.57
None
93.80
91.91
w/ Attention
39.8
92.77
91.22
MI.P
91.22
Using the same indicator
93.29
90.19
w/ Selective SSM
12.3
94.32
92.60
Using different indicator
94.32
92.60
5.3 Analysis and ablation study
To investigate the architecture design, we conduct ablation studies on ScanObjectNN [49] with both pre-trainee pre-training and fine tne-tuning. Default settings are marked in gray
The structural efficiency. We first discuss the efficiency of our method. To fully explore the potential of processing the long point tokens (sequence), we gradually increase the sequence length until the GPU (NVIDIA A800 80GB) memory explodes. The comprehensive efficiency comparisons are present in Fig. 1(b)-dd), where Compared with the most convincing Transformer- baseed method [33], our PointMamba demonstrates significantly improved inference speed and reduce the GPU usage and FLOPs, especially when facing the long sequence. For example, when the length increases to more than 32,768, we outperform PointMAE by 30.2x, 24.9x, and 5.2x terms of inference speed, GPU memory, and FLOPs, respectively. More importantly, even presenting impressive efficiency, we still achieve impressive performance on various point cloud analysis datasets.
The effect of each component. We then study the effectiveness of the proposed components of PointMamba as shown in Tab. 5. We can make the following observations: Directly utilizing random serialization, PointMamba only achieves 92.26% and 90.18% overall accuracy on OBJ-BG and OBJ-ONLY, respectively. It is reasonable as Mamba is hard to model the unstructured point clouds due to its unidirectional modeling. 2) By introducing the locality-preserved Hilbert scanning, PointMamba's ability to capture sequence information is enhanced, leading to performance improvements compared to random serialization. Further applying both Hilbert and Trans- scanning curves, PointMamba surpasses the random serialization by 1.20% and 1.73% on two datasets, respectively. 3) By using the order indicator to maintain the distinct characteristics
preserved Hilbert or Trans-

---

of the two different scanning strategies, we achieve notable improvement, resulting in 94.32% and 92.60% on OBJ-BG and OBJ-ONLY, respectively. Note that the order indicator extremely light (only 1.5k parameters), which will not introduce additional computational costs.
The effect of different scanning curves. We further explore the effect of using different scanning curves to construct serialized point tokens. Specifically. we select two widely used space-filling curves, including Hilbert and Z-order, along with their transposed variants, i.e., Trans-Hillerr and Trans-Z- order. As listed in Tab. 6, we empirically find that serializing point clouds with space-filling curves scanning can achieve better performance compared to random sequences. We argue that scanning sequences along a specific pattern of spatial locations offers a more logical sequence modeling order for SSM. We choose the combination of Hilbert and Trans-Hilbers for PointMamba due to their superior locality-preservnng properties.
The effect of Selective SSM. The key of S6 models or Mamba [14] is the SSM with the selective mechanism. We prove that as a unidirectional modeling method, SSM can be analogous to masked self-attention, ensuring each position can only attend to previous positions (the detailed proof can be found in the Appendix A.I). Thus, as shown in Tab. 7, we analyze the effect of selective SSM by removing it (i.e., identity setting) or replace with masked self-attention or MLP (an illustration is shown in Fig. 5). Compared with the identity setting, the selective SSM brings notable improvement, indicating the effectiveness of introducing global modeling from SSM. Note that while very recent method, MambaOut [59]. thinks the SSM of Mamba might negatively impact image classification tasks. our findings demonstrate that this is not the case for point cloud analysis tasks. Another interesting thing is that when Selective SSM is replaced with masked self-attention, the performance even lower than that of the identity setting. We argue the main reason that masked self-attention is hard to combine with the Gated MLP [46] used in default Mamba, leading to optimized difficulty, which might need to be explored in the future.
Analysis on order indicator. This part analyzes the effect of the order indicator. PointMamba applies Hilbert and Trans-Hilberr to recognize the key points, obtaining two types of serialized point tokens E and Eb' The order indicator used indicate the scanning strategy, As shown in Tab. using two different order indicators can improve 1.20% and 1.03% compared to no indicator on OBJ-BG and OBJ-ONLY respectively. However, using the same order indicator for both types of sequences without distinguishing between different scanning strategies does not yield positive results.
5.4 Limitation
Although PointMamba achieves promising results, there are some limitations: 1) We only focus on the point cloud analysis task in this paper while designing unified caabba-based foundation model for various 3D vision tasks (e.g.. 3D object mrassfictioodeeettoo/ssmmmmttiii
is a more appealing direction. 2) We only use the point clouds as training data while combining them with 2D images or language knowledge to improve the performance, which is also worthy of exploration. We left these in our future work.
Conclusion
In this paper, we present elegant, simple Mamba-based method named PointMamba for point cloud analysis. PointMamba utilizes space-filling curve-based point tokenizer and a plain, non- hierarchical Mamba architecture to achieve global modeling with linear complexity. Despite its structural simplicity. PointMamba delivers state-of-the-arr performance across various datasets, significantly reducing computational costs in terms of GPU memory and FLOPs. PointMamba success highlights the potential of SSMs, particularly Mamba, in handling the complexities of point cloud data. As newcomer to point cloud analysis, PointMamba is promising option for constructing 3D vision foundation models, and we hope it can offer new perspective for the field.
Acknowledgements
This work is supported in part by the NSFC (Grant. No. 62225603 and 623B2038), in part by the Taihu Lake Innovation Fund for Future Technology (HUST: 2023-A-1), and in part by the Fundamental Research Funds for the Central Universities (HUST: 52400036).
10

---

References
[1] Mohamed Afham, Isuru Dissanayake, Dinithi Dissanayake, Amaya Dharmasiri, Kanchana Thilakarathna. and Ranga Rodrigo. Crosspoint: Sclf-supervised eross modal contrastive learning for 3d point cloud understanding. In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, 2022.
[2] Ameee Ali, Itamar Zimerman, and Lior Wolf. The hidden attention mamba models. arXiv preprint arXiv:2403.01540 2024.
[3] Ali Behrouz and Farnoosh Hashemi. Graph mamba: Towards learning on graphs with state space models, In Proc. of ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 119-130, 2024,
[4] Tom Brown, Benjamin Mann. Nick Ryder. Melanie Subbiah. Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. In Proc. of Advances in Neural Information Processing Systems, 2020.
[5] Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rie 3d model repository. arXiv preprint 111.v1152..03012. 2015.
[6] Guangyan Chen, Mciling Wang. Yi Yang, Kai Yu, Li Yuan, and Yufeng Yue. Pointgpt: Auto-regressively generative pre-training from point clouds. In Proc. of Advances in Neural Information Processing Systems, 2023.
[7] Guo Chen, Yifei Huang, Jilan Xu. Baoqi Pei, Zhe Chen, Zhiqi Li, Jiahao Wang, Kunchang Li, Tong Lu, and Limin Wang. Video mamba suite: State space model as versatile alternative for vidco understanding. arXiv preprint arXiv:2403.0900 09626, 2024.
[8] Yukang Chen, Jianhui Liu, Xiangyu Zhang, Xiaojuan Qi, and Jiaya Jia. Voxelnext: Fully sparse voxelnet for 3d object detection and tracking. In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, 2023.
[9] Silin Cheng, Xiwu Chen, Xinwei He, Zhe Liu, and Xiang Bai. Pra-net: Point relation aware network for 3d point cloud analysis. IEEE Transactions on Image Processing, 30:4436-444 4444, 2021.
[10] François Chollet. Xception: Deep learning with depthwise separable convolutions. In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, 2017.
[11] Runpei Dong, Zekun Qi, Linfeng Zhang. Junbo Zhang. Jianjian Sun, Zheng Ge, Li Yi, and Kaisheng Ma. Autoencoders as gross-moda teachers: Can pretrained 2d image transformers help 3d representation learning? In Proc. of Ial. Conf. on Learning Representations, 2022.
[12] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, al. An image is worth 16x16 words: Transformers for image recognition at scale. In Proc. of Intl. Conf. on Learning Representations, 2021.
[13] Haoqiang Fan, Hao Su, and Leonidas Guibas. point set generation network for 3d object reconstruction from a single image. In Proc. of IEEE Insl, Conf. on Computer Vision and Pattern Recognition, 2017.
[14] Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. Conference on Language Modeling, 2024.
[15] Albert Gu, Tri Dao, Stefano Ermon, Atri Rudra, and Christopher Ré. Hippo: Recurrent memory with
optimal polynomial projections. In Proc. of Advances in Neural Information Processing Systems, 2020.
[16] Albert Gu, Karan Goel, Ankit Gupta, and Christopher Ré. On the parameterization and initialization of
diagonal state space models. In Proc. of Advances in Newral Information Processing Systems, 2022.
[17] Albert Gu, Karan Goel, and Christopher Re. Efficiently modeling long sequenees with structured state spaces. In Proc. of Intl. Conf. on Learning Representations, 2021.
[18] Albert Gu, Isys Johnson, Karan Goel, Khaled Saab, Tri Dao, Atri Rudra, and Christopher Ré. Combining recurrent, convolutional, and continuous-time models with linear state space layers. In Proc, of Advances in Neural Information Processing Systems, 2021.
[19] Albert Gu, Isys Johnson, Aman Timalsina, Atri Rudra, and Christopher Re How to train your hippo: State space models with generalized orthogonal basis projections. In Proc, of Intl. Conf. on Learning Representations, 2022.
11

---

[20] Meng-Hao Guo, Jun-Xiong Cai, Zheng-Ning Liu, Tai-Jiang Mu, Ralph R Martin, and Shi-Min Hu. Pet: Point cloud transformer. Computational Visual Media, 2021.
[21] Ziyu Guo, Renrui Zhang, Longtian Qiu, Xianzhi Li, and Pheng-Ann Heng. Joint-mae: 2d-3d joint masked autoencoders for 3d point cloud pre-training. In Proc. of Imtl. Joint Conf. Artificial Intelligence, 2023.
[22] Ankit Gupta, Albert Gu, and Jonathan Berant. Diagonal state spaces are as effective structured state spaces. In Proc. of Advances in Neural Information Processing Systems, 2022.
[23] Abdullah Hamdi, Silvio Giancola, and Bernard Ghanem. Mvtn: Multi-view transformation network for 3d shape recognition. In Porc. of IEEE Intl. Conf. on Computer Vision, 2021.
[24] Xu Han, Yuan Tang. Zhaoxuan Wang, and Xianzhi Li. Mamba3d: Enhancing local features for 3d point cloud analysis via state space model. In Proc. of ACM Multimedia, 2024.
[25] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, 2022.
[26] Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv preprint arXiv: 1xXiv:1606.084155 2016.
[27] David Hilbert. Über die stetige abbildung einer linic auf flächensttick. Dritter Band: Analysis- Grundlagen der Mathematik Physik Verschiedenes: Nebst Einer Lebensgeschichte, 1935.
[28] Cheng- Yao Hong, Yu-Ying Chou, and Tyng-Luh Liu. Attention discriminant sampling for point clouds. In Porc. of TEEE Intl. Conf. on Computer Vision, 2023.
[29] Vincent Tao Hu, Stefan Andreas Baumann, Ming Gui, Olga Grebenkova, Pingchuan Ma, Johannes Fischer, and Bjor Ommer. Zigma: Zigzag mamba diffusion model. Proc. of European Conference on Computer Vision, 2024.
[30] Kunchang Li, Xinhao Li, Yi Wang, Yinan Hc, Yali Wang, Limin Wang, and Yu Qiao. Videomamba: State space model for efficient vidco understanding. In Proc. of European Conference on Computer Vision, 2024.
[31] Shufan Li, Harkanwar Singh, and Aditya Grover. Mamba-nd: Selective state space modeling for multi- dinensional data, arXiv preprint Seti:::400205899.. 2024.
[32] Yangyan Li, Rui Bu, Mingchao Sun, Wei Wu, Xinhan Di, and Baoquan Chen. Pointenn: Convolution on x-transformed points. In Proc. of Advances in Neural Information Processing Systems, 2018
[33] Haotian Liu, Mu Cai, and Yong Jac Lec. Masked discrimination for self-supervised learning on point clouds. In Proc. of European Conference Computer Vision, 2022.
[34] Jiarun Liu, Hao Yang. Hong- Yu Zhou, Yan Xi, Lequan Yu, Cheng Li. Yong Liang, Guangming Shi, Yizhou Yu, Shaoting Zhang, et al. Swin-umamba: Mamba-based unet with imagenet-based pretraining. In International Conference on Medical Image Computing and Computer-Assisseed Intervention, pages 615-625. Springer, 2024.
[35] Yue Liu, Yunjie Tian, Yuzhong Zhao, Hongtian Yu, Lingxi Xie, Yaowei Wang, Qixiang Ye, and Yunfan Liu. Vmamba: Visual state space model. In Proc. of Advances in Neural Information Processing Systems, 2024.
[36] Jun Ma, Feifei Li, and Bo Wang. U-mamba: Enhancing long-range dependency for biomedical image segmentation. arXiv preprint arXiv:2401.04722 2024.
[37] Xu Ma. Can Qin, Haoxuan You, Haoxi Ran, and Yun Fu, Rethinking network design and local geometry in point cloud: A simple residual mlp framework. In Proc. of Intl. Conf. on Learning Representations, 2022.
[38] Yatian Pang, Wenxiao Wang, Francis EH Tay, Wei Liu, Yonghong Tian, and Li Yuan. Masked autoencoders for point cloud self-supervised learning. In Proc. of European Conference on Computer Vision, 2022.
[39] Anh Viet Phan, Minh Le Nguyen, Yen Lam Hoang Nguyen, and Lam Thu Bui. Dgenn: A convolutional neural network over large-scale labeled graphs. Neural Networks, 2018
[40] Charles R Qi, Hao Su. Kaichun Mo. and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, 2017.
12

---

[41] Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++:: Deep hierarchical feature learning on point in metric space. In Proc. of Advances in Neural Information Processing Systems, 2017.
[42] Zekun Qi, Runpei Dong, Guofan Fan, Zheng Ge. Xiangyu Zhang, Kaisheng Ma, and Yi. Contrast with reconstruct: Contrastive representation learning guided by generative pretraining. Proc. of Intl. Conf. on Machine Learning, 2023.
[43] Zekun Qi, Runpei Dong, Shaochen Zhang, Haoran Geng, Chunrui Han, Zheng Ge. Li Yi, and Kaisheng Ma. Shapellm: Universal 3d object understanding for embodied interaction. In ECCV, 2024.
[44] Guocheng Qian, Yuchen Li, Houwen Peng. Jinjie Mai, Hasan Hammoud. Mohamed Elhoseiny, and Bernard Ghanem. Pointnext: Revisiting pointnet++ with improved training and scaling strategies. In Proc. of Advances in Neural Information Processing Systems, 2022.
[45] Haoxi Ran, Jun Liu, and Chengjie Wang. Surface representation for point clouds. In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, 2022.
[46] Noam Shazeer. Glu variants improve transformer. arXiv preprint arXiv:2002.05202,220 2020.
[47] Shaoshuai Shi, Zhe Wang. Jiamping Shi, Xiaogang Wang, and Hongsheng Li. From points to parts: 3d object detection from point cloud with part-aware and part-aggregation network, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.
[48] Jimmy TH Smith, Andrew Warrington, and Scott Linderman. Simplified state space layers for sequence modeling. In Proc. of Intl. Conf. on Learning Reprexentutions, 2022.
[49] Mikacla Angelina Uy Quang-Hics Hieu Pham, Binh-Son Hua, Thanh Nguyen, and Sai Kit Young. Revisiting point cloud classification: new benehmark dataset and classification model real-world data. In Porc. of IEEE Intl. Conf. on Computer Vision, 2019.
[50] Chloe Wang, Oleksii Tsepa, Jun Ma, and Bo Wang. Graph-mamba: Towards long-range graph sequence modeling with selective state spaces, arXiv preprint urXiv:2402.00789 2024.
[51] Peng Soug-Shui Wang. Octformer: Octree-based transformers for point clouds. ACM Transactions Graphics, 2023.
[52] Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. Dynamic graph can for learning on point clouds. ACM Transactions ON Graphics, 2019.
[53] Zicheng Wang. Zhenghao Chen, Yiming Wu, Zhen Zhao, Luping Zhou, and Dong Xu. Pointramba: A hybrid transformer-mante framework for point cloud analysis. arXiv preprint arXiv:2405. 2024.
[54] Chengzhi Wu, Junwei Zheng, Julius Pfrommer, and Jürgen Beyerer. Attention based point cloud edge sampling. In Proc. of IEEE Insl. Conf. on Computer Vision and Pattern Recognition, 2023.
[55] Xiaoyang Wu, Li Jiang, Peng-Shuai Wang, Zhijian Liu, Xihui Liu, Yu Qiao, Wanli Ouyang, Tong He, and Hengshuang 7hao. Point transformer v3: Simpler, faster, stronger. In Proc. of IEFE Intl. Conf. on Computer Vision and Puttern Recognition, 2024.
[56] Xiaoyang Wu, Yixing Lao, Li Jiang, Xihui Liu, and Hengshuang Zhao. Point transformer v2: Grouped vector attention and partition-base based pooling. In Proc. of Advances in Neural Information Processing Systems, 2022.
[57] Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In Proc. of IEEE Inil. Conf. on Computer Vision and Pattern Recognition, 2015.
[58] Li Yi. Vladimir G Kim, Duygu Ceylan, 1-Chao Shen, Mengyan Yan, Hao Su, Cewu Lu, Qixing Huang, Alla Sheffer, and Leonidas Guibas. A scalable active framework for region annotation in 3d shape collections. ACM Transactions ON Graphics, 2016.
[59] Weihao and Xinchao Wang. Mambaout: Do we really need mamba for vision? arXiv preprint urXiv:2405.079% 2024.
[60] Xumin Yu, Lulu Tang. Yongming Rao, Tiejun Huang. Jie Zhou, and Jiwen Lu. Point bert: Pre-training 3d point cloud transformers with masked point modeling. In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, 2022.
13

---

[61] Yaohua Zha, Huizhen Ji. Jinmin Li, Rongsheng Li. Tao Dai, Bin Chen, Zhi Wang. and Shu-Tao Xia. Towards compact 3d representations point feature enhancement masked autoencoders. In Proc. of the AAAI Conf. on Artificial Intelligence, 2024.
[62] Yaohua Zha, Naiqi Li, Yanzi Wang. Tao Dai, Hang Guo, Bin Chen, Zhi Wang, Zhihao Ouyang, and Shu-Tao Xia. Lem: Locally constrained compact point cloud model for masked point modeling, Proc. of Advances in Neural Information Processing Systems, 2024.
[63] Yaohua Zha, Jinpeng Wang, Tao Dai, Bin Chen, Zhi Wang, and Shu-Tao Xia. Instance- aware dynamic prompt tuning for pre-trained point cloud models. In Porc. of IEEE Intl. Conf. on Computer Vision, 2023.
[64] Renrui Zhang, Ziyu Guo, Peng Gao, Rongyao Fang. Bin Zhao, Dong Wang, Yu Qiao, and Hongsheng Li. Point-m2ae: multi-scale masked autoencoders for hierarchical point cloud pre-training. In Proc. of Advances in Neural Information Processing Systems, 2022.
[65] Renrui Zhang, Liuhui Wang. Yu Qiao, Peng Gao, and Hongsheng Li. Learning 3d representations from 2d pre-trained models via image-to-poke masked autoencoders. In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, 2023.
[66] Tao Zhang, Xiangtai Li, Haobo Yuan, Shunping Ji, and Shuicheng Yan. Point could mamba: Point cloud learning via state space model. arXiv preprint 3xiv:2403.00700 2024.
[67] Hengshuang Zhao, Jiaya Jia, and Vladlen Koltun. Exploring self-attention for image recognition. In Proc. of IEEE Intl, Conf. on Computer Vision and Pattern Recognition, 2020.
[68] Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip HS Torr, and Vladlen Koltun. Point transformer. In Porc. of IEEE Intl. Conf. Computer Vision, 2021.
[69] Xiao Zheng, Xiaoshui Huang, Guofeng Mei, Yuenan Hou, Zhaoyang Lyu. Bo Dai. Wanli Ouyang, and Yongshun Gong. Point cloud pre-training with diffusion models. In Proc, IEEE Intl. Conf. on Computer Vision and Pottern Recognition, 2024.
[70] Xin Zhou, Dingkang Liang. Wei Xu, Xingkui Zhu, Yihan Xu, Zhikang Zou, and Xiang Bai. Dynamic adapter meets prompt tuning: Parameter-efficient transfer learning for point cloud analysis. In Proc. of IEEE Inrl. Conf. on Computer Vision and Pattern Recognition, 2024.
[71] Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang. Vision mamba: Efficient visual representation learning with bidirectional state space model. In Proc. of Intl. Conf. on Machine Learning, 2024.
14

---

Appendix A Theoretically Analysis
A.1 Closer look at Selective SSM
As described in Sec. 3, Selective SSM [14] considers parameters B,C, A in Eq. 2 as functions of the input. To be specific, given the input sequence [1,,.. ,o. matrices B,, C2 A can be computed as follows:
LL E 18 LL the per-time
B = LB())) C. = Lc(x), A = softphus (La(
(5)
where LB, Lo, La are linear projection layers, and softplus(x) log(1 + e*). The matrices A. B,, C₁, A can be obtained by taking Eq. into Eq. 2.
To simplify, we ignore the residual connection D and expand Eq. 1, the output ÿ [y,... , E IR L can be computed below:
Yt Cyhes
h4
Σ
II j=++1
A,
B
(6)
which can be further described in matrix form below:
h1
B
0
T1
h2
A,B
B₂
0
322
hr
AT
A.B
B
Drawing inspiration from the previous work [2]. we further provide an intuitive understanding. The Eq. resembles the self-attention mechanism with mask M, specifically causal self-attention. In this context, M is a lower triangular matrix with elements set 1. To further exam this, consider the transfer matrix W between and 8, i.e., (ÿ- W.:):
i
Wi = C,
A
B₁
(8)
k=j+1
C
II
exp
Ax))
B₁
(9)
kmj+1
C, exp
Σ
A
(10)
k=j+1
R C exp
Σ Rma+1 (A(FE
La (xx) A
Bj,
(11)
where W., represents the element in the i-th row and j-th column, the approximation in Eq. 11 done using ReLU instead of softplus. Consider the notation below:
(La (5x) A) (A)) A)
Q = C.,
Tr = exp
Σ k=S+1 La (x
K, = (B,)
(12)
Thus, the Eq. 11 can be simplified to:
W R QTTKK
(13)
This shows that the Selective SSM captures the influence of x; and xj using Q, and K,, respectively, while Túy molding the token significance from I; to Ty- Note that i S because W lower triangular matrix, indicating strong relationship with causal sclf-attention [4, 2].
15

---

A.2 Global modeling of PointMamba
In this subsection, we explain the global modeling of our PointMamba. Let's consider the total imput sequence as [ |',,, 21/22111/2211 ,zi] where the sequence has a length and is
even. i comes from Hilbert serialization and the other half, 1z. comes from
The large below:
X
0
matrix in Eq. 7 can be represented partitioned matrix
Y Z
B
I1
B
I A,B,
A A
B B
AB I7 A,B
I2 A, +1
B
(14)
Note that the block Y. highlighted in gray associated with both & (from B) and ls (from A), denoted Y (4, (2). The blocks X and Z only relate half of the sequence, denoted X (l,) and Z (12), respectively. Thus, the hidden space output (h1s hu [hi, h1/2:1//244 hi] can be compressed as below:
h1 = h2
X( r(,, + Z(i))
(15)
As in Fig. and Eq. 15, the serialized points from Trans-Hilbert can receive global information from Hilbert serialization.
Appendix B Concurrent Related Works
Table 9; Classification performance comparisons with other state space model methods on three variants of the ScanObjectNN [49]. All results are reported without voting.
ScanObjeetNN
Method
Param. (M)
FLOPs (G)
OBJ BG
OBJ OBL_ONLY
PB 1S_TSO_RRG RS
Point Cloud Mamba [66]
34.2
45.0
88.10
Mamha3D [24]
16.9
3.9
93.12
92.08
88.20
PoinTramba [53]
19.5
92.30
91.30
89.10
PointMamba (ours)
3.1
94.32
92.60
89.31
Some works on state space models for point cloud analysis appeared recently. This section discusses the differences between these methods and PointMamba.
Point Cloud Mamba (PCM) [66] combines an improved Mamba module, i.c., Vision Mamba [71], with PointMLP [37] (a strong point cloud analysis method), and incorporates consistent traverse serialization at each stage. To enhance Mamba's 's capability managing point sequences with varying orders, PCM introduces point prompts that convey the sequence's 's arrangement rules. While these techniques improve the performance state space models, they also introduce additional computational overhead and complexity in design.
Mamba3D [24] another recently proposed method. obtain better global features, Mamba3I introduces enhanced Vision Mamba [71] block, which includes both token forward SSM and
16

---

KNN
Tekenenbedlll lover
Zilbert
Vanilla Mandu Nock
Pooling
Seg
indiseer
Canan
Thons Ailben
Sampling
Concuterate
Gkthll Pasppint
Eantre
fistuee
Figure 6: The details of our PointMamba for segmentation task.
backward SSM that operates on the feature channel. And it proposes Local Norm Pooling block to extract local geometric features.
PointTramba [53] introduces hybrid approach that integrates Transformers and Mamba. It segments dependencies, while Mamba
point clouds into groups and utilizes Transformers to capture models siterggrupp relationships using bi bi-directional, importance aware ordering strategy.
As shown in Tab. 9, our PointMmmaa surpasses these concurrent methods by offering superior performance with reduced computational overhead. Note that method extremely simple and without complex design, which utilizes the vanilla Mamba block and abstains from incorporating modular designs from other baselines, thereby maintaining simplicity and efficiency in approach. We believe such method can better illustrate the potential of SSM in point cloud analysis tasks.
Appendix C More experimental resutls
C.1
Implement details
Table 10: Implementation details for pre-training and downstream tasks.
Pre-training
Classification
Segmentation
Configuration
ShapeNetCore
ModelNer40
ScanObjectNN
ShapeNetPart
Optimizer
AdamW
AdamW
Adam
AdamW
Learning rate
1c-3
3e-4
5c-4
2c-4
Weight decay
5e-2
5c-2
5e-2
5e-2
Learning rate scheduler
cosine
cosine
cosine
cosine
Training epochs
300
300
300
Warmup epochs
10
10
10
10
Batch size
128
32
32
16
Num. of encoder layers N
12
12
12
12
Num. of decoder layers
4
Input points M
1024
1024
2048
2048
Num. patches n
128
128
Patch size ls
32
32
32
32
Augmentation
Trale&Trans
Scale&Trans
Rotation
Pre-training Details. The ShapeNetCore dataset [5] is used for pre-training, including ~1 clean 3D sample across 55 categories. The 1,024 input points are divided into 64 point patches, with each patch consisting of 32 points. The pre-training process includes 300 epochs, with a batch size of 128. More detail can be found in Tab. 10.
Downstream tasks Details. Fig 2 shows the pipeline of PointMamba for classification tasks. We report the overall accuracy without voting on the challenging ScanObjectNN [49] using 2,048 input points, and on ModelNet40 [57] using 1,024 input points. For segmentation on ShapeNetPart [58], as shown Fig. 6, we use random, Hilbert, and Trans-Hilbert serializations, with order indicators serializations. Features from the 3-rd, 7-th, and last layer are pooled
applied on Hilbert/Trans- as global features after a simple feature fusion. These global features are then concatenated with per-point features and sent to the segmentation head. More detail can be found in Tab. 10.
17

---

Table 11: The effect of masking strategy. The pre-training loss (x 1000) along with fine- tuning accuracy (%) are reported.
Table 12: The effect of classification token.
Fine-tuning accuracy (%) are reported.
Methods
OBJ-BG
OBI-ONLY
Musking ratio
Loss
OBI-BG
OBI-ONLY
Before the sequence
93.63
91.05
0.4
2.01
92.60
90.70
After the sequence
94.32
90.19
0.6
1.97
94,32
92.60
Middle the sequence
93.98
90.71
0.8
2.33
93.46
90.17
Max
93.39
91.36
0.9
2.00
92.43
91.05
Average Pool
94.32
92.60
Input
Masking
Reconstruction
Input
Masking
Reconstruction
Figure 7: The qualitative results of mask predictions of PointMamba on ShapeNet validation set.
C.2
Additional ablation study
In this section, we do additional ablation studies on several syper-parameters
Masking strategy for pre-training. By employing serialization- based mask modeling paradigm, PointMamba achieves superior performance. To find a proper masking strategy for our method, we compare two types of masking with varying ratios. The block masking [60] masks geometrically proximate point cloud patches, leading a more challenging reconstruction target. In Tab. 11, experimentally find that masking 60% of point patches by randomly choosing can achieve good performance.
Usage of classification token. Previous works [12, 60, 38] often use classification token (CLS) global token for classification. As in Tab. 12, find that without (CLS) and utilizing only the average pooling of the final block's 's output yields the best results for PointMamba.
Appendix D Qualitative Analysis
D.1 Mask modeling visualization
As in Sec. 4.2, we customize simple yet effective serialization-base mask modeling paradigm. By randomly masking about 60% of serialization-bian based point tokens, an asymmetric vanilla Mamba autoencoder is utilized to extract the point feature, with simple prediction head for reconstruction. In Fig. we present qualitative results of mask modeling on ShapeNet validation set. Despite 60% masking ratio, our PointMamba effectively reconstructs the masked patches, providing a strong self-supervised knowledge for downstream tasks.
D.2 Part segmentation visualization
In this subsection, we present the qualitative results for part segmentation on the ShapeNetPart valida- tion set, including both the ground truth and the predicted results. As in Fig, 8, our PointMamba shows highly competitive results on part segmentation.
18

---

GT
Predicted
Airplane
Lamp
Lapnop
Figure 8: The qualitative results of part segmentation of our PointMamba on ShapeNetPart.
NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
See abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings. It is fine include aspirational goals as motivation long it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
See limitation part.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
19

---

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: See experiments.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proof's provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Answer: [Yes]
Justification: See experiments part.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
20

---

reproducible, they (c) If the contribution is new model (e.g.. a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described supplemental material?
Answer: [NA]
Justification:
The code will made available.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.c cc/
public/ cublic/guides. /ttdg//sooooeeuuuiiiisssssssssss SoodeSumisssionP conoolccy)) for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / /niiss.. http:puulllggddeeeee
Codeeubmiis sesisssssssssiiieeoe
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification:
See experiments part.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
See experiments part.
21

---

Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the eπror bar is the standard deviation or the standard error of the mean.
It is OK to report I-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (c.g. negative error rates).
show in tables
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification:
See experiments part.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that NeurlPS Code of Ethics https h/t.uerurip...e wwwpuariss.cpppppleeee httpp/blict..iiccccceeeeeeeeeeee
didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the
Answer: [Yes]
Justification:
We follow the NeurIPS Code of Ethic.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [NA]
22

---

Justification:
no societal impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (c.g., pretrained language models, image generators, scraped datasets)?
Answer: [NA]
Justification:
No such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We will release the code.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
23

---

The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors are encouraged reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We use the public assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
25


Reviews:
[{"review_id": "l2GONhhW2L", "reviewer": "Reviewer_Daqu", "summary": "This work utilizes the Mamba architecture for point clouds. It employs Hilbert and Trans-Hilbert curves to order the point clouds, thus addressing the unidirectional modeling nature of Mamba. Additionally, it replaces transformer blocks with Mamba blocks. The proposed PointMamba demonstrates reasonable performance following pre-training.", "questions": "See the weakness section.", "limitations": "See the weakness section.", "rating": 3, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. This work tries introducing a new network architecture to the point cloud domain, which is appreciated.\n2. The presentation is clear, accompanied by well-crafted figures.", "weaknesses": "1. While it is worthwhile to explore how the Mamba architecture performs when applied to point clouds, this work exhibits limited novelty or insights in architectural innovation for point clouds. The authors themselves acknowledge this by stating, “It should be noted that this paper does not claim algorithmic novelty but rather presents a simple and solid Mamba-based baseline.” In my opinion, works that directly adopt an existing network from other domains to point clouds without sufficient insights and innovation should not be accepted by top-tier conferences like NeurIPS.\n\n2. The authors only demonstrate the performance of **pre-trained** PointMamba by comparing it with other widely used baselines like PointNext. This implies that PointMamba cannot surpass previous methods when trained from scratch. To effectively showcase the solid merits of PointMamba, it would be more reasonable to provide comparisons without pre-training.\n\n3. In Table 4, an important baseline, PointNext, is omitted. According to the PointNext paper, PointNext exhibits significantly better performance than the proposed PointMamba (87% without pre-training vs. 84.4% with pre-training). This omission raises doubts about the effectiveness of PointMamba. To give a \"solid Mamba-based baseline\", comprehensive comparisons with previous methods should not be omitted.\n\n4. In Figure 1, it would be beneficial to include comparisons with widely used methods like PointNext to convincingly demonstrate the necessity of introducing Mamba for point clouds. While PointMamba likely offers inference advantages, this contribution stems from the original Mamba paper, not this work. Additionally, does PointMamba have lower training efficiency?\n\n5. The paper claims that the proposed Hilbert and Trans-Hilbert ordering is advantageous based on results from ScanObjectNN. However, I am not entirely convinced. I suggest the authors also present results on ShapeNet-Part and ModelNet40, for which only comparing against a random baseline is enough.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "- **To Weakness 1:**\n\n**Reply:** Thanks. We would like to emphasize that **simply replacing the Transformer with the Mamba cannot achieve ideal performance** due to the unidirectional modeling (as shown in below table). Thus, we customize the key designs to handle the point cloud data: **1)** The proposed **point scanning strategy** transforms the unstructured 3D point clouds into a regular sequence, which can provide diverse perspectives on spatial locality via scans from different curves; **2)** The **order indicator** maintains the distinct spatial characteristics of different scanning when training, preserving the integrity of the spatial information, which is crucial for the unstructured point cloud; **3)** **Serialization-based mask modeling** randomly choose a space scanning curves for masking, allowing the model exact the general local relationships from different scanning perspectives, better matching the requirement of unidirectional modeling of Mamba.  \n\nConsidering that this paper is the first work that discusses the potential of Mamba in point cloud tasks, it would bring new insights to the community and be interesting for many people. **A similar toy example is the paper you mentioned, PointNeXt**, **an empirical study**, that mainly studies the data augmentation in point cloud without specific new designs (90% of performance gains from data augmentation and hyper-parameter adjustment, while the remaining 10% comes from micro-designs such as residual connections), but many people still like such a simple yet effective paper. We hope the reviewer could reconsider the motivation and insights of our method (also see reply to weaknesses 2 & 3).\n\n| Method | OBJ-BG  | OBJ-ONLY  | PB-T50-RS | ModelNet40 |\n| --- | --- | --- | --- | --- |\n| Point-MAE | 92.77 | 91.22 | 89.04 | 93.2 |\n| Simply replace Transformer with Mamba in Point-MAE | 92.25 | 90.69 | 87.11 | 92.4 |\n| PointMamba (ours) | 94.32 | 92.60 | 89.31 | 93.6 |\n\n- **To Weaknesses 2 & 3:**\n\n**Reply:** Thanks! **It seems that the reviewer might be confusing different performance metrics** **on ShapeNetPart** (Inst. mIoU, 86.2%, and 87.0% for PointMamba and PointNeXt, respectively, are correct). Note that, **the performance of PointNeXt is somewhat misleading** and shouldn’t be directly compared due to the following reasons: **1)** PointNeXt adopts **extensive data augmentation**, while our method and other Transformer-based methods only adopt random rotation for ScanObjectNN and random scaling for ShapeNetPart; **2)** PointNeXt adopts the **voting strategy** and **manual refinement** on this dataset (i.e., ShapeNetPart), which is an **unfair post-processing** that is usually resisted by the  Transformer-based methods. \n\nAs such, no vanilla Transformer-based method can outperform PointNeXt when these unfair practices are applied. Therefore, we believe the reviewer should not deny a series of good Transformer-based works for these reasons.  For fair comparisons, we remove all tricks and pre-training, as shown below, where we perform better than PointNeXt on most datasets. \n\nOverall, PointNeXt is the MLP paradigm instead of Transformer-based, and it mainly discusses the effect of various data augmentation in vanilla PointNet. In this paper, we aim to unlock the potential of Mamba in point cloud tasks, discussing whether it can be a viable alternative to Transformers. Thus, the direct comparison of our PointMamba is the vanilla Transformer-based methods. **Through extensive experiments, we clearly outperform the SOTA Transformer-based counterparts and even include the cross-modal method (ACT, ICLR 23).** \n\n| Method | Data augmentation | Voting | ShapeNetPart (scratch/pre-training) |\n| --- | --- | --- | --- |\n| PointNeXt (NeurIPS 22) | Random rotation, Random scaling, Random translation, Random jittering, Normal Drop, Height appending | Yes  | 87.0/- |\n| PointNeXt (NeurIPS 22) | Random scaling | No | 85.7/- |\n| ACT (ICLR 23) | Random scaling | No | 85.8/86.1 |\n| PointGPT (NeurIPS 23) | Random scaling | No | 85.6/86.2 |\n| PointMamba | Random scaling | No | 85.8/86.2 |\n\n| Method |  Data augmentation | OBJ-BG (scratch/pre-training) | OBJ-ONLY (scratch/pre-training) | PB-T50-RS (scratch/pre-training) |\n| --- | --- | --- | --- | --- |\n| PointNeXt (NeurIPS 22) | Random rotation, Random scaling, Random translation | 90.88/- | 90.36/- | 87.7/- |\n| PointNeXt (NeurIPS 22) | Random rotation | 90.71/- | 89.50/- | 87.20/- |\n| ACT (ICLR 23) | Random rotation | -/93.29 | -/91.91 | -/88.21 |\n| PointGPT (NeurIPS 23) | Random rotation | -/93.39 | -/92.43 | -/89.17 |\n| PointMamba | Random rotation | 91.74/94.32 | 90.19/92.60 | 87.27/89.31 |\n\n- **To weakness 4:**\n\n**Reply:** Thanks! **1)** We will include the suggested methods in Figure 1 in the revised version. In addition, the clarification of PointNeXt is presented in reply to weaknesses 2&3;  **2) Although Mamba performs inference advantages, it can not work well in the point cloud by simply replacing Transformer with Mamba**. PointMamba not only keeps the efficiency of Mamba but also makes it achieve promising performance in the point cloud. **3)** Besides, as shown below table, PointMamba presents superior training efficiency (second/epoch) compared with the convincing Transformer-based method, especially when the length of point tokens increases.\n\n| Sequence length | 128 | 256 | 512 | 1024 | 2048 |\n| --- | --- | --- | --- | --- | --- |\n| Point-MAE | 97.24 | 128.99  | 194.56  | 351.64 | 797.78 |\n| PointMamba | 89.18 | 112.57  | 150.55  | 225.56 | 374.89 |\n\n- **To weakness 5:**\n\n**Reply:** Good suggestion! We now provide the analysis of our point-scanning strategy on ShapeNet-Part and ModelNet40. These results clearly prove the effectiveness of our proposed key designs when adopting Mamba to point clouds. We will release the code to ensure the results are convinced. \n\n|  | ModelNet40 | ShapeNetPart |\n| --- | --- | --- |\n| Random | 92.7 | 85.7 |\n| Hilbert + Trans-Hilbert | 93.6 | 86.2 |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OkrO6NeqJE", "reviewer": "Reviewer_QPV3", "summary": "This paper introduces PointMamba, an interesting method for point cloud analysis that utilizes a linear complexity state space model (SSM) instead of traditional Transformer architectures. PointMamba employs space-filling curves for point tokenization and features a simple, non-hierarchical Mamba encoder. Additionally, the authors propose an effective order indicator and serialization-based mask modeling strategy. Comprehensive evaluations across multiple datasets demonstrate PointMamba's superior performance and significantly reduced computational costs compared to Transformer-based methods.", "questions": "See weakness.", "limitations": "The authors have adequately discussed the limitations of their work.", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. The paper reads very well. I appreciate the motivation of the paper and agree that we should make an effective method as simple as possible. As the first Mamba-based work for point cloud tasks, the proposed method is simple, elegant, and effective, establishing a solid Mamba-based baseline for point cloud analysis tasks.\n\n2. The paper considers the limitations of Mamba's unidirectional modeling and proposes practical solutions like serialization-based mask modeling strategies and order indicators.\n\n3. The theoretical analysis of Mamba used in point cloud is reasonable, and the paper is easy to reproduce.\n\n4. The experiments are convincing and support the main idea of the paper. The authors provide thorough evaluations, including comparisons with SOTA methods, ablation studies, and analyses of each component.", "weaknesses": "1. Some experiments are missing. For example, from Table 11, it would be beneficial to include more masking ratios, especially the performance when masking 90% point patches. Besides, from Table 12, could the authors provide a more detailed analysis of why average pooling performs better? Additionally, what about the performance of max pooling?\n\n2. Have the authors considered the potential benefits of integrating PointMamba with Transformer architectures? For example, a transformer layer can be used as the final prediction head. Such a combination can take advantage of both architectures and will not introduce many computational costs.\n\n3. The point clouds have complex structures, and this paper only considers two orders. In my view, introducing more orders (e.g., introducing three serialization methods and tripling the input length) can better capture the geometry information of the point clouds, which might be beneficial for learning. \n\n4. During the pre-training strategy, do the authors use different order indicators or the same indicator? Figure 4 is somewhat ambiguous.\n\nMinor: Using space-filling curves to scan point clouds is an interesting attempt. Although it is common knowledge for some readers, it still would be helpful if the authors provided a more detailed introduction to space-filling curves in the preliminaries part.\n\ntypos: PointMAE -> Point-MAE", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "- **To Weakness 1: “Some experiments are missing…masking 90% point patches…the performance of max pooling?”**\n\n**Reply:** Good question! We conduct the mentioned missing experiments:\n\n(1) As shown in the table below, masking 90% point patches may harm performance, as a higher masking ratio makes reconstruction tasks too difficult during pre-training.\n\n| Masking ratio | Loss | OBJ-BG | OBJ-ONLY |\n| --- | --- | --- | --- |\n| 0.9 | 2.00 | 92.43 | 91.05 |\n\n(2) Average pooling takes into account the entire sequence with all the information, while max pooling selects the maximum value of each token in each dimension through a nonlinear process.  We empirically find that only use max-pooling may destroy the global information of the extraction and cause performance drop, as shown in the table below.\n\n|  | OBJ-BG | OBJ-ONLY |\n| --- | --- | --- |\n| Average pooling | 94.32 | 92.60 |\n| Max pooling | 93.39 | 91.36 |\n\n- **To Weakness 2: “… benefits of integrating PointMamba with Transformer architectures … a transformer layer can be used as the final prediction head …”**\n\n**Reply:** Good suggestion! Based on your advice, we kept the total number of blocks the same and replaced the last layer with a Transformer. As shown in the table below, we observed a confirmed performance drop. We argue that optimizing two modules simultaneously can be challenging, and developing an effective hybrid architecture is a promising topic for future research.\n\n|  | # TP | FLOPs | OBJ-BG | OBJ-ONLY |\n| --- | --- | --- | --- | --- |\n| 12 Mamba-block | 12.3 | 3.1 | 94.32 | 92.60 |\n| 11 Mamba-block + 1 Transformer-block | 13.1 | 3.4 | 91.98 | 89.64 |\n\n- **To Weakness 3: “…  introducing more orders …”**\n\n**Reply:** Thanks! Based on your suggestion, we conducted an experiment by adding an additional z-order sequence to triple the input length. As shown in Fig. 3 and Appendix A of our paper, using two orders enables global modeling for PointMamba; however, adding redundant information may lead to performance degradation.\n\n|  | FLOPs | OBJ-BG | OBJ-ONLY |\n| --- | --- | --- | --- |\n| Hilbert + Trans-Hilbert | 3.1 | 94.32 | 92.60 |\n| Hilbert + Trans-Hilbert + Z | 3.6 | 93.43 | 91.36 |\n\n- **To Weakness 4: “…Figure 4 is somewhat ambiguous.”**\n\n**Reply:** We apologize for the ambiguity. During pre-training, different serialized point tokens have different order indicators. We will clarify Fig.4 in the next version. \n\n- **To Minor & Typos:**\n\n**Reply:** Thanks for the comments! Here is a brief introduction to Hilbert space-filling curves. Denoting the coordinates of voxelized points as $(x,y,z)$ and convert them into binary format using $log_2 n$ bits ( e.g., $x\\to(x_{m}x_{m-1}...x_{0}),m=\\lfloor log_2 n \\rfloor$). $x_m,y_m,z_m$ is then iterated to $x_1,y_1,z_1$, making exchanges when the current bit is 0, and inversions otherwise. By concatenate the bits into $(x_{m}y_{m}z_{m}x_{m-1}y_{m-1}z_{m-1}\\ldots x_{0}y_{0}z_{0})$ and apply a 3$m$-fold Gray decoding, the traversal position $T$ can be obtained. All voxels are then sorted into a single sequence based on their traversal position $T$. By recording the traversal position for all potential voxel coordinates, the points can be serialized. We will add this introduction and carefully revise our paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "R981ExRC6m", "reviewer": "Reviewer_vmtW", "summary": "This paper propose a simple but effective Mamba-based method named PointMamba for point cloud analysis. This paper is the first paper that studies the Mamba-based method for point clouds. The experiments are comprehensive and the paper is in very good shape.", "questions": "1.\tIt has been experimentally determined that Point-MAE, Point-M2AE, and MaskPoint do not accurately replicate their reported performance, as observed in this study. It is important to investigate whether PointMamba exhibits consistent stability across all of these tasks. If not, providing mean and standard deviation values similar to those presented in Table 3 is crucial.\n2.\tAdditionally, it has been observed that conducting experiments with a larger number of points using Mamba results in an increase in training time. Have you encountered this phenomenon in your own experiment? If so, kindly elaborate on possible solutions to enhance its suitability for real-world applications, such as Auto-Driving.\n3.\tCould you please elaborate on the reasons why incorporating Hilbert and Trans-Hilbert techniques in your current version has led to improved performance compared to the reordering strategy implemented in the initial version? Moreover, Hilbert and Trans-Hilbert techniques seem to borrow from Point Transformer. If so, the main contribution lies in integrating Mamba into Point-MAE.\n4.\t(No need for experiments, just discuss.) There are several Mamba-based methods after this paper. Please just discuss the advantages and disadvantages compared with them.", "limitations": "1.\tThe authors have addressed the limitations and potential negative societal impact of their work.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1.\tThis paper demonstrates excellent writing quality, with a clear and evident motivation.\n2.\tFigure 1 is highly comprehensible, particularly in its comparisons with Point-MAE.", "weaknesses": "1.\t(No need for experiments) The use of validation datasets is prevalent in the field, with consistently high metrics reported. I encourage the inclusion of indoor segmentation and detection tasks, as exemplified in Point-M2AE and MaskPoint, in future iterations of your work. Additionally, tackling the classification task on Objaverse-LVIS appears to be a more demanding and stimulating challenge.\n2.\tTo be honest, there are numerous papers that adhere to the evaluation paradigm established by Point-BERT. However, I am genuinely interested in exploring novel discoveries in self-supervised learning specifically applied to point clouds.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "- **To weakness 1: “(No need for experiments)…encourage the inclusion of indoor segmentation and detection tasks…”**\n\n**Reply:** Thanks for this very constructive suggestion, and we will explore a unified Mamba-based foundation model for various 3D vision tasks (including indoor segmentation, detection tasks, and classification tasks on Objaverse-LVIS ) in the future. We also appreciate the reviewer's understanding that finishing such experiments in a limited time is difficult.\n\n- **To weakness 2: “…Point-BERT…genuinely interested in exploring novel discoveries in self-supervised learning…”**\n\n**Reply:** We totally agree with this insightful point. Actually, this paper also explores a new self-supervised pre-training paradigm named **serialization-based mask modeling** (Sec. 4.2). Specifically, different previous methods (e.g., Point-BERT) using random masking, we consider the unidirectional modeling of Mamba and propose to randomly choose one space-filling curve to generate the serialized point tokens for mask modeling. In addition, the proposed extremely simple order indicator effectively maintains the distinct characteristics of these different scanning strategies. As a starting point for this area, we hope this paper can provide useful insight for the community and encourage researchers to focus on the potential of Mamba in 3D vision tasks. \n\n- **To question 1: “… consistent stability across all of these tasks…providing mean and standard deviation…”**\n\n**Reply:** Thanks! We promise that all code will be released and the results can be easily reproduced. Following your suggestion, we report the mean and std in all datasets under three runs, as shown below. We can see that the proposed method achieves consistent stability across different tasks/datasets. \n\n| Method | OBJ-BG | OBJ-ONLY | PB-T50-RS | ModelNet40 | ShapeNetPart |\n| --- | --- | --- | --- | --- | --- |\n| PointGPT (NeurIPS 23) | 93.39 | 92.43 | 89.17 | 93.3 | 86.2 |\n| PointMamba (Ours) | 3 $\\times$ runs: [93.76, 93.98, 94.32] | 3 $\\times$ runs: [92.31, 92.60, 92.47] | 3 $\\times$ runs: [89.44, 89.31, 89.25] | 3 $\\times$ runs: [93.6, 93.6, 93.5] | 3 $\\times$ runs: [86.18, 86.12, 86.16] |\n\n- **To question 2. “… a larger number of points…training time…possible solutions…real-world applications…”**\n\n**Reply:** Thanks! Following your suggestion, we evaluate the training time (second/epoch) with different input lengths (from 128 to 2048) with a batch size of 16, as shown below. We find that with point tokens increasing, the advantages of our approach over Point-MAE will be even more pronounced, further indicating the efficiency of our PointMamba. In addition, to apply the PointMamba to Auto-Driving, a possible way is to design an efficient voxel-based tokenizer that can effectively transform the larger number of point clouds into a series of long point/voxel tokens, which is a promising future work.\n\n| Sequence length | 128 | 256 | 512 | 1024 | 2048 |\n| --- | --- | --- | --- | --- | --- |\n| Point-MAE | 97.24 s/epoch | 128.99 s/epoch | 194.56 s/epoch | 351.64 s/epoch | 797.78 s/epoch |\n| PointMamba | 89.18 s/epoch | 112.57 s/epoch | 150.55 s/epoch | 225.56 s/epoch | 374.89 s/epoch |\n- **To question 3: “…improved performance compared to…the initial version? …borrow from Point Transformer…the main contribution lies in integrating Mamba into Point-MAE”**\n\n**Reply:** Thanks! In fact, the improved performance compared to the initial version is not simply replacing the reordering strategy but rather the comprehensive new proposed technology. Specifically, compared with the initial version, we make the following new contribution: **1)** The **proposed point scanning strategy transforms the unstructured 3D point clouds into a regular sequence, which can provide diverse perspectives on spatial locality via scans from different clues; 2) The order indicator** maintains the distinct spatial characteristics of different scanning when model training, preserving the integrity of the spatial information, which is crucial for the unstructured point clouds ; **3)** **Serialization-based mask modeling** randomly choose a space scanning curves for masking, allowing the model exact the general local relationships from different scanning clue, better matching the requirement of unidirectional modeling of Mamba.\n\nFurthermore, we would like to clarify that our method are totally different from Point-MAE: **1)** For processing steps, Point-MAE directly generates the random tokens via a lightweight PointNet. In contrast, we propose a **point scanning strategy**, which leverages the space-filling curves to transform the unstructured point clouds into a regular sequence. Although Point Transformer also adopts space-filling curves, the motivation and objective are different. Point Transformer utilizes space-filling curves to **partition the point cloud,**  capturing spatial contexts. In contrast, our work mainly focuses on transferring the point clouds to serialization-based sequences and combines it with Mamba to **implement global modeling**. **2)** For feature extraction, Point-MAE employs a vanilla Transformer. We merge the vanilla Mamba with the proposed **order indicator**, making the Mamba preserve the integrity of the spatial information from different scanning curves;  **3)** For pre-training, the PointMAE is the traditional random mask modeling paradigm, while our PointMamba proposes **serialization-based mask modeling**, which employ serialization masking modeling, combining with the proposed order indicator; **4)** We also provide detailed **theoretical analyses** of why PointMamba work well in point cloud tasks, while it seems lack on PointMAE.\n\nOverall, the current version makes substantive improvements compared with Point-MAE and our initial version. Extensive experiments also validate the effectiveness of our method (See Fig.1 of the manuscript).\n\n**For question 4, please see the reply presented in the “comment” part.**", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "x0UDLSfNPJ", "reviewer": "Reviewer_MyQv", "summary": "In summary, PointMamba, an innovative state space model tailored for point cloud analysis, successfully harnesses the global modeling prowess of Mamba, a representative SSM from the NLP domain. By adopting a linear complexity algorithm, PointMamba addresses the computational challenges posed by traditional Transformer-based methods while maintaining their global modeling capabilities. Its key techniques include utilizing space-filling curves for effective point tokenization and employing a non-hierarchical Mamba encoder as the foundation. Extensive experiments across multiple datasets validate the superior performance of PointMamba while significantly reducing GPU memory usage and FLOPs. This work not only demonstrates the vast potential of SSMs in 3D vision tasks but also provides a simple yet effective baseline for future research in this domain.", "questions": "1. Based on weakness 4, is it possible that PointMamba actually only works well in Hiber conditions? As this very much affects the final results and conclusions of the paper.\n\n2. Intuitively, PointMamba seems to be just a replacement of the Mamba block (Equation 4) for the previous Transformer block, and even if this results in good performance (Figure 1), I am not convinced. The paper is theoretically and experimentally fleshed out, but I still don't find an obvious innovative contribution. Therefore, I would like to confirm whether the authors have designed a specialized Mamba technique for 3D point clouds?", "limitations": "Content-wise, the paper has no obvious limitations, and its core technology builds on Mamba, which is popular in other fields. Formatting-wise, the paper shows the mamba icon several times; please confirm whether this representation is appropriate in an academic paper.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. **Linear Complexity with Global Modeling**: PointMamba leverages state space models to achieve linear complexity while maintaining global modeling capabilities, overcoming the computational challenges of traditional Transformers.\n\n2. **Efficient Point Cloud Representation**: The use of space-filling curves for point tokenization enables efficient representation of point clouds, capturing spatial structure while facilitating global feature extraction.\n\n3. **Simple and Effective Mamba Encoder**: The non-hierarchical Mamba encoder provides a simple yet powerful backbone for PointMamba, enabling fast and accurate global feature modeling.\n\n4. **Superior Performance**: Comprehensive evaluations show that PointMamba achieves state-of-the-art performance across multiple datasets, demonstrating its effectiveness for point cloud analysis tasks.", "weaknesses": "1. Although PointMamba borrows structurally from Mamba, it may not take full advantage of the unique characteristics of point cloud data, such as spatial distribution, density variations, and local geometry.\n\n2. PointMamba, while inheriting the strengths of Mamba, may have inherited some of the limitations of its design for natural language processing tasks. These limitations may not be applicable to point cloud analysis tasks, such as the lack of specific preprocessing steps, feature extraction methods, or post-processing techniques for point cloud data. The feature extraction and processing steps in the paper are largely similar to those of the previous PointMAE.\n\n3. Although PointMamba achieves global modeling of linear complexity by introducing state-space models, there may be trade-offs between model complexity and performance in real-world applications. Because according to the experimental results of the thesis, the effect of partially using PointMamba becomes worse instead.\n\n4. According to Table 5 of the ablation experiment, the serialization operations Hiber and Trans-Hiber have the most obvious impact on the experimental results. Meanwhile, the framework diagram at the core of the paper, i.e., Figure 4, shows that the only change is Hiber, from the directly cited literature [27], and there is almost no additional description or setting in the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "- **To weakness 1&2: “… may not take full advantage of the unique characteristics of point cloud data…/…may not be applicable to point cloud…are largely similar to…Point-MAE.”**\n\n**Reply:** Thanks. We respect the reviewer's opinion. However, we believe this paper considers the characteristics of point clouds. Specifically, through a pilot of experiments, we first demonstrate that **simply replacing the Transformer with a Mamba can not achieve ideal performance** due to the unidirectional modeling. Based on this, we customize the key innovative designs to handle the point cloud data: **1)** The **proposed point scanning strategy** transforms the unstructured 3D point clouds into a regular sequence, which can provide diverse perspectives on spatial locality via scans from different clues; **2) The order indicator** maintains the distinct spatial characteristics of different scanning when training, preserving the integrity of the spatial information, which is crucial for the unstructured point cloud; **3)** **Serialization-based mask modeling** randomly choose a space scanning curves for masking, allowing the model exact the general local relationships from different scanning perspectives, better matching the requirement of indirection modeling of mamba. **Thanks to these proposed simple yet innovative key designs, our PointMamba works well and performs better than its Transformer-based counterparts in point cloud tasks**. We will make these technical contributions more clearly in the revised version.\n\nBesides, compared with the Point-MAE, the differences are also distinct: 1) **For processing steps**, Point-MAE directly generates the random tokens via a lightweight PointNet. In contrast, we propose a **point scanning strategy**, which leverages the space-filling curves to transform the unstructured point clouds into a regular sequence; 2) **For feature extraction**, Point-MAE employs a vanilla Transformer. In contrast, we **merge the** **vanilla Mamba with the proposed** **order indicator**, making the mamba preserving the integrity of the spatial information from different scanning curves;  3) **For pre-training**, the Point-MAE is the traditional random mask modeling paradigm, while our PointMamba proposes **serialization-based mask modeling**, allowing the model exact the general local relationships from different scanning curves via the proposed order indicator; 4）We also provide **detailed theoretical analyses** of why PointMamba work well in point cloud tasks, while it seems lack on Point-MAE.\n\nOverall, although our method presents a simple pipeline, the processing steps, feature extraction, pre-training, etc, are significantly different from the previous methods. **As R2 said, “…the proposed method is simple, elegant, and effective, establishing a solid Mamba-based baseline for point cloud analysis tasks…”**, we believe our method would be interesting and valuable for people in this area. We will deeply discuss the differences in the revised version.\n\n- **To weakness 3: “…trade-offs between model complexity and performance…the effect of partially using PointMamba becomes worse instead.”**\n\n**Reply**: Thanks. Please refer to Tab.1, Tab.2, and Tab.3 of the manuscript, where our method consistently performs better on all conducted datasets compared with the single-modal SOTA Transformer-based methods. Note that some methods (e.g., ACT) adopt multi-modal information (e.g., text description or 2D), which is unfair for comparison, but we still surpass it in most datasets. \n\n- **To weakness 4: “…have the most obvious impact on the experimental results…no additional description or setting in the paper.”**\n\n**Reply:** Thanks. We prove that directly replacing the transformer with mamba in Point-MAE can not achieve ideal performance due to the unidirectional modeling of mamba in Point Cloud. Therefore, we propose serialization operations, including point scanning strategy,  order indicator, and serialization-based mask modeling. Therefore, **serialization operations naturally reflect the performance gains, which verify the effectiveness of our proposed key designs.**\n\nBesides, we would like to clarify that Fig.4 is not the only proposed core. It just means the proposed serialization-based mask modeling paradigm and the comprehensive overall core designs are presented in Fig.2. Compared with the Transformer-based method (e.g., Point-MAE, PointGPT), we present substantive differences: point scanning strategy, order indicator, serialization-based mask modeling (please see the reply to weakness 1&2). Furthermore, the Hilber is a representative space-filling curve, which we adopt into our proposed key designs. The preliminary of Hilbert is listed in Sec. 3 of the manuscript, and we provide further descriptions in reply to the minor problem of Reviewer 2 (Please refer to it). \n\n- **To question 1: “…only works well in Hiber conditions?…”**\n\n**Reply:** Thanks. We have conducted experiments to analyze the effect of different scanning curves in our key designs, as shown in Tab.6 of the original manuscript. Specifically, we argue that using the space scanning curve sequences along a specific pattern of spatial locations offers a more logical sequence modeling order for SSM. The experiments also prove that it can achieve notable performance gains compared with the random paradigm. Among them, Hilbert achieves the best results due to their superior locality-preserving properties. \n\n- **To question 2: “…obvious innovative contribution…specialized Mamba technique for 3D point clouds?”**\n\n**Reply:** Good question! This paper is not just a replacement of the Mamba block (Equation 4) for the previous Transformer block, please see the reply to weakness 1&2.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear Reviewers,

We are grateful to the reviewers for their invaluable feedback and the time they dedicated to evaluating our work. We are excited to see that reviewers identified **the novelty of our technical contribution (R2), clear motivation (R2, R3), convincing experiments (R2), superior performance (R2, R3, R4), and well-written (R1, R2, R3).**

**We respond to each reviewer separately with a detailed analysis to answer all the questions**. We think that our following rebuttal has sufficiently addressed the concerns raised by the reviewers. Please reply if you have any further questions. Thank you again for your insightful feedback, and we look forward to continuing the discussion. 

Best regard,

Paper 940 Authors

Meta-review: This paper received mixed ratings of Reject, Strong Accept, and two Borderline Accepts.  Three Reviewers see the merits of this work and share positive opinions.  Reviewer Daqu (who suggested Reject) has concerns in the insights of this work in architectural innovation for point clouds, and comparison to baseline PointNext. After reading the authors rebuttal and the reviewers' comments, the AC agrees with the other 3 reviewers that introducing Mamba to point clouds in non-trival and useful to the community. Comparative results to PointNext have been provided in the rebuttal. The AC suggests Accept and encourages the authors to includes these additional importants results provided in Rebuttal into the camera ready version.
Decision: Accept (poster)