Exploring contextual modeling with linear complexity for point cloud segmentation

## Abstract

Abstract Point cloud segmentation is an important topic in 3D understanding. Traditionally, this task has been tackled using either the CNN or Transformer. Recently, a newcomer, Mamba, has emerged as a promising alternative, offering efficient long-range contextual modeling capabilities without the quadratic complexity associated with the Transformer’s attention mechanisms. However, despite Mamba’s potential, early efforts have all failed to achieve better performance than the best CNN-based and Transformer-based methods. Since each of these diverse architectures possesses unique strengths and weaknesses, it’s difficult to determine the most suitable approach for this task. In this work, we address this challenge by identifying the key components of an effective and efficient point cloud segmentation architecture. Specifically, we show that: 1) Spatial locality and robust contextual understanding are critical for strong performance, and 2) Mamba features linear computational complexity, offering superior data and inference efficiency compared to Transformers, while still being capable of delivering strong contextual understanding. Additionally, we further enhance the standard Mamba specifically for point cloud segmentation by identifying its two key shortcomings. First, the enforced causality in the original Mamba is unsuitable for processing point clouds that have no such dependencies. Second, its unidirectional scanning strategy imposes a directional bias, hampering its ability to capture the full context of unordered point clouds in a single pass. To address these issues, we carefully remove the causal convolutions and introduce a novel Strided Bidirectional SSM to enhance the model’s capability to capture spatial relationships. Our efforts culminate in the development of a novel architecture named Meepo , which effectively integrates the strengths of CNN and Mamba. Meepo surpasses the previous state-of-the-art method, PTv3, by up to +0.8 mIoU on key benchmark datasets, including ScanNet, ScanNet200, S3DIS, and nuScenes, while being 42.1% faster and 5.53 × \times × more memory efficient. Our code and data will be released.

## 1 Introduction

Point cloud segmentation is an important topic in 3D understanding that has gained significant attention from the research community in recent years. Numerous neural network architectures have been proposed for this task, with CNN-based and Transformer-based designs being the two most prominent. Currently, Transformer-based methods consistently deliver the highest performance across numerous benchmarks. Their success is frequently credited to the attention mechanism, which can adequately capture and model context. However, the quadratic complexity of self-attention in Transformers poses a significant challenge for point cloud processing, especially when handling a large number of points. To address this, researchers have explored more efficient strategies, such as aggressive downsampling (Zhao et al., ; Wu et al., ; Pan et al., ), efficient attention algorithms (Yang et al., ), and windowed attention mechanisms (Wang, ; Lai et al., ; Wu et al., ). While these methods help reduce computational costs, they achieve this at the expense of valuable spatial and geometric information, potentially weakening the Transformer’s modeling capability and hindering its contextual understanding of the point cloud (Shen et al., ).

Recently, State Space Models (SSMs) like Mamba (Lieber et al., ; Gu & Dao, ; Pióro et al., ; Wang et al., ) have emerged as a promising alternative to Transformers. These models combine aspects of recurrent neural networks (Cho et al., ; Hochreiter & Schmidhuber, ) and convolutional neural networks (LeCun et al., ) within a framework rooted in classical state space theory. Similar to Transformers, SSMs provide robust contextual modeling capabilities. However, unlike Transformers, which scale quadratically with sequence length, SSMs scale linearly and maintain constant memory usage during inference. This efficiency makes SSMs particularly advantageous for tasks that require handling extensive contexts.

Figure: (a) mIoU v.s. Latency on V100
Refer to caption: x1.png

**Table 1: Performance and Latency Comparison among representative Mamba-based, CNN-based and Transformer-based networks on ScanNet.**
| case | mIoU$\uparrow$ | latency$\downarrow$ |
| --- | --- | --- |
| Point Mamba (Liu et al., ) | 75.7 | 280 |
| OA-CNN (Peng et al., ) | 76.1 | 141 |
| PTv3 (Wu et al., ) (current best) | 77.5 | 183 |

Although numerous early attempts have been made to utilize SSMs for point cloud segmentation (Liang et al., ; Zhang et al., ; Liu et al., ), these efforts have been largely discouraging, as their performance significantly lags behind that of leading CNN-based and Transformer-based models. As illustrated in Tab. 1, the recently proposed Point Mamba network (Liu et al., ) not only incurs roughly twice the latency of the CNN-based OA-CNN (Peng et al., ) and Transformer-based PTv3 (Wu et al., ) but also underperforms them by 0.4 points and 2.2 points in mIoU on the ScanNet (Dai et al., ) validation dataset, respectively. These shortcomings underscore the ongoing debate regarding the optimal architecture for point cloud segmentation, leading to the pivotal question: What constitutes an efficient and effective model architecture for point cloud segmentation?

In this work, we aim to provide valuable insights for the design of point cloud segmentation architectures. To achieve this, we conduct a preliminary analysis in Sec. 3 to examine the properties of the three most popular architectures, namely the CNN-based, Transformer-based and Mamba-based networks. Using the meta-architecture presented in Fig. 2, which seamlessly integrates different operators from the CNN, Transformer and Mamba architectures, we compare these architectures across three key dimensions: contextual understanding capability, local sensitivity, and network efficiency. Our analysis reveals that while CNNs excel at local modeling, they lack the ability to capture broader context. Transformers can adequately capture contextual information but are inefficient due to unnecessary long-range attention and quadratic computational complexity. Mambas strike a balance by efficiently providing essential contextual understanding with linear complexity. Given the distinct strengths and limitations of each architecture, we hypothesize that an integrated architecture combining their best features could potentially yield a more powerful and efficient model.

Building on the insights gained, we systematically explore various block choices, placements, and quantities within the previously proposed meta-architecture to determine the optimal arrangement. Through this process, we identify the CNN-Mamba block as the optimal elementary block for our architecture. As depicted in Fig. 6(a), the CNN-Mamba block comprises a sequential stack of sparse convolution layers and Mamba modules. Notably, the Attention module is ultimately excluded from the final architecture because the more efficient Mamba module can already provide sufficient contextual understanding for this task.

With the macro-level design established, we turn our attention to the micro-level design, specifically assessing whether the standard Mamba module, originally designed for sequential processing, is suitable for point cloud segmentation. Our investigation shows that it is not. In particular, we identify two major shortcomings of the standard Mamba module when applied to this task:

- 1.
Loss of spatial information due to enforced causality: Mamba’s use of causal convolutions introduces unnecessary artificial dependencies that can disrupt the inherent spatial relationships in point cloud data, ultimately reducing its effectiveness for point cloud data.
- 2.
Directional bias due to unidirectional scan: Mamba’s unidirectional scan strategy inherently favors certain data points over others, creating a directional bias. This bias undermines its ability to fully grasp the context of unordered point clouds in a single pass.

To address these issues, we propose two corresponding improvements to the standard Mamba module. Specifically, we 1) remove the causality constraint, and 2) incorporate a novel Strided Bidirectional SSM to enhance its context and spatial understanding.

Our work culminates in Meepo, a novel point cloud segmentation architecture which seamlessly integrates Mamba’s efficiency and strong contextual understanding with the local sensitivity of CNN. As shown in Fig. 1, Meepo not only achieves significantly lower inference latency but also surpasses the performance of other leading segmentation models across both indoor and outdoor scenes. Specifically, it outperforms the previous best method, PTv3, by up to +0.8 mIoU across ScanNet, ScanNet200, S3DIS, and nuScenes datasets, with 42.1% smaller latency and 5.53$\times$ smaller memory usage. Meepo also scales much better with respect to the size of point cloud. As shown in Fig. 1(d), it achieves up to $12\times$ fewer GFLOPs than PTv3 when processing a scene with $2^{17}$ ($131,072$) points.

In summary, the contributions of our paper are as follows:

- 1.
We carefully analyze existing point cloud segmentation networks, identifying the importance of spatial locality and robust contextual understanding in achieving high performance. We then reveal that Transformer’s global attention is unnecessary and inefficient for this task as Mambas offer comparable contextual modeling with linear complexity.
- 2.
We propose two corresponding solutions to address the limitations of Mamba when applied to point cloud segmentation, namely the removal of the causality constraint and the incorporation of an innovative Strided Bidirectional SSM to enhance contextual understanding.
- 3.
We introduce Meepo, a novel architecture that solely utilizes the efficient sparse convolution to provide spatial locality and the efficient SSM to provide robust contextual understanding. Meepo not only consistently outperforms previous best method, PTv3 across multiple key benchmark datasets, but is also much faster and much more memory efficient.

## 2 Preliminaries

In this section, we briefly introduce state space model (SSM) to facilitate subsequent discussion.

SSM’s formulation. SSM is a type of sequence model (Gu & Dao, ) that maps an input sequence $x(t)\in\mathbb{R}$ to an output sequence $y(t)\in\mathbb{R}$ through a latent state $h(t)\in\mathbb{R}^{N}$:

$$ $h^{\prime}(t)=\mathbf{A}h(t)+\mathbf{B}\mathbf{x}(t),\quad y(t)=\mathbf{C}h(t).$ (1) $$

Using zero-order hold (ZOH) rule, we can discretize the continuous parameters $(\mathbf{\Delta},\mathbf{A},\mathbf{B})$ as:

$$ $\bar{\mathbf{A}}=\exp(\Delta\mathbf{A}),\quad\bar{\mathbf{B}}=(\Delta\mathbf{A })^{-1}(\exp(\Delta\mathbf{A})-\mathbf{I})\cdot\Delta\mathbf{B},$ (2) $$

where $\Delta$ is the discretization step size and $\mathbf{I}$ is the identity matrix. Under this discretization rule, the hidden state $h_{t}$ can be computed efficiently as a linear recurrence:

$$ $h_{t}=\bar{\mathbf{A}}h_{t-1}+\bar{\mathbf{B}}x_{t},\quad y_{t}=\mathbf{C}h_{t}.$ (3) $$

Mamba is a recent popular SSM model that sets the SSM parameters to be functions of the input:

$$ $\mathbf{B}_{t}=f_{B}(x(t))\quad\mathbf{C}_{t}=f_{C}(x(t))\quad\mathbf{\Delta}_ {t}=f_{\Delta}(x(t))$ (4) $$

This allows the model to selectively propagate or discard information based on the input. In practice, matrix $\mathbf{A}$ is typically set as a diagonal matrix, ensuring that all elements of $\bar{\mathbf{A}}=\exp(\Delta\mathbf{A})$ lie between 0 and 1. Consequently, $\bar{\mathbf{A}}$ can be viewed as a forget gate, which controls how much information from the previous hidden state, $\textit{h}_{t}$, is retained.

## 3 Analysis

Figure: Figure 2: Proposed meta-architecture and various block options used for analysis. The model that exclusively uses choice A is called Pure CNN, the model that exclusively uses choice B is called Pure Mamba, and the model that exclusively uses choice C is called Pure Transformer.
Refer to caption: x5.png

In this section, we delve into the key requirements for effective point cloud segmentation, showing the importance of capturing both local and contextual features via model architecture. Robust local modeling is essential for maintaining point-level consistency, especially when object boundaries are unclear, while strong contextual modeling is crucial for identifying occluded or ambiguously shaped objects. With these needs in mind, we evaluate various architectures, assessing their strengths and weaknesses in providing these important properties for effective point cloud segmentation.

#### Meta-architecture for analysis.

To facilitate our analysis, we first propose a meta-architecture for point cloud segmentation. This meta-architecture follows an encoder-decoder framework, featuring a 5-stage encoder and a 4-stage decoder. Following previous works (Zhang et al., ; Liu et al., ), the input points are first voxelized into non-overlapping segments and arranged into an ordered sequence using a locality-preserving space-filling curve (Morton, ; Peano, ). These voxels are then processed by an embedding module that employs a single submanifold sparse convolution layer (Graham et al., ). GridPooling and GridUnpooling operations are applied at the end and beginning of each encoding stage, respectively, to downsample and upsample the point cloud (Wu et al., ). Within this meta-architecture, we train three similarly sized models, each using either the CNN, Mamba, or Transformer blocks, as shown in Fig. 2, to evaluate their performance both qualitatively and quantitatively. These models are referred to as Pure CNN, Pure Mamba, and Pure Transformer, respectively. We hypothesize that the global attention of Pure Transformer and the long-range sequential modeling capabilities of Pure Mamba are well-suited for contextual modeling. Pure Mamba, in particular, offers additional advantages such as greater inference efficiency due to its linear complexity and improved data efficiency through inductive bias provided by the forget gate. In contrast, Pure CNN excels at local modeling, leveraging its inherent spatial locality.

**Table 2: PTv3 without spatial locality perform much worse on ScanNet val.**
| Case | mIoU$\uparrow$ |
| --- | --- |
| PTv3 (Wu et al., ) | 77.5 |
| Remove spatial locality | 69.3 (-8.2) |
| Remove contextual modeling | 73.2 (-4.3) |

1. Are both local and contextual modeling important for point cloud segmentation? To assess the individual contributions of local and contextual modeling in point cloud segmentation, we analyze the performance impact of removing key components from the current leading network, PTv3 (Wu et al., ). Specifically, we investigate the effects of eliminating the sparse convolution layers, which capture spatial locality, and attention modules, which model contextual relationships. As detailed in Tab. 2, removing the sparse convolution layers results in a substantial performance drop of -8.2 mIoU, while removing the attention modules leads to a decrease of -4.3 mIoU. These results highlight the critical importance of both local and contextual modeling in achieving effective point cloud segmentation.

Figure: Image
Refer to caption: x6.png

2. Which model is more effective at providing contextual understanding? To investigate this, we perform another experiment using PTv3, by modifying its full attention mechanism to a window-based approach. By varying the window size, we control the amount of context the model processes.

Figure: Figure 4: Analysis of a representative Transformer-based model, PTv3, demonstrates that additional context beyond a certain amount is unnecessary.
Refer to caption: x12.png

As shown in Fig. 4, increasing the window size progressively improves performance from 24 to 1024, peaking at 1024 before gradually declining as the window size grows further. This indicates that while contextual understanding is beneficial, too much context can become detrimental, underscoring the importance of finding a balance between local and global modeling. Meanwhile, Mamba emerges as a strong candidate due to its emphasis on local processing while still capturing broader context when necessary. To provide evidence for this, we visualize the performance of different models using a representative example in Fig. 3, which depicts an office room with a challenging-to-discern door in the bottom left corner. Try to focus on the boxed region in each image and identify the object category. This task is challenging because the door’s point cloud is partially occluded and its texture and color closely match the surrounding wall and floor, requiring a comprehensive understanding of the scene. In this scenario, where contextual understanding is crucial, Mamba demonstrates superior performance by effectively balancing local and global context. It outperforms the Transformer model, which in turn surpasses the CNN, thereby confirming Mamba’s efficacy in contextual understanding.

3. Which model is more effective at providing spatial locality? To compare the performance of the aforementioned model architectures in this regard, we visualize their predictions in Fig. 4 using another insightful example of a room with a centrally placed table whose boundaries are poorly defined due to severe overexposure. Without zooming in, the boundaries are hard to discern, requiring strong local feature extraction capabilities. In this challenging scenario, the results support our hypothesis regarding spatial locality. The CNN, equipped with sparse convolutional layers specifically designed to capture local spatial patterns, outperforms both Mamba and the Transformer. Mamba exhibits slight spillage of floor pixels onto the table, while the Transformer mislabels a significant portion of the table area. These results demonstrate the CNN’s superior capability in local modeling.

Figure: Image
Refer to caption: x13.png

4. Which model architecture is more efficient? Let $C$, $C_{\text{in}}$, and $C_{\text{out}}$ represent the channel sizes, $N$ the SSM’s state dimension, $E$ the SSM’s expansion factor, $L$ the number of points or input sequence length, $K$ the depthwise convolution kernel size in Mamba and $k$ the convolution kernel size in sparse convolutions within CNN, the computational complexities of the core operations in the Transformer, Mamba, and CNN models are as follows:

$$ $\displaystyle\Omega(\text{Transformer})$ $\displaystyle=$ $\displaystyle\hskip 11.38109pt\underbrace{4\cdot L\cdot C^{2}}_{\text{qkv and output projections}}$ $\displaystyle+$ $\displaystyle\hskip 11.38109pt\underbrace{2\cdot L^{2}\cdot C}_{\text{ attention}}$ $\displaystyle=$ $\displaystyle\hskip 5.69054ptO(L^{2}),$ (5) $\displaystyle\hskip 8.53581pt\Omega(\text{Mamba})$ $\displaystyle=$ $\displaystyle\hskip 19.91692pt\underbrace{9\cdot L\cdot C\cdot N}_{\text{SSM}}$ $\displaystyle+$ $\displaystyle\hskip 5.69054pt\underbrace{L\cdot C\cdot K}_{\text{depthwise conv1d}}$ $\displaystyle+$ $\displaystyle\underbrace{3\cdot L\cdot C^{2}\cdot E}_{\text{input and output projections}}$ $\displaystyle=$ $\displaystyle\hskip 5.69054ptO(L),$ (6) $\displaystyle\hskip 14.22636pt\Omega(\text{CNN})$ $\displaystyle=$ $\displaystyle\hskip 8.53581pt\underbrace{2\cdot C_{\text{in}}\cdot C_{\text{ out}}\cdot k^{3}\cdot L}_{\text{convolution}}$ $\displaystyle+$ $\displaystyle\hskip 5.69054pt\underbrace{L\cdot C_{\text{in}}\cdot C_{\text{ out}}}_{\text{bias addition}}$ $\displaystyle=$ $\displaystyle\hskip 5.69054ptO(L).$ (7) $$

Both Mamba models and CNNs scale linearly with respect to $L$, while Transformers scale quadratically. Consequently, for point cloud processing tasks involving hundreds of thousands of points, Transformers can be significantly slower. When comparing CNNs and Mamba models of similar sizes, CNNs are generally faster in practice. This is because CNNs typically have fewer layers when the total number of parameters is the same, as each 3D sparse convolution layer in a CNN contains significantly more parameters than a corresponding block in a Mamba model.

Key insights: Point cloud segmentation requires both effective local modeling and a comprehensive understanding of contextual information. While CNNs are effective for local modeling, contextual modeling demands a different approach. Although Transformers can handle contextual information, they are inefficient because very long-range attention is unnecessary, e.g., PTv3 performance peaks at a window size of 1024. Mambas, on the other hand, strike a balance between local and global modeling, providing the necessary contextual understanding with linear complexity instead of the quadratic complexity associated with Transformers. Given the distinct strengths and limitations of each architecture, it’s essential to explore models that can holistically integrate their strengths.

## 4 Method

Building on insights from previous analysis, we introduce Meepo, a novel model that is both effective and efficient for point cloud segmentation. Meepo adopts the same meta-architecture presented in Fig. 2 but incorporates the CNN-Mamba block as a core component to facilitate the seamless integration of local and contextual modeling. Additionally, we introduce two micro-level modifications to the standard Mamba module to address its limitations in point cloud segmentation.

### 4.1 Macro Architecture Design

Optimizing block choices for point cloud segmentation. To identify the most effective block configuration for integrating local and contextual modeling, we draw inspiration from the widely adopted sequential combination of local convolutional layers and contextual operators in previous point cloud segmentation studies (Wang, ; Wu et al., ) and evaluate two possible candidates: the CNN-Mamba block and the CNN-Transformer block. As depicted in Fig. 6, the CNN-Mamba block comprises a sparse convolution layer, followed by a Mamba module and an MLP layer. In contrast, the CNN-Transformer block follows the same structure but substitutes the Mamba module with an Attention module. Our comprehensive ablation study, presented in Tab. 7(b), demonstrates that replacing CNN-Transformer blocks with CNN-Mamba blocks throughout the network (22 blocks in total) achieves the best performance and efficiency. These findings validate our hypothesis that the Mamba module offers superior contextual understanding while maintaining efficiency.

Figure: Figure 6: Our proposed architecture, Meepo, integrates CNN-Mamba blocks throughout the proposed meta-architecture to harness their strengths in local and contextual modeling. To optimize for point cloud segmentation, Meepo modifies the standard Mamba by replacing causal convolutions with regular convolutions, preserving critical spatial information. Additionally, it introduces a novel Bidirectional Strided SSM, which enhances contextual modeling by minimizing directional bias.
Refer to caption: x19.png

### 4.2 Micro Architecture Design

Optimizing Mamba for point cloud segmentation. Despite its impressive speed and performance, Meepo without micro-level optimizations still does not surpass the leading point cloud segmentation network, PTv3, in accuracy. This aligns with previous research, which consistently demonstrates that attempts to apply Mamba have all failed to outperform well-established models for this task (Zhang et al., ; Liu et al., ). A closer analysis reveals that the standard Mamba, originally designed for sequential data, is inherently unsuited for processing unordered point clouds. We identify two key limitations in the standard Mamba and propose the solutions to overcome them.

1. Loss of spatial information due to enforced causality: Mamba was originally designed for sequential data with clear causal relationships. However, point clouds lack such dependencies, as their spatial relationships are multidimensional, requiring simultaneous consideration of points holistically. The causal convolutions in standard Mamba impose a causal dependency that disrupts these essential spatial relationships, making it ineffective for handling spatial data like point clouds.

Proposed Solution: Causal-Free Mamba. To address this issue, we propose replacing the causal convolution with a standard convolution, resulting in the Causal-Free Mamba module, as illustrated in Fig.6(b). This modification eliminates the limitations of the original Mamba architecture when processing spatial data, greatly enhancing its performance in point cloud segmentation tasks.

2. Directional bias due to unidirectional scan: Mamba’s unidirectional scanning method introduces a directional bias in the representation learning of point clouds because some parts are scanned first while others are scanned later. Such sequential processing is ill-suited for orderless point cloud data, where all information should be treated equally. This approach can lead to models prioritizing information from later stages of the scan, overlooking details captured earlier. The bias is further amplified by factors such as noise, occlusions, or reduced data density at the beginning of the scan, which can degrade the quality of the early data. As a result, important features captured early in the scan may be missed or inaccurately interpreted, leading to segmentation errors where boundaries are poorly defined, and features are incorrectly merged or omitted.

Proposed Solution: Bidirectional Strided SSM. To address this issue, we propose an innovative multi-directional scanning approach to reduce the directional bias of Mamba. Unlike the standard Mamba, which employs a unidirectional scan, this scanning method processes data in four distinct scanning directions: forward, backward, $n$-strided forward, and $n$-strided backward. In an $n$-strided forward scan, every $n$-th token is skipped, and the scan pattern restarts from the beginning once the end is reached. For example, given the sequence 1, 2, 3, 4, 5, 6, a 2-strided forward scan would process it as 1, 3, 5, 2, 4, 6. The backward scan operates similarly but in reverse order. This multi-directional scanning approach can effectively expand Mamba’s receptive field, reduce information loss, and improve overall performance by shortening the information flow path.

## 5 Experiments

**Table 3: Indoor semantic segmentation comparison on ScanNet, ScanNet200, S3DIS Area 5.**
| Method | ScanNet | ScanNet200 | S3DIS |  |  |
| --- | --- | --- | --- | --- | --- |
| Val | Test | Val | Test | Area5 |  |
| PCM (Zhang et al., ) | - | - | - | - | 63.4 |
| PointNeXt (Qian et al., ) | 71.5 | 71.2 | - | - | 70.5 |
| MinkUNet (Choy et al., ) | 72.2 | 73.6 | 25.0 | 25.3 | 65.4 |
| ST (Lai et al., ) | 74.3 | 73.7 | - | - | 72.0 |
| PTv2 (Wu et al., ) | 75.4 | 74.2 | 30.2 | - | 71.6 |
| OctFormer (Wang, ) | 75.7 | 76.6 | 32.6 | 32.6 | - |
| Point Mamba (Liu et al., ) | 75.7 | - | - | - | - |
| OA-CNNs (Peng et al., ) | 76.1 | 75.6 | 32.3 | 33.3 | 71.1 |
| Swin3D (Yang et al., ) | 76.4 | - | - | - | 72.5 |
| PTv3 (Wu et al., ) | 77.5 | 77.9 | 35.2 | 37.8 | 73.4 |
| Meepo(Ours) | 78.0 | 78.4 | 36.0 | 38.5 | 73.5 |

In this section, we begin by briefly describing our implementation details (Sec. 5.1). Then, we compare Meepo with state-of-the-art (SOTA) methods (Sec. 5.2) and ablate our proposed method (Sec. 5.3). Due to space limitation, detailed implementation details, more quantitative and qualitative results and additional ablations are presented in the Appendix A.

### 5.1 Implementation Details

#### Datasets.

We evaluate our proposed method on several indoor and outdoor semantic segmentation datasets using mean Intersection over Union (mIoU) metric. For indoor scenes, we use ScanNet (Dai et al., ), its extended version ScanNet200 (Rozenberszki et al., ), and S3DIS (Armeni et al., ). For outdoor scenes, we employ nuScenes (Caesar et al., ).

#### Training and Inference Details.

We follow all experimental settings of PTv3 (Wu et al., ) without any changes. For indoor segmentation, the number of epochs is 800, the learning rate is 0.006, and the weight decay is 0.05. For outdoor segmentation, the number of epochs is 50, the learning rate is 0.002, and the weight decay is 0.005. We train all our models using batch size of 12 and the AdamW optimizer. For efficiency evaluations, we use a single V100 with a batch size of 1.

### 5.2 Main Results

#### Indoor Semantic Segmentation.

Tab. 6 compares Meepo with leading methods on the indoor ScanNet, ScanNet200, and S3DIS Area 5 cross-val datasets. Meepo achieves new SOTA results with mIoU scores of 78.0, 36.0, and 73.5 on ScanNet val, ScanNet200 val, and S3DIS Area 5 cross-val, respectively, outperforming the second-best method, PTv3, by +0.5, +0.8, and +0.1 points.

#### Outdoor Semantic Segmentation.

Tab. 6 compares Meepo with leading methods on the outdoor nuScenes val dataset. Meepo achieves new SOTA result with mIoU score of 80.8 on nuScenes val, surpassing the second-best method, PTv3, by 0.4 points.

#### Model Efficiency.

In Fig. 1, we compare the average latency and memory usage of our model with multiple leading methods on the ScanNet val dataset. As shown in Fig. 1(a) and Fig. 1(b), Meepo exhibits better latency and memory consumption than many previous leading networks and achieves much higher performance. Remarkably, Meepo not only outperforms the previous top-performing method, PTv3, but is also $42.1\%$ faster and $5.53\times$ more memory efficient. Meepo particularly excels at ultra-long-range modeling, owing to its use of Mamba, which eliminates the quadratic complexity of attention typical in Transformer models. Meepo also scales well to scenes containing a large number of points. As illustrated in Fig.1(d), Meepo also achieves $12\times$ reduction in FLOPs when handling scenes containing $131,072$ points. Note that Tab. 6 gives the exact numbers corresponding to points in Fig. 1(a) and Fig. 1(b).

#### Data Efficiency.

In Tab. 6, we compare Meepo with leading methods on the ScanNet data efficiency benchmark, which evaluates models with limited reconstructions and annotations. The “Limited Reconstruction” setting uses a fraction of the available 3D reconstructions, while the “Limited Annotation” setting restricts the number of annotated points per scene. As shown, Meepo outperforms the second-best method, PTv3, by 0.6, 2.0, 1.3, and 1.1 points at 1%, 5%, 10%, and 20% reconstructions, and by 1.8, 0.9, 0.9, and 1.7 points at 20, 50, 100, and 200 annotations, respectively.

Table: Table 7: Ablation experiments on on ScanNet for evaluating different design choices used for Meepo. The entries marked in gray are the same, which specify the default settings.

### 5.3 Ablation Experiments

In this subsection, we conduct ablations for all components of our architecture using the ScanNet (Dai et al., ) val dataset. All experimental settings follow the settings used in the main results.

#### Effectiveness of Our Proposed Architecture.

Aside from the qualitative comparisons presented in Fig. 2 and Fig. 3, we also quantitatively compare our proposed method with other model architectures. As shown in Tab. 7(a), Meepo requires fewer parameters and delivers superior performance.

#### Effectiveness of CNN-Mamba Block.

In Tab. 7(b), we progressively replace some of the CNN-Mamba blocks in Meepo with CNN-Transformer blocks. The results show that increasing the number of CNN-Transformer blocks always leads to higher latency, more memory usage and reduced performance. This confirms the effectiveness of our proposed CNN-Mamba block, which can efficiently and effectively integrate local and contextual modeling for point cloud segmentation.

#### Effectiveness of Causal-Free Mamba.

The original Mamba uses causal depthwise convolution to preprocess input tokens before passing them to the SSM module. While this makes sense for sequence modeling, its necessity for 3D vision tasks, which lack causality, is unclear. To investigate this, we experiment with replacing the causal convolution with normal convolution. As shown in Tab. 7(c), causal convolution results in a performance improvement of +0.5 mIoU.

#### Effectiveness of Bidirectional Strided SSM.

In Tab. 7(d), we demonstrate the effectiveness of our proposed Bidirectional Strided SSM. As shown, it outperforms both the standard SSM and its bidirectional variant, resulting in a performance improvement of +0.8 mIoU. Additionally, we ablate the optimal stride for this module in Tab. 7(e), which shows that a stride of 2 yields the best performance.

#### Compatibility of Our Proposed Modules.

In addition to ablating our proposed modules individually, we perform an additive ablation study in Tab. 7(f) to demonstrate the compatibility of the enhancements. Incorporating CNN-Mamba blocks results in a significant improvement of +3.7 mIoU. Further upgrades to the standard Mamba module, namely introducing causal-free convolutions and employing multi-directional scanning, provide additional gains of +0.5 and +0.6 mIoU, respectively.

## 6 Related Work

Architectures for point cloud segmentation fall into three main categories: point-based (Thomas et al., ; Qi et al., ; ; Ma et al., ), voxel-based (Maturana & Scherer, ; Song et al., ), and projection-based (Chen et al., ; Lang et al., ; Li et al., ; Su et al., ). Although they differ in pre-processing strategies, all these methods are designed with careful consideration of the unique characteristics of point clouds. They mainly differ in how they integrate local and contextual features and manage irregular point distributions. Recently, transformer-based models (Wu et al., ; Robert et al., ; Yang et al., ; Lai et al., ) have emerged in this field, achieving higher accuracy but suffering from significant time and memory complexities relative to the size of the point cloud. To address this, more efficient attention mechanisms, such as vector attention (Zhao et al., ), grouped vector attention (Wu et al., ), local window-based attention (Lai et al., ), and memory-efficient attention (Yang et al., ), have been developed. However, these strategies approximate original attention, resulting in a loss of global modeling capability (Shen et al., ), which may impede performance on long range modeling task like point cloud segmentation. Our research examines the strengths and weaknesses of several widely-used architectures for point cloud segmentation and introduces a novel architecture that effectively integrates their best features, culminating in a new state-of-the-art model for point cloud segmentation.

State space models (SSMs) (Gu & Dao, ; Wang et al., ; Lieber et al., ) have emerged as a promising alternative to Transformers (Vaswani et al., ) in natural language processing (NLP) for capturing long-range dependencies. Unlike Transformers that scale quadratically with sequence length, SSMs (Gu et al., ; Nguyen et al., ; Smith et al., ) achieve linear scaling during inference. The seminal Mamba model (Gu & Dao, ) greatly improves the performance and efficiency of SSM by introducing input-specific parameterization and a scalable, hardware-optimized method, allowing it to outperform Transformers for the first time. Driven by the success of SSMs in NLP, recent works have also explored their application to visual tasks. S4ND (Nguyen et al., ) marks the introduction of SSM modules for processing visual data across 1D, 2D, and 3D domains. Following works, such as VMamba (Liu et al., ) and Vim (Zhu et al., ), address the directional sensitivity in SSMs with bi-directional and cross-scan mechanisms, showcasing the potential of SSMs to rival traditional CNN and Transformer models. The impressive performance of Mamba-based models has spurred further research into their applications for various vision tasks, such as image segmentation (Xing et al., ; Liu et al., ), image synthesis (Yan et al., ), graph modeling (Wang et al., ), and low-level vision tasks (Guo et al., ), underscoring their versatility and effectiveness. Despite some initial attempts to apply SSM architectures for point cloud segmentation, these models have failed to outperform existing architectures. In this work, we identify the shortcomings of SSM when applied to point cloud segmentation and propose simple solutions to significantly improve its performance in this domain.

## 7 Conclusion

In this work, we present a detailed analysis of existing network architectures for point cloud segmentation, highlighting their strengths and weaknesses. This evaluation provides valuable insights into designing more efficient and effective architectures for this task. Our findings emphasize the crucial role of spatial locality and robust contextual understanding in achieving strong performance. Specifically, we identify convolution and the Mamba module as essential components for efficient and accurate point cloud segmentation. Convolution provides spatial locality, while the Mamba module enhances the understanding of context. Additionally, we improve the standard Mamba module by removing the causality constraint and introducing Bidirectional Strided SSM, which further enhances its ability to capture and utilize contextual information. Following these design principles and applying targeted optimizations, we introduce Meepo, a novel architecture that outperforms previous state-of-the-art models across multiple key benchmark datasets and efficiency metrics.

Acknowledgements. This work is supported in part by the National Key R&D Program of China under Grant 2021ZD0140407, the National Natural Science Foundation of China under Grants 62321005 and 62276150, and the THU-Bosch JCML.

## Appendix A Appendix

For a thorough understanding of our proposed Meepo, we have compiled a detailed Appendix. The table of contents below offers a quick overview and will guide to
specific sections of interest.

### A.1 Implementation Details.

We implement our method using the Pointcept (Contributors, ) codebase. Detailed specifications of our implementation are provided in this section.

**Table 8: Indoor sem. seg. settings.**
| Config | Value |
| --- | --- |
| optimizer | AdamW |
| scheduler | Cosine |
| criteria | CrossEntropy (1) |
|  | Lovasz (1) |
| learning rate | 6e-3 |
| block lr scaler | 0.1 |
| weight decay | 5e-2 |
| batch size | 12 |
| datasets | ScanNet / S3DIS |
| warmup epochs | 40 |
| epochs | 800 |

#### Training Settings.

The experimental settings for indoor and outdoor semantic segmentation are outlined in Tab. 9 and Tab. 9. The numbers in brackets indicate the relative weight assigned to each criterion in the loss. The main differences between indoor and outdoor settings are in the learning rate, weight decay, warmup epochs and training epochs used.

**Table 10: Model settings.**
| Config | Value |
| --- | --- |
| positional encoding | None |
| embedding depth | 2 |
| embedding channels | 32 |
| no. of layers in Local Perceiver | 1 |
| no. of layers in Channel Modulator | 2 |
| encoder depth | [2, 2, 6, 2] |
| encoder channels | [64, 128, 256, 512] |
| encoder num heads | [4, 8, 16, 32] |
| decoder depth | [1, 1, 1, 1] |
| decoder channels | [64, 64, 128, 256] |
| decoder num heads | [4, 4, 8, 16] |
| down stride | [$\times$2, $\times$2, $\times$2, $\times$2] |
| drop path | 0.3 |

#### Model Settings

Detailed model configurations of our Meepo are listed in Tab. 10.

**Table 11: Data augmentations.**
| Augmentations | Parameters | Indoor | Outdoor |
| --- | --- | --- | --- |
| random dropout | dropout ratio: 0.2, p: 0.2 | ✓ | - |
| random rotate | axis: z, angle: [-1, 1], p: 0.5 | ✓ | ✓ |
|  | axis: x, angle: [-1 / 64, 1 / 64], p: 0.5 | ✓ | - |
|  | axis: y, angle: [-1 / 64, 1 / 64], p: 0.5 | ✓ | - |
| random scale | scale: [0.9, 1.1] | ✓ | ✓ |
| random flip | p: 0.5 | ✓ | ✓ |
| random jitter | sigma: 0.005, clip: 0.02 | ✓ | ✓ |
| elastic distort | params: [[0.2, 0.4], [0.8, 1.6]] | ✓ | - |
| auto contrast | p: 0.2 | ✓ | - |
| color jitter | std: 0.05; p: 0.95 | ✓ | - |
| grid sampling | grid size: 0.02 (indoor), 0.05 (outdoor) | ✓ | ✓ |
| sphere crop | max points: 102400 | ✓ | - |
| normalize color | p: 1 | ✓ | - |

#### Data Augmentation.

Data augmentations used for training Meepo are detailed in Tab. 11.

### A.2 Additional Quantitative Results

**Table 12: Results on ScanNet200 for semantic segmentation.**
| Method | Val |  |  |  |
| --- | --- | --- | --- | --- |
| Head | Comm. | Tail | All |  |
| MinkowskiNet (Choy et al., ) | 48.3 | 19.1 | 7.9 | 25.1 |
| SparseUNet (Wu et al., ) | - | - | - | 28.8 |
| LGround (Rozenberszki et al., ) | 51.5 | 22.7 | 12.5 | 28.9 |
| PTv2 (Wu et al., ) | - | - | - | 29.3 |
| OA-CNNs (Peng et al., ) | 51.3 | 28.0 | 17.7 | 32.3 |
| PTv3 (Wu et al., ) | 56.5 | 30.1 | 19.3 | 35.2 |
| Meepo | 56.6 | 30.7 | 20.7 | 36.0 |

#### Class Imbalance Analysis on ScanNet200 (Rozenberszki et al., 2022 ) .

In Tab. 12, we compare Meepo with other leading methods on the head, common, and tail subsets of the ScanNet200 validation benchmark. This comparison offers a more nuanced understanding of performance across different levels of class imbalance. As shown, Meepo achieves improvements of +0.2, +0.6, and +1.4 mIoU on these three subsets, respectively. Notably, it performs much better on the most challenging tail subset, indicating its potential in handling long-tail distributions.

**Table 13: Additional Evidences of Mamba’s Importance.**
| Case | Params (M)$\downarrow$ | mIoU$\uparrow$ |
| --- | --- | --- |
| Meepo | 45.6 | 78.0 |
| Replace Mamba with Attention | 42.7 | 77.4 |
| Remove Mamba | 38.2 | 73.5 |
| Use one additional sparse conv. layer | 69.0 | 75.7 |
| Use one additional block in each stage | 52.7 | 74.7 |
| Increase input channel size from 32 to 36 | 48.3 | 74.2 |

#### Additional Evidences of Mamba’s Importance.

To emphasize Mamba’s importance, we also test alternative methods to scale up the models without it. The results in Tab. 13 show that none of the Meepo variants without Mamba outperform the original configuration, clearly demonstrating its critical role in providing contextual modeling for point cloud segmentation.

### A.3 Additional Qualitative Results

Figure: Figure 7: Comparison between Meepo’s and PTv3’s (Wu et al., ) predictions. Black color are unlabelled points. Red boxes with dash-dotted lines are wrong predictions by PTv3.
Refer to caption: x20.png

### A.4 Limitations

While our work has significantly advanced the performance of point cloud segmentation models, many challenges and opportunities for improvement remain. For instance, although Meepo exhibits notable efficiency gains, there is still room for further optimization to enhance its computational and memory efficiency. Additionally, segmentation quality, especially in handling fine details and complex geometries within point clouds, can be further improved. Enhancing the model’s ability to accurately segment objects in diverse and densely populated scenes is another critical area for future research. Moreover, the potential of pretraining through self-supervised learning is unexplored in this work. Leveraging large-scale unlabeled point cloud data for pretraining could help the model learn more robust and generalizable features, ultimately boosting performance across various tasks and datasets. Future work should explore and integrate self-supervised learning techniques to harness this potential fully. Addressing these challenges will ensure that Mamba-based architectures continue to evolve and set new benchmarks in the field of point cloud segmentation.

### A.5 Broader Impacts

#### Accessibility and Resource Efficiency.

By demonstrating that our proposed method, Meepo can achieve superior performance with reduced latency and memory consumption, this research promotes the development of more accessible and resource-efficient machine learning models. This is particularly important for applications in resource-constrained environments, such as mobile and embedded systems, where computational power and memory are limited. As a result, more organizations and developers can leverage advanced point cloud segmentation techniques without requiring extensive computational resources.

#### Environmental Impact.

The reduction in computational cost and memory usage directly translates to lower energy consumption. Given the growing concern over the environmental impact of large-scale machine learning models, Meepo’s efficiency can contribute to more sustainable AI practices. By minimizing the energy required for training and inference, this work aligns with global efforts to reduce the carbon footprint of technology.

### A.6 Compute Resources

We run all experiments on a cluster with a mix of RTX3090, RTX4090 or 32GB V100 and 80GB A100 GPUs.

### A.7 Reproducibility

Our main results can be fully reproduced by running the training and evaluation scripts given in the attached code.

## Peer Reviews

confidence: 5
contribution: 3
initial_rating: 8
presentation: 4
questions: 1. In Figure 1(b), PTv3 consumes even more memory compared to PTv2, which seems unusual since window-based attention should be significantly more memory-efficient than neighborhood attention. The only reason I can imagine is that FlashAttention may be disabled while using a large kernel, resulting in larger matrix multiplications. Could you provide a more detailed explanation of this setup?

2. Maybe a shorter, more impactful title could make it easier for readers to remember? The current version is too lengthy.

3. Maybe the table arrangement could be improved? I don’t recommend using resizebox, as it can lead to uneven text sizes. You might consider referring to the LaTeX source code from the PTv3 paper for tips on adjusting table formatting.

4. It might be better to reduce the use of bold text? For instance, consider changing `\textbf{Proposed Solution:}` to` \textit{Proposed solution:}`, and avoid bolding certain numbers in the main text.

(Minor suggestions for reference only)
soundness: 3
strengths: 1. **[Analysis-driven Methodology]** First of all, I appreciate the paper's analysis-driven approach: rather than being experiment-driven, it presents insights based on previous work and derives its methodology from this understanding. This makes the paper easier to follow and the proposed method more convincing.

2. **[Strong performance]** The paper achieves strong performance on several major point cloud semantic segmentation benchmarks and appears to be efficient as well.
summary: This paper focuses on a simple yet practical target in 3D understanding: how to enhance the accuracy of a Mamba-based framework while preserving its efficiency rooted in linear complexity. The paper seeks to achieve this through an in-depth analysis of the components that contribute to the strong performance of PTv3, summarized as contextual understanding and spatial locality. These findings lead to the two major design elements of this work: Causal-free Mamba and Bidirectional Strided SSM. Overall, it is encouraging to see the method achieve solid performance on major scene-level point cloud semantic segmentation benchmarks.
weaknesses: The insightful analysis, convincing approach, and solid performance demonstrate that this manuscript makes **a valuable empirical contribution**. However, the following weaknesses **limit its broader impact**:

1. **[Scaling up]**  
**(a)** Many observations and analyses (such as the ablation on PTv3 window size) are based on training from scratch on ScanNet (1,500 samples), which is relatively small in scale. However, many model properties may change significantly when scaled up with more data. For instance, using the PTv3 window size ablation as an example, attention mechanisms are inherently adaptive to kernel size, meaning accuracy should not be negatively impacted by the window size. The degradation observed when increasing the window size beyond 1024 is likely due to insufficient data to support this adaptive capacity.  
**(b)** It is good to see the Mamba framework achieve both higher accuracy and efficiency compared to previous SOTA. However, the true value of superior scratch accuracy and efficiency lies in its capacity for further scaling. I am particularly curious about the model's accuracy and efficiency when scaling up training through multi-dataset joint training, as well as its performance when scaling up parameters with larger data volumes.

2. **[Go beyond semantic segmentation]**
Why do point cloud perception and representation learning prioritize semantic segmentation? This is because it is the simplest way to evaluate the quality of learned representations using a single linear layer. However, this does not mean that research on point cloud backbones should be limited to semantic segmentation alone. The claims of the paper would be stronger if more downstream tasks, such as instance segmentation and object detection, were included. It may not be necessary to achieve the highest performance; instead, demonstrating more properties of the proposed method could be more informative.

---

confidence: 4
contribution: 3
initial_rating: 5
presentation: 3
questions: [Q1] Could the authors provide more detailed explanations of W1, such as whether there is theoretical evidence to support it or if there are sufficiently extensive experiments across multiple datasets?
[Q2] Is the Bidirectional SSM mentioned by the authors consistent with VisionMamba (Zhu 2024)? I do not see a bidirectional structure in Fig(6b), only the Strided SSM.
[Q2.5] If the answer of Q2 is yes, the result you reported in the second line of Tab(7d) shows that this structure can only bring 0.1% increasement, which seems inconsistent with the effects reported in VisionMamba (Zhu 2024) regarding this structure. From an efficiency perspective, is the introduction of the indirection necessary if it only brings a 0.1% improvement?
[Q3] The casual-free conv block is not clearly explained. If this is something new you are proposing, it would be helpful to include a more detailed explanation or illustration.
[Q4] Standardize the notation for SSM. In the abstract and introduction, it is referred to as "Strided Bidirectional SSM," while later it is called "Bidirectional Strided SSM."
[Q5] There’s something wrong of your hyperlink in 3.3. It should be Fig 5 but yours is Fig 4.
soundness: 3
strengths: [S1] It is meaningful to compare the performance of the three types of blocks—CNN, Transformer, and Mamba—on the same task, point cloud segmentation.
[S2] The paper is detailed and easy to read.
[S3] The paper conducts detailed ablation experiments on the newly proposed module.
[S4] The paper achieves state-of-the-art results on multiple datasets.
summary: This paper focuses on the 3D point cloud segmentation task. It first compares the performance of three types of blocks: CNN, Transformer, and Mamba, and then improves the existing Mamba architecture. The authors provide extensive visualization results and ablation experiments.
weaknesses: [W1] The comparison of CNN, Transformer, and Mamba blocks lacks strong evidence. I believe that concluding "CNN is more effective for local modeling and Transformer is better at handling contextual information" based on just one example for each is insufficient.
[W2] The proposed method is more like a small fix to Mamba-based method. 
[W3] The two solutions proposed in this paper: 1) the casual-free block is described in very little detail and lacks a clear explanation; 2) bidirectional SSM seems to have already been introduced in VisionMamba (Zhu 2024), and this paper's work only adds n-stride.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: please refer to the weakness
soundness: 3
strengths: 1.	MEEPO introduces a new method by integrating CNN and Mamba components, achieving good contextual modeling with reduced computational costs.
2.	The paper thoroughly explores the limitations of existing architectures (CNN, Transformer, and SSM) and provides comprehensive ablations and comparisons to validate its design choices.
3.	The introduction of causal-free convolutions and bidirectional strided SSM in Mamba addresses some original limitations in Mamba for this task,.
summary: This paper presents a novel architecture named MEEPO, designed for point cloud segmentation with a focus on efficient contextual modeling. It introduces Mamba, a state-space model (SSM) that achieves linear complexity compared to traditional Transformers, which have quadratic complexity. The authors identify Mamba's limitations, such as enforced causality and directional bias, and propose solutions through causal-free convolutions and bidirectional strided state-space modeling. MEEPO outperforms previous models, like PTv3, in accuracy, latency, and memory efficiency across several benchmark datasets (e.g., ScanNet, S3DIS, nuScenes).
weaknesses: 1.	In Section 4.2, the authors claim that they propose a 'Bidirectional Strided SSM' method, however, Bidirectional SSMs have already been proposed in works like [1][2] to address the issue of forgetting unidirectional sequences. I believe that not referencing these articles while claiming authorship is not rigorous. Additionally, what baffles me is that in Table 7, a comparison is made for strides ranging from 2 to 16, showing a decreasing trend in performance. So, why not similarly compare and discuss the results for a stride of 1?
2.	The manuscript presents in Fig. 4 the mIoU performance of the Transformer-based PTv3 under different range windows. The advantage of the Mamba model lies in its better memory capabilities for long sequences. The authors emphasize this advantage in line 240, but they do not provide specific data to demonstrate the validation of this conclusion in point cloud data  (the author can show a similar format figure as Fig.4).
3.	Table 7 in the article indicates that the parameter volume of CNN+Mamba is smaller than that of pure Mamba. However, based on the author's description, compared to pure Mamba, the author has added a CNN module to Mamba and employed a bidirectional mechanism for computation. Why the parameter count is lower than pure Mamba needs further clarification from the author.
4.	In line 251, the referred figure is not Fig.4, it should be Fig. 5.


[1] Bi-Mamba+: Bidirectional Mamba for Time Series Forecasting
[2] Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: I am unsure whether the authors of this paper are aware of *Point Cobra*. I would like the authors to address the numerous similarities between the two papers and provide an explanation.

1. While the new model structure aligns with the previous version (NIPS2024), there are differences in latency and memory usage. I am interested in understanding the source of these improvements.

2. This paper introduces an additional CNN module to capture the local features of the point cloud. Could the Causal-Free Conv1D in the Mamba block be removed, given that these two modules appear to serve the same role in the model?
soundness: 3
strengths: 1. The model is highly efficient and achieves accuracy that exceeds prior state-of-the-art methods.
2. The design of the Strided Bidirectional SSM effectively enhances the model's understanding of spatial relationships.
summary: This paper analyzes several popular neural network architectures, including CNN-based and Transformer-based designs, as well as the recently introduced Mamba model. It proposes a new point cloud segmentation architecture, MEEPO, which combines the strengths of CNNs and Mamba, surpassing previous state-of-the-art methods like PTv3 on multiple key benchmark datasets.
weaknesses: 1. The proposed hyper-structure is based on the analysis of existing architectures, which makes the technical innovation somewhat limited.

---

confidence: 4
contribution: 2
initial_rating: 6
presentation: 3
questions: see above weakness
soundness: 2
strengths: 1. The paper is well-structured with a clear analysis-driven approach.
2. The experimental results are impressive.
summary: Point cloud segmentation is crucial for 3D understanding, traditionally tackled using CNNs or Transformers. Recently, Mamba has emerged as an efficient solution for contextual modeling, though it has struggled to outperform leading CNN and Transformer methods. This work identifies key requirements for effective segmentation: strong spatial locality and robust contextual understanding. Enhancing Mamba, the authors remove causality and introduce a Strided Bidirectional SSM to address directional biases in unordered point clouds. The resulting architecture, MEEPO, merges the strengths of CNNs and Mamba, achieving up to +0.8 mIoU over state-of-the-art methods on benchmarks like ScanNet and nuScenes, with notable gains in speed and memory efficiency.
weaknesses: 1. In lines 238-241, it's unclear why Mamba is considered effective for local processing, given its design goal of modeling long sequences with linear complexity. The reference to a "locally-biased forget gate" needs further explanation and detailed analysis or statistical data beyond visualization.

2. In lines 300-302, the statement that long-range attention is unnecessary may only partially reflect the issue. The core reason might be the sparsity of 3D point clouds. Transformers generally require extensive data and prolonged training to surpass CNNs, so it might be more accurate to say that insufficient data limits the full potential of Transformers rather than implying long-range attention is irrelevant.

3. The innovations appear somewhat limited, with minimal structural changes. The modifications, such as Causal-Free Mamba and Bidirectional Strided SSM, feel more like tricks.

4. Presenting results on downstream perception tasks like segmentation and detection would add further value.

Minor Errors:
- Line 251: Fig.4 should be Fig.5.
- clarify the point cloud processing order in Mamba. Is it similar to PTv3's order?

Meta-review: The paper receives 4 positive and 1 negative rating after rebuttal. Although the paper has some merits like competitive results with faster runtime and lower memory cost, the reviewers pointed out a few critical concerns about 1) technical contributions compared to other Mamba-based approaches, and 2) results other than semantic segmentation. After taking a close look at the paper, rebuttal, and discussions, the AC agrees with reviewers' feedback, especially on the minor architectural change of the existing Mamba methods but applying to the studied task. Without further explanations about the main contributions or additional results, e.g., other point cloud tasks, it is not convincing to show enough technical novelty, and hence the AC suggests the rejection decision. The authors are encouraged to improve the paper based on the feedback for the next venue.

Decision: Reject