MambaMOS: LiDAR-based 3D Moving Object Segmentation with Motion-aware State Space Model

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-27 06:12:06

Abstract: LiDAR-based Moving Object Segmentation (MOS) aims to locate and segment moving objects in point clouds of the current scan using motion information from previous scans. Despite the promising results achieved by previous MOS methods, several key issues, such as the weak coupling of temporal and spatial information, still need further study. In this paper, we propose a novel LiDAR-based 3D Moving Object Segmentation with Motion-aware State Space Model, termed MambaMOS. Firstly, we develop a novel embedding module, the Time Clue Bootstrapping Embedding (TCBE), to enhance the coupling of temporal and spatial information in point clouds and alleviate the issue of overlooked temporal clues. Secondly, we introduce the Motion-aware State Space Model (MSSM) to endow the model with the capacity to understand the temporal correlations of the same object across different time steps. Specifically, MSSM emphasizes the motion states of the same object at different time steps through two distinct temporal modeling and correlation steps. We utilize an improved state space model to represent these motion differences, significantly modeling the motion states. Finally, extensive experiments on the SemanticKITTI-MOS and KITTI-Road benchmarks demonstrate that the proposed MambaMOS achieves state-of-the-art performance. The source code of this work will be made publicly available.

Main paper: 
 related work: 
 Existing MOS methods can be categorized into two categories: Projection-based methods [5, 8, 20, 39] and Non-Projection-based methods [23, 24, 29, 30, 41] . Projection-based methods involve projecting a 3D point cloud onto a compact 2D plane as the model input, while Non-Projection-based methods are processed directly within the 3D point cloud space.Projection-based MOS methods can be divided into Range View (RV) methods [5, 8, 20, 39 ] and Bird's-Eye View (BEV) ones [32, 46] . There has been extensive work in the field of object detection and segmentation in 3D LiDAR data using RV images [9, 12, 21] , which use the original single scan point cloud through the spherical projection [31] to obtain 2D RV image as the model input. In motion perception tasks, the temporal information needed to perceive motion is usually provided by the residual images obtained from the residual processing of the RV images of the current scan and the past few scans [5, 8, 20, 39] . Chen et al. [5] directly concatenate the RV images and the corresponding multi-scan residual images as input, whereas Sun et al. [39] proposed a dual-branch model structure, which used two encoders to extract features from the RV images and the multi-scan residual images respectively. Different from [39] , Kim et al. [20] use a branch in its model to decouple the movable objects into moving objects and static objects using additional semantic labels, which enhances the model's capacity to understand the dynamic scenario. Cheng et al. [8] focus more on the feature extraction of motion features, which coincides harmoniously with our viewpoint and have achieved leading performance with additional semantic labels. Unlike the above RV-based methods, the BEV methods present the point cloud features from a top-down perspective, which maintains the consistency of the object scale in the point cloud and makes it easier to understand and process features [45] . Mohapatra et al. [32] first proposed moving object segmentation in BEV, which achieved faster running speed but lower accuracy than RVbased methods. Zhou et al. [46] employed polar coordinates to transform point clouds into Bird's-Eye View (BEV) representation. They utilized a dual-branch CNN to extract appearance and motion features from multiple BEV scans, resulting in improved accuracy and efficiency. Although the above projection-based methods are efficient, there is a loss of geometric information in the process of returning the final result to the 3D point cloud space, which limits the performance of such methods.Non-projection-based methods, which directly operate on point clouds in 3D space, circumvent the loss of geometric information inherent in projection-based approaches. Consequently, these methods hold the theoretical advantage of achieving superior segmentation performance. 4DMOS [29] inputs the voxelized point cloud superposition representation of multiple scans into a sparse 4D CNN and fuses the prediction results of multiple different scans of moving objects by a binary Bayesian filter as an additional post-process, which improved the confidence score of judging the moving object in the current scan and achieved excellent segmentation results. Similarly, InsMOS [41] is also based on a 4D point cloud as input, but they assist in segmenting moving objects by fusing BEV representations containing object instance information at different resolutions. Li et al. [23] proposed a dual-branch model that integrates 3D point clouds and 2D images, and employed Transformer [40] to fuse multi-scale point cloud and image features, aiming to enhance the coupling of temporal and spatial characteristics. Li et al. [24] utilized cylindrical coordinates to voxelize the aggregated point cloud input and employed a CNN to obtain moving object segmentation results and further applied MOS in the task of LiDAR-based localization to improve its robustness in dynamic scenes. MapMOS [30] improves that selecting fixed past scans will lead to some moving objects not being perceived due to occlusion. Therefore, a strategy of moving target perception based on a local map constructed by past scans is proposed, and state-of-the-art performance is achieved on the validation set of the SemanticKITTI-MOS benchmark. In addition to the learning-based methods mentioned above, there are also many non-learning-based methods, including map-cleaning methods [2, 19, 25, 26, 36] and map-based methods [6, 34, 37] . Mapcleaning methods remove the moving objects offline by the geometric information of the target [2, 19, 25, 26, 36] . Map-based methods, on the other hand, require a pre-built map to remove the objects that are moving throughout the mapping process [6, 34, 37] .In general, existing MOS methods have not thoroughly explored the coupling between temporal and spatial features, which limits their understanding of motion states. In contrast, our method achieves shallow coupling of temporal and spatial features during the embedding stage and deep coupling within each stage of the model. This deep coupling establishes a robust correlation between temporal and spatial clues, enhancing the model's comprehension of motion scenes. Importantly, our method achieves state-of-the-art performance without any post-processing modules on the MOS task. 
method 3.1 preliminaries: 
 State Space Model. SSM [14] is a sequential model that can map a one-dimensional input 𝑥 (𝑡) ∈ R sequence to an output sequence 𝑦 (𝑡) ∈ R. The process is represented by a series of continuous hidden states ℎ(𝑡) ∈ R 𝑁 of state size 𝑁 . In general, the SSM of a continuous-time system can be represented by the following linear Ordinary Differential Equation (ODE) as depicted in Equation ( 1),ℎ ′ (𝑡) = 𝐴ℎ(𝑡) + 𝐵𝑥 (𝑡) 𝑦 (𝑡) = 𝐶ℎ(𝑡) (1)where the parameters 𝐴 ∈ R 𝑁 ×𝑁 , 𝐵 ∈ R 𝑁 ×1 and 𝐶 ∈ R 1×𝑁 establish the correlation between the state and output variables.Discretization. It is essential that the original SSM equations be transformed into a discrete form to fit the discretized data in the task. The discretized SSM can be written as Equation ( 2),ℎ 𝑡 = Āℎ 𝑡 -1 + B𝑥 𝑡 𝑦 𝑡 = 𝐶ℎ 𝑡 (2)The discretization parameters Ā, B can be described by the Zero-Order Hold (ZOH) rule with timescale parameters Δ as Equation ( 3),EQUATIONSelective Scans Mechanism. Mamba [15] proposed a selective scan mechanism that effectively adjusts the parameters through the parameterized projection of the input sequence, enabling the SSM to selectively filter the input sequence features. This has advanced the research of SSM in the time-varying domain.MambaMOS Principles. To address the weak coupling of temporal and spatial information in existing MOS methods, we try to adapt Mamba from a Natural Language Processing (NLP) to MOS task. An intriguing discovery emerged: the MOS task inherently involves selecting a moving subset of elements from an unordered set, akin to the selective copying mechanism in NLP [1, 15] . Leveraging this insight, we introduced MambaMOS based on the selective copying mechanism. This enhancement equips Mamba to effectively address MOS tasks, enabling the model to adaptively select the moving target while reducing operational costs.Overview Architecture. The proposed MambaMOS leverages a U-Net [35] style overall architecture as shown in Figure 2 . Firstly, the 4D point cloud set as input will be transformed into an ordered sequence after the serialization process.Simultaneously, they are encoded through the meticulously designed TCBE (Sec. The overview of our proposed MambaMOS. The previous 𝐹 -1 scans, after undergoing viewpoint transformation, are overlaid with the current scan to form a 4D point cloud. This 4D point cloud is then serialized to obtain a sequence as input. After passing through TCBE, the coupling degree between temporal and spatial information in the input is enhanced and fed into a symmetric encoder-decoder architecture (the pink box). Each stage of the encoder/decoder consists of a pooling/unpooling layer and 𝑁 blocks (the blue box). MSSM serves as the core of each block to achieve deep-level coupling of temporal and spatial features. Finally, the MOS result in the current scan can be obtained from the output of the decoder by a linear layer.attention of the feature following the idea of most point transformer works [22, 43, 44] .The point cloud features after layer normalization will pass through the MSSM (Sec. 3.4), the core insight of the entire block, where the motion features of the objects will be enhanced. The final output of the block is the layer normalization and a multi-layer perceptron. And residual connections are extensively applied in each of our blocks to avoid vanishing gradients [16] . Finally, the logits of each point can be obtained by a linear layer. And points are deserialized to extract the segmentation result.Input Representation. At the current time (𝑡 = 0), given a LiDAR scan 𝑆 𝑡 = 𝑝 𝑖 ∈ R 4 𝑁 𝑡 -1 𝑖=0 with 𝑁 𝑡 points 𝑝 𝑖 = [𝑥 𝑖 , 𝑦 𝑖 , 𝑧 𝑖 , 1] 𝑇 represented by homogeneous coordinates. The goal is to segment the moving points in the current scan 𝑆 0 using the continuous point cloud set 𝑆 = {𝑆 𝑡 } 𝐹 -1 𝑡 =0 including the current scan 𝑆 0 and its past 𝐹 -1 scans. To aggregate the 𝐹 scans point cloud data into a 4D point cloud input containing temporal-spatial information and eliminate self-motion, we need to transform the past 𝐹 -1 scans to the perspective of the current scan and convert the homogeneous coordinates to Cartesian coordinates separately. Given the pose transition matrix 𝑇 0 𝑡 from scan 𝑡 to the current scan, the perspective transition from the point cloud at time 𝑡 to the current point cloud can be expressed as Equation (4) .EQUATIONThus, the 4D point cloud set𝑆 ′ = 𝑝 ′ 𝑖 ∈ R 4 𝑁 -1 𝑖=0 with 𝑁 = 𝐹 -1𝑡 =0 𝑁 𝑡 points can be represented as Equation (5) . To distinguish each scan in the 4D point cloud, we add the corresponding time step of each scan as an additional dimension of the point and obtain the spatio-temporal point representationEQUATIONSerialization. The SSM, as the core of MambaMOS, typically takes in a sequence of data, such as natural language. Therefore, it is necessary to obtain the sequence 𝑆 ′ 𝑜 from the unordered 4D point cloud set 𝑆 ′ by serialization. The serialization can be understood as a projection function Ψ that transforms the unordered set 𝑆 ′ into the sequence 𝑆 ′ 𝑜 . Thus, the process of serialization and deserialization can be described as Equation 6, where Ψ -1 is the inverse projection function. One approach to serialize point clouds is by sorting the coordinate of each point [27] . However, this serialization method fails to adequately preserve the local spatial relationships of the Space-filling curves are mathematical curves that can project data in 𝑁 -dimensional space to one-dimensional continuous space: Z 𝑁 → Z, which have been applied in recent 3D scene understanding works [42, 43] . Inspired by them, our serialization process utilizes z-order curves [33] and Hilbert curves [17] , which preserve neighborhood relationships in the original 3D point cloud effectively.EQUATIONPrevious methods [29, 30, 41] have not effectively emphasized the dominance of temporal information for each point. This is evident in their equal treatment of spatial occupied information obtained from LiDAR and the corresponding timestamp information from the scan aggregation process. However, the direct overlaying for temporal and spatial information, which belong to different modalities, does not fully exploit the supervisory role of one modality on the other. Therefore, we propose the Time Clue Bootstrapping Embedding (TCBE). It emphasizes temporal information over spatial information based on the principle that time evolution drives the motion of objects, thereby enhancing the coupling of temporal and spatial information.The structure of TCBE is illustrated in the bottom right of Figure 2 . Specifically, TCBE embeds the spatial and temporal information of each point in the ordered point cloud sequence 𝑆 ′ 𝑜 using 1D convolution to obtain the corresponding spatial feature 𝑓 𝑠 and temporal feature 𝑓 𝑡 in embedded dimension, both of which possess local characteristics. Firstly, the initial coupled temporal and spatial feature 𝑓 𝑐𝑜𝑢 is obtained by adding the temporal feature 𝑓 𝑡 and the spatial feature 𝑓 𝑠 , which serves as an alternative implementation of previous embedding methods. Then, in order to emphasize the dominance of temporal information over spatial information, 𝑓 ′ 𝑡 which reflects the local temporal evolution trends, obtained through 1D convolution without changing its channels, is multiplied elementwise by 𝑓 𝑠 . Finally, the time-guided spatial feature 𝑓 𝑇𝐺𝑆 is added to the initial coupled temporal and spatial feature 𝑓 𝑐𝑜𝑢 , resulting in the enhanced temporal and spatial coupling information 𝑓 ′ 𝑐𝑜𝑢 . After undergoing 1D convolution, batch normalization, and activation functions, 𝑓 ′ 𝑐𝑜𝑢 serves as the output of TCBE.Although there are some similarities in form between the selective copy task [1, 15] in NLP and the MOS task as mentioned before, the direct application of Mamba [15] cannot effectively exploit the temporal features. This is attributed to the fact that the original Mamba [15] is designed for one-dimensional natural language with a certain causal relationship. However, the serialized multi-scan point cloud sequence cannot reflect strong causality. Thus, we propose MSSM to compensate for the shortcomings of Mamba [15] on MOS.The main design idea of MSSM is to enhance the original Mamba's perception of temporal features regarding moving objects by using cross-product attention between single-scan features and multiscan features. As shown in the upper left of Figure 2 , it is mainly composed of linear layers, activation function 𝜎, and an SSM with the selective scans mechanism. Let the input point cloud feature with batch size 𝐵, sequence length 𝑁 , and number of channels 𝐶 be characterized by 𝑓 𝐼 ∈ R 𝐵×𝑁 ×𝐶 , which will go through three branches. We derive the main branch of Mamba [15] to obtain the upper and the middle branches of our MambaMOS. The upper branch is used to extract the appearance features of each object in the single-scan point cloud. And the middle branch focuses more on the temporal features of moving objects in the 4D point cloud. Since the MOS task only focuses on moving objects, we aim for the MSSM to assign lower attention to unmovable objects such as roads or tree trunks. Therefore, a feature weighting process is required. Inspired by the gated attention units [18] , we employ a simple gating mechanism as the bottom branch of MSSM to allocate weights to features in each hidden state, thereby determining whether the features are expressed.Specifically, to obtain the single scan feature 𝑓 𝑆 ∈ R 𝐵 ′ ×𝑁 𝑝 ×𝐶 with 𝐵 ′ = 𝐵 × 𝐹 at this time, the upper branch firstly performs Reversed Aggregation (RA), which separates each scan of 𝑆 ′ and concatenates them as a separate batch after 0-padding to 𝑁 𝑝 . Then the appearance features of the single scan 𝑓 ′ 𝐴 can be obtained through the process of 1D convolution and single scan aggregation. This process can be written as:EQUATIONThe middle branch employs 1D convolution to obtain the temporal and appearance features of moving objects in multiple scans. The output of this process is denoted as 𝑓 ′ 𝑀 . Subsequently, 𝑓 ′ 𝑀 is fused with the output of upper branch 𝑓 ′ 𝐴 through the cross-product attention to obtain 𝑓 𝑀𝐺 . The fusion process can be described as follows:EQUATION)In the subsequent design, we follow the idea of the original Mamba, that is, the final output 𝑓 ′ of the block is obtained by element-wise multiplication of the result of the main branch and the result 𝑓 𝐺 of the gated branch after a linear projection. This process is described as follows:𝑓 ′ = SSM (𝜎 (𝑓 𝑀𝐺 )) ⊗ 𝑓 𝐺 (9)Before performing the loss calculation, we first deserialize the obtained sequence segmentation results to correspond to the initial unordered point cloud set as Euqation 6. Afterwards, following the majority of 3D segmentation methods, we adopt the combination of cross-entropy loss (L ce ) and Lovász-Softmax loss [4] (L ls ) as the joint loss L = L ce + L ls for supervised training. 
experiment 4.1 experiment setups: 
 We perform a variety of experiments to verify the proposed Mam-baMOS on the SemanticKITTI-MOS dataset [3, 5] . Sequences 00∼07 and 09∼10 are used as the training set, sequence 08 is used as the validation set, and sequences 11∼21 are used as the test set, following the same division as previous MOS methods [8, 30, 41] .The KITTI-road dataset [13] is also utilized in the experiments for comparison with other MOS methods, and the same partitioning approach as [8] is maintained. The entire training is conducted on four NVIDIA RTX A6000 GPUs with 48G VRAM for 50 epochs with a batch size of 4. AdamW [28] with a weight decay of 0.005 is used as the optimizer, and the learning rate is set to 0.00032. A grid size of 0.09𝑚 is applied to voxelize the input aggregated point cloud, and scans of 𝐹 = 8 are used for input same as [5, 8, 39] . Moreover, common point cloud data augmentation approaches such as random rotation and random flipping are applied during training to enhance the generalization capacity of MambaMOS. All ablation experiments were conducted on eight NVIDIA GeForce RTX 3090 GPUs, using a four-scan input (𝐹 = 4), a batch size of 8, and completed using automatic mixed precision. We report the voxelized moving object IoU for ablations. Additionally, similar to [5, 8, 20] , we employed additional semantic labels for training as well. During the validation and testing stages, Intersection-over-Union (IoU) [11] is adopted as the metric to evaluate the performance. Following previous methods [23, 24, 30] , all experiments provide IoU for the moving objects as IoU 𝑀𝑂𝑆 as the main evaluation metric which can be described as Equation 10with True Positive 𝑇 𝑃, False Positive 𝐹 𝑃 and False Negative 𝐹 𝑁 :EQUATIONWe analyze the comparison with other SoTA methods from both quantitative and qualitative perspectives.Quantitative Analysis. Table 1 presents the quantitative comparison results of MambaMOS with state-of-the-art methods in the MOS benchmark [3, 5] . All results reported for each method are the best-reported results in their respective papers. Due to some methods [8, 39, 41] using the KITTI-Road dataset [13] as additional training data in their comparisons, we follow the principle of fair comparison by using consistent training data, distinguishing it with a symbol † from the original comparison on the table.Without using additional training data, MambaMOS successfully outperforms almost all methods on the benchmark. Specifically, MambaMOS surpasses Two-streamMOS [23] , which integrates point clouds and images as input, by 4.4% and 2.6% on the validation set and hidden test set. We attribute this significant improvement largely to the geometric losses present in non-projection-based methods. In the comparison with non-projection-based methods, MambaMOS achieves superior performance on the validation than LiDAR-IMU-GNSS [24] , which includes ground optimization as an additional pre-processing, by 3.3%, and leads it by 0.7% on the test set, owing to the stronger coupling of temporal and spatial information in MambaMOS. Despite MambaMOS using a fixed number of scans as input, it still surpasses MapMOS [30] , which leverages a local map instead, with a significant margin of 9.6% on the hidden test set. After incorporating additional training data, MambaMOS, when trained with the same settings as before, still outperforms other methods under comparable conditions. MambaMOS surpasses the state-of-the-art MF-MOS [8] by 3.4%, and outperforms InsMOS [41] ,Table 1 : Comparison with state-of-the-art methods on the SemanticKITTI-MOS benchmark. † denotes using additional KITTI-Road for training.IoU 𝑀𝑂𝑆 (%) Validation 08Test 11-21LiMoSeg [32] 52.6 -LMNet [5] 67.1 54.5 SSF-MOS [38] 70.1 -MotionSeg3D [39] 71.4 64.9 RVMOS [20] 71.2 74.7 4DMOS [29] 77.2 65.2 InsMOS [41] 73.2 -MF-MOS [8] 76.1 -MotionBEV [46] 76.5 69.7 MapMOS [30] 86.1 66.0 Two-streamMOS [23] 77.9 73.0 LiDAR-IMU-GNSS [24] 79.0 74. To further analyze the advantages brought by our method, we have conducted a detailed comparison of the segmentation performance of existing methods on the SemanticKITTI-MOS validation set for different distances, as shown in Table 2 . The metrics in Table 2 are either reported in their respective papers or determined using their publicly available weights. The weights for other methods such as RVMOS [20] , Two-streamMOS [23] , and LiDAR-IMU-GNSS [24] are either undisclosed or not reported in the papers, hence not included in the comparison.As known, the point cloud distribution becomes sparser as the object distance from the LiDAR increases. As shown in Table 2 , most MOS methods achieve satisfactory segmentation results at close distances. However, their segmentation performance sharply declines when the distance reaches the range of 20𝑚 and 50𝑚. Furthermore, beyond a distance of 50𝑚, some projection-based methods such as LMNet [5] , MotionSeg3D [39] , and MotionBEV [46] fail to discern the motion attributes of the objects. Although MF-MOS [8] , with its focus on motion features, surpasses the non-projection-based methods like 4DMOS [29] and InsMOS [41] in segmenting distant moving objects, it is still limited in recognizing the motion attributes of distant objects due to geometric losses caused by the projection process, which prevents the strong coupling of spatial information and temporal information for the objects. On the other hand, Mam-baMOS demonstrates precise segmentation of moving objects even in cases of extremely sparse point clouds. This indirectly supports Qualitative Analysis. As shown in Figure 3 , MambaMOS has achieved significant performance improvements compared to other methods on the SemanticKITTI-MOS validation set [3, 5] . We have conducted a detailed analysis of this phenomenon as shown in the figure. MF-MOS [8] , InsMOS [41] , and 4DMOS [29] all fail to correctly determine the motion attributes of the objects, resulting in a large number of false negative results. It can be attributed to they only rely on a weak coupling between spatial and temporal information for motion estimation. Slow-moving objects or distant moving objects do not exhibit obvious characteristics in terms of spatial information, so methods [8, 29, 41] that do not strengthen the temporal information perform poorly in their estimation. However, MambaMOS effectively addresses this problem by incorporating strong temporal information coupling. Furthermore, to reduce false positive predictions, we also categorize stationary vehicles as a movable class for training, following the method of [5, 8, 20] , which further enhances the model's understanding of motion scenes.Since the SSM receives sequential features, different spatial serialization combinations will have an impact on overall performance. We explore the influence of the serialization combination of 𝑧curve [33] and Hilbert curve [17] , which have good spatial locality characteristics, on the performance of MOS. As the spatial filling curves traverse the spatial points based on the order of 𝑥, 𝑦, and 𝑧, prioritizing 𝑦 will yield different serialization results compared to prioritizing 𝑥. We denote this variant with 𝑇 . As shown in Table 6 , richer serialization methods yield better performance on the validation set. This is because multiple serialization methods capture different contextual relationships of sequences, reducing overfitting while enhancing the model's understanding of dynamic objects.Table 4 presents two methods proposed in our study to enhance the coupling of temporal and spatial information. We conducted ablation experiments to demonstrate that the proposed modules can enhance the model's perception of moving objects. When MSSM is not used, we replaced it with the original Mamba block [15] , and we employed a simple 3D convolution for information embedding when TCBE is not applied. From the experimental results, it can be observed that when only MSSM or TCBE is applied compared to the baseline, the performance is improved by 1.86% and 1.41%. This indicates that both MSSM and TCBE can enhance the coupling of temporal and spatial features and improve the model's perception of motion features. However, when only MSSM is applied, the performance is improved by 0.45% compared to TCBE. This is because MSSM, based on the interaction between single-scan features and multi-scan features, focuses more on deep-level spatio-temporal information coupling and can learn the motion attributes of objects more comprehensively. Finally, by joining TCBE on the basis of MSSM, the emphasis on temporal information during the embedding phase is further enhanced, which aligns with the fundamental logic of motion recognition and achieves optimal performance, surpassing the baseline by 2.25%.Since the majority of the SemanticKITTI dataset [3] of MambaMOS, we fine-tune it on the KITTI-Road dataset [13] to evaluate its generalizability to new environments. We follow the same data partitioning strategy as MF-MOS [8] , InsMOS [41] , and MotionSeg3D [39] , and compared it with methods that use fixed scans as input with publicly available weights. The original weights of all methods shown in Table 5 are open-source and trained exclusively on the SemanticKITTI-MOS [3, 5] dataset. They were then fine-tuned for 10 epochs on the KITTI-Road training set [13] to obtain the final results. The results indicate that even with a small amount of data and minimal fine-tuning, MambaMOS still achieves better results than previous methods, demonstrating its excellent generalization capability for adapting to new environments. 
conclusion: 
 This paper introduces MambaMOS, a novel framework for moving object segmentation, aiming to address the issue of weak spatiotemporal coupling in existing methods. Specifically, we introduce the Time Clue Bootstrapping Embedding to achieve the shallow coupling of temporal and spatial information of the objects. Furthermore, we underscore the importance of temporal information as the primary cue for recognizing motion attributes, thereby enhancing the model's sensitivity to motion features. To achieve deeper spatio-temporal coupling, we propose the Motion-aware State Space  
: 
 effect" caused by object motion during mapping in simultaneous localization and mapping, resulting in a clean static map [7, 24] .Chen et al. [5] propose a learning-based MOS method that projects point clouds onto a planar representation and utilizes a sequence of these representations to incorporate temporal information for MOS. Similar paradigms like [8, 20, 32, 39, 46] achieve low latency but suffer from geometric loss introduced by projection, leaving room for improvement in terms of accuracy and generalization. Nonprojection methods [29, 41] perform feature extraction directly in the 3D space and have achieved precise segmentation results and excellent generalization. However, these methods cannot sufficiently couple the temporal-spatial features of multi-scan point clouds and suffer from the issue of "weak coupling between temporal and spatial information". Specifically, due to the changing spatial positions of moving objects over time, trailing artifacts will be formed in the aggregated point cloud. Without incorporating timestamp information to differentiate each scan in the aggregated point cloud, these artifacts may be confused with larger objects in terms of their similar appearance (e.g., moving cars and parked trucks). The evolution of timestamp information reflects the motion of objects, and the moving objects can also be identified through the evolution of their timestamp information.Based on the above observations, we hypothesize that the temporal information of objects is the dominant information for determining their motion, and strengthening the coupling between the temporal and spatial information of objects will facilitate the segmentation of moving objects. However, the aforementioned methods [29, 41] directly concatenate the timestamp information of each point with the spatially occupied information to form a 4D point cloud that contains temporal-spatial features and employ a Convolutional Neural Network (CNN) to learn these temporalspatial features, as shown in Figure 1 (a) . Although they are effective, the neglect of the dominant role of timestamp information and the lack of deeper coupling between temporal and spatial information hinder further improvement in their segmentation performance.In this work, we rethink the issue of effectively encoding shallow temporal and spatial features and facilitating sufficient interaction among deep temporal and spatial features. For cases where simply concatenating timestamp information with spatial information fails to highlight the importance of temporal information, we propose an effective embedding approach named Time Clue Bootstrapping Embedding (TCBE), which emphasizes the expressive power of temporal information through attention mechanisms and enhances the mutual coupling between temporal and spatial information by treating temporal information as an independent channel separate from spatial information.Although TCBE can enhance the coupling between temporal and spatial information to some extent compared to the previous embedding approach, it can only be applied to shallow layers and cannot further deepen the coupling between temporal and spatial In contrast, our method emphasizes the primacy of temporal information at each point more through our designed TCBE and achieves a deeper coupling of temporal and spatial information with MSSM, which aligns more closely with the fundamental principles of motion recognition.information. Recently, the work of [23] introduced a projectionbased method and for the first time incorporated the self-attention mechanism core of transformers [40] into MOS, achieving better performance far surpassing the methods of the same paradigm. However, studies [10, 15] have shown that the quadratic computational complexity of the transformer model in managing large input sequences presents challenges in achieving a balance between training cost and accuracy. Fortunately, the State Space Model (SSM) introduced by Mamba [15] offers a promising solution, providing us with the opportunity to achieve comparable long-range context modeling capabilities to the transformer [40] while maintaining linear time complexity. Inspired by this advancement, we proceed to develop the Motion-aware Space State Model (MSSM). In the designed MSSM, we decouple the aggregated point cloud features into multiple single-scan features and learn the appearance features expressed by single-scan features and the motion features expressed by the aggregated features separately. Then, by using the cross-product attention between these two features, we achieve spatial appearance interpretation of multiple-scan features from single-scan features and temporal information supplementation of single-scan features from multiple-scan features, thereby enabling the deep-level coupling between temporal and spatial information with the assistance of SSM and achieving linear complexity.Through extensive experiments, we demonstrate that the combination of TCBE and MSSM can effectively achieve strong coupling between temporal and spatial information, and achieve state-ofthe-art performances on SemanticKITTI-MOS [3, 5] and KITTI-Road [13] benchmarks. Our contributions are summarized as follows:• We rethink the problem of the weak coupling between temporal and spatial information that existed in the previous methods and propose a novel LiDAR-based moving object segmentation framework, here in MambaMOS. To the best of our knowledge, this work represents the first attempt to utilize SSM in MOS, providing directions for future extensions of SSM in the MOS domain. • An effective Time Clue Bootstrapping Embedding method (TCBE) is introduced, which enhances the coupling capability of temporal and spatial information to some extent, improving the expressive power of motion object features. • A novel temporal-spatial information coupling module based on SSM (MSSM) is proposed, which enables deep-level coupling between temporal and spatial features and enhances the perception of moving objects through the complementary nature of single-scan and multiple-scan features. 
 
 Appendix: LMNet [5] 87.4 MotionBEV [46] 80.5 4DMOS [29] 81.0 InsMOS [41] 83.9 MF-MOS [8] 87.9 MambaMOS 89.4Model, which facilitates interaction between single-scan and multiscan features. Leveraging the SSM's linear complexity and strong contextual modeling capability, the MSSM achieves strong spatiotemporal coupling of features. Extensive experiments validate the effectiveness of our method, demonstrating state-of-the-art performance on both SemanticKITTI-MOS and KITTI-Road datasets. Additionally, this paper marks the pioneering application of SSM to the MOS task, and establishes a significant connection between point cloud segmentation in 3D vision and natural language tasks, offering valuable insights for future research directions.  


Decision: the paper proposes a LiDAR-based 3D Moving Object Segmentation with Motion-aware State Space Model. All reviewers agree the proposed method improves performance.
The rebuttal addresses the key questions raised by the reviewers.

Reviews: review: paper_summary: paper_summary: This paper proposes a transformer-based framework to segment 3D moving objects among LiDAR point cloud frames by emphasizing temporal information and coupling spatial-temporal cues.
strength_weakness: strength_weakness: 1. A LiDAR-based sequential framework is designed for the 3D moving object segmentation task.
limitations: limitations: 1. The motivation and method demonstration for "Time Clue Bootstrapping Embedding" in Section 3.3 are confusing. Explaining why temporal information is more important than spatial information for 3D Moving Object Segmentation would help to better illustrate your novelty. Additionally, the method demonstrations of this module in Figure 2 and in text lines 49-508 need to be much clearer for the audience. Formula expressions are necessary for understanding.

2. The same issue exists for the "Motion-aware State Space Model" in Section 3.4: the method demonstration is hard to follow.

3. The advantage of the Mamba-based transformer is its comparable performance with less computational complexity. This feature is mentioned in line 153 of the text, but there are no experiments provided to demonstrate the efficiency of this work.

4. Other questions: I am curious about the differences between 3D moving object segmentation and 3D scene flow estimation/segmentation.
paper_summary: paper_summary: This paper aims to achieve strong coupling of temporal and spatial information in LiDAR sequences to improve the accuracy of 3D Moving Object Segmentation. 
The main contributions include an Time Clue Bootstrapping Embedding method which enhances the shallow coupling of temporal and spatial information, a temporal-spatial information coupling module that enables deep-level coupling. 
The proposed method is the first attempt to ultize SSM in MOS and achieves state-of-the-art performance on the SemanticKITTI-MOS and KITTI-Road benchmarks.
strength_weakness: strength_weakness: The proposed method utilizes the Time Clue Bootstrapping Embedding and temporal-spatial information coupling to enhance the shallow and deep coupling of temporal and spatial information, respectively. 
It achieves state-of-the-art performance on the SemanticKITTI-MOS and KITTI-Road datasets.
limitations: limitations: 1. Do not use the same numbering for multiple formulas, such as lines 300-302, lines 308-310, etc.
2. The serialization process in Section 3.2 is unclear. The space-filling curves project the data in 𝑁-dimensional space to one-dimensional continuous space. In this paper, the z-order curves and Hilbert curves project the 4D point cloud to 1D? In Figure 2, it can be seen that the results are a two-dimensional point cloud with size of nx5. In addition, the Equation (6) is too concise to reflect the point cloud conversions.
3. Compared to the existing moving object segmentation methods, what is the time cost of the proposed method?
paper_summary: paper_summary: In summary, thish paper discussed the challenges of learning-based point cloud MOS and proposed a method for moving object segmentation, which utilize an improved state space model to represent these motion differences, modeling the motion states.
strength_weakness: strength_weakness: - Excellent performance performance, basically exceeds the existing solution.
- The paper is well-written and presents a novel contribution to the field of point cloud MOS.
limitations: limitations: There are some key points and suggestions that the authors should address:

- L342, what do [2,2,2,6,2] and [2,2,2,2] represent respectively? What is 6?
- I would like to confirm with the author whether the input of the point cloud only contains the xyz coordinate position and not the intensity information of the point cloud?
- Looking at Fig2, the loss function only supervises the current frame t? Or t-1, t-2 will be supervised at the same time?
- Why does the final experiment use 8 frames as input, while the ablation experiment uses 4 frames as input? Have you also done ablation experiments on the number of input frames? The more input frames the better? 4DMOS seems to input 10 frames, while LMNet, MotionSeg3D, inputs 8 frames, just as you said.
- This article does not give the inference time/speed of the proposed method, but this is compared and reported in 4DMOS LMNet and MotionSeg3D. It is recommended that the author provide a fair model inference speed for reference.
- The video of additional materials cannot show the comparison with other solutions, and the comparison with GT, which has almost no reference value.