Fore-Mamba3D: Mamba-based Foreground-Enhanced Encoding for 3D Object Detection
Authors: Zhiwei Ning
Xuanang Gao
Jiaxi Cao
Runze Yang
Huiying Xu
Xinzhong Zhu
JIE YANG
Wei Liu
Venue: ICLR 2026 Poster (2026)
Keywords: 3D Object Detection
Mamba
Foreground Representation
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: Linear modeling methods like Mamba have been merged as the effective backbone for the 3D object detection task. However, previous Mamba-based methods utilize the bidirectional encoding for the whole non-empty voxel sequence, which contains abundant useless background information in the scenes. Though directly encoding foreground voxels appears to be a plausible solution, it tends to degrade detection performance. We attribute this to the response attenuation and restricted context representation in the linear modeling for fore-only sequences. To address this problem, we propose a novel backbone, termed Fore-Mamba3D, to focus on the foreground enhancement by modifying Mamba-based encoder. The foreground voxels are first sampled according to the predicted scores. Considering the response attenuation existing in the interaction of foreground voxels across different instances, we design a regional-to-global slide window (RGSW) to propagate the information from regional split to the entire sequence. Furthermore, a semantic-assisted and state spatial fusion module (SASFMamba) is proposed to enrich contextual representation by enhancing semantic and geometric awareness within the Mamba model. Our method emphasizes foreground-only encoding and alleviates the distance-based and causal dependencies in the linear autoregression model. The superior performance across various benchmarks demonstrates the effectiveness of Fore-Mamba3D in the 3D object detection task.

Published as a conference paper at ICLR 2026
FORE-MAMBA3D: MAMBA-BASED FOREGROUND- ENHANCED ENCODING FOR 3D OBJECT DETECTION
Zhiwei Ning!, Xuanang , Jiaxi Caol? Runze Yang!24 Huiying Xu², Xinzhong Zhu Jii Yangl.23' & Wei Liu 1,23,"
1 School of Automation and Intelligent Sensing, Shanghai Jiao Tong University
2 Institute of Image Processing and Pattern Recognition. Shanghai Jiao Tong University
3 Institute of Medical Robotics, Shanghai Jiao Tong University
School of Computing, Macquarie University
5 School of Computer Science and Technology, Zhejiang Normal University
6 Beijing Geekplus Technology Co., Ltd
(zwning ng, fangkuar, caojiaxi, yunze.y)@sjun edu. cn
[xhy, 2x edu. (jieyang, weiliucv)es jt edu. https: /tt///gitthlll wwwp:paatt-wwinnnn///rtaaaaaaaaa
ABSTRACT
Linear modeling methods like Mamba have been merged as the effective back- bone for the 3D object detection task. However, previous Mamba-based methods utilize the bidirectional encoding for the whole non-empty voxel sequence, which contains abundant useless background information in the scenes. Though directly encoding foreground voxels appears to be plausible solution, it tends to degrade detection performance. We attribute this to the response attenuation and restricted context representation in the linear modeling for fore-only sequences. To address this problem, we propose a novel backbone, termed Fore-Mamba3l) to focus the foreground enhancement by modifying Mamba-based encoder. The fore- ground voxels are first sampled according to the predicted scores. Considering the response attenuation existing in the interaction of foreground voxels across dif- slide window (RGSW) to prop-
ferent instances, we design regional-to-globe agate the information from regional split to the entire sequence. Furthermore, a semantic-assion and state spatial fusion module (SASFMamba) is proposed to enrich contextual representation by enhancing semantic and geometric awareness within the Mamba model. Our method emphasizes foreground-only encoding and alleviates the distance-based and causal dependencies in the linear autoregression model. The superior performance across various benchmarks demonstrates the effectiveness of Fore-Mamba3D the 3D object detection task.
INTRODUCTION
3D object detection is critical task in computer vision with broad applications in autonomous driv-
ing (Mao et al. 2023; Grigorescu et al. 2020), embodied intelligence (Gupta et al. [021
Huang
jet al. 2022 Zhang et al. 2025), and robotic navigation (Gul et al.] 2019 Xu et al, 2023 Pre-
vious LiDAR based methods (Shi et al. 2020b Qi et al., 20176] Li et al. 2021 Yan et al. 2018)
achieve remarkable performance, which utilize sparse convolutional neural network (SpCNN) (Liu jet al., 2015) or the Transformer (Vaswani] 2017) architecture as their backbones. However, the hardware incompatibility of SpCNN and the quadratic computational complexity of Transformer present substantial obstacles to their deployment in real-time detection applications. Recently, sev- eral Mamba-base methods (Gu Dao 2023 Liu al. 2024a Zhang et al., (024c) empirically show that integrating bidirectional scanning mechanism with the state space model (SSM) could achieve promising performance in 2D image recognition tasks with linear computational cost.
"The corresponding authors.

---

Published as a conference paper at ICLR 2026
Inspired by the effectiveness in 2D computer vision tasks, some studies extend SSM 3D object detection (Liu et al. 2024b Zhang et al., 2024c). These methods can be briefly classified into group based and group-fre free approaches. As shown the left of Figure I group-based works (Liu et al. 2024b Wang et al. 2023) partition the 3D voxel features into multiple groups along the X/Y-axis order for linear modeling. In contrast, group-free methods (Zhang et al. (202c) directly flatten all the non-empty voxels in the scene via space-filling curves, such as Hilbert (Hilbert & Hilbert 1935) or Z-order (Orenstein) 1986) curves. Though these methods encode the entire scene in various ways. the informative foreground occupies only small portion, as illustrated on the right of Figure This naturally motivates focusing on the effective foreground-only encoding technique.
Nevertheless, foreground-centee -cnttric detectors confront inherent challenges. Primarily, imprecise or incomplete foreground sampling risks omitting critical structural information, thereby necessitating specialized prediction and sampling strategy to ensure representational integrity. More critically, the sparse distribution of foreground voxels across distinct instances hinders conventional linear encoders from capturing long-range dependencies, resulting in response attenuation. Given that group-based methods excel at local modeling while group-free methods provide stronger global interaction, it is intuitive to mitigate response attenuation through regional-to- coonnal----goballl encoding strategy that leverages the advantages of both paradigms. In addition, enhancing the semantic and geometric awareness of the state variables in Mamba further enrich contextual understanding and lead to superior overall performance.
In this paper, we propose Fore-Mamba3D to achieve encod-
Empty Voxel
Background
Foreground
Background
Foreground
Voxel
Voxel
foreground-enhanceed ing for 3D object detection. Specifically. for higher sampling accuracy, we predict the fore- ground score for each non-empty voxel and flatten the top-k voxels into a ID sequence via the Hilbert (Hilbert & Hilbert 1935) space-filling curve template. However, directly applying the vanilla Mamba to foreground features leads
Foreground/Bare Background Ratio
84.6
Mannbu-Boo Baved Encods
77.4
Group
22.6
(a) Group-Based
15.4
Mambe- Bosod
Encode
nuSeenes
KITTI
Compurison GPU Usage and
suboptimal performance due to response at- tenuation and insufficient textural representation. Therefore, we integrate the linear encoder with regional-to-global sliding win- dow (RGSW), which aggregates information from the regional patch to the entire sequence for sufficient long-range interaction. Then, we introduce the SASF- Mamba component for better semantic and geometric understanding in state variables, which consists of demantic-assiste fusion (SAF) and state spatial fusion (SSF) modules. As shown the right of the Figure [ the proposed combinations reduce the memory usage with promising performance achieved simultaneously. The main contributions of our paper are summarized follows:
Group-Free
82.2
Oves (KTT)
Background
KITTI (maP))
Vanilla
Das (nuScase))
Lalvnn Bessing
Vasille Maniae
Flatten
Foreground
10
(c) Foreground-only (Ours)
Usage
Figure 1: Comparison of previous group-based and group-free Mamba methods with our approach.
We propose Fore-Mamba3D model, novel Mamba-based approach that focuses the effective linear encoding of foreground features for superior 3D detection performance.
A segional-to-globee sliding window strategy is designed to aggregate and propagate lo- cal information the global sequence address the deficiency of global interaction in previous autoregression models.
The SASFMamba component is introduced to leverage both comantic-assister fusion and selective state spatial fusion in state variables, which achieves non con-casual encoding with semantic and geometric understanding.
2

---

Published as a conference paper at ICLR 2026
2
RELATED WORK
LIDAR-BASED 3D OBJECT DETECTION
LiDAR is an important sensor for 3D perception due to its precise and intuitive geometric repre-
sentation of scenes. Most of the AIDAR-based 3D detection methods (Shi al. 2020a Qi al.)
b Vora et al., 2020 Yan et al. 2018 Chen et al. 2022) can be typically categorized into based and voxel-based approaches. Point-based methods (Qi et 2017a Shi et al. 20] directly take raw points as input, which assemble the regional context through set abstraction, and progressively downsampling for point encoding. The above process in point-based methods is com- putationally inefficient. In contrast, voxel-based methods (Deng et al. 2021] Zhou & Tuzel [0118]
Chen et al. 2023) first divide point clouds into regular voxel grids, and then extract features via 3D
SpCNN Chen et al. 2023 2022
Wu et al. 2022) or Transformer encoder (Wang et al. 2022b
Bai et al. 2022 Sheng et al. 2021 D. 3D SpCNN-based methods extract the voxel features by nor-
mal convolution kernels, limiting the capability to capture the larger or global scene context. The
Transformer- based method groups the voxels in sequence and encodes the information in a self- attention or cross-attention mechanism, which is hindered by the quadratic complexity. Therefore, reducing computational complexity while achieving global recognition remains a challenge.
2.2
MAMBA FOR 3D DETECTION
Similar to the process in structured 2D images, the 3D points cloud or voxel representation can also be flattened into serials. Most of the 3D Mamba-based methods (Liang et al.] [2024] Liu et al.) 2024b, Zhang et al. 2024c, Ning et al. 2024) can be categorized into group-based and group-free classes. PointMamba (Liang et al. 2024) is the pioneer in grouping the point cloud by faster points sampling (FPS) and serializing all the non-empty voxels. LION (Liu et al. 2024b) enables sufficient feature interaction in much larger group than previous Transformer-bast
methods, such as DSVT (Wang et al.) 2023). contrast, Voxel-Mamba (Zhang et al. 2024c) is a group-free approach, which serializes the whole space of voxels into single sequence and enhances the spatial proximity of voxels by designing a dual-scale SSM. MambaDETR (Ning et al. 2024) serializes the queries in the current 3D scene and aggregates the sequential in different frames. However, the linear encoding for all the non-empty voxels is unnecessary and time-cost, while the semantic and spatial relation in the state space is lacking. Our approach tackles these limitations by performing foreground-forr -ocusse encoding to substantially reduce redundancy.
2.3 FOREGROUND SAMPLING AND ENCODING
Foreground sampling and encoding have become central components in recent 3D object detection approaches (Zhang et al. 2022 Wang et al. 2022a Zhang et al.) 2023), particularly for LiDAR- based methods. For example, IA-SSD (Zhang et al. 20222) introduces instance-aware downsampling to hierarchically select foreground points, while RBGNet (Wang al.) 2022a) employs foreground- biased sampling to capture more object-surface points and then applies ray-based feature grouping for improved bounding box prediction. DSASA (Zhang et al. 2023) further develops series of FPS-based strategies to increase foreground coverage while balancing point density across instances. Nevertheless, persistent challenge lies in maintaining sufficient representation when the sampled foreground points are sparse and spatially scattered. In this work, we mitigate this issue by designing the specific module to alleviates the response attenuation and information loss arising from sparse foreground-ond sequences.
METHOD
As shown in Figure a), the 3D backbone of Fore-Mamba3D consists of four stages and each contains an instance selection block and downsampling block. The instance selection block in Figure ([(b) is the main component in our method, which includes foreground voxel sampling, regional-to-glee to-glbbll sliding window (RGSW) strategy in Figure ((c), and a semantic-assisted and state spatial fusion Mamba (SASFManha) in Figure
3

---

Published as a conference paper at ICLR 2026
(n) Framework
SStagel
ASinged
Pre Pre-praaa
D Sumplin
Backbone
Detection
Instance Selection Block1
Instance Selection
BEV
Head
Dowe
Block4
LiDAR Input
Predicted Ouput
(b) Instance Selection Black
Xyp
GT Labels
Loss
SASPManbo
Lecal Progugate
Devile
Siide
Predicted Scores
PE
Mask
b
Local Token
(c) RGSW
Iteration x C
Hilber Interpolation
GT--
Mrsh
Rot
Reverse
Sparse2Lincar
x'
RGSW
PS
Xy
ConviD
DwCCony
Xy.o
Rot 0g
Top-k
SSSS
Reurrange
Lincar2Sparse
Add
Concati
Monris Muliiply
Interpolate
Scmantic-
State Spatial
Empty
Background
Foreground
Train-unly
SASFMamba
Fusion
Figure The overall framework Fore-Mamba33. (a) Framework: the backbone of Fore- Mamba3D consists of four stages and each contains instance selection block and downsam- pling block. (b) Instance Selection Block: we predict the foreground score for each voxel and select the top-k foreground voxels for further linear encoding.
(c) RGSW: we utilize regional-to-glon sliding window process for iterative encoding, (d) SASFMamba: semantic-assistee and state spatial fusion modules are designed enhance the semantic and geometric recognition of state variables.
3.1 FOREGROUND VOXEL SAMPLING AND FLATTENING
In autonomous driving scenarios, background voxels typically account for substantial portion of the entire scene (e.g., about 80% in the nuScenes or KITTI datasets, as shown in Figure Encoding all nonempty voxels would significantly increase the computational cost and memory usage. To address this challenge, we propose foreground voxel sampling and encoding method. Given voxel features X' R
where L x H W denotes the spatial resolution and indicates the channel dimension, we predict foreground score F for each nonempty voxel via submanifold convolution. Then, we add the positional embedding PE with X' hementtwisely, followed by sparse convolution operator to obtain the updated features. Subsequently. the top- (= a) updated features are sampled to form our foreground features Xy E xDND according to their predicted scores F in a descending order. The parameter N indicates the number of sampled voxels and the background voxels are defined as Xi Then, we resort X2 based on the predefined Hilbert curve.
However, there usually exists 'regional truncation' issue in the Hilbert curve, illustrated the bottom of Figure b)). Voxels (c.g., V1 and U2) that are close in the original 3D coordinates may be far from each other in the sequence, which cannot be resolved by bidirectional encoding in previous works (Zhu et al.] [2024] Zhang et al., (024c). Therefore, we rotate and flatten the original scene along the Z-axis for multiple times ensure the truncated neighbor voxels can be closer in the sequence, such as the v/ and v/g. Specifically, given the grid coordinates of sampled voxels P RN we rotate the entire scene with an angle 0 in the bird's eye view (BEV). The initial coordinate p (z, 2T) is transformed to R(0,p) p) ([æcost + ysind], Lycost zsind) ) =T where [.] indicates the floor function. Subsequently. the voxels are linearized according the index of their rotated coordinates in the Hilbert curve template. The flattened features can be defined as:
Xy = H(X,, (R(e, p)lp E P)) E R-N.D.
(1)
where {R(0,p) P} represents the rotated coordinates set and H(-) refers to the Hilbert curve interpolation function. The sequence features Ny within different 0; are fed into the subsequent encoding modules, followed by sum and multi-layer perception (MIP) operation. The encoded
4

---

Published as a conference paper at ICLR 2026
features then concatenated with the background features X, to produce the final output X', The above process is formulated as follows:
x = Caa[MML ,,,(X
(2)
1=1
where r denotes the rotation times (default as 2). The approach effectively alleviates the regional truncation issue in the Hilbert curve template and improves the robustness of the model across different viewpoints.
3.2 REGIONAL-TO
GLOBAL TO-LLOBAL SLIDING WINDOW STRATEGY
To address the response attenuation in the foreground voxels across different instances, conduct a RGSW strategy for effectively propagating information from the region to the entire sequence, as illustrated in Figure c). We first divide the N-length input sequence X into M patches to enable the parallel computation for efficiency. However, due to the causal and auto-regressive nature of Mamba, the model inherently prevents each voxel from accessing later contextual information within the sequence. To compensate for this limitation and eliminate the need for computationally intensive bidirectional encoding, introduce local token T; E R inserted into the end of each patch to acquire the expanded feature R M & +1)x which is further sent our SASFMamba model obtain the encoded feature z On account of the autoregression property of linear modeling, the encoded local token T naturally aggregates the comprehensive regional information within the patch 24- Subsequently, we propagate this summarized context T back to the preceding voxel features within the patch via similarity- -based weighting, defined as:
Fi. = F. + Sim(1i. T.) T
where 2i5 E R denotes the j-th vector in the i-th patch and Sim(-) refers to the cosine similarity.
Although the above approach ensures regional information within each patch be captured, in- teraction between patches remains unaddressed. To facilitate global understanding, we implement a sliding window mechanism to update the patch by combining the later half of z; with the former half of 1 1 to obtain the new sliding patch x," formulated as:
N x = CCul(sil22) :), T 2M
N
where 217 represents the middle position in z Given the obtained x², it is further fed into SASF- Mamba for information propagation. The above process repeated for t times to enable information propagation across patches. The technique enables information propagation across patches, which a critical limitation prevalent the previous group-based approaches.
N
3.3 SASFMAMBA FOR LINEAR ENCODING
To mitigate the limited contextual representation caused by incomplete and imprecise foreground sampling, we incorporate both geometric and spatial cues and design the SASFMamba encoder. This proposed encoder consists of two critical components: semantic-assisted fusion (SAF) and state-spatial fusion (SSF).
3.3.1 SEMANTIC-ASSISS FUSION
As illustrated in Figure dd) given the input sequence x, we first employ lightweight MLP to pre- dict the semantic categories S. Simultancously, feed into SSM to obtain the state variables h. The relationship between and a standard SSM is formulated as:
h4 BABB A
II A.
(5)
where h; and Fj denote the i-th state variable and the j-th input variable, respectively. A, and B, represent the discretized state-transition and input matrices.
5

---

Published as a conference paper at ICLR 2026
To incorporate semantic guidance, we perform semantic rearrangement on h. Specifically, reorder the state variables h by grouping them according to their predicted categories S, while strictly preserving the original relative order within each category sequence. This process aligns voxels with similar semantics regardless of their original positions. Subsequently, a 1D convolution with effective receptive field (ERF) is applied this rearranged sequence aggregate semantic context, followed by reverse operation to restore the original voxel order, yielding the updated state variables h'.
We provide the following derivation to theoretically justify how SAF facili- tates longerange information interaction. According to the above process, the updated state variable hg (at original index i) can be formulated as weighted sum of its semantic neighbors:
Theoretical Analysis:
Ne(i)
h4 Eaghneeo Σ KEK 5=1
Σ
ax A. (.) Byag,
where K- (- K} denotes the ERF and K refers kernel size in the convolution operation. Crucially, (i( represents the original index of the feature that is spatially adjacent to i in the rearranged semantic domain. 0g the learnable convolution weight. By expanding (N),
calculate the association score Mij between the updated state li and the input zj as:
Maj = Σ aLA By; Ki, = {1 E JC| Nx(1) 5}.
(7)
REKiL
where K., denotes the subset of the kernel where the semantic neighbor's original index Nx(1)) appears after the input index j. From Equation 7 it is evident that if there exists a valid k E K,,,- then the cumulative transition Ax J.N..()) # O. Since or and By are non-zero, this ensures M.j * 0. This theoretically demonstrates that the SAF module enables the current state hi to effectively capture information from distant inputs 2j (where j < (i) that share similar semantics, overcoming the locality bias of standard lincar encoders.
3.3.2 STATE SPATIAL FUSION.
To solve the geometric distortion from 3D to ID sequence, we propose the SSF module. Given the state variables h' after the SAF module, we first map each variable into the 3D space based on its original coordinate in the feature space to create new sparse 3D tensor in the state space. Since there are inherent spatial deficiency in the linear modeling, we apply simensionnwise convolution (He al., 2024) with large kernel along different axis achieve spatial recognition. Then, we flatten the 3D representation back into sequence h".. Thus, the overall design of the state space fusion can be expressed as follows:
h" (D ))),
where "L2S" and "S2L" refer to the transformation from linear features to a sparse 3D tensor and the reverse process, respectively. "DwConv" denotes the dimension-wise convolution operation. Based on the observation equation in SSM, we multiply h" by the dynamic output matrix Ć to acquire the output features z'. The mechanism in the SSF is similar SAF, which can ensure the non-casual and geometrically correlated encoding.
3.4 LOss FUNCTION
To improve the precision of the scores FF in foreground prediction and semantic category S in the SAF module, we design two loss functions L, and L, for supervision. Moreover, we define voxels within enlarged bounding boxes-obbainn
by expanding the original boxes by 0.5 m along the X/Y axes and 0.25 along the 7. axis-as foreground, in order to preserve ambiguous boundary information. Considering the number imbalance between the predicted categories, we select the focal loss to calculate C and L., which are defined as:
C, Σ B1 - y:loopp)))
(9)

---

Published as a conference paper at ICLR 2026
where C' defines the number of output classes. The focusing parameter y is introduced to em- phasize -o-cssssyy samples. 8 and p; represent the weight and probability of the i-th category. 9: indicates whether the ground-truth label matches the i-th category. After acquiring Cy and L, from the 31) backbone, we integrate them with the classification loss Lits and regression loss freg from the detection head with weight w = 2. Lats and freg are computed with the widely used cross-entropy loss and smooth-LI loss, respectively (Yan et al. 2018 Shi et al., (2020a). The final loss C is defined as:
C = w(C L.) - Lets + freg.
(10)
4 EXPERIMENTS
4.1 DATASET AND METRICS
nuScenes dataset. The nuScenes dataset (Caesar al. 2020) is large-scale 3D detection bench- mark with perception range of up to 100 meters. The dataset includes 700 training scenes, 150 validation scenes, and 150 testing scenes, Detection performance is evaluated using the normal met- rics: mean average precision (mAP) and the nuScenes detection score (NDS), as previous works Yin et al.
2021 Bai al. 2022).
KITTI dataset. The KITTI dataset (Geiger et al.) 2012) contains 3,712 paired training samples, 3,769 paired validation samples, and 7,518 paired lest samples. The standard metries include 3D and BEV average precision (AP) under 40 recall thresholds (R40), with three different difficulty levels. this work, we evaluate our method across three major categories with IoU thresholds of 0.7 for cars, and 0.5 for both pedestrians and eyclists, which is the same previous methods (Shi ct al.
2020a Yan et al. 2018 Zhou & Tuzel 2018).
Waymo Open Dataset. The Waymo dataset (Sun al. 2020 includes 230k annotated samples and the whole scene covers a large reception range of 150 meters. The evaluation metrics contain the average precision (AP) and its variant by weighted heading accuracy (APH). The detection difficulty of each object is divided into two categories: Level (L1) for objects containing more than five points and Level 2 (L2) for those containing at least one point.
4.2 IMPLEMENTATION DETAILS
Our model is trained by the Adam optimizer on eight NVIDIA RTX 4090D GPUs with cosine learning rate of 3e-3. For the nuScenes dataset, our method builds upon the TransFusion framework Bai et al. 2022) and it is trained for 36 epochs with 2 batch size. For the KITTI dataset, we replace the 3D backbone of the SECOND network (Yan et al. 2018) with our proposed method and train the model for 50 epochs with a batch size of 4. For the Waymo dataset, our method is employed based on the CenterPoint (Yin et al., 2021) with the channel dimension equal to 64 and it is trained for 24 epochs with 2 batch size. We adopt similar data augmentation configuration in the training stage as previous works (Chen et al.] 2023 Yan et al.] 2018 Yin et al., 2021 Bai et al., 2022). Notably, the Hilbert curve templates generated offline and directly loaded into each stage for both training and inference acceleration.
4.3 COMPARISON WITH THE PREVIOUS METHONS
Performance on the nuScenes dataset. In Table I we compare Mamba3D with previous methods on both the nuScenes validation and test sets. For fair comparison, all experiments are conducted without class-balanced ground-truth sampling or model ensembling. Fore-Mamba3D achieves state-of-the-arr performance among all existing LiDAR-only approaches.
Performance on the KITTI dataset. We compare Fore- MateMMam with other methods based on various 3D backbone architectures, including MLP (Shi et al. 2019a), SpCNN (Zhou & Tuzel 2018). Transformer (Wang et al. 2023). and Mamba (Liu et al. 2024b), to highlight the effective- ness of model's structure. The result in Table 2 shows that our method achieves satte-of-the-nn performance on the KITTI dataset, yielding an average improvement of 1.7% over the second- best method Maxel (Zhang et al. 2024c).
7

---

Published as a conference paper at ICLR 2026
Table 1: Performances on the nuScenes validation and test set. 'C.V., 'Ped., and 'T.C.' are short for construction vehicle, pedestrian, and traffic cone, respectively. The first and second best results are in bold and underlined, respectively. All the results are conducted without class-baland ground- truth sampling (CBGS).
Method
Present
mAP
NDS
Car
Track
Bus
Trailer
C.V.
Fed.
Moter.
Bike
T.C.
Bariier
Performance the validattin dataset
Contec
ein al.
2021
CVPR21
59.2
66.5
84.9
574
70.7
38.1
85.1
59.0
42.0
69.8
68.3
TransFusion
CVPR22
65.5
70.1
$6.9
60.8
73.1
43.4
25.2
87.5
72.9
57.3
77.2
70.3
VoxelNeXx
Che
CVPR23
64.5
70.0
84.6
53.0
64.7
55.8
28.7
85.8
73.2
45.7
79.0
74.6
DSVT
Wang clil M775
CVPR23
66.4
71.1
62.6
75.9
42.1
25.3
88.2
74.8
58.7
77.9
71.0
HRDNNet
Zhing B. 2174h
NIPS23
66.7
714
87.7
60.5
77.8
50.7
28.9
87.1
74.3
56.8
76.3
66.9
SAFDNe
CVPR24
66.3
71.0
87.6
60.8
78.0
43.5
26.6
87.8
75.5
58.0
75.0
69.7
Mambl-Mambi
77.1
LJON
(L aie Miz440
NIPS24
68.0
72.1
87.9
64.9
77.6
44.4
28.5
89.6
75.6
59.4
80.8
71.6
FSHNer
Liu ef 2012
CVPR25
68.1
71.7
58.7
61.4
79.3
47.8
26.3
89.3
76.7
60.5
78.6
72.3
Fore-
68.4
72.3
88
Performance on the test dataset
HEDNeer Zhang et ml 30744
NIPS24
67.7
72.0
87.1
56.5
70.4
63.5
33.6
87.9
70.4
44.8
85.1
78.1
SAFDN&
Ziaan
Moncl-Mamba Zllng cl 20246
NIPS24
69.0
73.0
86.8
57.1
68.0
63.2
35.4
89.5
74.7
50.8
86.9
77.3
LION (Lit et S.. 20230
NIPS21
69.8
73.9
87.2
61.1
68.9
65.0
36.3
90.0
74.0
49.2
$7.3
79.5
Fore
70.1
74.0
87.1
60.5
70.5
63.9
34,3
73.7
53.1
88.6
78.6
Table 2: Comparison of the performance on the KITTI validation set with an average recall of 11.
 indicates that the result is reproduced by us. The first and second best results are in bold and underlined, respectively.
Methods
BackBone
Car Mod
Pedestrion
Cyclists
Fasy
Hard
Easy
Easy
PointPillars
(Lang et
2019
MLP
79.1
75.0
68.3
52.1
43.5
41.5
75.8
59.1
52.9
IA-SSD
Zhung al 2022
MLP
88.3
80.1
75.0
46.5
39.0
35.6
78.4
61.9
$5.7
VoxelNet
Zaou & Luzer
2018
SpCNN
77.5
65.1
57.7
39.5
33.7
31.5
61.2
48.4
44.4
DSVT
Wang 2075
Transformer
87.8
77.8
76.8
06.1
59.7
55.2
66.7
63.2
DGT-DDe
Ken ot 31
211p
Transformer
89.6
80.6
78.8
82.1
68.9
61.0
LION
Lu X074h
Mamba
88.6
78.3
77.2
67.2
60.2
55.6
83.0
68.6
63.9
VoxeIMamha
Zhiing mil Z024c
Mamba
89.1
80.8
78.
66.0
59.7
53.7
84.2
69.1
64.8
Hore-Mambe3D (Ours)
Mamba
90.3
82.2
79.5
67.8
62.2
57.0
86.4
69.5
66.3
Table 3: Performance on the Waymo dataset (trained on the 20% training dataset and evaluated on the full validation dataset). t denotes that the results of previous work are implemented by OpenPCDet (OpenPCDet Development Team 2020). The first and second best results are in bold and underlined. respectively.
Vehicle
Pedestrian
Cyclist
Methods
AP
APH
AP
APH
AP
APH
mAP
L2
LI
L2
L2
L2
SECOND
Yan al.
2018
71.0
62.6
70.3
62.0
65.2
57.2
54.2
47.5
55.6
53.5
58.3
PVECONN
Shi et al
2021a
75.4
67.4
74.7
66.8
72.0
63.7
61.2
54.0
65.8
63.4
64.3
61.8
64.8
Part-A2
Shr al.
2119
74.7
65.8
74.1
65.3
71.7
62.5
62.2
54.1
62.8
64.1
ConterPoint
Yin ét
2021
71.3
63,2
70.8
62.7
72.1
64.3
65.5
58.2
68.7
66.1
67.4
64.9
64.5
VoxeI-RCNN
Dong al
2021
76.1
68.2
75.7
67.7
78.2
69.3
72.0
63.6
70.8
68.3
69.7
67.2
68.6
IA SSD
Zhang
M024
70.5
61.6
69.7
60.8
69.4
60.3
58.5
50.7
67.7
65.0
65.3
62.7
62.3
LION
Lau et al. 20246
67.0
66.6
75.4
70.2
71.9
71.0
71.4
Fore Mare-Mamha3D (Ours)
76.3
67.8
75.8
67.4
75.6
75.6
70.0
72.8
72.2
71.3
70.6
71.9
Performance Waymo dataset. We further evaluate Fore-Mamba3D on subset of the Waymo Open Dataset, training with only 20% the training set and testing on the full validation set. As shown in Table 3 our method achieves competitive results with 71.9% mAP in the L2 level, out- performing the CenterPoint baseline (Yin et al., (2021) by 7.4%. Moreover, all the results in the L1 level surpass those of previous methods, further highlighting the effectiveness of our approach.
4.4 ABLATION STUDY
In this section, we conduct a series of ablation study on the nuScenes and KITTI validation sets to evaluate the effectiveness and efficiency of all the proposed components.
8

---

Published as a conference paper at ICLR 2026
Foreground Sampling and Efficiency. The results in Table demonstrate the effectiveness of our foreground voxel sampling strategy across different sampling ratios, along with its impact on computational efficiency. Specifically, //2, and select the top-k (=a) features as the foreground. When a equals to 0.2, our model achieves the best performance, with 72.3 NDS on the nuScenes dataset and 82.2 Car mAP the KπTTI dataset. The performance gain can be attributed to the value of approximating the true distribution. Moreover, further compare the efficiency under single-GPU setting with batch size of 1. Our approach reduces FLOPs by 43.7% and increases FPS by 23.9% compared with the LION backbone. The FLOPs in Mamba are determined by the sequence length, hidden dimension, selective-son operators, and causal convolution components. In addition, the supervised foreground scores ensure the alignment between the prediction and the ground-truth. Visualization of the heatmap in the BEV is provided in the Appendix.
rotate the entire
around the Z-axis with 0 = 0 and
Ablation on the Combination of Different Components. To clearly demonstrate the effective- ness of each component proposed in method, fix the sampling ratio at 0.2 and progres- sively integrate individual components into the vanilla Mamba encoder. The results are depicted in Table Employing Hilbert curve flattening with multiple rotations achieves certain detection accu- racy, Moreover, the RGSW strategy strengthens long-range interactions in the sequence, leading to further performance gains. The SAF and SSF modules focus on capturing semantic and geometric relationships in the foreground representations, respectively. When both of them are incorporated, our model achieves the final promising result.
Regional-to-Glon Sliding Window. As depicted in Table 6 explore the validity of the RGSW strategy in mitigating response attenuation. Without either encoding mechanism, the model exhibits suboptimal performance, achieving only 70.2% mAP. Introducing local token insertion or the sliding-window encoding independently improves detection performance by 0.6% and 0.7%, respectively. Notably, the global sliding-window approach provides more remarkable gains for large objects, such as cars (1.2%)). Meanwhile, the local token insertion is particularly benefi- cial for smaller and sparser instances, improving the detection of pedestrians (+0.93%) and eyclists (+0.35%). Furthermore, applying more than two iterations of the sliding strategy yields negligible additional gains, indicating saturation point. Consequently, we adopt 2 as the default setting to achieve a trade-off between accuracy and efficiency.
Table 5: Performance the various combi- nations of proposed components. 'HF' de- fines the Hilbert flattening.
Table 4: Efficiency under different sampling ratios and comparison with LION (Liu et al. 2024b).
a
Car
KITTI Ped.
Cyc.
nuSeenes mAP
NDS
FLOPs (G)
FPS
HF
RGSW
SAF
SSF
Car
Ped.
Cyc.
0.1
0.2
82.2
81.0
62.2
60.9
69.5
68.7
68.4
67.4
72.3
71.0
26.04
22.62
67
70
79.4
59.2
66.0
0.5
81.8
61.1
70.0
68.0
71.8
38.62
58
80.6
60.5
66.8
1.0
81.5
61.5
69.7
67.8
71.6
52.17
50
81.8
61.9
67.3
81.0
61.3
68.2
LION
78.3
60.2
68.0
72.1
82.6
62.2
Table 6: Ablation on the regional-to-glee slid- ing window. 'Regional' denotes the insertion of local tokens. 'Global' represents the slide win- dow operation. 't' denotes the iteration number.
Table 7: Comparison of applying different ker- nel sizes in the SAF module. 'Sim' denotes the similarity score between the predicted matrix M and the label.
Regional
Global
Car
Ped.
Cyc.
mAP
80.97
60.94
68.66
70.2
K
mAP
NDS
Mems (G)
FLOPs (G)
Sim
81.56
61.87
69.01
70.8
68.1
71.8
22.6
0.27
v
82.12
61.54
68.91
70.9
68.3
72.1
23.9
0.45
7
68.4
72.3
5.5
26.0
0.64
82.16
62.23
69.46
71.3
15
68.2
72.4
30.4
0.68
81.45
61.95
69.84
71.1
Effectiveness of the State Variable Fusion. We also investigate the effectiveness of the state vari- able fusion in enhancing the contextual understanding. As illustrated Figure we visualize the association matrix M in vanilla SSM, semantic-assisted strategy with different kernel sizes, and

---

Published as a conference paper at ICLR 2026
(a) SSM
(b) SAF (K
(e) (K 15)
(d) Label
Figure 3: The association matrix in (a) vanilla SSM, (b, c) semantic-assisted strategy with different kernel sizes K and (d) the semantic association labels. We define the semantic labels follow Gaussian distribution for the distance.
the semantic association labels. The association matrix M the vanilla SSM a lower triangular matrix. After applying semantic-assisted fusion strategy, the current output can attend to sub- sequent inputs within similar semantics. Furthermore, as the kernel size increases, the response in the matrix extends further. This demonstrates that model could focus the semantic rela- tionship across the entire sequence. We further compare the detection performance under different kernel sizes in Table 7 which motivates adopt a kernel size of 7 as the default setting balance effectiveness and efficiency.
CONCLUSION
In this work, we propose Fore-Mamba3D novel framework that integrates foreground voxel selection with a hierarchical regional-to-global sliding window strategy to effectively capture inter- instance dependencies in 3D scenes. Furthermore, we introduce the SASFMamba module to en- hance both semantic cues and geometric structures, enabling non-causal interactions in the whole linear sequence. Our approach demonstrates substantial performance improvements over existing Mamba-based foreground-basee methods, achieving state-of -frc--r-thh-rtttt results across various tonomous driving benchmarks. Comprehensive ablation studies further validate the effectiveness of each proposed component, highlighting their contribution in advancing the 3D) detection task.
ACKNOWLEDGMENTS
This work is partially supported by National Natural Science Foundation of China (Grant No. 62376153, 62402318, 242990200676, 62376252), Zhejiang Province Lead- ing Geese Plan(2025C02025. 2025C01056) and Zhejiang Province Province-Land Synergy Program(2025SDXXOO009))
REFERENCES
Xuyang Bai, Zeyu Hu, Xinge Zhu, Qingqiu Huang, Yilun Chen, Hongbo Fu, and Chiew-Lan Tai. Transfusion: Robust lidar-camera fusion for 3d object detection with transformers. CVPR, pp. 1090-1099,
Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: multimodal dataset for autonomous driving. In CVPR, pp. 11621-11631, 2020.
Yukang Chen, Yanwei Li, Xiangyu Zhang, Jian Sun, and Jiaya Jia. Focal sparse convolutional networks for 3d object detection. CVPR, pp. 5428-5437, 5437, 2022.
Yukang Chen, Jianhui Liu, Xiangyu Zhang, Xiaojuan Qi, and Jiaya Jia. Voxelnext: Fully sparse voxeinet for 3d object detection and tracking. CVPR, pp. 21674-21683, 2023.
Jiajun Deng, Shaoshuai Shi, Peiwei Li, Wengang Zhou, Yanyong Zhang, and Houqiang Li. Voxel r-cnn: Towards high performance voxel-hased 3d object detection. In AAAI, volume 35, pp- 1201-1209, 2021.
Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In CVPR, pp. 3354-3361. IEBE, 2012.
10

---

Published as a conference paper at ICLR 2026
Sorin Grigorescu, Bogdan Trasnea, Tiberiu Cocias, and Gigel Macesanu. A survey of deep learning techniques for autonomous driving. Journal of field robotics, 37(3):362-388 -36, 2020.
Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00755 2023.
Faiza Gul, Wan Rahiman, and Syed Sahal Nazli Alhady, A comprehensive study for robot navigation techniques. Cogent Engineering, 6(1):1632046, 2019.
Agrim Gupta, Silvio Savarese, Surya Ganguli, and Li Fei-Fei. Embodied intelligence via learning and evolution. Nature communications, 12(1))5721, 2021.
O Hayeon, Chanuk Yang, and Kunsoo Huh. Sesame: Simple, easy 3d object detection with point- wise semantics. In ACCV, pp. 211-227. Springer, 2024.
Chenhang He, Ruihuang Li, Guowen Zhang, and Lei Zhang. Scatterformer: Efficient voxel trans- former with scattered linear attention. In European Conference Computer Vision, pp. 74-92. Springer, 2024.
David Hilbert and David Hilbert. Über die stetige abbildung einer linie auf ein fläähenstüück. Drit- ter Band: Analysis- Grundlagen der Mathematik- Physik Verschiedenes: Nebst Einer Lebens- geschichte, pp. 1-2, 1935.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8): 1735-1780, 1997.
Keli Huang, Botian Shi, Xiang Li, Xin Li, Siyuan Huang, and Yikang Li. Multi-modal sensor fusion for auto driving perception: A survey, arXiv preprint arXiv:2202.022700 2022.
Alex H Lang, Sourabh Vora, Holger Caesar, Lubing Zhou, Jiong Yang, and Oscar Beijbom. Point- pillars: Fast encoders for object detection from point clouds. In CVPR, pp. 12697-12705, 2019.
Jiale Li, Hang Dai, Ling Shao, and Yong Ding. From voxel to point: lou-guided 3d object detection for point cloud with voxel-to-point decoder. In ACM MM, pp. 4622-4631, 2021.
Dingkang Liang, Xin Zhou, Xinyu Wang, Xingkui Zhu, Wei Xu, Zhikang Zou, Xiaoqing Ye, and Xiang Bai. Pointmamba: A simple state space model for point cloud analysis. arXiv preprint arXiv:2402.1077 10739, 2024.
Baoyuan Liu, Min Wang, Hassan Foroosh, Marshall Tappen, and Marianna Pensky. Sparse convo- lutional neural networks. In CVPR, pp. 806-814 2015.
Shuai Liu, Mingyue Cui, Boyang Li, Quanmin Liang, Tinghe Hong, Kai Huang, and Yunxiao Shan. Fshnet: Fully sparse hybrid network for 3d object detection. In Proceedings of the Computer Vision and Paltern Recognition Conference, pp. 8900-8909, 2025.
Yue Liu, Yunjie Tian, Yuzhong Zhao, Hongtian Yu, Lingxi Xie, Yaowei Wang. Qixiang Ye, and Yunfan Liu. Vmamba: Visual state space model. arXiv preprint arXiv:2401.10166 2024a.
Zhe Liu, Xin Zhao, Tengteng Huang, Ruolan Hu, Yu Zhou, and Xiang Bai. Tanet: Robust 3d object detection from point clouds with triple attention. In AAAI, volume 34, pp. 11677-11684, 2020.
Zhe Liu, Jinghua Hou, Xinyu Wang, Xiaoqing Ye, Jingdong Wang, Hengshuang Zhao, and Xi- ang Bai. Lion: Linear group rnn for 3d object detection in point clouds. arXiv preprint arXiv:2407.18233 2024b.
Jiageng Mao, Shaoshuai Shi, Xiaogang Wang, and Hongsheng Li. 3d object detection for tonomous driving: A comprehensive survey. IJCV, 1999-9999,,-196, 2023.
Tong Ning, Ke Lu, Xirui Jiang, and Jian Xue. Mambadetr: Query-based temporal modeling using state space model for milti-view 3d object detection. arXiv preprint arXiv:2411.13625 2024.
Q OpenPCDet Development Team. Openpedet: An open-source toolbox for 3d object detection from point clouds, 2020.
11

---

Published as a conference paper at ICLR 2026
Jack A Orenstein. Spatial query processing in an object-oriented database system. In ACM SIG- MOD, pp. 226-336, 1986.
Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung. Matteo Grella, et al. Rwkv: Reinventing mns for the transformer arXiv preprint arXiv:2305. 13048, 2023.
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In CVPR, pp. 652-660, 2017a.
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++:: Deep hierarchical feature learning point sets in metric space. Advances in NeurIPS, 30, 2017b.
Siyuan Ren, Xiao Pan, Wenjie Zhao, Binling Nie, and Bo Han. Dynamic graph transformer for 3d object detection. Knowledge-Ban Based Systems, 259: 10085, 2023.
Hualian Sheng, Sijia Cai, Yuan Liu, Bing Deng, Jiangiang Huang, Xian-Sherp Hua, and Min-Jian Zhao. Improving 3d object detection with channel-wise transformer. ICCV, pp. 2743-2752, 2021.
Shaoshuai Shi, Xiaogang Wang, and Hongsheng Li. Pointrenn: 3d object proposal generation and detection from point cloud. In CVPR, pp. 770-779, 20199a,
Shaoshuai Shi, Zhe Wang, Xiaogang Wang, and Hongsheng Li. Part-a" net: 3d part-aware and ag- gregation neural network for object detection from point cloud. arXiv preprint arXiv: arXiv: 1907.03670, 2(3), 2019b.
Shaoshuai Shi, Chaoxu Guo, Li Jiang, Zhe Wang, Jianping Shi, Xiaogang Wang. and Hongsheng Li. Pv-renn: Point-voxel feature set abstraction for 3d object detection. In CVPR, pp. 10529-10538, 2020a.
Shaoshuai Shi, Zhe Wang, Jianping Shi, Xiaogang Wang, and Hongsheng Li. From points to parts: 3d object detection from point cloud with part-aware and part-aggregation network. TPAMI, 43 (8):2647-2664, 2020b.
Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, et al. Scalability in perception for au- tonomous driving: Waymo open dataset. In CVPR, pp. 2446-2454, 2020.
Yutao Sun, Li Dong, Shaohan Huang, Shuming Ma, Yuqing Xia, Jilong Xue, Jianyong Wang, and Furu Wei. Retentive network: A successor transformer for large language models. arXiv preprint arXivv2307.08622. 2023.
Vaswani. Attention all you need. Advances NeurlPS, 2017.
Sourabh Vora, Alex H Lang, Bassam Helou, and Oscar Beijbom. Pointpainting: Sequential fusion for 3d object detection. In CVPR, pp. 4604 4612, 2020.
Haiyang Wang, Shaoshuai Shi, Ze Yang, Rongyao Fang, Qi Qian, Hongsheng Li, Bernt Schiele, and Liwei Wang. Rbgnet: Ray-based grouping for 3d object detection. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 1110-1119, 2022a.
Haiyang Wang, Chen Shi, Shaoshuai Shi, Meng Lei, Sen Wang, He, Bernt Schiele, and Liwei Wang. Dsvt: Dynamic sparse voxel transformer with rotated sets. In CVPR, pp. 13520-13529, 2023.
Yue Wang, Vitor Campagnolo Guizilini, Tianyuan Zhang, Yilun Wang. Hang Zhao, and Justin Solomon. Detr3d: 3d object detection from multi-view images via 3d-to-2d queries. In CoRL, pp. 190-191. PMLR, 2022b.
Hai Wu, Jinhao Deng, Chenglu Wen, Xin Li, Cheng Wang, and Jonathan Li. Casa: A cascade attention network for 3-d object detection from lidar point clouds. TGRS, 60:1-11, 2022.
12

---

Published as a conference paper at ICLR 2026
Zhefan Xu, Xiaoyang Zhan, Yumeng Xiu, Christopher Suzuki, and Kenji Shimada. Onboard dynamic-objed detection and tracking for autonomous robot navigation with rgb-d camera. IEEE Robotics and Automation Letters, 9(1):651-658 2023.
Yan Yan, Yuxing Mao, and Bo Li. Second: Sparsely embedded convolutional detection. Sensors, 18(10)3333,, 2018.
Tianwei Yin, Xingyi Zhou, and Philipp Krahenbuhl. Center-based 3d object detection and tracking. In CVPR, pp. 11784 11793, 2021.
Gang Zhang, Jumnan Chen, Guohuan Gao, Jianmin Li, Si Liu, and Xiaolin Hu. Safdnet: A simple and effective network for fully sparse 3d object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14477-14486, 2024a
Gang Zhang, Chen Junnan, Guohuan Gao, Jianmin Li, and Xiaolin Hu. Hednet: A hierarchical encoder-decoder network network for 3d object detection in point clouds. Advances NeurIPS, 36, 2024b.
Guowen Zhang, Lue Fan, Chenhang He, Zhen Lei, Zhaoxiang Zhang, and Lei Zhang. Voxel mamba: Group-free state space models for point cloud based 3d object detection. arXiv preprint arXiv:2406. 10700, 2024c.
Peng Zhang. Xin Li, Xin Lin, and Liang He. A new literature review 3d object detection on autonomous driving. Journal Artificial Intelligence Research, 82:973-1015, 2025.
Tingyu Zhang, Jian Wang, and Xinyu Yang. Boosting 3d object detection with density-aware semantics-augmented set abstraction. Sensors, 23(12):5757, 2023.
Yifan Zhang, Qingyong Hu, Guoquan Xu, Yanxin Ma, Jianwei Wan, and Yulan Guo. Not all points are equal: Learning highly efficient point-based detectors for 3d lidar point clouds. Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 18953-18962, 2022.
Yin Zhou and Oncel Tuzel. Voxelnet: End-to-end learning for point cloud based 3d object detection. In CVPR, pp. 4499, 2018.
Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang. Wenyu Liu, and Xinggang Wang. Vi- sion mamba: Efficient visual representation learning with bidirectional state space model. arXiv preprint arXiv:2401. 009417, 2024.
13

---

Published as a conference paper at ICLR 2026
APPENDIX
1 PERFORMANCE IN DIFFERENT MODEL. SCALES
To investigate the impact of the channel dimension in the 3D backbone, compare the performance of different model scales on the KITTI dataset. As shown Table g the model with a channel dimension of 128 outperforms the model with channel dimension of 64 by 1.25% and 1.31% mAP in 3D and BEV moderate level detection, respectively.
3D
BEV Mod
Model Scale
Easy
Mod
Hard
Easy
Hard
-T
88.53
80.91
78.46
90.87
87.78
87.27
-B
90.32
82.16
79.54
92.27
89.09
88.11
(+1.80)
(+1.25)
(+1.08)
(+1.40)
(+1.31)
(+0.84)
Table 8: Performance on different model scales. '-T' indicates the channel dimension in the back-
bone 64, and '-B' denotes the channels dimension set as 128.
.2 DERIVATION OF STATE VARIABLE FUSION
Defining the initial state bn in state transition as zero, we rewrite the state transition equation and observation equation in the recursive form as follows:
h, = 1, + B,T;
A,4 + 1 + Brx;
-A;A,-14442 1/4 + A, ri:-1 + BiFi
=A A, - A.... A2 +
(1I)
+ A,A,-1 Bi-22i-- - A.Bi-II;- + Bi
Σ A,) = = Byazi
JSi (=j+1
JSi
V/ =C,h, + Dix;
The output matrix D, in the observation equation is also omitted, so the output in Equation 6 can be simplified as follows:
= C,hss ECA
(12)
JSA
Assuming the matrix M' indicates the association between the output sequence y [y1, V2. Vnl and the input sequence x [I1, .2, .In), which is defined by yı M, Combining Equa- tion 7 with the observation equation, M', can be calculated as:
M Σ KEK'
(13)
where the (t) could define the semantic or geometric neighbor index. Actually, there usually exists satisfies (t) > j to ensure A (.N and M,,, not equal to 0. Therefore, when the (2) represents the index of a semantically neighboring feature, the output is adaptively recalibrated to prioritize subsequent input features x, exhibiting high semantic similarity with 24. In contrast, when (i) indicates the index of geometrically neighboring feature, the output y, attends to the later input žj with spatial proximity to Zz.
Moreover, visualize the process from the h to /t' in the SAF module, as shown in the Figure 4 The entire pipeline contains the rearrangement, the 1D convolution operator, and the reverse process. The illustration also corresponds the transform from Equation 5 to Equation 6.
.3 VISUALIZATION ON THE FOREGROUND PREDICTION
14

---

Published as a conference paper at ICLR 2026
N-1(0 No(O
N.(
1+
1+3
rearrangement
a-1
Conv X = [-1,0,1]
241
reverse
h'
category
category
category
Figure 4: The detailed pipeline in the SAF module, which contains the rearrangement, the ID convolution, and the reverse process.
1.0
We visualize the BEV heatmap of the predicted foreground scores and compare it with the ground truth (GT). Notably, we choose the foreground scores from the third stage in the backbone and accumulate them along the 7-axis. As illustrated in Figure 5 the predicted scores closely match the ground truth values, demonstrating clear separation be- tween the foreground and background voxels. This demonstrates the high accuracy of our foreground prediction, thereby validat- ing the effectiveness of the
0.9
0.8
0.7
0.6
0.5
(a) GT
0.4
0.2
0.1
0.0
(b) Prediction
Figure 5: The accuracy in foreground scores prediction. We accumu- late the scores along the z-axis and visualize the heatmap in the BEV. proposed foreground sampling strategy.
.4 MORE QUALITATIVE COMPARISON
Results on the KITTI test dataset. We also evaluate Fore-Mamba3D on the KITII dataset. The result in Table 9 shows that our method surpasses the previous work and achieves promising results in LEDAR-ooly 3D detection.
Impact of the Rotation and Serialization Strategy. As shown in Table 10 we freeze a = 0.2 and change the rotation angle. The result with (0, x x/2) outperforms the result without rotation or with single rotation, demonstrating the necessity of multiple rotations in flattening for linear encoding. The comparison on the bottom of Table 10 indicates that the simple convolution module outperforms the foreground merging module (Liu et al. 2024b) in the downsampling block. Fur- thermore, we evaluate the effectiveness of different serialization strategies in Table II Compared with the X/Y-raster the Z-order curve, the Hilbert curve enables the best spatial continuity for precise object detection. Meanwhile, the performance different Hilbert encoding strategies shows better adaptability to rotation scheme compared with bidirectional selection.
15

---

Published as a conference paper at ICLR 2026
Table Comparison of the performance on the KITTI test dataset with an average recall of 40. All the results are reported in the moderate difficulty level.
Car
Ped.
Cyc.
Methods
3D
BEV
3D
BEV
3D
BEV
PointRCNN
Shi et al
2019a
75.64
87.39
39.37
46.13
58
67.24
PointPillars
Lang el m
2019
74.99
86.10
43.53
50.23
59.07
62,25
VoxelNett
Zhou & Tuzel
2018
65.11
79.26
33.69
40.74
48.36
54.76
SLSCOND Yan
2018
73.66
42.56
53.85
TANet
(Lu al. 2720
75.94
86,54
44.34
51.38
59.44
63.77
SeSame Haycon et al
2024
76.83
87.49
35.34
41.22
54.46
61.70
Ours
77.88
88.06
45.60
50.68
63.97
68.02
Table 10: Ablation the foreground sampling and flattening. "DS' represents the downsam- pling ways. "a" denotes the sampling ratio and Rot. 0' is the rotation angle along the Z-axis.
Table 11 Performance under different scanning patterns and Hilbert encoding strategies.
Scan Ways
mAP
NDS
FLOPs (G)
Lataney (ms) J
DS
Rot,
Car
Ped.
Cyc.
mAP
X/Y-raster
0.2
81.31
62.16
68.23
70.6
Z-order
67.9
71.7
25.9
16.0
0.2
x/2
Hilbert Bid
68.0
72.0
25.4
15.1
0.2
x/2)
82.16
62.23
69.46
71.3
Hilbert Rot
68.4
72.3
26.0
15.48
Merging
0.2
(0. x/2)
81.22
61.71
66.46
69.7
Comparison of different sequence modeling approaches. We also compare the performance different sequence modeling alternatives, which is shown in Table 12 Specifically, we only replace the sequence encoding part with existing methods (e.g. RetNet/Sun al. (2023), RWKV al. (2023). and LSTM Hochreiter & Schmidhuber (1997)) and maintain other components consistent with our original pipeline. We train these alternatives from scratch in the nuScenes dataset with the same training configuration. The Mamba model achieves the highest scores in both NDS and mAP metries. Meanwhile, the computation of Mamba also efficient compared with the recent RetNet and RWKV approaches. Given the theoretically and empirically validated capability of Mamba in sequence modeling. we adopt as the default linear encoder throughout our paper.
Table 12: Comparison of the different sequence modeling, including RetNet, RWKV, LSTM, and Mamba.
Model
NDS
mAP
FLOPs
FPS
RetNet
72.1
67.7
31.12
59
RWKV
71.9
67.1
36.15
47
LSTM
70.8
65.9
23.92
78
Mamba
72.3
68.4
26.04
67
Robustness the foreground sampling noise. As shown in the Table 13 our method exhibits robustness to the noise of foreground scoring. Specifically, we directly replace a portion of the sampled foreground voxels (from 5% 15%) with random background voxels during inference. We then compare the overall detection mAP and NDS results in the nuScenes dataset, well as the accuracy of specific categories in the nuScenes dataset. It can be observed that our model maintains its performance with replacement ratio under 10%, which is a relatively large ratio for real-world applications.
Table 13: Comparison of the different ratios of noise in the sampled foreground voxels.
noise ratio
NDS
mAP
Car
Truck
C.V.
Bus
Trailer
Barrier
Motor
Bike
Ped.
T.C.
w/o
72.3
68.4
88.4
65.2
28.2
48.0
71.2
75.7
57.7
89.3
80.0
5%
71.9
67.9
87.9
65.1
30.2
77.4
48.1
71.5
74.3
56.0
88,9
10%
71.6
67.4
87.4
64.4
30.3
77.0
47.4
72.0
73.2
55.4
88.4
78.1
15%
71.0
66.5
86.7
63.4
30.1
76.4
46.7
72.2
70.8
54.3
87.6
76.9
16

---

Published as a conference paper at ICLR 2026
GT
(b) SECOND
LION
(d) Ours
Figure 6: Visualization of the failure cases on the KITTI dataset. Compared with SECOND Yan al. 2018) and LION (Liu et al. 2024b). our method can predict the targets more consistently with the ground truth.
LION
Ours
Figure 7: Visualization of the failure cases between our method and the previous method in the nuScenes dataset.
Visualization of Distinct Cases. As illustrated in Figure 6 we present qualitative comparison of the detection results produced by our method and several representative baselines across diverse driving scenarios the KITTI dataset. The baseline model SECOND Yaan et al. 2018) frequently misclassifies background clutter, such as trees or poles, pedestrians or cyclists, leading to high false positive rates. Similarly, LION (Liu et al.) 2024b) exhibits misdetections in cluttered scenes, proposed method generates predictions that are highly consistent with the ground truth (GT), significantly reducing spurious detections. These visualizations highlight the superior robustness and precision of our approach, especially in challenging cases involving dense and occluded pedestrian instances. We also visualize representative false positive and false negative cases from the nuScenes dataset, as shown in Figure 7 The green boxes denote the ground-truth annotations, while the red and blue boxes represent the predictions of our model and previous methods, respectively. We observe that our approach occasionally produces false negatives on smaller object categories, such as pedestrians and barriers, and certain confusing scenarios may lead to false positives. Nonetheless, previous method LION also frequently fails on these challenging cases.
often producing false positives when detecting pedestrians and vehicles. In contrast,
17


Reviews:
[{"review_id": "VcvZ9cIFvD", "reviewer": "Reviewer_cXEa", "summary": "This paper introduces a Mamba-based backbone for 3D object detection focused on more effective foreground voxel encoding. The method selects top-scoring foreground voxels, serializes them using a rotation-augmented Hilbert curve, and processes them with RGSW for propagating local and global context, and SASFMamba to enhance context representation via state space modeling. Experiments are performed on nuScenes, KITTI, and Waymo benchmarks, along with ablation studies to justify each component.", "questions": "1. How robust is the method to imperfect, noisy, or ambiguous foreground scoring? Do detection performance or ablations degrade gracefully if the scoring network fails to highlight the correct voxels?\n2. How do the authors envision handling the omission of background information in domains where background context could be a valuable cue (e.g., occlusions, instance support)?\n3. Please elaborate on any observed failure modes (missed detection, false positives/negatives), especially with reference to dataset bias, rare classes, or environmental conditions.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper clearly identifies a major inefficiency in prior Mamba-based 3D object detectors: unnecessary computation over background voxels, and the resulting context loss when naïvely restricting to the foreground.\n2. Foreground selection is performed using a trainable scoring mechanism with effective sampling and serialization, alleviating regional truncation problems. It is well-motivated both by experiments and design illustrations.\n3. The RGSW and the SASFMamba fusion modules are described in substantial detail, with concrete mathematical rationale, and the implementation roadmap is well clarified.\n4. The method is compared against a rich spectrum of modern baselines on three benchmarks. Relative improvements are numerically strong.", "weaknesses": "1. Limited discussion of causal sequence modeling approaches: Although the method addresses linear sequence encoding with foreground focus, it omits explicit comparison to other causal sequence modeling alternatives or other advanced state-space models.\n2. Modeling limitations: The method largely treats background voxels as non-informative, but in autonomous driving, background structure or clutter can contain subtle contextual cues (for object contextual priors, occlusion estimation, etc.). The effect of completely de-emphasizing background is not discussed in terms of potential blindspots for categories with ambiguous boundaries.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ErkQJQH8P4", "reviewer": "Reviewer_7P9E", "summary": "3D object detection is a critical task in computer vision with broad applications in autonomous driving. In this paper, the authors proposed a novel LiDAR based 3D object detector, named as FORE-MAMBA3D. It demonstrates the better performance in the KITTI, NuScenes, and Waymo datasets.", "questions": "The following points are suggested for further improvement.\n​​Q1. In the Introduction, the mention of \"incomplete and imprecise sampling\" seems disconnected from the preceding context, which might confuse readers about the purpose of this sampling operation. Please clarify its objective. For instance, if foreground-only encoding is used, does it require sampling foreground voxels? Please explain this clearly.\n​​Q2. In the Related Work section, the authors highlight the limitations of existing methods. It would be better to further emphasize how the proposed method in this paper specifically addresses these limitations.\n​​Q3. In Section 3.2, the authors introduce a local token to summarize information within each patch. It is recommended that the authors explain (1) their motivation (e.g., to address response attenuation) (2) explain  the need for a local token to re-summarize information, (3) finally delve into the specific computational details. Currently, the local token is introduced somewhat abruptly.\n​​Q4. Section 3.3.1 does not clearly explain the computational process and the motivation behind the proof, even with supplementary materials. The authors need to clarify:\na. How exactly are voxels rearranged using the semantic categories S? Is cosine similarity used for this rearrangement?\nb. From the diagram, it appears that x undergoes only one SSM operation and one 1D convolution to obtain h'. However, Equation 6 seems confusing. For example, is Nk(i) \"the original index of the semantically neighboring feature\" (as in Section 3.2) or the \"index of semantically neighboring feature\" (as in the supplementary material)? Furthermore, in Equation 6, is index j the index before or after rearrangement? Also, does merely changing the order allow the transformation from Equation 5 to Equation 6, given that the SSM operation is performed on the original sequence?\nc. It would be beneficial to separate the description of the network's computational process from the theoretical proof in Section 3.2. At the very least, indicate to readers that the subsequent formulas are primarily for theoretical justification and are not necessary during network inference.\n​​Q5. Please specify under what configuration (one 4090D? 1batch size?) the network's inference speed was evaluated in the experiments. Furthermore, I am curious about how the FLOPs for the Mamba module within the 3D backbone were computed.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper is well-structured and supported by sufficient experiments, showing great potential for acceptance. \n1. Experiments are conducted in the mainstreams datasets, such as KITTI, NuScenes, and Waymo datasets. These results show the effectinvess of the proposed method.\n2. Overview of the proposed framework is very clear. It shows the novelty of the proposed method.\n3. Motivation of the proposed method is also clear.", "weaknesses": "The following points are suggested for further improvement.\n​​Q1. In the Introduction, the mention of \"incomplete and imprecise sampling\" seems disconnected from the preceding context, which might confuse readers about the purpose of this sampling operation. Please clarify its objective. For instance, if foreground-only encoding is used, does it require sampling foreground voxels? Please explain this clearly.\n​​Q2. In the Related Work section, the authors highlight the limitations of existing methods. It would be better to further emphasize how the proposed method in this paper specifically addresses these limitations.\n​​Q3. In Section 3.2, the authors introduce a local token to summarize information within each patch. It is recommended that the authors explain (1) their motivation (e.g., to address response attenuation) (2) explain  the need for a local token to re-summarize information, (3) finally delve into the specific computational details. Currently, the local token is introduced somewhat abruptly.\n​​Q4. Section 3.3.1 does not clearly explain the computational process and the motivation behind the proof, even with supplementary materials. The authors need to clarify:\na. How exactly are voxels rearranged using the semantic categories S? Is cosine similarity used for this rearrangement?\nb. From the diagram, it appears that x undergoes only one SSM operation and one 1D convolution to obtain h'. However, Equation 6 seems confusing. For example, is Nk(i) \"the original index of the semantically neighboring feature\" (as in Section 3.2) or the \"index of semantically neighboring feature\" (as in the supplementary material)? Furthermore, in Equation 6, is index j the index before or after rearrangement? Also, does merely changing the order allow the transformation from Equation 5 to Equation 6, given that the SSM operation is performed on the original sequence?\nc. It would be beneficial to separate the description of the network's computational process from the theoretical proof in Section 3.2. At the very least, indicate to readers that the subsequent formulas are primarily for theoretical justification and are not necessary during network inference.\n​​Q5. Please specify under what configuration (one 4090D? 1batch size?) the network's inference speed was evaluated in the experiments. Furthermore, I am curious about how the FLOPs for the Mamba module within the 3D backbone were computed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YEZLrJdAbu", "reviewer": "Reviewer_jtLi", "summary": "Fore-Mamba3D is a Mamba-based 3D perception backbone for LiDAR-based object detection. It focuses on foreground voxels predicted by a learned scoring function. It propose a Regional-to-Global Sliding Window module which enhances long-range voxel interaction to prevent information loss from sparse foregrounds. It also introduce a Semantic-Assisted State Fusion module for fusing semantic and geometric features. It achieves efficient linear modeling and how state-of-the-art performance on nuScenes, KITTI, and Waymo.", "questions": "Have the authors evaluated how alternative sequence ordering strategies, such as random shuffling or different spatial traversal patterns, affect the performance of the Regional-to-Global Sliding Window (RGSW)? If so, how sensitive is the model to the chosen voxel sequence order, and does it significantly influence spatial consistency or detection accuracy?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The paper presents an interesting and effective idea by selectively adapting foreground voxels for Mamba-based modeling. This approach substantially reduces computational overhead while maintaining high representational quality, showing a smart trade-off between efficiency and performance.\n\n2. The method is validated across multiple benchmark datasets (nuScenes, KITTI, Waymo) with consistent performance gains. The visual results Fig3 are clear and insightful, effectively illustrating how the proposed model can improve the contextual understanding of fused voxel features.", "weaknesses": "1. Although the proposed method introduces innovative foreground-focused encoding, the overall performance gain compared to prior Mamba-based or Transformer-based 3D detectors is relatively modest. The improvements, while consistent, may not fully justify the added architectural complexity or the additional design components.\n\n2. Unclear Spatial Consistency in the Regional-to-Global Sliding Window (RGSW): In the proposed RGSW strategy, the authors directly fuse the later half of the sequence with the previous half, which raises concerns about spatial coherence. Since the voxel sequence is constructed by selectively sampled voxels along a Hilbert curve, their ordering does not guarantee true spatial adjacency. This makes the fusion operation confusing, the merged regions may not correspond to physically neighboring spatial areas. As a result, the process may not fully respect local spatial correlation, and in extreme cases, it could behave similarly to random voxel concatenation, weakening the interpretability of the regional-to-global transition.\n\n3. The authors completely discard background voxel feature extraction, directly concatenating the processed foreground features. While this design choice further reduces computational cost, it introduces a potential issue, semantic strength inconsistency between foreground and background regions. In fact, background voxels play an important role in defining object boundaries and separating adjacent instances. Ignoring background information entirely may lead to inaccurate boundary localization, especially in cluttered or occluded scenes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JEfSyBI6Qc", "reviewer": "Reviewer_3Dbr", "summary": "This paper proposes Fore-Mamba3D, a novel 3D object detection framework that focuses on foreground-enhanced encoding using Mamba-based architectures. The key insight is that previous Mamba-based methods encode entire non-empty voxel sequences containing abundant background information, while foreground voxels occupy only a small portion of the scene. The method introduces two main components: (1) Regional-to-Global Sliding Window (RGSW) strategy to address response attenuation in foreground-only sequences, and (2) Semantic-Assisted and State Spatial Fusion Mamba (SASFMamba) to enrich contextual representation with semantic and geometric awareness.", "questions": "1) How does the method perform when foreground prediction accuracy is low?\n2) How sensitive is the method to the choice of space-filling curves (Z-order vs. Hilbert)?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1) The motivation for foreground-focused encoding is compelling and well-supported by empirical analysis.\n2) The RGSW strategy effectively addresses the response attenuation problem in foreground-only sequences through a regional-to-global information propagation mechanism. \n3) The method achieves state-of-the-art performance across multiple benchmarks (nuScenes: 72.3 NDS, KITTI, Waymo), demonstrating consistent improvements over existing Mamba-based approaches while reducing computational overhead.", "weaknesses": "1) While the combination is novel, individual components are relatively incremental. The foreground sampling is essentially top-k selection based on predicted scores, and the Hilbert curve rotation is a straightforward solution to regional truncation issues.\n2)  The paper lacks comprehensive ablation studies on key hyperparameters. How sensitive is performance to the number of rotations r,  and the foreground sampling ratio α? The impact of different space-filling curves beyond Hilbert is not explored.\n3) The paper doesn't discuss failure cases or limitations of the approach. How does performance vary with different foreground/background ratios across datasets?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)