PTNET: A PROPOSAL-CENTRIC TRANSFORMER NET- WORK FOR 3D OBJECT DETECTION
Authors: Jianping Zhong
Zhaobo Qi
Kaiwen Duan
Xinyan Liu
Beichen Zhang
Weigang Zhang
Qingming Huang
Venue: ICLR 2026 Poster (2026)
Keywords: 3D Object Detection
Point Clouds
Two-stage
Transformer
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: 3D object detection from LiDAR point cloud data is important for autonomous driving systems. Recent two-stage 3D object detectors struggle to achieve satisfactory performance due to limitations in proposal quality, stemming from the degradation of geometric detail information in the generated proposal features caused by high sparsity and uneven distribution of point clouds, as well as a lack of effective exploitation of surrounding contextual cues in the independent proposal refinement stage. To this end, we propose a Proposal-centric
Transformer Network (PTN), which includes a Hierarchical Attentive Feature Alignment (HAFA) module and a Collaborative Proposal Refinement Module (CPRM). More concretely, to obtain multi-granularity proposal representations, HAFA employs a dual-stream architecture that extracts both coarse-grained voxel features and fine-grained point features to enhance proposal features, then harmo-
nizes them through a feature alignment network in a unified space. The CPRM first generates object queries for all objects and then establishes contextual-aware interactions to extract complementary information from semantically similar and spatially relevant proposals. PTN achieves promising performance on large-scale Waymo and KITTI benchmark, demonstrating the superiority of PTN.

Published as a conference paper at ICLR 2026
PTNET: A PROPOSAL-CENTRIA TRANSFORMER NET- WORK FOR 3D OBJECT DETECTION
Jianping Zhong' Zhaobo Qi
Kaiwen Duan²
Xinyan Liu'
Beichen Zhang' Weigang Zhang! 1.3 Qingming Huang?
Harbin Institute of Technology, Weihai
Univeeriity of Chinese Academy of Sciences
Harbin Institute of Technology (Weihai) Qingdao Research Institute
ji sianningzhongesttn stu. wwt.hit.dd en, www.kaawndannuttlooooooooooo com,
{qi zb, xinyl iu, beiczhang, (yzzhan)@hti @hit edu. cn, qmhuang@ucas acc.ccccccooo
ABSTRACT
3D object detection using LiDAR point cloud data is critical for autonomous driv- ing systems. However, recent two-stage detectors still struggle to deliver satisfac- tory performance primarily due to inadequate proposal quality, which stems from significant geometric detail degradation generated proposal features caused by high sparsity and uneven distribution of point clouds, as well complete failure to exploit surrounding contextual cues during independent proposal refinement, losing complementary details from adjacent proposals. To this end, we propose a Proposal-centric Transformer Network (PTN), which includes a Hierarchical Attentive Feature Alignment (HAFA) and Collaborative Proposal Refinement Module (CPRM). More concretely, HAFA employs a dual-stream architecture extract multi-granularity proposal representations, including coarse- coarse-graing grrssed multi-scale voxel features and fine-grained coordinate point features to enhance proposals' object geometric representation ability. CPRM first generates hybrid object queries for all objects and then establishes contextual-aware interactions through the 3D parameter-guide aggregate spatial location cues and category-specific information across propos- als that are spatially adjacent and semantically correlated. Extensive experiments the large-scale Waymo and KITTI benchmarks demonstrate the superiority of PTN. The code is available at htttps://gittub... com/ 2wwpg/ionnnnnnnnnn con ptnet.git
deformable attention mechanism to effectively
INTRODUCTION
3D object detection serves as foundational task for environmental perception autonomous driv- ing, aiming precise localization and classification of objects in 3D scenes. Recently, achieve a trade-off between performance and efficiency, researchers have focused on the two-stage 3D ob- ject detector paradigm Deng al. 202T); Shi al. (2023): this paradigm first employs the region proposal network (RPN) to obtain proposals, then extracts proposal features via ROI pooling [eng al. (2021), and ultimately produces detection outputs by refining these proposal features in the subsequent refinement stage. However, existing two-stage 3D object detectors are constrained by suboptimal proposal quality, which arises from two key underlying issues.
The primary concern revolves around the degradation of geometric detail in the generated proposal features. In particular, prevailing two-stage detectors typically progressively expand receptive fields through pooling operations to generate proposal features. For areas of objects with few points, the pooling operation tends filter out high-frequency geometric features such as surface details and edge sharpness, resulting in blurred proposal boundaries and structural incompleteness. As shown in Figure certain objects have only a few points or sparsely distributed points. After multi-pooling operations, the detailed information inherent in these points is lost, leading to inaccurate predictions (denoted by green boxes). Although recent research has explored the use of ineeround-agnostie
"Corresponding Authors.

---

Published as a conference paper at ICLR 2026
(a)
(b) object with
object with a few points
sparse distribution points
(c)
cal
TEb
Qualitative results under the bird's-eye s-eye view (BEV) Waymo. The red and black 3D boxes are annotated by humans, and the red points represent the object points. The green 3D boxes are predicted by VoxelNext. Some predicted boxes are not accurate due corresponding objects having only a few points (a) or sparsely distributed points (b). (c) The car is occluded by the tree. sampfing tethhods to generate
Figure 1:
raw points for compensating detailed information (hi al. (2020) 2023), these methods tend to overlook sparse foreground points, which undermines the com- pleteness of foreground representation and ultimately impairs the detection performance.
Another issue stems from the inefficient exploitation of surrounding contextual cues during the pro- posal refinement stage. Current two-stage methods optimize each proposal independently, only using its local features and failing to leverage complementary details from adjacent proposals with similar object characteristics. This limitation proves especially problematic in 3D scenes where oc- clusions are present. As illustrated in Figure [ (c), since a car is occluded by tree, its internal point clouds tend to be split into disjoint segments, with each segment independently predicted as separate proposal. Due to the absence of cross-interaction with other proposals and the failure to integrate complementary information, the refined proposals suffer from inaccurate localization
To address the above challenges, we propose Proposal-centric Transformer Network (PTN), which integrates hierarchical attentive feature alignment module to enhance proposals' object geometric representation ability and collaborative proposal refinement module to effectively aggregate com- plementary information among spatially adjacent and semantically correlated proposals.
The hierarchical attentive feature aligmment module employs dual-stream feature extraction archi- tecture to capture complementary guatuigrraullrrity features to enhance proposal features. First, we propose coarse-grained voxel feature extraction module to derive multi-scale semantic proposal features directly from voxel features, which enhances discriminative power for classification tasks. Concurrently, we design fine-grained point feature retrieval module recover intricate geometric details of proposals from unsampled raw foreground point clouds, thereby preserving precise spatial cues critical for regression refinement. Following the extraction of these dual-graaullriig features, we introduce a feature alignment module to harmonize them within a unified feature space, which ensures synergistic integration while maintaining their complementary strengths.
The collaborative proposal refinement module is designed to first generate object queries for all objects and then establish proposal contextual-aware interactions to extract complementary infor- mation from proposals that are spatially relevant and semantically similar. Specifically, we first select the top-K proposals from the output of RPN based on their classification confidence scores to construct the basic proposal queries. Concurrently, tively explore objects that may have been overlooked, such as fully occluded or smaller objects. These hybrid queries integrated object queries. Next, employ 3D parameter-guides de- formable attention mechanism to perform interaction between each object query and all generated proposals, which enables each object query capture beneficial spatial location cues and category- specific semantic features from relevant proposals, thereby ultimately improving performance.
introduce learnable random queries to proac-
We conduct abundant experiments on large-scale 3D object detection benchmarks Waymo Sun al. (2020) and KITTI Geiger et al. 2012)). Experimental results demonstrate the effectiveness of PTN. In summary, our contributions are as follows:
2

---

Published as a conference paper at ICLR 2026
We introduce hierarchical attentive feature alignment module, producing high-quality proposals with complementary multi-grained features.
We propose an effective collaborative proposal refinement module, which adaptively aggregates crucial surrounding regions and performs proposal-level interaction.
PTN achieves promising performance for 3D object detection on the large-scale datasets.
RELATED WORK
2.1 LIDAR-BASED 3D OBJECT DETECTION
Existing point cloud-based 3D object detection methods Fan et al. (2022b; 2024) predominantly fall into two technical paradigms: point boint-based and voxel-based approaches Zhong et al. (2025a b Point-based methods Chen et al. 2022 typically employ architectures like PointNet++ Qi et al. (017)) to directly extract features from unordered point clouds, followed by single-stage or two- stage detection frameworks generate predictions. Given the massive scale of raw point clouds, such methods commonly adopt metric space sampling strategies to select representative point sub- sets for computational efficiency. Although point-based methods demonstrate superior performance on small datasets, their computational complexity scales linearly with point cloud cardinality. To address the efficiency bottleneck point point-based methods, researchers predominantly adopt voxel- based methods Liu et al. (2024); Jin et al. (2025); Zhang al. (2024); [L et al. (2021); Fan et al. (2022)) to balance computational cost and performance. VoxeINet Zhou & Tuzel (2018) pioneers the transformation of irregular point clouds into structured voxel grids. Voxel R-CNN R-CNNDDeng et al. (202I) optimizes the two-stage detection pipeline, substantially reducing computational costs while maintaining accuracy. Our PTN is similar Voxel R-CNN but introduces core innovation: con- tinuously refining the geometric quality and semantic consistency of proposals through hierarchical feature alignment and dynamic receptive field adjustment.
2.2
3D OBJECT DETECTION WITH DETR
Recent advancements in Transformer-bseed architectures have motivated extensive exploration of DETR Carion al. (2020) frameworks for 3D point cloud object detection, particularly focus- ing on two critical design aspects: query initialization strategies and feature aggregation mecha- nisms. TransFusion Bai et al. (222) leverages heatmap-guided localization identify BEV feature peaks as initial queries, while CMT Yan et al. (2023) implements geometrically anchored learn- able queries combined with global cross-attention for feature integration. Alternative solutions ad- dress specific limitations through innovative mechanisms. ConQueR Zhuu et al. (2023) introduces contrastive query refinement to suppress false detections, and FocalFormer3D Chen et al. (2023) employs multiphase heatmap filtering alongside adaptive attention mechanisms to enhance both query selection efficiency and context modeling. However, existing DETR-inspired approaches still underperform compared some -n-trassfforrr detectors. Compared to existing DETR-based ap- proaches that rely on dense feature matching, we explicitly treat object proposals as learnable queries the DETR framework. By enabling dynamic interaction between proposals through deformable attention mechanisms, we achieve more effective feature representation and information exchange.
METHODOLOGY
3.1 OVERVIEW
The framework of our PTN shown in Figure form them into regular voxel representation. Next, we utilize 3D backbone network Yan et al. (2018) to extract voxel features at three scales: 2x, 4x, and 8x downsampled features denoted as F.N., where N, E (1. Subsequently, we transform the Få features into the Bird's Bird's-Eye -yee View (BEV) space and generate BEV features. A region proposal network (RPN) is then applied to produce proposals B {b.).. from the BEV features. Those proposals also called region of interest (ROIs). For each proposal b,, we employ the Hierarchical Attentive Feature Alignment (HAFA) module to enhance the proposal features, obtaining ROI features fo. Afterward, we lever-
2 Given the point clouds F, as input, we first trans-
3

---

Published as a conference paper at ICLR 2026
F
Tuxxaaae CTC
RPN
Fb
Input Point Clouds
Sparse Voxels
Dulpat Bounding Boxes
Clessitication
Cearso quaindd Vaxel Feature Extration
XX
M (
Prsposal
Peatures
Regrassion
Pustite
Defomaale Bos
A Alestor
Decodar
Layer
Tize grazeed Pornt Feeture Retneral
Self Attrrtonn
Heatrop HM
K--240
K=1.80
Top-
K-+300
Region Proposal
Network
eerreiical Atlentive Fee ture Alignment
Callsborative Proposal Reinement Midule
Figure 2: The overall architecture of PTN. consists of 3D backbone, an RPN, an HAFA module, and CPRM module. Specifically, the HAFA uses dual-stream feature extraction architecture to capture multi-granularity proposal features. The CPRM explicitly establishes contextual interactions among proposals through hybrid query generation mechanism.
age the Collaborative Proposal Refinement Module (CPRM) to facilitate cross-proposal interaction among the proposals B. Finally, feed-forward network (FFN) is used to predict the output.
3.2 HIERARCHICAL ATTENTIVE FEATURE ALIGNMENT
In this section, propose a dual-stream feature extraction architecture to capture complementary multi-granularity features to enhance the proposal features. First, we propose a coarse-grained voxel feature extraction module to derive multi-scale proposal features from voxel features. Concurrently, we design a fine-graine point feature retrieval module to recover intricate geometric details from the unsampled raw point clouds. Following the dual- qual-granularity feature extraction, fusion module to harmonize them within unified feature space.
a feature
3.2. COARSE-GRAINES VOXEL FEATURE EXTRACTION.
For each proposal we first use discrete grid points represent it and then extract the correspond- ing grid point feature from the voxel feature by trilinear interpolation. Finally, we feed these grid point features into a Transformer-bise based encoder enable gross-grid feature interaction within the proposal.
(ar, y. 2, w. h, 0), where (x, y. 2),, (1, w. h), and are the center, size, and rotation angle of the proposal. We uniformly divide the proposal into g x g X grid, and use the gd grid point gb ((92, 9=)} to represent the proposal following Voxel R-CNN Deng et al. 2002). After obtaining g" we generate the grid points feature based on the voxel features F.N.. Specially, we remap the grid points into voxel feature maps with different downsampling factors and
Given a proposal
apply trilinear interpolation to extract grid features (fu.i, ) and concatenate them to get multi- scale grid point features ( Then, we treat those multi-scale grid points as tokens and use them
the query content Q. fbiing fi=1 Finally, send the grid point features into the transformer
encoder and MLP Layers to get the coarse- -eeeeerrandddd voxel feature ff as follows:
f MLP(Encoder((Q.. P2)),
where P3 the position embedding. We employ a learned absolute position embedding function encode the grid points position Py - o(gz, By 9z) Rdd, d is the channel dimension of fb.i For simplicity, F. = {SE} represent the coarse proposal features of B.
4

---

Published as a conference paper at ICLR 2026
@
(bb
(c)
objjet
objjett
Figure 3: Three heatmaps for pedestrians. The left (a) displays heatmap generated by applying local NMS to the predicted heatmap, while the middle (b) and right (c) show the predicted and ground-truth heatmaps, respectively. Notably, after applying local NMS, the red-circled area in left heatmap filters out many correct predictions compared to the right heatmap.
3.2.2 FINE-GRAINED POINT FEATURE RETRIEVAL.
Given proposal b and the point clouds P (px, Pys Pz. fa). where fa E TRC, the intensity and timestamp features. we first select the foreground points P {Prr P=)} whose locations are inside b. Then encode the geometric information of proposal into these foreground points to eliminate size ambiguity Li et al. (2021). Finally, we fuse those enhanced foreground points as the fine-grained point feature.
Specifically, for each proposal b, we translate its foreground point clouds into the coordinate system of the proposal and rotate them along the proposal direction angle 0, through P Rg (P' (P"-))). where Rg denotes the rotation matrix and Tß = (x, z) denotes the center of the proposal b. After obtaining the transformed foreground point clouds, we calculate the Euclidean distances from the transformed foreground points P" to the six surfaces of the proposal bounding box where they are located. The feature vector for each transformed foreground point is constructed as:
fC Concat (p'+ PTy p2),, Sa, (di, dr, y da, de, da)),
(2)
where d(.) represents the Euclidean distance metric. These features are then processed by multi-layer perceptrons (MLPs) and max-poolinn max-pooling layers generate fine- meeggrinedd point features fā:
fl Maxpool(MLP((f]) F, {
For simplicity, we use F, } to represent the coarse proposal features of B.
3.2.3 FEATURE ALIGNMENT.
For multi-granule features, we first concatenate them into composite feature representation and subsequently employ a convolutional network to project them into a unified feature space. The final proposal features fo are formulated as follows:
fb Cony(Concat( ft), Fb (f6)
3.3 COLLABORATIVE PROPOSAL REFINEMENT MODULE
In this section, we propose CPRM to generate high-quality proposal queries and random queries object queries and then establish proposal contextual interactions with all proposals complete contextual knowledge to achieve context awareness. Specifically, we first suppress spatial redundan- cies RPN proposals B via NMS obtain candidate proposals Br Then we dynamically select the top K classification confidence proposals from B,ms as high-quality proposal queries Qp. Fur- thermore, we introduce M learnable random queries Q, that serve as potential objects. These hybrid queries are integrated as object queries Q = IQp, Q.). Finally, a deformable gross-attention is ap- plied to make interactions between hybrid object queries and the complete contextual knowledge Pb that is preserved in B.
3.3.1 PROPOSAL. QUERY GENERATION.
We first generate the candidate proposals Bams using the NMS. Generally, the proposals gen- erated by RPN often exhibit high overlap ratios. The conventional methods address the overlap
5

---

Published as a conference paper at ICLR 2026
issue by using Non-Local NMS Yin et al. (2221a). They utilize the centers of proposals for post- processing filtering, as shown in Figure However, the classification performance is not very good in the early training; such methods tend to filter out true positive proposals that close together (the red dashed circle area in V To address this issue, we use the box of proposals for post- processing filtering. In particular, we employ box NMS with low NMS threshold (e.g., 0.5) to select highly diverse candidate proposals Bams. The corresponding features are Fims.
Then we select the top K proposals from Bums as proposal queries Qp- As the object counts in 3D scenes are inherently uncertain, it is suboptimal to select fixed proposal queries. To achieve dynamic selection, we propose a query number estimation mechanism based on the classification score heatmap HM in RPN.
Specifically, we first model the probability distribution of object counts CNT in the scene as,
y (HM), if epoch T, C
CNT
otherwise,
where (HM) = SUM(HM > st) is the function to calculate the number of objects. st = 0.3 is the classification confidence to determine whether proposal is object. is the epoch to apply HM to estimate object counts. is the maximum object count.
Then we categorize CNT into three intervals {[n min .ik m and adaptively the proposal queries number K according to:
K1,
if r:jiu < CNT jaax K2; if n2 CNT n max
K
K3,
if nigin S CNT
< <n max
Finally, we select the proposal based on the classification confidence of the Bums and K as follows:
2p topx (Fums; HM), P,. <ottop (Bams; HM))
where top (u, v) means select top K queries from u according P. is the position embedding. K is determined by Equation 6 HM the classification score heatmap in RPN.
3.3.2 RANDOM QUERY GENERATION.
After getting the figh-quality object queries, we preserve random queries B, retrieve some over- looked objects by Qp. In particular, the BEV space is uniformly partitioned into X x Y grids (the BEV feature map size X x Y x C), where each grid center initializes an auxiliary query. The centers of random queries B, align with grid positions, and their scales are uniformly set to (0.05L, 0.05W, O.5HH. By incorporating such spatially prior-constrained random queries, our method effectively recalls TPs over-suppressed by NMS while maintaining high detection preci- sion. The random queries content Q, and position embedding P, are defined as follows:
Qr E R P. = o
(8)
where d is the channel dimension of the fo.
The object queries are donated as Q Concat(Q Qp. Qr), and the position embedding Pg Concat( Pp, P.).
3.3.3 PROPOSAL TRANSFORMER DECODER.
This module aims establish an interaction between the generated object queries and the complete contextual knowledge B to achieve object-level context awareness. A direct method is to follow the DETR by using cross-attention. However, such a paradigm neglects the inherent differences between 2D images and 3D point clouds. Unlike 2D images where objects may occupy most of the image, 3D objects exhibit spatial sparsity and occupy minimal area. Consequently. 3D proposal interactions should focus exclusively on neighborhoods, which reduces computational redundancy while enhancing geometric relationship modeling. Specifically, for each object query, we utilize its 3D bounding box parameters (e.g., position, dimensions, orientation) generate spatial attention

---

Published as a conference paper at ICLR 2026
Table I: Comparison with prior methods on the Waymo Open dataset (single-frame setting). Met-
rics: mAP/mAPH (%)t for the overall results, and AP/APH (%)t for each category. $ two-stage method. t: detr-like methods. results are not published.
Results validation dotaser
mAP/mAPH
Vehicle AP/APH
Pedestrian AP/APH
Cyclist AP/APH
Methods
L2
L1
L1
L2
CenterPoint Yin et al.
bb
68.2/65.8
74.2/73.6
66.2/65.7
76.6/70.5
68 08.8/63.2
72.3/111
69.7/68.5
PV-RCNNI Shi cT 2020
69.6/67.2
78.0/77.5
69.4/69.0
79.2/73.0
70.4/64.7
71.5/70.3 -/-
69.0/67.8 +
TS? Fan ct al. 2022a
76.2/75.8
68.0/67.6
81.4/74.0
72.8/65.9
SWFormer el al.
77.8/77.3
29.2/68.8
80.9/72.7
72.5/64.9
-/-
PillarNet-34
2022
70,9/68.4 4
79.1/78.6
70.9/70.5
80.6/74.0
72.3/66.2
72.3/71.2
69.7/68.7
CenterFormer
Zhou
2022
71.1/68.9
75.0/44.4
69.9/69.4 .4
78.6/73.0
73.6/68.3
72.3/71.3
69.8/68.8
PV-RCNN++1 Sin el at.
A125
71.0/64,9
78.8/78.2
70.3/69.7
76.7/76.2 +
68.5/59.7
69.0/67.6 +
66.5/65.2
TransFusionf
Bai et 2022
-164.9
-/-
-/65.1
-163.7
-/65.9
Wang et al. 2023
73.2/71.0
79.3/78.8
70.9/70.5
82.8/77.0
75.2/69.8
76.4/75.4
73.6/72.7
ConQueRi Zhu of 2023
70.3/67.7
1/.1/
68.7/68.2
79.0/72.3
70.9/64.7
73.9/72.5
71.4/70.1
FlatFormer Lau el 2023
69.7/67.1
-/--
69.0/68.6
71.5/65.3
6/67.
Shift-SSD Chen
64.8/61.1
74.1/73.6
65.1/64.6
72.3/ 62.3 +
63.4/ 54.5
68.2 /66.4 -/-
66.0/ 64.2
LiDAR-PTQ/Zhou et 2224
67.6/65.1
-/-
66.2/65.7
67.9/62.2
68.6/67.5
Huang el al 2024
71.0/68.6
78.5/78
70.0/69.5
81.0/75.1
72.2/66.7
73.4/72.5 5
70.7/69.7
PASS-PV Chen 2025
72.0/65.7
78.3/78.8
70.5/70.0
76.2/66.9
67.2/58.8
7188/0.7
69.4/68.3
PTN
73.5/71.2
76.7/77.1
68.7/68.2
84.2/78.6
76.8/71.4
77.7/76.5
75.0/73.9
Results on testing dataset
mAP/mAPH
Vehicle AP/APH
Pedestrian AP/APH
Cyclist
AP/APH
Methods
L2
L1
L2
L1
L2
PV-RCNN++ Shi ct al. 2023
72.4/70.2
81.6/1.2
73.9/73.5
80.4/75.0
74.1/69.0
71.9/7088
69.3/68.2
PillarNet et al. 2022
70.1/67.1
81.1/80.6
73.6/73.2 -/-
78.3/70.2 +
72.2/64.6
67.2/66.0
64.7/63.6
PillarNeXti cl. 2023
72.2/69.6 -/-
-/-
Fade3D al. 2025
77.7/77.2
69.9/69.5
+
PTN
72.7/70.6
80.2/79.8
72.5/72.1
82.0/77.0
76.0/71.2
72.2/71.1
69.6/68.5
weights. These weights dynamically adjust the sampling offset of deformable convolution kernels. This enables the network to autonomously capture the structural features of neighboring objects and to enrich each object query with scene-level dependencies. During proposal refinement, the detector jointly optimizes each object query by leveraging these scene-level dependencies. The enhanced object queries Q' Decoder( Q, Pq. Fb) are used for the subsequent detection head.
4 EXPERIMENTS
4.1 DATASETS AND EVALUATION METRICS
Waymo Open Dataset (Waymo). It includes 798, 202, and 150 scenes for the training, valiclation, and testing sets. It provides three categories: vehicle, pedestrian, and cyclist. The evaluation uses mean average precision (mAP) and mAP weighted by heading accuracy (mAPH). Objects classified into two levels: LEVEL 1 (L1) for more than 5 point clouds and LEVEL 2 (L2) for more than 1.
KITTI. There are 7481 and 7518 samples for training and testing. The dataset includes three cate- gories: car, pedestrian. and cyclist. The 7481 training samples are divided into two parts: 3769 and 3712 samples for the training and validation sets. We use 3D mAP as the evaluation metric.
4.2 IMPLEMENTATION DETAILS
Table Effectiveness of PTN on Waymo validation set using multi-frame inputs.
For Waymo and KITTI, we apply PTN to Voxel R-CNN. The setting aligns with prior works Deng et al. ). In CPRM, categorize CNT into three intervals [o, 20). (20,40), 40), and 200], and set K1
Methods
FramesmAP/mAPII (L.2)
CenterPoint
70.8/69.4
CenterFormer Zhou et al. 2002 PillarNet Shi et al. 2022
74.7/73.2
72.2/68.4
the parameters 240, K2, = 300. Additional results are provided in the supplementary materials.
180, K₂
PTN PTN
74.5/73.2
75.6/74.1
7

---

Published as a conference paper at ICLR 2026
Table 2: Comparison KITTI. Metrics: mAPt for the overall results.
results not published.
Results on the testing data set
Car
Cyclist Moderate
Methods
Fasy
Moderate
Hard
mAP
Easy
Hard
HVPR et 2021
86.38
77.92
73.04
79.11
SASACChen Chen
2022
88.76
82.16
77.16
82.69
IA-SS SSaizinng et di 2022
88.34
80.13
75.04
81.44
78.35
61.94
55,70
68.30
Voxel CNNN Deng el sll
2021
88.09
80.99
76.50
81.86
76.42
62.01
55.94
64.79
PASS-PV Chen
87.65
81.28
76.79
81.90
80.43
68.45
60.93
69.93
DPFusion Mo al. 2025
90.98
82.35
77.26
83.53
79,96
66.47
58.47
68.30
PTN
91.60
82.77
77.96
84.11
83,38
70.30
62.63
72.11
Resdts the validation data
Car
Cyclist
Methods
Easy
Moderate
Hard
mAP
Easy
Moderate
Hard
mAP
EPNe Huang et al.
2020
88. 76
78.65
78.32
81.91
83.88
65.60)
62.70
70.72
Pointtormer Pan et
2021
87.13
77.06
69.25
77.81
75.01
59.80
53,99
62.93
IA-SSD Zhang
91.88
83,41
80.44
85.24
88.42
70.14
65.99
74.85
VIF Li al
2024
92.31
85,51
82.92
86.91
89.40
73.12
69.86
77.46
Voxel CNN Deng al.
2021
92.53
85.03
82.56
86.70
89.52
72.62
68.32
76.82
Fade31) Ye
83.47
92.74
85.92
87.17
91.87
72.66
68.32
77.61
4.3 STATE-OF-THE-ART COMPARISON
Waymo. Results in Table demonstrate that PTN surpasses most detectors. PTN shows sig- nificant improvement in pedestrian. On one hand, the point clouds of pedestrians typically exhibit sparse distributions, and the use of coarse-grained voxel features tends to cause lo- calization inaccuracies. HAFA addresses this by incorporating fine-grained point cloud fea- tures to enhance precise localization. On the other hand, CPRM facilitates interaction between current objects and surrounding ones, effectively mitigating occlusion and overlapping issues. The results obtained by using multi-fran data as input in Table B demonstrate that PTN out- performs existing detection approaches.
70
PPVRCNN-
LerpeKennettD
We compare PTN with other methods in terms of performance and inference speed, as de-
Performnne (MAPH(L2)
PV RCNN
66
VaxSes
Censer Moccl
picted Figure 4 Remarkably, PTN achieves good trade-off between performance and in- ference speed. All models are evaluated on the
SS7
Vocet-HUUNN
*
62
NVIDIA A100 GPU.
KITTI.
The results on KITTI are shown in Ta- PTN achieves promising results the
PhintPillem
ble 4.1
Second
KITTI test set, particularly in detecting cyclists. This can be attributed to the fact that most of the cyclists' area is empty, making it difficult for the model to detect them with limited points. In contrast, PTN introduces fine-graine mre-ngri points to complete the appearance of these objects, mak- ing them easier to detect. PTN is generalizable and can be applied to most of the datasets.
8
13
23
28
Running Speed
Figure 4: Performance and speed of our PTN and other leading performance detectors on Waymo.
4.4 ABLATION STUDIES
The efficiency each component. In Table we present an ablation study for different com-
Table 4: Ablation study of each component on Waymo.
ponents on Waymo. To improve efficiency, we use 25% of the data for training and validation. In the following section, we use this as the de- fault setting unless otherwise stated. CPRM ex- plicitly introduces the relation between differ- proposals, which is beneficial for the detector filter out background noise and improve preci-
HAPA
CPRM
Vehicle
3D APIAPH (1.2)
Pedestrian
Cyclist
mAPhmAPH
(L.2)
59.09/58.57
62.73/61.33 58/20/52.47
60.00/57.
60. 60.86660.38
65 65.13/57.13
64.58163.08
63.52/60
61.91/61449
65 626528866
65 65.82444.54
64.45/61.57
8

---

Published as a conference paper at ICLR 2026
Table 5: Ablation study for each component in PTN on Waymo. Np and N, represent the number of object queries and random queries, respectively.
3D AP/APH (L2)
mAP/mAPH
N₂
N,
Vehicle
Pedestrian
Cyclist
(1.2)
200
54.57/54.01
62.81/53.04
60.05/58.47
19.14/55.17
300
58,35/57.88 83
64.96/55. 34
61.94/60.21
61.75/57.79
400
55,59/55.03
62.88/53.53
60.89/59.22 ,28
59.78/55.94
300
100
61.91/61.49
65.62/58.69
65.82/64.54
64.45/61.57
sion. HAFA assists in locating the bounding box of objects, thereby improving the accuracy of the detector in determining the location and shape of objects.
The number of object queries. When the number of proposal queries increases (e.g., N₂ 400 vs N2 = 300), the similarity among queries rises significantly, leading to feature redundancy and con- sequent degradation in classification performance (from 57.79% mAPH 55.94% mAPH) as shown in Table 5 Conversely, reducing the number of proposal queries (c.g.. N 200 vs Ng 300), results in missed detection of low-score true positive (TP) samples, thereby lowering the recall (from 57.79% mAPH to 55.17% mAPH). This indicates fundamental trade-off between the diversity and similarity of proposal queries. To mitigate this conflict, we introduce random queries, which enable the recovery of overlooked low-score TPs.
The results in sparse scenarios. T we test the performance in sparse scenarios. We first construct sparse scenarios and then mea- sure performance by counting the number of True Positives (TPs). Specifically, we sort ob- jects in descending order according to their internal point counts. We then use the last 2.5% of objects to construct sparse scenarios. We count the number of TPs on both the baseline and PTN 1193(-10
Table 6: Number of true positives under different IoU in sparse scenarios on Waymo.
Tol
R-CNN PTN
10327
7017
1075
10656(+3.19) 1%) 4401(+544%
4712(-7.7%)
different IoU thresholds. The results demonstrate the efficiency of PTN in sparse scenarios. It is noteworthy that PTN exhibits significantly enhanced performance under high IoU thresholds. This improvement attributed the HAFA, which restores essential details aiding localization.
The performance on occluded scenes. We di- vide the scene into heavily occluded scenarios (the objects whose internal point cloud num- ber less than or the distance from the Li- DAR sensor to them exceeds 50 meters) and lightly occluded scenarios (other objects). Re- sults in Table 7 indicate that PTN shows a rel- atively smaller decrease in recall compared to the Voxel R- R-CNN et al. (202T). When the top K proposals do not include these objects, it becomes challenging for the object queries to re- call them effectively. For the objects missed in the proposals, we add random queries the object queries to interact with complete contextual knowledge, thereby improving recall.
Table 7: Recall at different occluded scenes on Waymo. -: random queries. LO and HO represent lightly and heavily occluded, respec- tively. VR represents Voxel R-CNN.
Setting] VR PTN PTN PTN improv. PTN improv.
LO 10.255 0.856 0.874
3.1% 4.9%
HO 6.6634 0.685 0.722
5.1%
8.8%
The efficiency of the HAFA. When only using the fine-grained point features to refine the pro- posals, the performance is better than using the voxel features, as shown in Table g This is be- cause the classification results are encoded into the object query content, while the proposal lo- cations are encoded into the object queries' po- sition embedding.
Table 8: Ablation study of HAFA on Waymo. CVFE, FPFR, and FA represent coarse-gras grained voxel feature extraction, fine- fine-grained point feature retrieval and feature alignment, respectively.
CVFE
EPER
Vahiele
3D APIAP Pedestrion
Cyolist
mAPlmAPH (L2)
39,09158.57
50,0057.45
3
60.66/99.21
58 3823652.55
13.54/1.19
41/4/77.64
58.91/58.37
56/73/50.35
61. 61.74/60.12 12
12/56.228
60.86/6.338
65.1357.13
64.58663088
63.52660.1
CONCLUSION
In this paper, we propose PTN, a novel Proposal-centric Transformer Network for 3D object detec- tion. Since the performance of existing two-stage detectors is limited by the quality of proposals in terms of fine- ame-grained information decay and the lack of effective exploitation address these issues with PTN. PTN aims enhance the proposal features for accurate 3D de- tection. Specifically, we use a dual stream feature extraction module to extract coarse grained voxel features and fine-grained point features, and align them to enhance the representation of proposals.
contextual cues,
9

---

Published as a conference paper at ICLR 2026
Furthermore, we propose collaborative proposal refinement module to explicitly establish contex- tual interactions among proposals through a proposal transformer decoder. Extensive experiments on the KITTI and Waymo benchmarks demonstrate the effectiveness of PTN. Future work will focus on improving proposal quality through learnable mechanisms with minimal cost.
REPRODUCIBILITY STATEMENT
To ensure the reproducibility of our work, we provide general details on the datasets and experi- mental settings in Section 4. Comprehensive information on the model architecture, datasets, and training strategies can be found in Appendix A
ACKNOWLEDGEMENT
This work is partially supported by the National Natural Science Foundation of China under Grants 62441232, 62476068, 62306092, and 62502115, 15, and supported by Shandong Provincial Natural Science Foundation under Grants ZR2025ZD01, ZR2024QF066, and /R22250QC516..
REFERENCES
Xuyang Bai, Zeyu Hu, Xinge Zhu, Qingqiu Huang, Yilun Chen, Hongbo Fu, and Chiew-Lan Tai. Transfusion: Robust lidar-camera fusion for 3d object detection with transformers. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1090-1099, 2022.
Holger Caesar, Varun Bankiti, Alex H Lang. Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijhom. Nuscenes: A multimodal dataset for au- tonomous driving. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pp. 11621-11631, 2020.
Nicolas Carion, Francisco Massa, Gabriel Synnaeve. Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European conference computer vision, pp. 213-229. Springer, 2020.
Chen Chen, Zhe Chen, Jing Zhang, and Dacheng Tao. Sasa: Semantics-augmented set abstrac- tion for point-based 3d object detection. In Proceedings of the AAAI Conference Artificial Intelligence, volume 36, pp. 221-229, 2022.
Shitao Chen, Haolin Zhang, and Nanning Zheng. Leveraging anchor-based lidar 3d object detection via point assisted sample selection. IEEE Transactions on Intelligent Transportation Systems, 2025.
Yilun Chen, Zhiding Yu, Yukang Chen, Shiyi Lan, Anima Anandkumar, Jiaya Jia, and Jose M Alvarez. Focalformer3d: Focusing on hard instance for 3d object detection. Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 8394-8405. 2023.
Zhili Chen, Kien T Pham, Maosheng Ye, Zhiqiang Shen, and Qifeng Chen. Cross-cluste shifting for efficient and effective 3d object detection in autonomous driving. In 2024 IEEE International Conference Robotics and Automation, pp. 4273 4280. IEEE, 2024.
Jiajun Deng, Shaoshuai Shi, Peiwei Li, Wengang Zhou, Yanyong Zhang, and Houqiang Li. Voxel r-cnn: Towards high performance voxel-based 3d object detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 1201-1209, 2021.
Lue Fan, Ziqi Pang. Tianyuan Zhang. Yu-Xiong Wang. Hang Zhao, Feng Wang, Naiyan Wang. and Zhaoxiang Zhang. Embracing single stride 3d object detector with sparse transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022a.
Lue Fan, Feng Wang, Naiyan Wang, and Zhao-Xiang Zhang. Fully sparse 3d object detection. Advances in Neural Information Processing Systems, 35:351-366 2022b.
Lue Fan, Feng Wang, Naiyan Wang. and Zhaoxiang Zhang. Fsd v2: Improving fully sparse 3d object detection with virtual voxels. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
10

---

Published as a conference paper at ICLR 2026
Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are ready for autonomous driving? the kitti vision benchmark suite. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3354-3361, 2012.
Tengteng Huang, Zhe Liu, Xiwu Chen, and Xiang Bai. Epnet: Enhancing point features with image semantics for 3d object detection. In European Conference on Computer Vision, pp. 35-52, 2020.
Xun Huang, Hai Wu, Xin Li, Xiaoliang Fan, Chenglu Wen, and Cheng Wang. Sunshine to rainstorm: Cross weathhre knowledge distillation for robust 3d object detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 2409-2416 2024.
Xin Jin, Haisheng Su, Kai Liu, Cong Ma, Wei Wu, Fei Hui, and Junchi Yan. Unimamba: Uni- fied spatial-channel representation learning with group-efficient mamba for lidar-based 3d object detection. arXiv preprint arXiv:2503. 12009, 2025.
Jinyu Li, Chenxu Luo, and Xiaodong Yang. Pillarnext: Rethinking network designs for 3d object detection in lidar point clouds. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 17567-17576, 2023.
Yanwei Li, Xiaojuan Qi, Yukang Chen, Liwei Wang, Zeming Li, Jian Sun, and Jiaya Jia. Voxel field fusion for 3d object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 1120-1129, 2022.
Zhichao Li, Feng Wang, and Naiyan Wang. Lidar r-cnn: An efficient and universal 3d object detec- tor. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 7546-7555, 2021.
Zhe I Liu, Jinghua Hou, Xiaoqing Ye, Tong Wang, Jingdong Wang, and Xiang Bai. Seed: A simple and effective 3d detr in point clouds. In European Conference on Computer Vision, pp. 110-126. Springer, 2024.
Zhijian Liu, Xinyu Yang, Haotian Tang, Shang Yang, and Song Han. Flatformer: Flattened window attention for efficient point cloud transformer. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 1200-1211, 2023.
Yujian Mo, Yan Wu, Junqiao Zhao, Jijun Wang, Yinghao Hu, and Jun Yan. Enhancing lidar point features with foundation model priors for 3d object detection, 2025. URL httttt:///axxi
org/ borg/abs/5555 2207. 13899
Jongyoun Noh, Sanghoon Lee, and Bumsub Ham. Hvpr: Hybrid voxel-point representation for single-stage 3d object detection. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14605-14614, 2021.
Xuran Pan, Zhuofan Xia, Shiji Song, Li Erran Li, and Gao Huang. 3d object detection with point- former. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recogni- tion, pp. 7463-7472, 2021.
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical fea- ture learning on point sets in a metric space. Advances in neural information processing systems, 30, 2017.
Guangsheng Shi, Ruifeng Li, and Chao Ma. Pillamet: High-performance pillar-based 3d object detection. In European Conference on Computer Vision, 2022.
Shaoshuai Shi, Chaoxu Guo, Li Jiang, Zhe Wang, Jianping Shi, Xiaogang Wang. and Hongsheng
Li. Pv-renn: Poxnt-voxel feature set abstraction for 3d object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10529-10538, 2020.
Shaoshuai Shi, Li Jiang, Jiajun Deng, Zhe Wang. Chaoxu Guo, Jianping Shi, Xiaogang Wang, and Hongsheng Li. Py-renn++: Point-voxel feature set abstraction with local vector representation for 3d object detection. International Journal of Computer Vision, 131(2):531-551, 2023.
11

---

Published as a conference paper at ICLR 2026
Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, et al. Scalability in perception for au- tonomous driving: Waymo open dataset. In Proceedings of the IEEE/CVF conference com- puter vision and pattern recognition, pp. 2446-2454, 2020.
Pei Sun, Mingxing Tan, Weiyue Wang, Chenxi Liu, Fei Xia, Zhaoqi Leng, and Drago Anguelov. Swformer: Sparse window transformer for 3d object detection in point clouds. In European Conference on Computer Vision, 2022.
Haiyang Wang. Chen Shi, Shaoshuai Shi, Meng Lei. Sen Wang, Di He, Bernt Schicle, and Li- wei Wang. Dsvt: Dynamic sparse voxel transformer with rotated sets. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 13520-13529, 2023.
Junjie Yan, Yingfei Liu, Jianjian Sun, Fan Jia, Shuailin Li, Tiancai Wang, and Xiangyu Zhang. Cross modal transformer: Towards fast and robust 3d object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 18268-18278, 2023.
Yan Yan, Yuxing Mao, and Bo Li. Second: Sparsely embedded convolutional detection. Sensors, 18(1))3333,, 2018.
Wei Ye, Qiming Xia, Hai Wu, Zhen Dong, Ruofei Zhong, Cheng Wang. and Chenglu Wen. Fade3d: Fast and deployable 3d object detection for autonomous driving. IEEE Transactions on Intelligent Transportation Systems, 2025.
Tianwei Yin, Xingyi Zhou, and Philipp Krahenbuhl. Center-based 3d object detection and tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17784-1193,, 2
Tianwei Yin, Xingyi Zhou, and Philipp Krahenbuhl. Center-based 3d object detection and tracking- the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021b.
In Proceedings the
Gang Zhang, Junnan Chen, Guohuan Gao, Jianmin Li, Si Liu, and Xiaolin Hu. Safdnet: simple and effective network for fully sparse 3d object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14477-14486, 2024.
Yifan Zhang, Qingyong Hu, Guoquan Xu, Yanxin Ma, Jianwei Wan, and Yulan Guo. Not all points are equal: Learning highly efficient point-based based detectors for 3d lidar point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18953-18962. 2022.
Jaaniing Zhong, Zhaobo Qi, Kaiwen Duan, Yuanrong Xu, Weigang Zhang, and Qingming Huang. Multi-modal 3d object detector with object-guided fusion and hierarchical sample selection. ACM Transactions on Multimedia Computing, Communications and Applications, 2025a.
Janning Zhong, Zhaobo Qi. Kaiwen Duan, Yuanrong Xu, Weigang Zhang, and Qingming Huang. Vpa: Multi-modal virtual point augmentation for 3d object detection. JEEE Transactions on Circuits and Systems for Video Technology, 2025b.
Sifan Zhou, Liang Li, Xinyu Zhang, Bo Zhang, Shipeng Bai, Miao Sun, Ziyu Zhao Xiaobo Lu, and Xiangxiang Chu. Lidar-ptq: Post-training quantization for point cloud 3d object detection. arXiv preprint arXiv:2401.15866 2024.
Yin Zhou and Oncel Tuzel. Voxelnet: End-to-end learning for point cloud based 3d object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4490- 4499, 2018.
Zixiang Zhou, Xiangchen Zhao, Yu Wang, Panqu Wang, and Hassan Foroosh. Centerformer: Center-based transformer for 3d object detection. In European Conference on Computer Vision, pp. 496-513, 2022.
Benjin Zhu, Zhe Wang, Shaoshuai Shi, Hang Xu, Lanqing Hong, and Hongsheng Li. Conquer: Query contrast voxel-detr for 3d object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9296-9305, 2023.
Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable transformers for end-to-end object detection. 2021.
12

---

Published as a conference paper at ICLR 2026
Deformable Box A ttention
Proposals and the grid points
Voxel features
Learned offset
bilinear bilinearinterplation interpolation
Fish Offset
Proposal feolures
MLPs
MLPs
Grid features
A ttention score
Figure 5: The detail of the deformable box attention.
APPENDIX
THE USAGE OF LLMS
In this work, large language models (LLMs) are employed solely during the manuscript preparation stage to assist with translation and language refinement. Beyond this purpose, they are not utilized for any other aspects of the study.
A DETAILS OF THE PTN
Details of DBA. The detailed structure of the Deformable Box Attention (DBA) module is illus- trated in Figure The core idea of DBA is divide each proposal into G2 grid points and learn an offset for every grid, enabling it to aggregate features from neighboring proposals. The grid features are then fused through learnable attention weights to obtain the final interaction-enhanced proposal representation. This grid-based decomposition provides two key benefits: (1) using multi- ple grid locations offers richer semantics and spatial information than relying solely on the proposal center; (2) the grid layout implicitly encodes proposal size, which strengthens both regression and classification. Specially, given a proposal b, = (x, y: 2, L, h, w. 8), where (I, y, 2) denotes the center coordinates. (1, h, w) represents the dimensional size, and 0 indicates the orientation, normalize its coordinates to range [0, 1] relative to point cloud range (æo, yo, 20, 41, 1/1 Z1 as,
first
Tog
y y/
2 2
To
V1
y0
21
20
h
w
0 + π
y/
2T
Subsequently, we convert the proposal into set of G grid points Pg. as shown in Figure 5 These grid points are projected onto the proposal voxel feature map extract the corresponding grid features Fg = 1
1 G² For all grids, we first compute the normalized offset Og (MLP((F,) from Fg. where a denotes the sigmoid function normalize the offset into [0.1]. The final sampling locations are the normalized offset locations O plus the initial grid locations Pg. Based on the sam- pling locations, we apply bilinear interpolation on the voxel feature map to obtain the sampled grid features F, = ( G The learned offsets allow grids to sample features from nearby proposals, naturally enabling pross-proposal feature interaction. In parallel, we compute attention weights for all grids: A so/ LmaaaMMLP (FF)) {ax ) Finally, the enhanced proposal representation is obtained by weighted over all sampled grid features: Fp Σk=1 ax f This design ables rich feature exchange between proposals and implicitly encodes geometric structure, leading stronger proposal refinement.
13

---

Published as a conference paper at ICLR 2026
B DETAILS OF THE EXPERIMENTAL SETTING
The intervals setting of the ground truth. In the CPRM, we categorize the estimated object num- ber CNTT into three intervals |ng 7i xxx kk=11 In the experimental implementations, we set them as [0, 20), [20, 40), and [40, 200 This setting is based on the object distribution of the dataset. As shown in Figure 6 we analyzed the distribution of object counts in the Waymo validation set and calculated the 30%, 60%, and 90% percentiles, which are roughly around 20, 40, and 100, respec- tively. Additionally, the maximum number of objects in single scene in the dataset is reached 200. For other datasets, we can also set the intervals based on the object count distribution.
Distribution of Ground Truth Object Counts
Histogram
KDE
30%; 18
17.5
Probability Density(x0.0r
12.5
10.0
60%; 43
5.0
99%% 101
2.5
100%: 200
0.0
10
50
60
70
100 110
120
130
140
150
160
170
180
200
Number Ground Truth Objects per Sample
Figure 6: The object number distribution on the Waymo.
C
MORE EXPERIMENT RESULTS
The speed and cost of each com- ponent. Table 9 details the per- formance and cost of the individ- ual modules Voxel-RCNN Deng jet al.) (20TT). The results indicate that adding HAFA improves perfor- mance by 3.52/2.74 mAP/mAPH
Table Inference and speed of each component.
HAFA CPMMMAAH (I.2) Speed (FPS) (emory())
57.45
4533
101.8
60.19
18.0
2431
21.7
61.57
17.6
2027
63.4
63.58
16.5
2995
69.6
while reducing speed by 1.5 FPS. Adding CPRM improves performance by 3.35/4.12 mAP/mAPH
and reduces speed by 1.9 FPS. When both modules are integrated, PTN yields the best performance of 66.40/63.58 mAP/mAPH with speed of 16.5 FPS. PTN strikes a balance between performance and inference speed. Furthermore, PTN exhibits reduced computa- tional complexity, with fewer FLOPs and lower memory usage compared to the baseline.
The generality of PTN. To validate the effec- tiveness of HAFA and CPRM, we applied them to PV-RCNN, with the results summarized in Table TO The experimental results demon- strate that incorporating only HAFA yields sig- nificant improvements, particularly for pedes- trians (ped) and cyclists (cyc). This is attributed to the sparse point cloud distribution of these objects, where edge details prone being lost during downsampling operations. HAFA ef-
Table 10: Performance PV-RCNN on Waymo.
3D AP/APH (I.2)
Methods
Veh
Ped
Cyc
PV-RCNN(PV)
68.0/67.5
67.6/61.6
67.7/66.5
PV+HAFA PV+CPRM
68.3/67.9
68 68.4/62.3
68. 1/67/67.2
68.8/68.2
69.2/63.2
68.7/67.6
PV4PTN
69.1/68.6
70.3/64.4
69.0/67.9
fectively mitigates this by recovering these critical details. When CPRM is incorporated solcly, performance gains observed in all categories, with the most pronounced improvements in pedes-
14

---

Published as a conference paper at ICLR 2026
trians (ped) and vehicles (vehicle). The abundance of training samples for these categories allows CPRM to facilitate enhanced learning of neighboring contextual information, thereby optimizing both classification and regression performance. The concurrent integration of both modules leads to optimal performance. This result convincingly demonstrates the generalizability of our approach.
The threshold value of score st. We pre- liminarily estimate the number of objects in scene through the score distribution of first- stage heatmaps, thereby providing prior knowl- edge for the query quantity of objects CPRM. During this process, we first apply local Non-Maximum Suppression (NMS) to the heatmap to re- move overlapping points. We then set a score threshold: regions with scores exceeding this threshold are considered target objects, while others are rejected. To obtain accurate object counts, we test dif- ferent score thresholds (experimental results shown in Table The results indicate that when the threshold is too low, the estimated quantity significantly exceeds the actual number, conversely, an excessively high threshold tends to miss targets. Experiments demonstrate that at a score thresh- old of 0.3, the gap between estimated and actual counts is minimized. Therefore, set the score threshold to 0.3 in the experiments.
Table 11: Mean average error on estimated object number with different score threshold on Waymo.
st
0.1
0.2
0.3
0.5
0.7
MAE
13
10
6
10] 17]
The Process of Overlapping Proposals. To achieve high recall, we do not handle overlap-
Table 12: Performance of PTN using different IoU thresholds in the NMS on Waymo.
ping boxes in HAFA, as those overlapping pro- posals serve as keys and values in the subse- quent decoder. Instead, we address this issue in CPRM: before selecting the top proposals as object queries, we apply NMS post-processing to suppress redundant regions, thereby alleviating conflicts during Hungarian matching. The results in [able[[2]indicate a balance between overlap and differences. As the loU increases, the overlap among retained proposals becomes greater while their differences decrease. This leads to more pronounced conflicts among the classification branches during Hungarian matching, ultimately limiting overall performance.
IoU
0.3
0.4
0.5
0.6
0.7
MAPH
60.68 62.84
63.58
61.41
59.96
The Robustness of PTN. In DETR- meth- ods Liu et al. (2024); Carion et al. (2020); Zhu et al. (021), the number of object queries rep- resents the maximum number of objects that the detector can predict in scene and is closely tied to the dataset's object distribution. Prop- erly adjusting the number of object queries is crucial. Too many may complicate training and
Table 13: Ablation study for CPRM component in PTN on KITTI. No and N, represent the object queries number from proposal and random gener- ation, respectively.
N₂
N.
Car
Cyc
120
0
0,835
0.703
180
0
0.859
0.726
240
0
0.853
0.698
increase computational resource requirements, while too few may result in missing some ob- jects. Existing methods, such as Deformable-DETR, SEED, and TransFusion, confirm this notion by demonstrating that an appropriate number of queries enhances model performance across vary- ing scenarios. In practical applications, we design the number of queries based on the object count distribution to improve robustness and reduce sensitivity. For instance, we set 180 queries for the KITTI dataset and 400 queries for the Waymo [Sun et al. (2020) dataset, reflecting their differing ob- ject counts (see Table 13 and the main paper). The performance of PTN on the occluded scenarios (Table 7 in the main paper) also demonstrates the robustness of our PTN.
180
50
0,844
0.687
In addition, evaluate the proposal and ran- dom queries on the nuScenes Caesar et al. (2020) dataset. We first analyze the distribu- tion of object counts in the nuScenes dataset and calculate the 30%, 60%, and 90% per- centiles, which are approximately 40, 60, and 100, respectively. Subsequently, we catego- rize the estimated object count CNT into three intervals: 40), [40, 60), and [60, 100 according to the object distribution statistics of nuScenes. The number of queries then set to Ki 180, K₂
Table 14: Performance on the nuScnes. Nr and Nr represent the number of proposal queries and random queries, respectively.
N
N,
NDS
mAP
44.55
36.84
240
0
44.84
36.93
300
0
45.85
37.11
400
44.49
36.41
300
100
47.81
38.93
240, and
K3 = 300 for these intervals. respectively, following the same strategy as in Waymo.
15

---

Published as a conference paper at ICLR 2026
The results presented in Table 14 demonstrate that the adaptive query number method (Np = 300) outperforms the baseline, validating its robustness on the nuScenes dataset.
The design of adaptive K. We replace our discretized method with a dense (without dis- cretized intervals) prediction strategy. Re- sults in Table 13| show that the estimated ob- ject count becomes more accurate when us- ing the dense strategy, but the detection per- formance decreases (from 64.45 mAP(L2) to
Table 15: Performance with different object num- ber estimators. MAE represents the mean abso- lute error of the estimated object count.
Methods
mAP/mAPHIL2)
MAE
with discretized intervals (ours)
64.45/61.57
6
without discretized intervals
64.11/60.97
3.9
64.11 mAP(L2)). This is because the model tends to overemphasize the adaptive K estimation branch, which interferes with the optimization of the detector.
The number of sampled keys per query. For each query, we sample g x g keys. The re- sults are presented in Table I6 With increas- ing number of the sampled keys, the detection performance of PTN can be consistently im- proved. However, the corresponding compu- tational costs are also increasing due to more sampled features being performed for query interaction, leading to more latency. Therefore, in our paper, we choose proper 5x5 default to trade off the detection performance and latency.
Table 16: Performance on the number of proposal transformer decoder layers Waymo.
Grids
mAP/mAPH (1.2)
Latency(ms)
3x3
61. 61.00/60.6
55.2
64.45/61.5
56.8
7x7
64.52/61.61
59.3
The number of the proposal trans- former decoder layer. As shown in Table 17 we evaluate the perfor- mance of PTN on different proposal transformer decoder layers. When only one proposal transformer de- coder layer is applied, PTN achieves a relatively poor performance with 61 1/.0600.6 mPP/mAPH at 1.2. As the number of layers increases, the performance improves. In this paper, we set the number of proposal transformer decoder layers to 6 to achieve better performance.
Table 17: Performance and cost on Waymo.
Layers
3D AP/APH (L2)
mAPPmAPH (L22
Memory(M) FLOPs(G)
Vort
Ped
1
59.7/61.1
63.6/59.3 59.7/61.3
51.0/60.6
2021
23,5
3
59.4/60.9
63.1/59.2 62.5/62.4
61.3/00.8
2025
39.5
61.9/114
65.6/58.6 65.8/64.5 64.4/61.5
2027
63.4
The efficiency of the CPRM. We illustrate the ac- curacy improvements of PTN over VoxeI-RCNN at various distance ranges in Figure 7 Firstly, PTN showed significant improvements over VoxeI-RCNN on the pedestrian category, where the size of the ve- hicle is 3 times larger than that of the pedestrian and there are a few points inside the pedestrians. This highlights the importance of capturing fine-grained information (such as high-frequency edges and local boundary variation) for accurately detecting small objects. Furthermore, PTN achieved larger perfor- mance gains on distant objects compared with ob- jects closer to the LiDAR sensor across all three cat- egories (especially for the pedestrian, as the point cloud distributions within pedestrian instances are typically sparse and irregular). We believe this because distant objects with fewer point clouds require more contextual information for accurate detection. Overall, these results demonstrate the effectiveness of our proposed method in detecting small and distant objects.
Overall
Vehicle
L2 mA7
Pedestrian
Cyclist
0-30
50-inf
Distance LIDAR sonsor (m)
Figure 7: Performance improvement on PTN compared with baseline on Waymo.
Computational complexity of CPRM. We evaluate the computational complexity on NVIDIA GeForce RTX 4090 GPU with a batch size of 1. Results in Table [8] indicate that de- formable cross- aotss-atteniion reduces memory costs by approximately fivefold compared to stan- dard cross- cross-attention, while the FLOPs are slightly higher due to the bilinear interpolation operation.
Table 18: Computation complexity of the de- formable and standard cross-attention.
Attention
FLOPs(G)
Memory(MiB)
deformahle crnss-attention
2995
standard cross-attention
40.4
16729
16

---

Published as a conference paper at ICLR 2026
Figure 8:
The visualization results using PTN on the KITTI. The black points represent the point clouds, with yellow boxes, green boxes, and red boxes represent pedestrians, vehicles, and eyclists respectively. This visualization demonstrates that our PTN can accurately detect objects in the scene.
Impact of RPN proposals on the object queries. The quality of RPN proposals influ- ences the convergence speed and performance of the detector. Specifically, we use the high- quality proposals to initialize potential objects (object queries), which have been proven in prior works Liu et al. all.ll(.2222. (2024)) Bai et al. (202)) to be beneficial for accelerating convergence in DETR- like frameworks. To evaluate this effect, we replace the RPN proposals with zeros-initialized queries (where the RPN proposals are extremely poor) following the standard setting of vanilla DETR. The results shown in Table 19 indicate that short schedule training with better performance than long schedule training with zeros- crros-nniiialzzed queries.
Table 19: Performance on Waymo with different queries initialization.
Methods
mAP/mAPH(L22)
proposal-initializaiioon (12 epoch)
64.45/61.57
zeros-initialization (100 epoch)
61.23/58.51
achieves
Comparison with FSD and FSDv2. When us- ing the same sampling interval as input, PTN
Table 20: Comparison with other SOTA methods on Waymo.
achieves better performance than FSD Fan et al.)
Methods
sample interval
mAPmAAPHLL2))
(2022b) as shown in Table 20 While PTN de- livers performance comparable to FSDv2, we attribute the performance of FSDv2 Fan et al.
PSD Pan 22226 PSD.
72./708
75.6/73.5
PIN
733//112
75.1/728
(2024) to its introduction of virtual voxels into the detection head, which improves the mAP L2 by 2.7%. This blug-and-play technique could be integrated into the PTN to achieve further improve- ments. On the other hand, when trained with fewer data (i.e., at sample interval of 2), PTN still achieves better performance than FSD.
The Influence of Different Coordinates CPRM. To better model occlusion relation- ships, we enhance the positional encoding by incorporating polar coordinates alongside the Cartesian features on CPRM: PE = MLP((,, y,
Table 21: Performance with different coordinates on Waymo.
3D AP/APH (L2)
Methods
Veh
Ped
Cyc
Baseline CPRM(Cartesian)
59.09/58.57 58. 20/52.47 62.73/61.33
2, 1, h, w, 0. p. Φ), where p = / x2 + y² rep- resents the radial distance from the sensor and @ atan2 x) is the azimuth angle. The explicitly establishes depth ordering from the sensor's 's perspective, enabling direct modeling of oc- clusion: when two objects overlap in angular and height coordinates (Φ. z), the object with smaller p occludes the one with larger p. Experimental results in Table show that CPRM with polar coordi- nates achieves a mAP/mAPH (L2) of 64.73/61.88 1.82, outperforming the bascline (60.00/57.43 and the Cartesian-only CPRM (64.45/61.57) 157). The improvement is particularly notable for pedestrian and cyclist classes, where occlusion handling is critical. This demonstrates that polar coordinates more naturally represent occlusion relationships and reduce model learning complexity.
61.91/61.49 65. 62/58.69 69 65.82/64.54
CPRM (Polar)
61.87/61.4 .45 65.77/58.84 66.56/65.17
The visualization results. The visualization results using our PTN on KITTI Geiger et al. (2012) shown in Figure From the results, we can see that our PTN can detect most objects, especially for small objects (such as pedestrians). This is attributed to the HAFA module, which efficiently retrieves the lost detail information in the downsampling process. This information is important for the detector to accurately locate small objects.
17


Reviews:
[{"review_id": "ImTQgoaKEc", "reviewer": "Reviewer_uUA1", "summary": "This paper introduces PTNET, a novel Proposal-centric Transformer Network designed to enhance two-stage 3D object detection from LiDAR point clouds. PTNET targets two key limitations of current two-stage methods: 1) the degradation of geometric details in proposal features due to point cloud sparsity and pooling operations; and 2) the failure to leverage contextual clues from neighboring proposals during the refinement stage, which traditionally treats each proposal independently. Experimental results demonstrate that PTNET achieves state-of-the-art (SOTA) performance on the Waymo and KITTI large-scale benchmarks.", "questions": "- How robust is the adaptive query number estimation (Eq. 5–6) to datasets with different object count distributions, such as nuScenes or Argoverse?\n- The CPRM employs deformable cross-attention among proposals. What is its computational complexity compared with standard cross-attention, and how does it scale when the number of proposals increases?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The paper is well-motivated, introducing DETR into the second stage of 3D detectors to aggregate information from the full ROIs\n- The authors conduct extensive experiments on two major autonomous driving benchmarks (Waymo and KITTI), demonstrating state-of-the-art or highly competitive performance across multiple categories, especially for pedestrians and cyclists. The multi-frame input results further validate the model's robustness.", "weaknesses": "- While the combination of HAFA and CPRM is well executed, the individual ideas (multi-granularity alignment and proposal interaction through attention) resemble previous works such as PV-RCNN++[1] and ConQueR[2]. The contribution may thus be seen as an engineering refinement rather than a conceptual breakthrough.\n- The fine-grained branch (FPFR) in HAFA is meant to recover geometric detail from \"raw foreground point clouds.\" As described in Section 3.2.2, this module \"first selects foreground points $\\text{P}'$ whose locations are inside the proposal $b$.\" This introduces a limitation: if an RPN proposal $b$ is poor (e.g., a tiny bounding box due to heavy occlusion), FPFR can only access points within that incomplete proposal. It can therefore only \"sharpen\" the features within the existing, poor boundary, but seems inherently unable to \"complete\" the true geometric shape by accessing points that belong to the object but lie outside the RPN's initial prediction.\n\n[1] Shi, Shaoshuai, et al. \"PV-RCNN++: Point-voxel feature set abstraction with local vector representation for 3D object detection.\" International Journal of Computer Vision 131.2 (2023): 531-551.\n[2] Zhu, Benjin, et al. \"Conquer: Query contrast voxel-detr for 3d object detection.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PkvC0yLgKy", "reviewer": "Reviewer_1kqr", "summary": "This paper proposes a Proposal-centric Transformer Network (PTN) for 3D object detection from LiDAR point clouds. PTN addresses two main challenges in two-stage 3D detection: degradation of proposal geometric details due to point sparsity and ineffective use of contextual cues during refinement. PTN introduces two core modules: Hierarchical Attentive Feature Alignment (HAFA), which extracts and aligns coarse voxel and fine point features within each proposal, and Collaborative Proposal Refinement Module (CPRM), which integrates contextual interaction among spatially and semantically related proposals via hybrid queries and deformable attention. Extensive experiments on the Waymo and KITTI benchmarks demonstrate performance improvements over representative prior methods.", "questions": "Is the position embedding based on xyz only? How can such position embedding represents \"occlusion\" between proposals?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The Collaborative Proposal Refinement Module is the first proposal-centric transformer to refine the bounding boxes generated from RPN. The idea is interesting, and the performance gain is significant. With the random queries, it can recall weak proposals caused by distance or occlusion.\n2. The final performance on the Waymo Open dataset (single-frame setting) is impressive, especially on small objects.\n3. Ablation studies isolate the effect of each module, providing transparency on their contributions to accuracy and speed. Table 9 offers inference speed comparisons per module, supporting claims of balanced efficiency and performance.\n4. The paper contextualizes itself against a range of competing methods, including both DETR-style and non-Transformer 3D detectors, and references and discusses most major prior works.", "weaknesses": "1. One of the effects of CPRM is that it is an end-to-end module. However, the authors still apply NMS before the module, which weaken the end-to-end feature. Moreover, I would like to see a comparison between CPRM and classic RCNN module (e.g. Voxel-RCNN) with NMS.\n2. Section 3.3.3 introduces a “3D parameter-guided deformable attention,” but omits critical specifics: how offsets are computed per proposal, how attention ranges are constrained spatially, and what prior (if any) is imposed by the box parameters. A clear symbolic or algorithmic formulation is needed for reproducibility and transparency.\n3. The multi-frame performance is not superior to pervious work CenterFormer. Does the proposal collect features from the \"tail\" caused by object movements?\n4. Current state-of-the-art works, such as FSD and FSDv2, are not cited and compared.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QY0Utc5bGn", "reviewer": "Reviewer_Hw6U", "summary": "The paper tackles the proposal-quality bottleneck in two-stage LiDAR 3D detectors by introducing PTN, a proposal-centric transformer composed of two key parts: Hierarchical Attentive Feature Alignment (HAFA) and a Collaborative Proposal Refinement Module (CPRM). HAFA is a dual-stream design that fuses coarse, multi-scale voxel features with fine, foreground point features and then aligns them in a unified space to strengthen geometric detail in proposals. CPRM creates hybrid object queries—top-K proposal queries plus learnable random queries—and performs 3D parameter-guided deformable attention across proposals to share contextual cues, especially helpful under occlusion and sparsity. Experiments on Waymo and KITTI report consistent gains; e.g., Waymo test mAP/mAPH (L2) of 72.7/70.6 with strong category results, and competitive KITTI performance, alongside ablations justifying each component.", "questions": "1. Deformable attention specifics: How are reference points and offsets parameterized from 3D boxes? What’s the exact number of sampled keys per query and the computational/memory cost per decoder layer?  \n2. Query budgeting: Beyond global K, did you try class-aware (per-class) query budgets or adaptive K per scene without discretized intervals? How sensitive is performance to K and the random-query count M across datasets?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. Clear problem framing: identifies two concrete issues—loss of geometric detail in proposals and lack of cross-proposal context—and directly designs modules to address each.\n2. Well-motivated dual-stream features: combining grid-sampled voxel tokens (coarse) with raw foreground point cues (fine) is intuitive and technically grounded, with an explicit alignment step.  \n3. Ablations support claims: both HAFA and CPRM contribute; component-wise tables and studies on query counts/NMS thresholds/decoder depth are informative.", "weaknesses": "1. Added complexity & runtime: HAFA + CPRM introduce nontrivial overhead and multiple hyperparameters (e.g., grid sizes, query counts, NMS/thresholds); the paper discusses trade-offs qualitatively but detailed compute/memory costs and scaling behavior are limited.\n2. RPN dependence: although CPRM adds random queries, the pipeline still leans on RPN quality; failure modes when RPN proposals are poor (domain shift, long-range sparse objects) are not deeply analyzed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)