RAPTR: Radar-based 3D Pose Estimation using Transformer
Authors: Sorachi Kato
Ryoma Yataka
Pu Perry Wang
Pedro Miraldo
Takuya Fujihashi
Petros Boufounos
Venue: NeurIPS 2025 poster ()
Keywords: radar perception
3D pose estimation
transformer
deformable attention
Primary area: applications
TL;DR: Estimate 3D human poses from multi-view radar data using 2D image-plane keypoints and 3D BBox labels, rather than more expensive 3D keypoint labels.

Abstract: Radar-based indoor 3D human pose estimation typically relied on fine-grained 3D keypoint labels, which are costly to obtain especially in complex indoor settings involving clutter, occlusions, or multiple people. In this paper, we propose \textbf{RAPTR} (RAdar Pose esTimation using tRansformer) under weak supervision, using only 3D BBox and 2D keypoint labels which are considerably easier and more scalable to collect. 
Our RAPTR is characterized by a two-stage pose decoder architecture with a pseudo-3D deformable attention to enhance (pose/joint) queries with multi-view radar features: a pose decoder estimates initial 3D poses with a 3D template loss designed to utilize the 3D BBox labels and mitigate depth ambiguities; and a joint decoder refines the initial poses with 2D keypoint labels and a 3D gravity loss.
Evaluated on two indoor radar datasets, RAPTR outperforms existing methods, reducing joint position error by $34.3$\% on HIBER and $76.9$\% on MMVR. Our implementation is available at \url{https://github.com/merlresearch/radar-pose-transformer}.

RAPTR: Radar-based 3D Pose Estimation using Transformer
Sorachi Kato 1.2- Ryoma Yataka Pu (Perry) Wang's ! Pedro Miraldo',
Takuya Fujihashi", Petros Boufounos'
Mitsubishi Electric Research Laboratories (MERL). USA The University of Osaka, Japan
Information Technology R&D Center (ITC), Mitsubishi Electric Corporation, Japan
Abstract
Radar-based indoor 3D human pose estimation typically relied on fine-grained 3D keypoint labels, which are costly to obtaín especially in complex indoor set- tings involving clutter, occlusions, or multiple people. In this paper, we pro- pose RAPTR (RAdar Pose esTimation using tRansformer) under weak supervi- sion, using only 3D BBox and 2D keypoint labels which are considerably eas- ier and more scalable to collect. Our RAPTR is characterized by a two-stage pose decoder architecture with pseudo-3D 3 deformable attention enhance (pose/joint) queries with multi-view radar features: a pose decoder estimates initial 3D poses with 3D template loss designed to utilize the 3D BBox la- bels and mitigate depth ambiguities; and a joint decoder refines the initial poses with 2D keypoint labels and 3D gravity loss. Evaluated on two indoor radar datasets, RAPTR outperforms existing methods, reducing joint position error by 34.3% HIBER and 76.9% MMVR. Our implementation is available https://gii httpaaa....meeeeeeeeereeeee////e ppoee- transf orrer..
Introduction
Accurate human perception is essential for indoor applications, including elderly monitoring, smart building management, and robotic navigation, Although vision sensors offer high spatial resolution, they raise privacy concerns and perform poorly under low light, occlusions, and hazardous conditions (fire or smoke). In contrast, radar provides penetration capability, robustness to adverse conditions, and low deployment cost, ideal for privacy-preserving indoor sensing [44, 18, 23, 30, 26).
By processing 4D radar tensors, RF-Pose 3D [48] demonstrated through-ther theth-lth-wwlll wall 3D pose estimation with convolutional neural network (CNN), while HRRadarPose [11] employed an hourglass neural network HRNet [34]. mRI [1] is a multi-modal 3D human pose estimation dataset that integrates Wawe radar, RGB-D cameras, and inertial sensors to facilitate research in human pose estima- tion and action detection. QRFPose [33] a novel approach that adopts a DETR [3]-style query mechanism for end-to-end 31) regression using multi-view radar perceptions. Existing pipelines often rely on expensive fine-grained 3D keypoint labels [35], typically collected using non-portable 3D motion capture systems such as VICON, using LiDAR, which can still suffer from occlusions and incomplete observations.
Collecting cheaper, lower-cost labels, such as fine-graine grained 2D keypoints in the image plane and/or coarse gresse-rainedddee 3D bounding boxes (BBoxes), is considerably easier and more scalable particularly in complex indoor settings (c.g., cluttered, occlusion, multi meutio, compared with acquiring dense
"The work was initiated during his internship at MERL. I Project Lead.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Input radar heatmaps
RAPTR: Radar Pose Estimation using Transformer
mated
Training
Coordiaae
System
Poses
Teauformsion
Hortzontal
3D
Parage
counse gra ned
3D
Vertical
Pose Queries (Pose Decoder)
Joint Queries (Jint Decoder)
3D World
chouper label
spediic
Cooncinate: System
Figure 1: RAPTR takes multi-vier view radar heatmaps inputs and performs novel Pscudo-3D deformable attention between (pose and joint) queries and multi-view radar features a two-stage decoder to estimate 3D human poses in 3D coordinate system. Rather than relying expensive, environment-specifee fine- tne-grained 3D keypoint labels, RAPTR makes use of cheaper, more scalable labels such coarse-gre saanse-griieddd 3D BBoxes and fine- seee-grrind 2D keypoints train the model.
3D keypoints labels. Examples include RF-Pose [47], HuPR [15], and, more recently, MMVR [24] datasets. To the best of our knowledge, the use of 2D keypoints and 3D BBoxes, as a substitute for costly 3D keypoints, for radar-based 3D human pose estimation has not been systematically investigated in the literature before.
To address this gap, we propose RAPTR (RAdar Pose esTimation using (Ransformer) in Fig. 1, radar-based pipeline designed to take multi-view radar heatmaps inputs and estimate 3D human poses under weak supervision using only 3D BBox and 2D keypoint labels. RAPTR builds on the two-stage (pose and joint) decoder architecture of the state-of-ther ofte--of-the-ar art RGB-based 2D pose estimation PETR framework [28] and introduces a structural loss function that designed to utilize weak supervision labels to mitigate the depth ambiguity. RAPTR also lifts the 2D deformable attention in PETR to pseudo-3D deformable attention, wherein reference points (dots in Fig. 1) and offsets (arrows in Fig. I) are proposed in the 3D radar coordinate system and projected onto multiple radar views (dots on the radar heatmaps in Fig. 1) to eliminate redundant per-view offset estimation and offer better scalability as the number of radar views increases. Our model outperforms list of radar-based 3D pose baselines over two indoor radar datasets: HIBER [35] and MMVR [24]. The main contributions of this work are:
To the best of our knowledge. RAPTR is the first radar- based 3D human pose estimation framework to explicitly utilize low-cost weak supervision in the form of 3D BBoxes and 2D keypoints, rather than relying on fine-grained 3D keypoint labels.
We introduce structured loss function that tightly couples the two-stage decoder architecture to enable 3D pose estimation under weak supervision. Specifically, we design a 3D Template Loss, which utilizes the 3D BBox labels at the pose decoder, and combined 3D Gravity and 2D Keypoint Loss at the pose decoder, allowing RAPTR to effectively learn geometrically consistent 3D poses from weak supervision.
We further introduce a pseudo-3D deformable attention mechanism to bridge the 3D spatial domain and 2D radar views, enabling scalable view association while preserving pose estimation performance.
Related Work
Human Pose Estimation with RGB Image: Human pose estimation from images involves localiz- ing body joints for multiple subjects and associating them for each subject. Existing architectures fall into two main paradigms: top-down and bottom-up. The top-down methods first detect each person using detectors such as Faster R-CNN [25] or Mask R-CNN [10], then applying single-person pose estimator to each cropped region. These approaches achieve state-of-the-art accuracy with models like Stacked-Houree Hourglass [22], HRNet [31], and DarkPose [46]. In contrast, bottom-up methods such as OpenPose [2], Hit [6], and SAHR [19] bypass the detection step by predicting all joint candidates across the entire image and grouping them into individuals. PETR [28] introduces

---

end-to-end pose estimation framework using a query-based, two-stage transformer decoder architec- ture. Beyond 2D, recent methods addresses 3D pose from RGB or RGB-D inputs, either by directly regressing 3D joints [20] or by lifting 2D predictions into the 3D space through geometric reasoning or weak supervision [4, 5, 21, 32].
Human Pose Estimation with radar or radio frequency signals: Recent studies have shown that information extracted from commercial radars is sufficiently informative to perform fine-grained human pose estimation, both for 2D and 3D. Despite the coarse-grained nature of the radar point clouds (PCs), deep neural pipelines have achieved multitude of performance gains [29, 27, 38, 1, 41,36, 12, 14, 39, 42, 8, 7. 43]. On the other hand, methods using raw radar measurements and radar heatmaps have been widely explored [47, 49, 15, 35, 11, 24, 33, 45, 37]. RF-Pose [47] pioneered multi-view 3D CNNs for through-wall 2D estimation. HuPR [15] refines such heatmaps via graph convolutional network (GCN). HRRadarPose [11] adopts an HRNet-style [34] single-stage head for 3D output. QREPose [33], based on a DETR-style Transformer [3] for end-to-end query-based 3D pose estimation, is the closest baseline to ours. It differs by applying verwview 2D deformable
attention and using a single decoder for all keypoints, followed by grouping. In contrast, our method employs pseudo-3D 3D deformable attention and two-stage decoder.
Preliminary
Multi-View Radar Heatmaps: As shown in Fig. 2, two synchronized radar arrays (horizontal and vertical) collect re- flected pulses that form a 3D data cube per array (ADC sam- ples x pulses x elements). A 3D FFT converts each cube into a Lange-oopleerannile spectrum, whose angle dimen- sion is azimuth for the horizontal array and elevation for the vertical array. After Doppler-axis integration to boost SNR
axondrate
Intortonaaal Hexampp
Punge (m]
Depth PP
Armuuh ((ag)
Coodinate Trantborata atar
Horbor
Merten Heatrap
(signal-te -onnal-t--noiee ratio), we obtain two polar 2D heatmaps Longe-azimuth and range-elevation). These are mapped to the Cartesian space: Y hoz(()) (t) RWxD for horizontal- depth and Y Yee(()) E H for vertical-depth at frame t. The temporal context is captured by stacking T consecutive frames, giving Yoor E IR X WxD and Y E R
Rerge (nl
Dest
Eleadoon (dhg)
Vencal mn
Figure 2: Multi-view radar heatmaps.
Problem Formulation: The 3D pose estimation task takes consecutive radar frames, Yoor and
Yvar, as input and estimates poses P Pvorid in the 3D world coordinate system,
(P P Pvorld Trze (P radar) Tr (or. Yvar)).
where f represents the 3D pose estimation pipeline in the 3D radar coordinate system, and Trzv is a known radar-to- anda-tt-worrdd coordinate transformation that converts the estimated 3D poses into the 3D world coordinate system. Rather than relying on costly, non-scalable fine mme-grained 3D keypoint labels P Pworid. consider cheaper, more scalable labels such coarse-grand -aasine 3D BBoxes Byorld and fine-grained 2D keypoints P Pinage for supervision, as shown in Fig.
4 RAPTR: Radar-based 3D Pose Estimation using Transformer
We present the RAPTR architecture in Fig. 3, following left-to-right order, and highlight radar- specific modifications. Refer to Appendix A for detailed architecture and computational complexity.
4.1 Architecture
Backbone:
Given Yoor E TxXxDD and Y E RTxHxD a shared backbone network (e.g.. ResNet [9]) generates separate multi-scale horizontal-view and vertical-view radar feature maps: Znor = { 19 backbone (Ynor) hor) and Zver = [Zvar.). S
backbone YY,r), where the i-th scale feature maps Zhor,: E W.Dxxd and Zuaa, E H, xd have a spatial dimension of W D, H, x D, and a feature dimension of d, and S is the number of scales.
3

---

Joint Reg Head
Ograncty
deuntey JON
Excesone Aon
Refined estimates
pro Iaage
Pseudo-30 Joint Decoder
Pwrrk
Crossriie Encodeer
Lagt20
Liuss
20 Keypaints
Reg. Head
druncou at
Initial pose estimates
Crmppee
Pseudo-3D 3D Pose Decoder
30 BBoxes
Pose Estimates
Labels
Figure 3: The RAPTR architecture consists of: 1) Cross-view Encoder that extracts multi-scale radar features; 2) Pseudo-3D Pose Decoder that enhances pose queries via pseudo-3D deformable attention and predicts initial 3D poses; and 3) Pseudo-3D Joint Decoder that further refines joint queries and outputs final 3D poses. terms of loss function, RAPTR leverages 3D BBox and 2D keypoint labels through coarse-grained 3D loss (gravity and template) and 2D keypoint loss.
Cross- View Encoder a Transformer encoder with Lane layers that fuses the horizontal- and vertical- view radar features. Each layer runs a shared cross-attention twice: first with Zhor as key/value and Zeer as query, then vice versa. This bidirectional exchange embeds complementary cues, while residual connections keep view-specific details, producing refined features F (()) i 1, 1, ... Lene.
F(.) = F(i11) + CrossAttn (F(11)) F,,,F () (), E {hhor ver), (ver, hor)),
(2)
((U) where hor Zpori Flu = Zver, and ArossAttn(:; denotes the deformable cross-attention [50] following [28] with fixed positional embeddings added beforehand for efficiency. After Tonn iterations, the encoded features Foor and F Ever are obtained at the output of the cross-view encoder.
Pseudo-3D Pose Decoder associates N pose queries Qpose N (embedding dimension d) with encoded radar features (Froz Frez), where each query corresponds to reference pose refined through pseudo-3D deformable attention over Lyose layers. We define the 1-th decoder layer function Drose that updates both the pose queries and reference poses in the 3D radar space:
(Q P(!)
poss> P = D(!) pose Opose" Foor;
Ever, ((1-1))
redar 1
where P (0)
radar initialized by passing Qpoza through MLP. Reference poses iteratively refined
by applying predicted coordinate offsets AP radar in the normalized scale:
(1))
redar E Nx3K = o ((Ptl))) radar + AP(l-1)) I= 1,..,, Iiposa:
where a and o denote the Sigmoid function and its inverse. The predicted offsets AP (I)
rader
1l pose Qpese) are obtained by passing pose queries each layer shared regression head Hpose.
We convert the initial pose estimates P Prador = ((lga))) from the radar coordinate system to the world coordinate system via Pveria (Pa r. along with the corresponding confidence scores č. We defer the pseudo-3D deformable attention to Section 4.2.
Pseudo-3D Joint Decoder associates joint queries Qjoint E KR with encoded radar features (F bor F ). where each query corresponds a single joint refined by pseudo-3D deformable attention over Jooit layers. Here, K joint queries correspond to the same subject. We define the (-th
decoder layer as a function (!!) joint that updates both the joint queries and corresponding joints:
QQ joint: ((0)) (arader) =
(1)
joint f Fnor. F Fver. P P!
(5)
where Paraddrr R K =, l...., .N is one specific pose in the poses, and P v is i-th pose prediction from the pose decoder. Joints in the reference pose are iteratively refined by applying predicted coordinate offsets Ap,
(1)
(0)
p ( E TR. Kx3 a(aa (pp
(1-1 ) + Ap
(1-1) ( I= 1. Lyoinee
(6)

---

where the predicted offsets are given Pirradar H joint (Q with a shared regression head.
(digiiat) J.N rararr Ji=1 and convert
We collect refined reference poses from the joint decoder Predar them into the 3D world coordinate system as P Pvorld Teau (redar).
{P,
4.2 Pseudo-3D Deformable Attention
Our two-stage decoder incorporates a pseudo-3D deformable attention mod- ule, where "pseudo" highlights that ref- erence points and sampling offsets are defined in 3D space, while feature sam- pling occurs on the 2D radar views, as illustrated in Fig. 4.
sampting (ax,
Spprodt
Austo
UCA
Cits
Consider 3D reference point (z, y, 2) in the 3D radar space with a correspond- ing query q Rd (from either pose queries in the pose decoder or joint queries in the joint decoder). We first feed q into linear projection layer to predict a set 3D sampling offsets (Ar,, Ay, } Nattur Given the 3D reference point and sampling offisets, we can locate the 3D sampling coor- dinates and project them onto the two radar views, extracting deformable multi-view radar features:
Adjustec Intertoo we ghits
Aguitaalic
3D
W
Curnut
Figure 4: The pseudo-3D deformable attention operates on 3D reference point and 3D sampling offsets that are projected to different radar views for pseudo-3D attention between multi-view radar features and the query.
(i) Fnor (x: + Ar,; = + Az;), hor
p()) ver
Fver (y + Ayi, + Az;) i= i= 1,... Notract
We group deformable multi-view radar features Fatta ( (1)
(Nattro)) (NNottor)) Tbor
Eyar
Meanwhile, multi-view attention weights Aatun E NN-occoonx (where corresponds the two radar views) are proposed by linearly projecting the query and applying softmax normalization. These weights capture the relative importance of radar features across the two views a unified manner.
Given the deformable multi-view radar features F Fatta and the multi-view attention weights Aattn deformable multi-view attention features can be calculated as
Nottous
F aattn
(Ar.0 WF (2i-1) + A,, WF Ftt))
(8)
where A, and 1 are the attention weights in Aatta for the i-th deformable radar feature in the hori- zontal and, respectively, vertical radar views, and W E RCxC' is learnable weight matrix. We denote the overall pseudo-3D deformable attention
F Fartn DeformableAttn( Fvar, Foors (x, 2), q).
Appendix provides implementation details of Def Deformablettal plexity comparison with the decoupled 2D deformable attention used in QRFPose [33], demonstrating better scalability of the proposed pscudo-3D attention as the number of radar views increases. Ap- pendix C describes optional view mask module (top right of Fig, 4) that adds flexibility in selecting multi-view radar features per query. For example, an all-zero mask can be applied to exclude features from specific radar view.
and a computational com-
4.3 Structural Loss Function
As illustrated in Fig. RAPTR utilizes weak supervision labels: coarse-grained BBox labels Bzorld in the 3D world coordinate system and fine-grained 2D keypoint labels P inage in the image plane. The loss function calculated between these labels (B P Pinage] } and the initial and refined 3D pose estimates (P vorld: P Pooraa} with details included Appendix D.
3D Template (T3D) Loss at Pose Decoder utilizes coarse-grained 3D BBox labels B For each Byorid, we construct 3D keypoint template by computing the centroid of the corresponding 3D BBox, which serves the 3D gravity center label gaorld [ |x3
5

---

Then, given a keypoint template defined at the coordinate origin K E Ix the corresponding template pose Terrid computed as T yorld Kerrd Ezorld. As illustrated in the lower right of Fig. 3, the T3D loss Lromplate is defined as the Fuclidean distance between the template poses
Tooria and the initial 3D pose estimates P voria at the pose decoder.
Combined 3D Gravity (G3D) Loss and 2D Keypoint (K2D) Loss at Joint Decoder utilizes both the coarse-grained 3D BBox labels B Werld and the fine-grained 2D keypoint labels P Pinage in the image plane, as illustrated in the upper right of Fig. 3
For the G3D loss, the refined 3D pose estimate Perri is collapsed into its centroid as gworld R RIx3 by averaging the keypoint coordinates along each spatial axis. The resulting G3D loss Lgrrvviyy is then defined as the Euclidean distance between the predicted and ground-truth 3D gravity centers, Exorid and gxarld.
For the K2D loss, the refined 3D pose estimate P Pradar in the radar coordinate system are first transformed into the 3D camera coordinate system via a calibrated coordinate transformation:
P cannraa RP redar + 11t t where R and denote the calibrated 3D rotation matrix and the translation vector, respectively. The resulting 3D camera-space pose estimates are then projected onto the 2D im- age plane via a known 3DD-t-22D projection: P imago proj inage (P PPannrra. Finally, the fine-grained 2D loss combines the image-plame Euclidean error L2pe2D and the object keypoint similarity (OKS) loss Laks [28] between P imaae and P imago-
Structural Loss Function: Following the set-based loss [3], we employ bipartite matching to associate predictions {gvorid, Pworld: Pvoria] with their ground labels (gvorid, T Tworid; P Pvoria). Based on these associations, define the structural loss function
N
C
Σ (A1 Leemplate + >22ggrvvtty + A2 + A4 + A5 N
where N' is the number of matched pairs, A; is the corresponding weighting factor for each loss term, and Leis is the classification loss of the focal loss [17] with the confidence scores of the matched estimates.
5
Evaluation
5.1 Settings
Datasets: We assess the performance of RAPTR and baseline models the HIBER dataset² [35] and the MMVR dataset' [24], both of which are publicly available multi-viww mm Wave radar datasets designed for indoor human perception tasks. The HIBER dataset includes wo-view radar heatmaps from 10 different viewpoints, the corresponding 3D keypoint labels, and the 3D BBox labels. We use data protocols "MULTT" and "WALK", and use views 2 through 10 for training, validation, and testing. The MMVR dataset includes two-view radar heatmaps in various indoor scenarios, the corresponding 2D keypoint labels, and the 3D BBoxes. We use data split "PISI", single-person case in an open space. A detailed description of the datasets is provided in Appendix E.
Parameter Settings for RAPTR:
We use T 4 consecutive frames input to RAPTR network. For the point decoder, the number of pose queries N is 10. For the joint decoder, the number of joint queries K depends on the dataset be evaluated: K = 14 for HIBER and K 17 for MMVR. The parameters relating to model training are summarized in Appendix F.
Baselines: We consider the following competitive radar/RF-based 3D pose estimation baselines:
Person-in- WiFi 3D [40], HRRadarPose [11], and QRFPose [33]. We evaluate Person-in- WiFi 3D and HRRadarPose using their open-souree implementations. As QRFPose has public code,
reimplement it from scratches and verify similar performance to the original report [33] using 3D
keypoint labels. For fair comparison, we adopt loss function, similar to RAPTR, combining 2D
keypoint loss and 3D gravity loss. Baseline implementation details are provided Appendix G.

---

Table 1: 3D pose estimation performance on HIBER (MPJPE: cm).
Env
Method
Head
Neck
Shoulder
Elbow
Wrist
Hip
Knee
Ankle
Overall
(b)
(v)
(d)
Person-in- Wirsonin-WiFi 3D
54.28
57.01
54.18
54.81
59.98
53.98
60.32
68.86
58.25
25.60
23.94
36.20
MULTI WALK
QRFPose
42.23
34,21
37.37
38.05
41.25
31.24
34,39
46.87
38.20
14.78
13.40
26.76
HRRadarPose
30.23
25.44
42.33
27.71
31.55
40.46
33.96
15.14
13.13
19.85
RAPTR (ours)
21.75
17.41
20.72
23.23
26.55
18.97
21.06
26.10
22.32
8.41
4.85
17.73
Person-in in WiFi 3D
88.48
85.14
89.44
84.33
84.29
88.69
81.70
81.53
85,25
34,06
28.57
58.93
QRFPose
49.49
44.48
45.54
46.77
49.06
40.99
41.87
$1.57
46.11
18.20
14.13
34.39
HRRadarPose
30.24
24.24
35.17
44.34
28. 28.76
31.38
35.31
33.19
16.77
10.75
RAPTR (ouns)
18.39
13.13
16.44
20.12
24.62
15.01
17.76
23.22
18,99
7,80
4.38
14.54
Metrics: We employ Mean Per Joint Pose Error (MPJPE) with the unit of centimeters in the world coordinate. In addition, we evaluate this MPJPE for each body joint and along each 3D axis, horizontal (h), vertical (v), and depth (d), independently. For MMVR, since 3D keypoint labels are not available, we construct a 3D bounding box (BBox) that encloses the estimated 3D keypoints and then use the distance between the center points of this BBox and the 3D BBox labels, as well as the absolute error in the lengths of the edges along each axis of the box, as metrics to approximate the 3D pose estimation performance. Detailed evaluation metrics are described in Appendix H.
5.2 Main results
Table 1 shows the performance of 3D pose estimation for HIBER, using 2D and coarse 3D labels for baselines and our RAPTR. The qualitative results are provided in Fig. 5a.
HIBER:
For WALK, RAPTR achieves a significantly lower overall MPJPE of 22.32 em and outperforms all other baselines in the metric. More specifically, RAPTR reduces the overall error by 61.7%, 41.6 41.6%, and 34.3% compared to Person-in- WiFi 3D, QRFPose, and HRRadarPose, respectively. The per-joint breakdown demonstrates that RAPTR maintains its performance on relatively challenging joints, such as the wrist and ankle, where other baselines exhibit significant degradation. For example, HRRadarPose reports a wrist error of 42.33 cm, whereas RAPTR reports an error of 26.55 cm. Moreover, RAPTR maintains the error gap between the best- and worst-estimated joints within 10 cm, showing a consistent level of accuracy throughout the body. In terms of directional components, RAPTR shows much lower errors in the horizontal and vertical dimensions than baselines, indicating that RAPTR estimates well cell-proportioned 3D poses across all axes.
For MULTI, the more challenging multi overall MPJPE of 18.98 cm and shows a substantial margin compared to the second-best HRRadar- Pose at 33. 19 cm. RAPTR reduces the overall error by 77.7%, 58.8%, and 42.7% compared to WinF 3D, QRIPose, and HRRadarPose, respectively. Although the overall accuracy of Person-in-Wifi 3D and QRFPose, noticeably degrades on the MULTI split compared to WALK, likely due to the increased complexity of handling multiple objects, RAPTR maintains a nearly consistent level of performance.
scenario, RAPTR continues to outperform with an
Referring to the qualitative results provided in Fig. 5a, RAPTR estimates structurally consistent 3D poses that match the 3D labels in both position and orientation, while baselines often suffer from misaligned limbs and implausible joint configurations. While baselines often fail to maintain human- like pose structure in the MULTI setting despite performing well in WALK, RAPTR consistently produces plausible estimates in both scenarios, indicating its robustness to multi mulli-persoon scenes.
Table 2: Pose estimation performance on MMVR (PISI).
Edge length error (cm)
Method
Center distance (em)
(v)
Person-in-Wiis 3D
136.14
33.18
95.43
242.86
QRFPose
210.75
38.12
73.69
409.38
HRRadarPise
164.46
37.84
74.00
313.81
RAPTR (ours)
31.41
22.90
10.66
50.56
MMVR: Table 2 shows the performance comparison for baselines and RAPTR with MMVR, and Fig. 5b provides qualitative results. Although we cannot directly evaluate the precise 3D pose estimation performance for MMVR due to the absence of 3D pose labels, the results demonstrate that RAPTR effectively preserves reasonable human pose and location accuracy in the 3D space. Specifically, RAPTR shows improvements in center distance by 76.9%, 85.1%, and 80.9% compared to Person-in-Wind 3D, QRFPose, and HRRadarPose, respectively. As shown in Fig. 5b, other
7

---

Labels
RAPTR (ours)
QRFPose
HRRadarPose
Person-in- 33
3D Labels
R
A
(a) Visualization of 3D pose estimation by RAPTR and baseline methods on the HIBER dataset.
Labels RAPTR (ours)
QRFPose
HRRadarPose
Person-In-WiFl
(b) Visualization 3D pose estimation by RAPTR and baseline methods on the MMVR dataset.
Figure 5: Qualitative results. Blue lines indicate the keypoint labels, blue rectangles indicate the 3D BBox labels, and red lines indicate the predictions.
baselines exhibit degraded performance due to structural collapse in 3D space, caused by overfitting to 2D alignment when projected onto the image plane. We assume that RAPTR effectively avoids this issue by not directly predicting the keypoints, but instead refining the final output through 2D keypoint supervision applied to each joint a template pose that is placed the 3D space.
Ablation Study
In this section, present ablation studies of our RAPTR the HIBER dataset. Unless otherwise stated, all reported evaluation results are reported as the mean + standard deviation, computed over three random seeds. Additional ablation results and visualizations are provided in Appendix I.
Visualization Pose Refinement Process: Fig. illustrates refinement process of prediction through the two-stage decoder architecture. The pose decoder first establishes coarse 3D structures of the human body under the constraint of the 3D template loss (Ist row). Subsequently, the joint decoder fine-tunes the keypoints to better capture the subject orientation and limb configuration (2nd row), while preserving the structure consistency provided by the pose decoder.
Effect of Loss Terms: Table 3 provides an ablation study on the effect of different combinations of loss terms in the two-stage decoder. When only the K2D loss applied at the joint decoder (row 1), the 3D pose estimation suffers from depth ambiguity due to the absence of any 3D constraint, resulting substantial increase MPIPE 381. cm and 375. cm on the WALK and MULTI splits, respectively. From rows to 4 in Table remove modify loss term at a time from

---

Joint Decoder
20 Labels
Figure 6: Pose Refinement Process: the pose prediction is first constrained by the 3D template the pose decoder, and subsequently refined at the joint decoder.
the proposed structural loss. Removing the T3D loss at the pose decoder (row 2). replacing the T3D with the K2D+G3D loss at the pose decoder (row 3), or removing the G3D loss at the joint decoder (row 4) leads to a noticeable degradation in 3D pose estimation.
Table 3: Effect of loss terms for RAPTR (MPJPE:cm)
Loss
WALK
MULTI
Notes
Pose Dec.
Joint Dec.
K2D
381.18
0.28 375.73 + 6.31
2D keypoint loss only at joint decoder
K2D+G3D
28.54 = 4.57
57.90
9.81
without 3D template loss at pose decoder
K2D+G3D
X2D+G3D
27.49 = 3.40
23.43 + 3.44
with 2D keypoint + 3D gravity loss at both decoders without 3D gravity loss at joint decoder
T3D
K2D
25.96 4.95
25.83 + 3.87
T3D
K2D+G3D
22.32
0.06
18.99 1 0.16
proposed structural loss
K2D 2D Keypoint loss, T3D 3D Template loss, G3D 3D Gravity loss
Effect of Deformable Attention Mechanisms: Table presents an ablation study on the effect of the deformable attention mechanism for RAPTR. In this study, the pseudo-3D deformable attention is replaced with the decoupled 2D deformable attention used in QRFPose [33], while keeping the cross-view encoder, two-stage decoder architecture, and the proposed structural loss unchanged. The results show that the pseudo-3D deformable attention yields marginal performance improvements, approximately 4% and 2.5% on the WALK and MULTI splits, respectively.
Table 4: Effect of deformable attention mechanisms for RAPTR (MPJPE: cm).
Attn.
WALK
MULTI
Notes
2D 23.25 + 1.38 19.47 + 0.95
RAPTR with decoupled 2D deformable attention
22.32 + 0.06 18.99 + 0.16
RAPTR with pseudo-3D deformable attention
Comparison with 2D-to-3D Pose Uplifting Model: We further compare RAPTR with baseline that first estimates 2D keypoints in the image plane and subsequently lifts them to 3D space using pre- trained 2D-to-3D pose uplifting model [21] trained on vision-based datasets such as Human3.66 6M [13]. To ensure fair comparison, this baseline adopts the same network architecture as RAPTR, but both the pose and joint decoders are supervised only by the 2D keypoint loss. Because the 3D poses predicted by the uplifting model are defined in pelvis-centered coordinate system, additionally estimate a translation offset to align the estimated poses with their correct position in the world coordinate system. As shown in Table 5, the pose uplifting baseline performs significantly worse than RAPTR, with MPJPES of 43.43 cm and 41.76 em on the WALK and MULTI splits, respectively.
Given that the process of refining the template to the actual pose in the joint decoder is supervised by the 2D keypoint labels, the accuracy of the 3D pose estimation highly dependent on the precision of the labels in the image plane. In this context, since the 2D keypoint label lacks the
Limitation:

---

Table Comparison with 2D-to-3D pose uplifting model (MPJPE: cm).
Loss
WALK
MULTI
Pose Dec.
Joint Dec.
Pose Lifting [21] RAPTR (ours)
K2D T3D
K2D
43.43 + 2.66
41.76 + 6.85
K2D+G3D
22.32 0.06 18.99 + 0.16
ability to discern whether the person is facing forward or backward to the camera, the estimated 3D) poses may have joints that are bent in the opposite direction in depth from the actual pose. In addition, real-world conditions such as occlusion and human-to-humen interference can further degrade the pose estimation performance. These effects become more pronounced in crowded or interactive environments.
7 Conclusion
We introduced RAPTR, radar-based 3D human pose estimation system using reliable 2D keypoint labels and 3D BBoxes as the coarse-grained 3D information. We designed the network architecture and the loss function to integrate multi-view radar features and consistently represent human poses in the 3D space, whose effectiveness was demonstrated through experimental results.
Broader Impacts: Indoor radar perception technologies, such as RAPTR, provide diverse indoor applications. These technologies may improve the safety and energy efficiency of indoor systems while preserving privacy. However, it is paramount that the perception results remain secure and private prevent misuse.
References
Sizhe An, Yin Li, and Umit Ogras. mRI: Multi-modal 3D human pose estimation dataset using Wave, RGB-D, and inertial sensors. In Advances in Neural Information Processing Systems, pages 27414-27426. 2022.
[2] Zhe Cao, Tomas Simon, Shih-En Wei, Yaser Sheikh. Realtime multi-person 2D pose estimation using part affinity fields. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2017.
[3] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European Conference Computer Vision, page 213-229, 2020.
[4] Ching-Hane Chen and Deva Ramanan. 3D human pose estimation = 2D estimation + matching. In IEEE Conference Computer Vision and Patiern Recognition (CVPR), pages 5759-5767, 2017.
[5] Ching-Hang Chen, Ambrish Tyagi, Amit Agrawal, Dylan Drover, Rohith Mv, Stefan Stojanov, and James M Rehg. Unsupervised 3D pose estimation with geometrie self-supervision. In IEEE/CVP Conference on Computer Vision and Pattern Recognition (CVPR), pages 5714-5724, 2019.
Bowen Cheng, Bin Xiao, Jingdong Wang, Honghui Shi, Thomas Huang, and Lei Zhang. HigherHRNet: Scale-aware representation learning for bottom-up human pose estimation. In IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), 2020.
[7] Fangqiang Ding, Zhen Luo, Peijun Zhao, and Chris Xiaoxuan Lu. milliFlow: Scene flow estimation on mm Wave radar point cloud for human motion sensing. In Furopean Conference on Computer Vision (ECCV), pages 202-221, 2024.
[8] Junqiao Fan, Jianfei Yang. Yuecong Xu, and Lihua Xie. Diffusion model good pose estimator from 3D RF-vision. In European Conference Computer Vision, page 1-18, 2024.
[9] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE Conference Computer Vision and Pattern Recognition (CVPR), pages 770-778 2016.
[10] Kaiming He, Georgia Gkioxari, Piotr Dollar, and Ross Girshick. Mask R R-CNN, IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(2):386-33 2020.
10

---

[11] Yuan-Hao Ho, Jen-Hao Cheng, Sheng Yao Kuan. Zhongyu Jiang, Wenhao Chai, Hsiang- Wei Huang. Chih-Lung Lin, and Jenq-Neng Hwang. RT-Pose: 4D radar tensor-based 3D human pose estimation and localization benchmark. In European Conference on Computer Vision, page 107-125, 2024.
[12] Shuting Hu, Siyang Cao, Nima Toosizadeh, Jennifer Barton, Melvin G. Hector, and Mindy J. Fain. mmPose-FK: A forward kinematics approach to dynamic skeletal pose estimation using mm Wave radars. IEEE Sensors Journal, 24(5):6469-644 1, 2024.
[13] Catalin Ionescu, Dragos Papava, Vlad Olaru, and Cristian Sminchisescu. Human3. 6.:: Large scale datasets and predictive methods for 3D human sensing in natural environments. IEEE Transactions on Pattern Analysis and Machine Intelligence, (3)5)::335--339
2014. 2014.
[14] Niraj Prakash Kini, Ruey-Horng Shiue, ryan chandra. Wen-Hsiao Peng. Ching- Wen Ma, and Jenq-Neng Hwang. TransHuPR: Cross-view fusion transformer for human pose estimation using mm Wave radar. In British Machine Vision Conference, 2024.
[15] Shih-Po Lee, Niraj Prakash Kini, Wen-Hsiao Peng, Ching- Wen Ma, and Jenq-Neng Hwang. HuPR: A benchmark human pose estimation using millimeter wave radar. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 3715-57244 2023.
[16] Jicfeng Li, Siyuan Bian, Ailing Zeng, Can Wang, Bo Pang, Wentao Liu, and Cewu Lu. Human posc regression with residual log-likelihood estimation. In JEEE/CVF International Conference on Computer Vision (ICCV), 2021.
[17] Tsung-Yi Yi Lin, Priya Goyal, Ross Girshick, Kaiming Hc, and Piotr Dollar. Focal loss for dense object detection. IEEE Transactions Pattern Analysis and Machine Intelligence, (2(2):318-322 2020.
[18] Chris Xiaoxuan Lu, Stefano Rosa, Peijun Zhao, Bing Wang, Changhao Chen, John A. Stankovic, Niki Trigoni, and Andrew Markham. See through smoke: robust indoor mapping with low-cost mmwave radar. In International Conference on Mobile Systems, Applications, and Services (MobiSys), page 14-27, 2020.
[19] Zhengxiong Luo, Zhicheng Wang. Yan Huang, Liang Wang. Tieniu Tan, and Erjin Zhou. Rethinking the heatmap regression for bottom-up human pose estimation. In JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 33259-13268, 2021.
[20] Sebastian Lutz, Richard Blythman, Koustav Ghosal, Matthew Moynihan, Ciaran Simms, and Aljosa Smolic. Jointformer: Single-frame lifting transformer with error prediction and refinement for 3D human pose estimation. In International Conference on Pattern Recognition (ICPR), pages 1156-1163, 2022.
[21] Julicta Martinez, Rayat Hossain, Javier Romero, and James J Little. A simple yet effective baseline for 3D human pose estimation. In 2017 IEEE International Conference Computer Vision (ICCV), pages 2659 2668, 2017.
[22] Alejandro Newell, Kaiyu Yang, and Jia Deng, Stacked hourglass networks for human pose estimation. arXiv [cs.CV], 2016.
[23] Ashish Pandharipande, Chih-Hong Cheng. Justin Dauwels, Sevgi Z. Gurbuz, Javier Thanez-Guzman, Guofa Li, Andrea Piazzoni, Pu Wang, and Avik Santra. Sensing and machine learning for automotive perception: A review. IEEE Sensors Journal, 23(1 1097-111111111 2023.
[24] M. Mahbubur Rahman, Ryoma Yataka, Sorachi Kato, Pu (Perry) Wang, Peizhao Li, Adriano Cardace, and Petros Boufounos. MMVR: Millimeter-wave multi-view radar dataset and benchmark for indoor perception. In European Conference Computer Vision (ECCV), pages 306-322, 2024.
[25] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions Pattern Analysis and Machine Intelligence, 39(6):1
2017.
[26] Avik Santra, Pu Wang, George Shaker, Bhavani Shankar Mysore, Guido Dolmans, Yan Chen, Negin Shariati, and Ashish Pandharipande. Machine learning-powered radio frequency sensing: A review. IEEE Sensors Journal, 25(13):23164 23183, 2025.
[27] Arindam Sengupta, Feng Jin, Renyuan Zhang, and Siyang Cao. mm- mm-Pose: Real-time human skeletal posture estimation using mm Wave radars and CNNs. IEEE Sensors Journal, 20(17):10032-1000 2020.
[28] Dahu Shi, Xing Wei, Liangqi Li, Ye Ren, and Wenming Tan. End-to-end multi-person pose estimation with transformers. In JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11059 1 1068, 2022.
11

---

[29] Akash Deep Singh. Sandeep Singh Sandha, Luis Garcia, and Mani Srivastava. RadHAR: Human activity recognition from point clouds generated through millimeter wave radar. In ACM Workshop on Millimeter- Wave Networks and Sensing Systems, pages 51-56, 2019.
[30] Mikael Skog, Oleksandr Kotlyar, Vladimír Kubelka, and Martin Magnusson. Human detection from 4D radar data in low-visibility field conditions. arXiv:2404.05307, 2024.
[31] Ke Sun, Bin Xiao, Dong Liu, and Jingdong Wang. Deep high-resolution representation learning for human pose estimation. In JEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 5686-5699 2019.
[32] Denis Tome, Chris Russell, and Lourdes Agapito. Litting from the deep: Convolutional 3D pose estimation from a single image. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
[33] Hong Wan, Ruiyuan Song, Chunyang Xie, Zhi Lu, Qi Chen, Zhi Wu, Dongheng Zhang, Yang Hu, and Yan Chen. QRFPosc: Query-based 3D pose estimation using radio signals. In International Conference on Wireless Communications and Signal Processing (WCSP), pages 1271 1-277, 2024.
[34] Jingdong Wang. Ke Sun, Tianheng Cheng, Borui Jiang. Chaorui Deng. Yang Zhao, Dong Liu, Yadong Mu, Mingkui Tan, Xinggang Wang. Wenyu Liu, and Bin Xiao. Deep high-resolution representation learning for visual recognition. IEEF Transactions on Pattern Analysis and Machine Intelligence, 43(10):339933364, 2021.
[35] Zhi Wu, Dongheng Zhang, Chunyang Xie, Cong Yu, Jinbo Chen, Yang Hu, and Yan Chen, RFMask: A simple baseline for human silhouette segmentation with radio signals, IEEE Transactions Multinedia, 25:4730-4741, 2023.
[36] Qian Xie, Qianyi Deng, Ta Ying Cheng, Peijun Zhao, Amir Patel, Niki Trigoni, and Andrew Markham. mmPoint: Dense human point cloud generation from mm mave.v. In British Machine Vision Conference (BMVC), pages 194-196, 2023.
[37] Qian Xie, Xinyu Hou, Qianyi Deng, Amir Patel, Niki Trigoni, and Andrew Markham. mmDiffusion: mm mmWave Wave diffusion for sequential 3d human dense point cloud generation. In 2025 International Conference on 3D Vision (3DV), pages 781-790, 2025.
[38] Hongfei Xue, Yan Ju, Chenglin Miao, Yijiang Wang, Shiyang Wang, Aidong Zhang, and Lu Su. mmMesh: towards 3D real-time dynamic human mesh construction using millimeter wave. In International Confer- ence on Mobile Systems, Applications, and Services (MobiSys). page 269-282, 2021.
[39] Hongfei Xue, Qiming Cao, Yan Ju, Haochen Hu, Haoyu Wang, Aidong Zhang, and Lu Su. essh: mmWave-based 3D human mesh construction for multiple subjects. In Proceedings of the 20th ACM Conference on Embedded Networked Sensor Systems, pages 391-406, 2022.
[40] Kangwei Yan, Fei Wang, Bo Qian, Han Ding, Jinsong Han, and Xing Wei. Person-in- WiFi 3D: End-to-end multi-person 3D pose estimation with Wi-Fi. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 969-978, 2024.
[41] Jianfei Yang, He Huang, Yunjiao Zhou, Xinyan Chen, Yuecong Xu, Shenghai Yuan, Han Zou, Chris Xi- aoxuan La, and Lihua Xie. MM-Fi: Multi-modal non-intrusive 4D human dataset for versatile wireless sensing. In Advances in Neural Information Processing Systems (NeurIPS), pages 18756-18766 2023.
[42] Jiarui Yang, Songpengcheng Xia, Yifan Song, Qi Wu, and Ling Pei. mmBaT: A multi-task framework for maWave-based human body reconstruction and translation prediction. IEEE International Conference on Acoustics, Speech and Signal Processing, pages -450, 2024.
[43] Jiarui Yang, Songpengcheng Xia, Zengyuan Lai, Lan Sun, Qi Wu, Wenxian Yu, and Ling Pei. mmDEAR: mmWave point cloud density enhancement for accurate human body reconstruction. In IEEE International Conference on Robotics and Automation, pages 11227 -1233, 2025.
[44] Shanliang Yao, Runwei Guan, Zitian Peng, Chenhang Xu, Yilu Shi. Weiping Ding. Eng Gee Lim, Yong Yue, Hyungjoon Seo, Ka Lok Man, Jieming Ma, Xiaohui Zhu and Yutao Yue. Exploring radar data representations in autonomous driving: A comprehensive review, arXiv:2312.0486/ 2024.
[45] Ryoma Yataka, Adriano Cardace, Pu (Perry) Wang, Petros Boufounos, and Ryuhei Takahashi. RETR: Multi-view radar detection transformer for indoor perception. Neural Information P Processing Systems, 37: 19839- 19869, 2024.
12

---

[46] Feng Zhang. Xiatian Zhu, Hanbin Dai, Mao Ye, and Ce Zhu. Distribution-aware coordinate representation for humaa pose estimation. In /EEE/CVF Conference Computer Vision and Puttern Recognition (CVPR). pages 7091-7100, 2020.
[47] Mingmin Zhao, Tianhong Li, Mohammad Abu Alsheikh, Yonglong Tian, Hang Zhao, Antonio Torralba. and Dina Katabi. Through-we human pose estimation using radio signals. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pages 7356-7365 2018.
[48] Mingmin Zhao, Yonglong Tian, Hang Zhao, Mohammad Abu Alsheikh, Tianhong Rumen Hristov, Zachary Kabelac, Dina Katabi, and Antonio Torralba. RF-based 3D skeletons. In Conference of the ACM Special Interest Group on Data Communication (SIGCOMM), page 267-281, 2018.
[49] Peijun Zhao, Chris Xiaoxuan Lu, Bing Wang, Niki Trigoni, and Andrew Markham. CubeLearn: End-to-end lcarning for human motion recognition from raw mm Wave radar signals. IEEE Internet of Things Journal, ((12):10336-12444 2023.
[50] Xizhou Zhu, Weijie Su, Lewei Lu,, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable DETR: Deformable transformers for end-to-end object detection. In International Conference Learning Representations, 2021.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly outline the major challenge and our contributions on that in 3D human pose estimation achieved by the proposed architecture using bullet points. Specifically, following each bullet point, Section 4 and Section 5 provide detailed descriptions of the proposed architecture and its experimental results, demonstrating the effectiveness of the proposed method. Therefore, the overall content is consistent.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We included the discussion about the limitation in Section 6. Besides, analyzed the limitations by visualizing the failure cases in Appendix I.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
14

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: We provide the theoretical computational complexity of our proposed system in Appendix A. Although we mainly demonstrate empirical results in this paper, we show that our proposed method is broadly applicable by performing evaluations with two different datasets.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide the main parameters regarding the dataset, model architecture and model training in Table 7 in Appendix F. In addition, we provide a detailed description of the network architecture in Appendix A. This allows us reproduce the main experimental results the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm.
(b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
15

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We plan release source code online. In addition, the link the datasets we used in the evaluations are provided in Section 5.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
//nips.cc/ppt wttpcccppppcccceeee cttlic//giieee des/de////eeee CodeSubmi ssisnnni ssionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We included the specifications of data and hyper parameters. Refer Table 7 in Appendix F.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Due to the high computational cost, statistical results are not included. We instead report more comprehensive experimental results by conducting experiments with multiple datascts. Please refer to Section 5 and Appendix
16

---

Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Refer Table in Appendix F including descriptions for computer resources used.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https http://eeeuuppeeeeeee cttppc/bct///thheeeeueeeeeeaeed
Answer: [Yes]
Justification:
We carefully confirmed it.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
17

---

Answer: [Yes]
Justification:
We disucuss the social impact of our proposal in Section 7.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairesss considerations (e.g.. dep.lymmlo of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications,
authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: The datasets we use in our evaluation are distributed through regular procedures. Besides, we do not use any pre-trained models, and thus there no such risk.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes)
The papers for HIBER and MMVR dataset are cited; see Section 5 that the papers are eited in the proper context.
Justification:
Guidelines:
The answer NA means that the paper does not existing assets.
18

---

The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- re-packaged, both the original license and the license of the derived asset (if has changed) should provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We plan to release the source code required to reproduce the evaluation results in the main content.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
19

---

Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our L.LM policy (https: for what should or should not be described.
20


Reviews:
[{"review_id": "23ozilTiyz", "reviewer": "Reviewer_bgMt", "summary": "This paper introduces RAPTR, a novel method for radar-based 3D human pose estimation. The core innovation of the proposed method is that it uses only low-cost, easily accessible 3D bounding boxes (BBoxes) and 2D keypoints for supervision, instead of relying on expensive and difficult-to-collect 3D keypoint labels. The whole framework is built on PETR to output initial 3D pose and refined 3D pose, which is supervised by the proposed 3D template loss, and 2D keypoint and 3D gravity losses, respectively. Experiments on two public radar datasets demonstrate the effectiveness of the proposed method.", "questions": "In weakness.", "limitations": "In weakness.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "**Strengths**\n\n1.The proposed method significantly reduces data annotation costs. The proposed method is trained using low-cost 2D keypoints and 3D BBox labels instead of expensive 3D keypoint labels, which allows the model to be trained on larger-scale and more diverse scenarios and enhances the scalability of the proposed method.\n\n2.This paper is well-written and easy to understand.\n\n**Weakness**\n\nThe methodological novelty of the proposed method is somewhat limited. The overall framework of RAPTR is mainly based on PETR. Its primary modification is to \"lift\" the 2D deformable attention mechanism into a \"pseudo-3D\" space. However, as shown in Table 3, pseudo-3D deformable attention is harmful to the model performance, which cannot show the effectiveness of the proposed component. What is the performance of +(1)+(3).", "was_revised": true, "final_justification": "The rebuttal addresses my concerns and I keep my rating for Borderline accept", "rebuttal": "> **Summary \\& Strengths**\n> * The core innovation of the proposed method is that it uses only low-cost, easily accessible 3D BBoxes and 2D keypoints for supervision, instead of relying on expensive and difficult-to-collect 3D keypoint labels. The whole framework is built on PETR to output initial 3D pose and refined 3D pose, which is supervised by the proposed 3D template loss, and 2D keypoint and 3D gravity losses, respectively...\n> * The proposed method significantly reduces data annotation costs...allows the model to be trained on larger-scale and more diverse scenarios and enhances the scalability of the proposed method.\n\n**Response**  \nWe sincerely thank the reviewer for the positive feedback and recognition of our core motivation—**advancing 3D radar perception under weak supervision**—which, to the best of our knowledge, is the first work to explore this direction in the radar domain. We appreciate your acknowledgment of the broader impact of our approach, particularly our effort to enable scalable training and deployment using inexpensive labels, which hopefully encourages more extensive radar data collection without the need for costly 3D annotations.  We will release the full training and evaluation code, along with the pretrained model checkpoints, to support reproducibility and future open research in 3D radar perception. \n\n> **Weakness**  \n> *The methodological novelty of the proposed method is somewhat limited. The overall framework of RAPTR is mainly based on PETR. Its primary modification is to \"lift\" the 2D deformable attention mechanism into a \"pseudo-3D\" space. However, as shown in Table 3, pseudo-3D deformable attention is harmful to the model performance, which cannot show the effectiveness of the proposed component. What is the performance of +(1)+(3)?\n\n**Response**  \nThanks for raising this concern, which allows us to clarify our core contributions more clearly. We also conducted **additional ablation studies** on the HIBER dataset, evaluating a comprehensive set of configurations involving architectural modules and loss functions with random seeds, including your suggested combination of **+(1)+(3)**. \n\nFirst, while we do not claim the two-stage decoder as a core contribution, we **deliberately adopt PETR's two-stage design as a strong, empirically validated baseline for our purpose**. Rather than reinventing the wheel for the sake of novelty, we plug-and-play PETR's backbone and enable us to focus on advancing 3D radar perception under weak supervision.\n\nSecond, **PETR with only 2D keypoint loss completely fails to generalize to 3D radar perception**, as evidenced by the first row of Table 1 below, which shows drastically worse performance—with an error exceeding 3 meters. The core issue is *depth or depth-scale ambiguity*, a limitation well illustrated by the behavior of a *pinhole camera or flashlight projection*: A larger object that is farther away can appear identical in 2D to a smaller object that is closer. In such cases, 2D keypoint supervision alone provides *no constraint on absolute scale or depth*, making 3D recovery fundamentally under-constrained—especially in weakly supervised settings.\n\n#### Table 1: Effect of our structural loss function on PETR\n##### Results with ± values are newly added and report **mean ± std** from at least 3 runs with different random seeds.\n| Decoder | Encoder | Attn | Pose Loss | Joint Loss | MPJPE (MULTI)  | Notes                                                     |\n| ------- | ------- | ---- | --------- | ---------- | ----------- | --------------------------------------------------------- |\n| 2S      | SV      | 2D   | K2D       | K2D        | 372.03      | PETR with 2D Keypoint Loss only                                                |\n| 2S      | SV      | 2D   | K2D+G3D   | K2D+G3D    | 21.03       | PETR with 3D Gravity Loss  |\n| 2S      | SV      | 2D   | T3D       | K2D+G3D    | **18.99 $\\pm$ 0.16**   | PETR with 3D Gravity Loss+ 3D Template Loss |\n\n### Symbol Definitions\n\n- **Decoder:**  \n  `1S` = one-stage, `2S` = two-stage\n\n- **Encoder:**  \n  `SV` = separate-view encoder, `CV` = cross-view encoder\n\n- **Attn (Attention):**  \n  `2D` = 2D deformable attention, `3D` = pseudo-3D deformable attention\n\n- **Loss Terms:**  \n  `T3D` = 3D Template loss (weak-supervsion label)   \n  `G3D` = 3D Gravity loss  (weak-supervsion label)  \n  `K2D` = 2D Keypoint loss (weak-supervsion label)  \n\nThird, it is **our structured design of the loss functions**—particularly the use of 3D Gravity Loss and 3D Template Loss across the two decoder stages—that leads to substantial performance gains. As shown in the second and third rows of Table 1, these additions reduce the MPJPE of PETR from over 370 cm to under 20 cm—a more than **10× improvement**. We also observe in **Table 2** that our structured loss function significantly outperforms other loss combinations, further validating its effectiveness on RAPTR.\n\n#### Table 2: Effect of our structural loss function on RAPTR\n| Decoder | Encoder | Attn | Pose Loss | Joint Loss  |  MPJPE (MULTI) | Notes                                  |\n| ------- | ------- | ---- | --------- | ----------  | -------------- | ---------------------------------------|\n| 2S      | CV      | 3D   | --        | K2D+G3D     | 50.96          | (A) in Fig.7                           |\n| 2S      | CV      | 3D   | K2D+G3D   | K2D+G3D     | 27.36          | (B) in Fig.7                           |\n| 2S      | CV      | 3D   | T3D       | K2D         | 30.06          | new result (suggested by Reviewer vQxv)|\n| 2S      | CV      | 3D   | T3D       | K2D+G3D     | **18.99 $\\pm$ 0.16**      | RAPTR                                  |\n\nFourth, **Pseudo-3D attention works better with stronger, view-consistent radar features—such as those provided by the cross-view (CV) encoder.** Thanks to your suggestion to evaluate **+(1)+(3)**, we include the result in the third row of Table 3 below, alongside the original Table 3 from the main paper (we re-run **+(1)** and **+(1)+(2)+(3)** with 3+ random seeds and report **mean ± std** for more rigorous evaluation). It is observed that using either the cross-view encoder (CV, 3rd row) or pseudo-3D attention (3D, 2nd row) alone results in slightly worse performance than the base two-stage setup (first row). However, when pseudo-3D attention is paired with the cross-view encoder (last row), it achieves the best performance at the two HIBER data splits (WALK, MULTI). This suggests that the effectiveness of pseudo-3D attention **depends heavily on the quality and consistency of the underlying radar features**, which are better provided by the cross-view encoder than the separate-view encoder. We provide the following justification by looking into the pseudo-3D deformable attention in Fig. 4 of the main paper. \n- Pseudo-3D attention predicts 3D spatial offsets—unlike the decoupled 2D offsets in QRFPose—to align and extract spatially consistent features across radar views.\n- These 3D offsets are projected into each radar view to determine *where* features should be sampled.\n- The sampled features from different views are then aggregated using normalized attention weights to form the pseudo-3D deformable attention output.\n- For this mechanism to function effectively, the underlying radar features need to be **spatially coherent and consistent across views**—a property better provided by the cross-view (CV) encoder.\n- In contrast, when pseudo-3D attention is combined with the separate-view (SV) encoder, the lack of shared multi-view context may limit its ability to extract reliable 3D features, leading to less powerful aggregated multi-view radar feature at the pseudo-3D deformable attention output. \n\n#### Table 3: Ablation Study on RAPTR architectural modules (Encoder, Decoder, Attention)\n| Decoder | Encoder | Attn | Pose Loss | Joint Loss | MPJPE (WALK) | MPJPE (MULTI) | Notes                                            |\n| ------- | ------- | ---- | --------- | ---------- | ------------------ | -------------------- | ------------------------------------------------ |\n| 2S      | SV      | 2D   | T3D       | K2D+G3D    | 22.95  $\\pm$ 1.21             | 20.33  $\\pm$ 0.73               | **+(1)** in Table 3                              |\n| 2S      | SV      | 3D   | T3D       | K2D+G3D    | 22.83              | 20.78                | **+(1)+(2)** in Table 3                          |\n| 2S      | CV      | 2D   | T3D       | K2D+G3D    | 24.73              | 20.57                | **+(1)+(3)** (suggested by Reviewer bgMt) |\n| 2S      | CV      | 3D   | T3D       | K2D+G3D    | **22.32 $\\pm$ 0.06**              | **18.99 $\\pm$ 0.16**            | **+(1)+(2)+(3)** in Table 3 (RAPTR (ours))       |\n\n\n## Summary\n\nWe thank Reviewer **bgMt** for the thoughtful and constructive feedback. Your positive evaluation of our work's **quality, clarity, significance, and originality** (all rated 4) affirms our core contributions to weakly supervised 3D radar perception. We hope that the added clarification on pseudo-3D deformable attention, additional ablations (including your suggested **+(1)+(3)**), and new results with random seeds further strengthen the impact and completeness of our submission."}, {"review_id": "KbAwwqyECv", "reviewer": "Reviewer_vQxv", "summary": "This paper addresses 3D human pose estimation using multi-view millimeter wave radar with 3D bounding boxes and 2D keypoints as weak supervision. The framework consists of pseudo-3D pose and joint deformable transformers as decoders to predict poses and joints. The authors then enforce the gravity and template loss to regularize predicted 3D keypoints. The authors have conducted experiments on the HIBER and MMVR dataset to show the effectiveness of the proposed method.", "questions": "**Important questions and concerns for the authors to address. Responses will strongly influence the final rating**\n- Elaborate on the intuition of using gravity loss. \n- Conduct the experiment of training the method with full supervision (3D keypoints) to show the upper bound performance for comparison.\n- Conduct the ablation study of training with and without gravity loss.\n- Explain why pseudo-3D deformable attention mechanism becomes a contribution regarding performance and complexity, specifically compared to QRFPose [32].\n\n**Optional questions and concerns for the authors to address. Good responses can increase the the final rating**\n- Address the concerns listed in the minor weakness section.\n\nPlease find the relevant details of some questions in the weakness section.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths:**\n- Using learnable queries to represent 3D keypoints and offsets in deformable transformer is interesting. \n- Using weak supervision makes this work more practical in real-world applications. \n- The experimental results are promising, highlighting the potential of using weak supervision.\n\n**Weaknesses:**\n- **Lack of clarity in the proposed method**\n    - Template loss and fine-grained 2D Loss regularize the predicted 3D/2D keypoints.  Both losses are stricter than gravity loss which only encourages the predicted and bounding box centroids to be closer, but not the individual keypoints. The authors should elaborate on the intuition of using gravity loss. \n- **Lack of essential experiments and unconvincing results**\n    - The authors should conduct the experiment of using 3D keypoints as supervision. The proposed method uses template loss, meaning that the model has a certain amount of access to 3D keypoints. Training RAPTR with 3D keypoints provides reviewers with a clear understanding of how well the method can be regarding full supervision. \n    - The authors did not provide the ablation study on with and without using gravity loss in Figure 7 and therefore, it raises concerns about the effectiveness of gravity loss. \n    - Table 3 shows that 3D-pseudo deformable attention is not useful, achieving approximately 1cm worse in MPJPE. Because the pseudo-3D deformable attention is the main difference between RAPTR and QRFPose [32], the inferior results make the proposed design less impactful. The authors should explain why pseudo-3D deformable attention mechanism becomes a contribution regarding performance and complexity. \n- **Minor weaknesses**\n    - What is SNR in line 91?\n    - In Figure 3, the arrow from Pose Decoder to Joint Decoder is missing, as Joint Decoder takes the output from Pose Decoder as inputs (line 135 - 136).", "was_revised": true, "final_justification": "The authors have addressed most of my concerns. Specifically, the complexity analysis demonstrates that the proposed method does offer advantages over the baselines. However, in my opinion, the improvement of using pseudo-3D deformable attention still falls within the range of statistical bias. \n\nOverall, I consider this a borderline paper, slightly leaning toward acceptance.", "rebuttal": "> **Summary \\& Strengths**\n\n**Response**  \nWe thank the reviewer for recognizing our core motivation—advancing 3D radar perception with weak supervision—which, to our knowledge, is the first to explore this direction. We appreciate your summary of our contributions, including pseudo-3D attention, structured loss design, and evaluations on two large indoor radar datasets. We will release full code and pretrained checkpoints to support future research.\n\n> **Weaknesses & Questions**\n> * Elaborate on the intuition of using gravity loss and conduct the ablation...with and without gravity loss.\n\n**Response**  \nFirst, we highlight that the motivation of using a hierarchical and structured loss function:\n- **(a)** a **3D Template Loss (T3D)** at the first-stage pose decoder, using coarse 3D bounding boxes and a human keypoint template to guide initial 3D pose estimates, and  \n- **(b)** a combination of **2D Keypoint Loss (K2D)** and **3D Gravity Loss (G3D)** at the second-stage joint decoder to refine the pose predictions.\n\nSecond, to clarify the role of **gravity loss**, we follow your suggestion and include an ablation (last row of Table 1). The second-stage joint decoder refines poses using 2D keypoint loss (K2D), but K2D alone cannot constrain depth due to **depth ambiguity** (a limitation illustrated by the behavior of a *pinhole camera or flashlight projection*: A larger object that is farther away can appear identical in 2D to a smaller object that is closer). Adding **3D gravity loss (G3D)** regularizes depth and scale. Comparing the 3rd and last rows of Table 1 shows that removing G3D leads to over 50% performance drop (e.g., 18.99 cm → 30.06 cm on MULTI), confirming the necessity of using G3D in the second-stage decoder.\n\nThird, we justify the **template loss** at the first-stage decoder using Table 1. Comparing rows 1–3 shows that adding 3D shape priors reduces MPJPE from 50.96 cm and 27.36 cm to 18.99 cm. The template not only localizes poses near the 3D box center but also encourages **human-like keypoint structures**, avoiding degenerated or implausible shapes produced without this human shape prior. \n\n#### Table 1: RAPTR with Weak Supervision (2D Keypoint Loss (K2D),  3D Gravity Loss (G3D), and 3D Template Loss (T3D))\n##### Results with ± values are newly added and report **mean ± std** from at least 3 runs with different random seeds.\n| Decoder | Encoder | Attn | Pose Loss | Joint Loss | MPJPE (WALK) | MPJPE (MULTI) | Notes                                            |\n| ------- | ------- | ---- | --------- | ---------- | ------------------ | -------------------- | ------------------------------------------------ |\n| 2S      | CV      | 3D   | --        | K2D+G3D    | 29.12              | 50.96                | (A) in Fig.7                                     |\n| 2S      | CV      | 3D   | K2D+G3D   | K2D+G3D    | 23.77              | 27.36                | (B) in Fig.7                                     |\n| 2S      | CV      | 3D   | T3D       | K2D+G3D    | **22.32  $\\pm$ 0.06**              | **18.99  $\\pm$ 0.16**            | \\(C) in Fig. 7 (RAPTR)                   |\n| 2S      | CV      | 3D   | T3D       | K2D        | 40.92              | 30.06                | new result (suggested by Reviewer vQxv)          |\n---\n##### symbol are defined below Table 1 of the response to Review bgMt. \n\n> **Weaknesses**\n> * Conduct the experiment of training the method with full supervision (3D keypoints) to show the upper bound performance for comparison.\n\n**Response**  \nSuggestion has been taken. We have included the evaluation of **RAPTR under full supervision** (using 3D keypoint loss, K3D) in **Table 2** below. We agree that this setting provides a meaningful upper bound on achievable performance, and allows us to quantify the performance gap between weak and full supervision.\n\nUnder full supervision, RAPTR achieves an MPJPE of *8.93 cm*, whereas our best weakly supervised variant—leveraging 3D Template Loss (T3D) and 3D Gravity Loss (G3D)—achieves *18.99 cm*. The weakly supervised model yields about a loss of ~10 cm in MPJPE, compared with the full supervision result. For completeness, we also include additional weakly supervised variants from Table 1 to highlight how each loss term contributes to final performance. \n\n### Table 2: RAPTR with Weak and Full Supervision\n| Decoder | Encoder | Attn | Pose Loss | Joint Loss | MPJPE (MULTI) | Notes                                    |\n| ------- | ------- | ---- | --------- | ---------- | ---------- | ---------------------------------------- |\n| 2S      | CV      | 3D   | --        | K2D+G3D    | 29.12     | RAPTR + Weak Supervision (K2D+G3D)                |\n| 2S      | CV      | 3D   | T3D       | K2D        | 40.92     | RAPTR + Weak Supervision (K2D+T3D)                 |\n| 2S      | CV      | 3D   | T3D       | K2D+G3D    | 18.99  $\\pm$ 0.16     | RAPTR + Weak Supervision (K2D+T3D+G3D)                |\n| 2S      | CV      | 3D   | K3D       | K3D        | 8.93      | RAPTR+Full supervision (suggested by Reviewer vQxv) |\n\n### Symbol Definitions\n- **Loss Terms:**  \n  `T3D` = 3D Template loss (weak-supervsion label)   \n  `G3D` = 3D Gravity loss  (weak-supervsion label)  \n  `K2D` = 2D Keypoint loss (weak-supervsion label)  \n  `K3D` = 3D Keypoint loss (Full-supervsion label)                      \n\n> **Weaknesses**\n> * ...The authors should explain why pseudo-3D deformable attention mechanism becomes a contribution regarding performance and complexity...\n\n**Response**  \nThanks for this insighful comment. **Pseudo-3D attention works better with stronger, view-consistent radar features—such as those provided by the cross-view (CV) encoder.** Thanks to your suggestion to evaluate **+(1)+(3)**, we include the result in the third row of Table 3 below, alongside the original Table 3 from the main paper (we re-run **+(1)** and **+(1)+(2)+(3)** with 3+ random seeds and report **mean ± std** for more rigorous evaluation). It is observed that using either the cross-view encoder (CV, 3rd row) or pseudo-3D attention (3D, 2nd row) alone results in slightly worse performance than the base two-stage setup (first row). However, when pseudo-3D attention is paired with the cross-view encoder (last row), it achieves the best performance. This suggests that the effectiveness of pseudo-3D attention **depends heavily on the quality and consistency of the underlying radar features**, which are better provided by the cross-view encoder than the separate-view encoder. We provide the following justification by looking into the pseudo-3D deformable attention in Fig. 4 of the main paper. \n- Pseudo-3D attention predicts 3D spatial offsets—unlike the decoupled 2D offsets in QRFPose—to align and extract spatially consistent features across radar views.\n- These 3D offsets are projected into each radar view to determine *where* features should be sampled.\n- The sampled features from different views are then aggregated using normalized attention weights to form the pseudo-3D deformable attention output.\n- For this mechanism to function effectively, the underlying radar features need to be **spatially coherent and consistent across views**—a property better provided by the cross-view (CV) encoder.\n- In contrast, when pseudo-3D attention is combined with the separate-view (SV) encoder, the lack of shared multi-view context may limit its ability to extract reliable 3D features, leading to less powerful aggregated multi-view radar feature at the pseudo-3D deformable attention output. \n\nIn terms of **complexity**, we provide such an analysis of RAPTER in Appendix A. As shown in Fig. 9 of Supplemental, we don't expect big differences between the 2D and pseudo-3D deformable attention as pseudo-3D saves one parameter for one offset prediction. \n\n#### Table 3: Ablation Study on RAPTR architectural modules (Encoder, Decoder, Attention)\n\n| Decoder | Encoder | Attn | Pose Loss | Joint Loss | MPJPE (WALK) | MPJPE (MULTI) | Notes                                            |\n| ------- | ------- | ---- | --------- | ---------- | ------------------ | -------------------- | ------------------------------------------------ |\n| 2S      | SV      | 2D   | T3D       | K2D+G3D    | 22.95  $\\pm$ 1.21             | 20.33  $\\pm$ 0.73               | **+(1)** in Table 3                              |\n| 2S      | SV      | 3D   | T3D       | K2D+G3D    | 22.83              | 20.78                | **+(1)+(2)** in Table 3                          |\n| 2S      | CV      | 2D   | T3D       | K2D+G3D    | 24.73              | 20.57                | **+(1)+(3)** (suggested by Reviewer bgMt) |\n| 2S      | CV      | 3D   | T3D       | K2D+G3D    | **22.32 $\\pm$ 0.06**              | **18.99  $\\pm$ 0.16**            | **+(1)+(2)+(3)** in Table 3 (RAPTR)                   |\n---\n\n> **Minor Weaknesses**\n> * What is SNR in line 91?\n\n**Response**  \nSNR denotes *Signal-to-Noise Ratio*. Raw radar waveforms are often affected by random thermal noise from the hardware and incoherent multi-path reflections. Integrating radar waveforms over the Doppler domain typically improves SNR. \n\n> **Minor Weaknesses**\n> * In Figure 3, the arrow from Pose Decoder to Joint Decoder is missing, as Joint Decoder takes the output from Pose Decoder as inputs (line 135 - 136).\n\n**Response**  \nWe will revise the figure in the updated manuscript. \n\n## Summary\nWe appreciate the reviewer’s constructive feedback, which has helped us clarify our technical contributions and strengthen the manuscript. With the added ablation studies, full-supervision upper bound, and improved explanation of the loss design and pseudo-3D attention, we hope that our paper now has clearly demonstrates its quality, clarity, significance, and originality."}, {"review_id": "aJCyw1ZtkQ", "reviewer": "Reviewer_WQeC", "summary": "This paper introduces RAPTR, a radar-based 3D human pose estimation framework that avoids costly 3D keypoint labels by instead using cheaper 3D bounding boxes and 2D keypoints. RAPTR builds upon PETR, a vision-based 2D pose transformer, and adapts it for radar input via a pseudo-3D deformable attention mechanism and a two-stage decoding strategy. The authors propose novel 3D losses (template and gravity loss) and demonstrate competitive performance on two indoor radar datasets (HIBER and MMVR), outperforming prior radar-based baseline methods. The approach is shown to be scalable, privacy-preserving, and suitable for various challenging environments.", "questions": "* **1) How does RAPTR handle human-to-human occlusions and radar signal interference in multi-person scenes?**\nIs there a mechanism to explicitly model or mitigate inter-person radar reflections?\n\n* **2) To what extent does the fixed 3D human pose template limit generalization to diverse body types or motion patterns?**\nWould a learned or dynamic template set improve robustness?\n\n* **3) Is pseudo-3D deformable attention significantly better than simple view-wise fusion or 3D CNN alternatives?**\nCan ablations with more diverse attention mechanisms help validate this choice?", "limitations": "* **1) Limited originality in core methodology:**\nRAPTR is largely a radar-adapted version of PETR, and while technically solid, the core innovation lies more in adaptation than conceptual novelty.\n\n* **2) Lack of effective modeling for occlusion and multi-person interference:**\nRadar-based human-to-human occlusion remains a key challenge that RAPTR does not explicitly solve.\n\n* **3) Rigid prior assumptions in the 3D template:**\nThe reliance on a fixed pose template may reduce generalization to out-of-distribution poses or body shapes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n* **1) Efficient Use of Weak Supervision:** The method relies on cheap and scalable labels (3D BBoxes + 2D keypoints) rather than expensive 3D motion capture data, improving data accessibility for real-world radar applications.\n* **2) Extensive Experiments and Ablation Studies:** The work presents comprehensive experimental results, including detailed per-joint error analysis, directional breakdowns, and ablations on architecture and loss functions.\n* **3) Solid Engineering and Architectural Design:** The proposed pseudo-3D deformable attention and two-stage decoding pipeline demonstrate good adaptation of vision transformer ideas to radar-based pose estimation.\n* **4) Robust to Multi-Person and Cluttered Scenes:** The method shows good performance in multi-person scenarios, which are typically challenging for radar-based perception systems.\n\n**Weaknesses**\n* **1) Limited Methodological Novelty:** The method is a modified version of PETR, a 2D pose transformer, adapted to radar data. While practical, this reduces its originality, especially since most contributions are engineering extensions.\n* **2) Occlusion and Human-to-Human Interference Remain Unsolved:** The paper does not adequately address the inherent limitations of radar, particularly signal interference in multi-person occlusion-heavy scenarios—one of the core challenges in radar-based perception.\n* **3) Strong Assumption on Template Priors:** The use of a fixed 3D human pose template in the decoder can be limiting, especially when generalizing to diverse body shapes, poses, or actions. A more flexible or learned template bank might improve robustness.\n\nIn a word, RAPTR is a well-executed system that makes radar-based 3D pose estimation more feasible under weak supervision. However, its conceptual originality is modest considering that the novelty bar is high in NeurIPS, and it leaves open key challenges like occlusion robustness and template generality.", "was_revised": true, "final_justification": "Despite some shortcomings, I am impressed by the strong empirical results and extensive experimental tests of this paper. Therefore, I promise to maintain the positive evaluation. I ask the authors to clarify the concerns we discussed in the final version.", "rebuttal": "> **Summary \\& Strengths**\n\n**Response**  \nWe thank the reviewer for recognizing our core motivation—**advancing 3D radar perception under weak supervision**, which we believe is the first to explore this direction. We appreciate the positive feedback on our 3D loss design and experiments. To support reproducibility, we will release full code and pretrained checkpoints.\n\n> **Weaknesses**\n> * Limited Methodological Novelty:... While practical, this reduces its originality, especially since most contributions are engineering extensions.\n\n**Response**  \nWe thank the reviewer for raising this concern, which allows us to clarify our core contributions more clearly below. With additional ablation studay, we summarize our contribution in two folds: (1) the structural design of loss functions under weak supervision; and (2) the architectural modifications including cross-view (CV) radar encoder and pseudo-3D deformable attention. \n\nFirst, **our structured design of the loss functions**—particularly the use of 3D Gravity Loss and 3D Template Loss across the two decoder stages—**is key to the substantial performance gains**. Along with the suggestion from Reviewer vQxv, **Table 1 below provides a complete picture of the significance of each loss function and the decoder stage where it is applied**.\n\n- The **3D template prior at the first-stage decoder** helps localize initial pose estimates near the correct spatial region (e.g., close to the 3D bounding box center) and promotes **human-like keypoint configurations**, avoiding degenerate or implausible shapes.\n- The **second-stage joint decoder** refines initial pose estimates by projecting 3D keypoints to the 2D image plane and minimizing 2D keypoint loss (K2D), which leverages the fine granularity of 2D annotations. However, **2D keypoint loss alone cannot constrain absolute scale or depth** due to *depth ambiguity*—where a distant large object may appear identical in 2D to a closer small object. To resolve this, we **combine 2D keypoint loss with 3D Gravity Loss**, allowing the model to learn depth- and scale-aware corrections during pose refinement.\n\n#### Table 1: Effect of our structural loss function on RAPTR\n##### Results with ± values are newly added and report **mean ± std** from at least 3 runs with different random seeds.\n| Decoder | Encoder | Attn | Pose Loss | Joint Loss  |  MPJPE (MULTI) | Notes                                  |\n| ------- | ------- | ---- | --------- | ----------  | -------------- | ---------------------------------------|\n| 2S      | CV      | 3D   | --        | K2D+G3D     | 50.96          | (A) in Fig.7                           |\n| 2S      | CV      | 3D   | K2D+G3D   | K2D+G3D     | 27.36          | (B) in Fig.7                           |\n| 2S      | CV      | 3D   | T3D       | K2D         | 30.06          | new result (suggested by Reviewer vQxv)|\n| 2S      | CV      | 3D   | T3D       | K2D+G3D     | **18.99  $\\pm$ 0.16**      | RAPTR                                  |\n---\n##### symbol are defined below Table 1 of the response to Review bgMt. \n\nSecond, our structural loss design is also compatible with PETR. As shown in Table 2 (new results), **PETR with only 2D keypoint loss completely fails to generalize to 3D radar**, with MPJPE over 370 cm due to *depth ambiguity*. Adding gravity and template losses reduces the error to under 20 cm—a **10× improvement**.\n\n#### Table 2: Effect of our structural loss function on PETR\n| Decoder | Encoder | Attn | Pose Loss | Joint Loss | MPJPE (MULTI)  | Notes                                                     |\n| ------- | ------- | ---- | --------- | ---------- | ----------- | --------------------------------------------------------- |\n| 2S      | SV      | 2D   | K2D       | K2D        | 372.03      | **PETR** with 2D Keypoint Loss                                                 |\n| 2S      | SV      | 2D   | K2D+G3D   | K2D+G3D    | 21.03       | **PETR** with 2D Keypoint Loss + 3D Gravity Loss (ours)  |\n| 2S      | SV      | 2D   | T3D       | K2D+G3D    | **19.52**   | **PETR** with 2D Keypoint Loss + 3D Gravity Loss (ours) + 3D Template Loss (ours)   |\n\nThird, **pseudo-3D attention works better with stronger, view-consistent radar features—such as those provided by the cross-view (CV) encoder.** Thanks to Reviewer bgMt's suggestion to evaluate **+(1)+(3)**, we include the result in the third row of Table 3 below, alongside Table 3 from the main paper (we re-run **+(1)** and **+(1)+(2)+(3)** with 3+ random seeds and report **mean ± std** for more rigorous evaluation). It is observed that using either the cross-view encoder (CV, 3rd row) or pseudo-3D attention (3D, 2nd row) alone results in slightly worse performance than the base two-stage setup (first row). However, when pseudo-3D attention is paired with the cross-view encoder (last row), it achieves the best performance. This suggests that the effectiveness of pseudo-3D attention **depends heavily on the quality and consistency of the underlying radar features**, which are better provided by the cross-view encoder than the separate-view encoder. We provide the following justification by looking into the pseudo-3D deformable attention in Fig. 4 of the main paper. \n\n#### Table 3: Ablation Study on RAPTR architectural modules (Encoder, Decoder, Attention)\n| Decoder | Encoder | Attn | Pose Loss | Joint Loss | MPJPE (WALK) | MPJPE (MULTI) | Notes                                            |\n| ------- | ------- | ---- | --------- | ---------- | ------------------ | -------------------- | ------------------------------------------------ |\n| 2S      | SV      | 2D   | T3D       | K2D+G3D    | 22.95  $\\pm$ 1.21             | 20.33  $\\pm$ 0.73                | **+(1)** in Table 3                              |\n| 2S      | SV      | 3D   | T3D       | K2D+G3D    | 22.83              | 20.78                | **+(1)+(2)** in Table 3                          |\n| 2S      | CV      | 2D   | T3D       | K2D+G3D    | 24.73              | 20.57                | **+(1)+(3)** (suggested by Reviewer bgMt) |\n| 2S      | CV      | 3D   | T3D       | K2D+G3D    | **22.32  $\\pm$ 0.06**              | **18.99  $\\pm$ 0.16**            | **+(1)+(2)+(3)** in Table 3 (RAPTR (ours))       |\n\n> **Weaknesses**\n> * Lack of effective modeling for occlusion and multi-person interference:...\n\n**Response**  \nFor **occlusion**, our preliminary investigation shows that RAPTR with weak supervision has some capability in recovering 3D poses even when some body parts are occluded (e.g., by furniture or other people). This is particularly observed on the MMVR dataset where the subjects are often partially occluded by furniture. We observed that:\n\n- The 2D keypoint labels—generated by pre-trained vision models—often extrapolate occluded keypoints accurately in the image pixel domain.\n- Our weakly supervised framework lifts these 2D keypoints to 3D using the combination of 2D keypoint loss, 3D template loss, and 3D gravity loss.\n\nThese results appear to suggest that RAPTR can handle small-to-moderate occlusion via strong image-based 2D supervision loss and structured 3D regularization (template + gravity loss). However, due to the absence of ground-truth 3D keypoints in MMVR, we cannot provide a quantitative analysis under varying occlusion levels.\n\nFor **multi-person interference**, this is a good point. In both the HIBER and MMVR datasets, we did not observe strong interference effects, potentially due to the  absorption of radar waveforms at 77 GHz at human body and clothing, and multiple-bouncing attenuated signals, resulting in weak ghost targets. One possible direction is to incorporate **temporal dynamics** of human motion. We leave this as a future topic.\n\n> **Weaknesses**\n> * Rigid prior assumptions in the 3D template: The reliance on a fixed pose template may reduce generalization to out-of-distribution poses or body shapes.\n\n**Response**  \nThanks for the insightful comment on the 3D template prior. In implementing our 3D Template Loss, we used a standard standing pose of a 1.6 m-tall person (see Fig. 3 in the main paper). To address this comment, we conducted an ablation study evaluating the impact of different 3D templates at the first-stage pose decoder. Results are shown in **Table 4** below. Specifically, we experimented with:\n- A standing pose scaled by two factors: **0.5×** and **1×**\n- A **sitting pose** of a 1.6 m-tall person\n- A **learned scaling factor** applied to the standard standing pose\n\nTable 4 suggests that the choice of 3D template has *some minor impact on final MPJPE*, likely due to the *refinement capability of the second-stage decoder*, as long as the first-stage decoder generates a *reasonable, human-like initial pose*. \n\n#### Table 4: The effect of 3D Template \n| Decoder | Encoder | Attn | Pose Loss | 3D Template                   | Joint Loss | MPJPE (MULTI) | Notes                        |\n| ------- | ------- | ---- | --------- | ------------------------------- | ---------- | ---------- | ---------------------------- |\n| 2S      | CV      | 3D   | T3D       | Standing (scale=1)        | K2D+G3D    | 18.99  $\\pm$ 0.16      | existing result                 |\n| 2S      | CV      | 3D   | T3D       | Standing (scale=0.5)        | K2D+G3D    | 19.49      |   new result                 |\n| 2S      | CV      | 3D   | T3D       | Sitting (scale=1)          | K2D+G3D    | 19.86      | new result  |\n| 2S      | CV      | 3D   | T3D       | Standing (learned scale)   | K2D+G3D    | 19.71      | new result |\n---\n\n> **Questions**\n> * ...Can ablations with more diverse attention mechanisms help validate this choice?\n\n**Response**  \nWe refer the reviewer to **Table 3 above**, which evaluates the effect of RAPTR's architectural modules—including the attention mechanisms (2D vs. pseudo-3D)—in response to your concern. We appreciate your suggestion and will consider more attention mechnisms in the update paper."}, {"review_id": "CXLFwqsLUp", "reviewer": "Reviewer_Zb8G", "summary": "This paper proposes an advanced 3D human pose estimation method using radar signals and Transformers. Annotations are provided in the form of bounding boxes and 2D keypoints on images, offering significantly better scalability compared to conventional 3D joint coordinate annotations.\nThe proposed Transformer model, RAPTR, consists of an encoder that applies deformable cross-attention to multi-view radar signals and a pose/joint decoder that performs similar attention operations in a pseudo-3D space. A custom loss function is also introduced to accommodate both 2D pose and 3D bounding box annotations.\nExtensive experiments on two datasets demonstrate that the method significantly outperforms existing approaches.", "questions": "A comparison that clearly demonstrates the effectiveness of the proposed architecture, which I pointed out as a weakness, is highly desirable. Additionally, if my understanding of the template loss is incorrect, I would appreciate it if the authors could clarify or correct it.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- By using a scalable annotation scheme with 3D bounding boxes and 2D joint positions, the method achieves very high accuracy and demonstrates strong practical applicability in real-world scenarios. It also makes a valuable contribution toward enabling future large-scale dataset development.\n- As shown in Table 1, the proposed annotation setting clearly improves accuracy compared to existing methods.\n- The results in Figure 7 show that the proposed loss function is highly effective, confirming its strong suitability for scalable annotation settings.\n- The paper is well-written, and the figures are very clear and easy to understand. The methodology is also clearly explained.\n\nWeaknesses:\n- The technical contributions of the paper mainly consist of the architecture and the loss function. However, the effectiveness of the proposed architecture remains unclear. A close look at the ablation study suggests that the performance gains are primarily due to the two-stage structure of the pose decoder and joint decoder. As shown in Figure 9 of the supplementary material, the proposed decoder using pseudo 3D space essentially merges the offset computation branches of the deformable attention used in the existing method QRFPose. This architectural change appears to be incremental. Although the author claims that the proposed method is simpler than the existing QRFPose, a comparison regarding how much the switch from QRFPose to RAPTR improves model size (memory usage) or inference time is insufficient. \n> Although adding pseudo-3D deformable attention slightly worsens the performance, the error gain is only about 1 cm, and this rather contributes to the aspect that the attention mechanism for two-view radar input becomes simpler.\n\n- To fairly demonstrate the effectiveness of the architecture, it would be necessary to apply the {Pose + Joint} decoder structure to QRFPose and evaluate the resulting performance.\n- (Minor) Regarding the proposed Template Loss, the paper does not describe how the template pose is calculated for each subject. I understand that global translation is computed using 3D gravity centers, but the paper does not clearly specify how the template keypoint at the coordinate origin is calculated.", "was_revised": true, "final_justification": "The authors have adequately addressed my concerns by demonstrating the effectiveness of pseudo-3D attention in terms of both accuracy and computational complexity. They also showed that their scalable training strategy, based on 3D bounding boxes and a template loss, achieves only minimal performance degradation compared to training with full 3D keypoint ground truth. Based on these clarifications and improvements, I have raised my score and now consider the paper a borderline accept.", "rebuttal": "> **Summary \\& Strengths**\n\n**Response**  \nWe thank the reviewer for recognizing our motivation to use weak supervision for 3D radar perception—an unexplored direction in this domain. We also appreciate your positive feedback on our loss function ablation (Fig. 7) and experiments (Table 1). As noted, our work has implications for future large-scale dataset development. We will release full training/evaluation code and pretrained checkpoints to support such efforts.\n\n> **Weaknesses**\n> * ...However, the effectiveness of the proposed architecture remains unclear. A close look at the ablation study suggests that the performance gains are primarily due to the two-stage structure of the pose decoder and joint decoder. \n\n**Response**  \nThanks for raising this concern. While we do not claim the two-stage decoder as a core contribution, we deliberately adopt PETR's two-stage design as a strong, empirically validated baseline for our purpose - 3D radar perception under weak supervision.  \n\nWith additional ablation studay, we summarize our contribution in two folds (as you pointed out): (1) the **structural design of loss functions** under weak supervision; and (2) the **architectural modifications** including cross-view (CV) radar encoder and pseudo-3D deformable attention. \n\nFirst, **our structured design of the loss functions**—particularly the use of 3D Gravity Loss and 3D Template Loss across the two decoder stages—**is key to the substantial performance gains**. Along with the suggestion from Reviewer vQxv, Table 1 below provides a complete picture of the significance of each loss function and the decoder stage where it is applied.\n\n- The **3D template prior at the first-stage decoder** helps localize initial pose estimates near the correct spatial region (e.g., close to the 3D bounding box center) and promotes **human-like keypoint configurations**, avoiding degenerate or implausible shapes.\n- The **second-stage joint decoder** refines initial pose estimates by projecting 3D keypoints to the 2D image plane and minimizing 2D keypoint loss (K2D), which leverages the fine granularity of 2D annotations. However, **2D keypoint loss alone cannot constrain absolute scale or depth** due to *depth ambiguity*—where a distant large object may appear identical in 2D to a closer small object. To resolve this, we **combine 2D keypoint loss with 3D Gravity Loss**, allowing the model to learn depth- and scale-aware corrections during pose refinement.\n\n#### Table 1: Effect of our structural loss function on RAPTR\n##### Results with ± values are newly added and report **mean ± std** from at least 3 runs with different random seeds.\n\n| Decoder | Encoder | Attn | Pose Loss | Joint Loss  |  MPJPE (MULTI) | Notes                                  |\n| ------- | ------- | ---- | --------- | ----------  | -------------- | ---------------------------------------|\n| 2S      | CV      | 3D   | --        | K2D+G3D     | 50.96          | (A) in Fig.7                           |\n| 2S      | CV      | 3D   | K2D+G3D   | K2D+G3D     | 27.36          | (B) in Fig.7                           |\n| 2S      | CV      | 3D   | T3D       | K2D         | 30.06          | new result (suggested by Reviewer vQxv)|\n| 2S      | CV      | 3D   | T3D       | K2D+G3D     | **18.99  $\\pm$ 0.16**      | RAPTR                                  |\n---\n##### symbol are defined below Table 1 of the response to Review bgMt. \n\nOur structural loss design is also compatible with PETR. As shown in Table 2 below, **The original PETR with 2D keypoint loss alone completely fails to generalize to 3D radar perception**, with MPJPE over 370 cm due to *depth ambiguity*. Adding gravity and template losses reduces the error to under 20 cm—a **10× improvement**.\n\n#### Table 2: Effect of our structural loss function on PETR\n| Decoder | Encoder | Attn | Pose Loss | Joint Loss | MPJPE (MULTI)  | Notes                                                     |\n| ------- | ------- | ---- | --------- | ---------- | ----------- | --------------------------------------------------------- |\n| 2S      | SV      | 2D   | K2D       | K2D        | 372.03      | PETR with 2D Keypoint Loss only                                                |\n| 2S      | SV      | 2D   | K2D+G3D   | K2D+G3D    | 21.03       | PETR with 3D Gravity Loss  |\n| 2S      | SV      | 2D   | T3D       | K2D+G3D    | **19.52**   | PETR with 3D Gravity Loss+ 3D Template Loss |\n\nSecond, **pseudo-3D attention works better with stronger, view-consistent radar features—such as those provided by the cross-view (CV) encoder.** Thanks to Reviewer bgMt's suggestion to evaluate **+(1)+(3)**, we include the result in the third row of Table 3 below, alongside Table 3 of the main paper (we re-run **+(1)** and **+(1)+(2)+(3)** with 3+ random seeds and report **mean ± std** for more rigorous evaluation). It is observed that using either the cross-view encoder (CV, 3rd row) or pseudo-3D attention (3D, 2nd row) alone results in slightly worse performance than the base two-stage setup (first row). However, when pseudo-3D attention is paired with the cross-view encoder (last row), it achieves the best performance. This suggests that the effectiveness of pseudo-3D attention **depends heavily on the quality and consistency of the underlying radar features**, which are better provided by the cross-view encoder than the separate-view encoder. We provide the following justification by looking into the pseudo-3D deformable attention in Fig. 4 of the main paper. \n\n#### Table 3: Ablation Study on RAPTR architectural modules (Encoder, Decoder, Attention)\n| Decoder | Encoder | Attn | Pose Loss | Joint Loss | MPJPE (WALK) | MPJPE (MULTI) | Notes                                            |\n| ------- | ------- | ---- | --------- | ---------- | ------------------ | -------------------- | ------------------------------------------------ |\n| 2S      | SV      | 2D   | T3D       | K2D+G3D    | 22.95  $\\pm$ 1.21 | 20.33  $\\pm$ 0.73                | **+(1)** in Table 3                              |\n| 2S      | SV      | 3D   | T3D       | K2D+G3D    | 22.83              | 20.78                | **+(1)+(2)** in Table 3                          |\n| 2S      | CV      | 2D   | T3D       | K2D+G3D    | 24.73              | 20.57                | **+(1)+(3)** (suggested by Reviewer bgMt) |\n| 2S      | CV      | 3D   | T3D       | K2D+G3D    | **22.32  $\\pm$ 0.06**              | **18.99  $\\pm$ 0.16**            | **+(1)+(2)+(3)** in Table 3 (RAPTR (ours))       |\n\n> **Weaknesses**\n> * As shown in Figure 9 of the supplementary material, ...This architectural change appears to be incremental...a comparison regarding how much the switch from QRFPose to RAPTR improves model size (memory usage) or inference time is insufficient.\n\n**Response**  \nAs you pointed out, Figure 9 in the supplementary material illustrates key differences between the QRFPose and RAPTR: \n\n- **decoupled 2D offsets in QRFPose** versus **3D offsets in RAPTR**. While conceptually simple, our 3D offsets better align with radar physics—ensuring that the same human keypoint projects to consistent depths across multiple radar views. In contrast, the 2D offsets in QRFPose may select unrelated features at different depths, limiting cross-view consistency.\n- **decoupled attention weights in QRFPose** versus **integrated attention weights in RAPTR** to aggregate the view-vise features or cross-view radar features.  \n\nIn terms of memory footprint and inference time, we don't expect big differences between the 2D and pseudo-3D deformable attention as pseudo-3D saves one parameter for one offset prediction. \n\n> * ...it would be necessary to apply the {Pose + Joint} decoder structure to QRFPose and evaluate the resulting performance.\n\n**Response**  \nSuggestion has been taken. As shown in **Table 4**, the second row reports the MPJPE for the {Pose + Joint} decoder structure added to QRFPose with 2D keypoint and gravity losses, showing significant gains over the base one-stage QRFPose (first row).\n\nThis {Pose + Joint} structure is further improved by applying our structured loss design (e.g., 3D template loss at the first-stage decoder), as shown in the third row.\n\nWith additional architectural enhancements—such as the cross-view (CV) encoder and pseudo-3D deformable attention (3D)—the last row achieves the best performance across all architecture and loss configurations on the HIBER (MULTI) dataset.\n\n#### Table 4: QRFPose versus RAPTR (with 3D template loss)\n\n| Decoder | Encoder | Attn | Pose Loss | Joint Loss | MPJPE (MULTI) | Notes                                                            |\n| ------- | ------- | ---- | --------- | ---------- | ---------- | ---------------------------------------------------------------- |\n| 1S      | SV      | 2D   | K2D+G3D   | --         | 56.02      | **Base** QRFPose of Table 3 |\n| 2S      | SV      | 2D   | K2D+G3D   | K2D+G3D    | 21.03      | QRFPose with 2-stage decoder (suggested by Reviewer Zb8G) |\n| 2S      | SV      | 2D   | T3D       | K2D+G3D    | 20.33  $\\pm$ 0.73      | **+(1)** in Table 3                              |\n| 2S      | CV      | 3D   | T3D       | K2D+G3D    | **18.99  $\\pm$ 0.16**  | **+(1)+(2)+(3)** in Table 3 (RAPTR (ours))       |\n\n> * (Minor) ...the paper does not describe how the template pose is calculated for each subject...\n> \n**Response**  \nThanks for pointing out the calculation of 3D template. In implementing our 3D Template Loss, we used a standard standing pose of a 1.6 m-tall person (see Fig. 3 in the main paper). More specifically, we used the ground-truth 3D keypoint labels at one frame to construct the template (minus its center position).  \n\nRelated to your comment, we also conducted an ablation study evaluating the impact of different 3D templates at the first-stage pose decoder (in response to a comment from Reviewer WQeC,). Please refer to Table 4 in our response to Reviewer WQeC."}]

Author rebuttal:

Meta-review: This paper introduces RAPTR, a novel method for radar-based 3D human pose estimation. The core innovation of the proposed method is that it uses only low-cost, easily accessible 3D bounding boxes (BBoxes) and 2D keypoints for supervision, instead of relying on expensive and difficult-to-collect 3D keypoint labels. The whole framework is built on PETR to output initial 3D pose and refined 3D pose, which is supervised by the proposed 3D template loss, and 2D keypoint and 3D gravity losses, respectively. Experiments on two public radar datasets demonstrate the effectiveness of the proposed method 

Strenghts include:
Well-written paper
Significant reduction in data annotation costs
Interesting ideas - Using learnable queries to represent 3D keypoints and offsets in deformable transformer
Solid Engineering and Architectural Design 
Extensive Experiments and Ablation Studies 
Promising results - Robust to Multi-Person and Cluttered Scenes
Efficient Use of Weak Supervision

Weaknesses include:
Somewhat limited methodological novelty
Clarity could be improved
Experimental validation could be improved
Occlusion and Human-to-Human Interference Remain Unsolved
Strong Assumption on Template Priors

The authors provide solid rebuttals. There was an engaging discussion among authors and reviewers after the rebuttal, after which all four reviewers were positive - grades 4,4,4,4. I recommend accepting the paper.
Decision: Accept (poster)