From Sparse to Dense: Spatio-Temporal Fusion for Multi-View 3D Human Pose Estimation with DenseWarper
Authors: Ling Li
Changjie Chen
Yuyan Wang
Jiaqing Lyu
Kenglun Chang
Yiyun Chen
Zhidong Deng
Venue: ICLR 2026 Poster (2026)
Keywords: 3D pose estimation; Spatiotemporal; Sparse Interleaved Input; Epipolar Geometry;
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We propose a coefficient-interleaved multi-view input scheme and DenseWarper module for efficient spatiotemporal 3D pose estimation, achieving state-of-the-art performance with sparse inputs.

Abstract: In multi-view 3D human pose estimation, models typically rely on images captured simultaneously from different camera views to predict a pose at a specific moment. While providing accurate spatial information, this traditional approach often overlooks the rich temporal dependencies between adjacent frames. We propose a novel 3D human pose estimation input method: the sparse interleaved input to address this. This method leverages images captured from different camera views at various time points (e.g., View 1 at time $t$ and View 2 at time $t+\delta$), allowing our model to capture rich spatio-temporal information and effectively boost performance. More importantly, this approach offers two key advantages: First, it can theoretically increase the output pose frame rate by N times with N cameras, thereby breaking through single-view frame rate limitations and enhancing the temporal resolution of the production. Second, using a sparse subset of available frames, our method can reduce data redundancy and simultaneously achieve better performance. We introduce the DenseWarper model, which leverages epipolar geometry for efficient spatio-temporal heatmap exchange. We conducted extensive experiments on the Human3.6M and MPI-INF-3DHP datasets. Results demonstrate that our method, utilizing only sparse interleaved images as input, outperforms traditional dense multi-view input approaches and achieves state-of-the-art performance.

Published as a conference paper at ICLR 2026
FROM SPARSE TO DENSE: SPATIO-TEMPORAL FU- SION FOR MULTI-VIED 3D HUMAN POSE ESTIMA- TION WITH DENSEWARPER
Ling Li Changjie Chen Yuyan Wang", Jiaqing Lyu', Kenglun Chang", Yiyun Chen', Zhidong Deng 1,* Depparmeent of Computer Science, THUAI, BNRist, Tsinghua University, Beijing, China
2 Apple, Beijing, China
Dalian University of Technology, Dalian, China
*Hong Kong University of Science and Technology (Guang Zhou), Guang Zhou, China
University of Manchester, Manchester, UK
"Equal contribution
"Corresponding author: michael @ @tsinghua.edu.cn
ABSTRACT
In multi-view 3D human pose estimation, models typically rely on images cap- tured simultaneously from different camera views predict pose specific moment. While providing accurate spatial information, this traditional approach often overlooks the rich temporal dependencies between adjacent frames. We propose a novel 3D human pose estimation input method: the sparse interleaved input to address this. This method leverages images captured from different cam- era views at various time points (e.g., View at time and View 2 at time t + 8), allowing our model to capture rich spatio-temporal information and effectively boost performance. More importantly, this approach offers two key advantages: First, it can theoretically increase the output pose frame rate by N times with N cameras, thereby breaking through single-view frame rate limitations and en- hancing the temporal resolution of the production. Second, using a sparse subset of available frames, method reduce data redundancy and simultaneously achieve better performance. We introduce the DenseWarper model, which lever- ages epipolar geometry for efficient spatio-temporal heatmap exchange. We con- ducted extensive experiments on the Human3.6.. 6.6 and MPI-INF-3DHP datasets. Results demonstrate that our method, utilizing only sparse interleaved images as input, outperforms traditional dense multi-view input approaches and achieves state-of-the-art performance. The source code for this work available at htttps://gitiii com www.c
ICLR2026
INTRODUCTION
3D Human Pose Estimation (3DHPE) (Li et al. 2023 Baumgartner & Klatt 2023 Bridgeman
jet al. 2019 Qiu et al. 2019b Zheng et al., (020)) has broad applications in fields such as dance
synthesis (Wang al. 2024a Yin et al., 2024: Wang et al. (024b), action recognition (Karim)
let al. 2024; Manakitsa et al.) 2024). and virtual reality (Lampropoulos ool & Kinshuk 2024 de Lur- des Calisto Sarkar 2024). Especially in multi-view settings, utilizing images captured by multiple synchronized cameras can provide more accurate and stable pose estimation results than singlc-view methods (Hyla 2016 BBidgemaan et al., 2019 Zheng et al. 2021a Pavilo et al.) (019b). However, existing methods commonly adopt a single- moment dense input paradigm, where the model must receive synchronized images from all views at each time step to predict the pose. Although this ap- proach provides sufficient spatial information, its single-moment input structure presents three key bottlenecks: redundant computational overhead, insufficient utilization of temporal information (Xu et al. 2024b] Xie et al. 2024), and the inability to break through camera frame rate limitations. This inefficient input mode leads to unnecessary computational waste and temporal information discon-
tinuity (Han et al. 2024 Liu al. 2020a).

---

Published as a conference paper at ICLR 2026
Frame Frame Frame Frames
Frame Frame Frame Frame
Frame
Frame
Frame Frame Frame Frame 2
Time
View
View
View
View
Interpolation
Sparse Interleaved
(b) Dense Full Frame Input
(c) Skeleton Interpolation
(d) Monocular Image Input
View
Input (Ours)
Input
Figure Common approaches for multi-view pose estimation. proposed sparse interleaved input, where each view selects single temporally interleaved image as input to leverage spatio-temperal information across views fully; (b) illustration of dense, full-frame multi-view input: (c) keypoint interpolation input, which entances the output frame rate; (d) illustration singlo-view image input.
We propose a novel 3D human pose estimation input paradigm to solve this challenge: sparse in- terleaved input. This method breaks the limitations of traditional synchronous input by cleverly using images captured from different camera views at various time points as input. More specifi- cally, Camera 1 captures an image at time f, followed by Camera 2 at + 6, and forth, with the sequence culminating in Camera N at t + (N - 1) x 6. This innovative method brings two core ad- vantages. First, it efficiently utilizes spatio-temport information and can be mathematically viewed as joint of multi-view inputs to reconstruct a higher-frequency pose output signal the temporal dimension. Second, it fundamentally breaks through the single-camera frame rate limitation. Specifically, for N lxed-fraaee-aaee cameras with fixed sampling rate of F, the sampling interval N x 5. Due the interleaved sampling between views, the pose output interval is ó, which raises the camera's effee- tive sampling rate to N x F, opening up new path for 3D tasks requiring high temporal resolution. To solve the computational latency problem in practical applications, optimization strategy, allowing the model to sample and process data instantly without waiting for all cameras to complete their interleaved sampling, achieving efficient and real-time processing.
sampling. The method cleverly leverages the temporal phase differences
adopt sliding window
To achieve this goal, we designed an end-to-end framework named Dense Warper. This framework efficiently convert sparse interleaved inputs into dense outputs with high spatio-temporal consis- tency. Dense Warper includes two core modules: spatial rectification and temporal fusion modules. First, we use spatial transformation module based on epipolar geometry to spatially rectify and fuse 2D heatmaps, generating a preliminary dense heatmap. Next, we use a temporal fusion module based on deformable convolution to learn and complete the temporal information in the heatmaps implicitly. Finally, a precise 3D pose can be obtained through triangulation.
Our main contributions are as follows:
Pioneering Task Paradigm: We are the first to propose and define the 3D pose estimation task based on sparse interleaved multi-view input. We provide novel paradigm for efficiently utilizing spatio- temporal information and potentially inspiring research in other multi-view 3D perception tasks.
Dense DenseWarper Framework: We designed Dense Warper, which uses an innovative
fusion mechanism to convert sparse interleaved inputs into dense pose outputs with high spatio- temporal consistency. breaks the frame rate limitations of traditional 3D tasks and, combined with the sliding window strategy, achieves low-latency processing.
Rigorous Experimental Validation and Benchmark Establishment: We conducted comprehensive experiments on the Human3.6 ..M (Ionescu et al., 2013) and MPI-INE-3DHP (Mehta al. (017) datasets. We reproduced numerous state-of-the-an 3D pose estimation algorithms and tested their performance the MPI-INF-3DHP dataset. Using unified benchmark for the 2D part, we fill a gap in testing benchmarks on MPI-INF-3DHP providing a rigorous and reliable reference for algorithms in this field.
2

---

Published as a conference paper at ICLR 2026
Timeltue
2
Frames
Deuse Warser
View
View
Triangulation
View
View
Sliding Window Movement
2D Pose Estimation
2D Heatmap
Original Heatmap
Spatial Fusion
Temporal Fusion
Output
Input
Vigure 2: Overview of the Dense DenseWarper Warper architecture. A sliding window is used to sample spurse interleaved images. with 2D pose estimation model generating initial heatmaps for each Missing information filled create and corrected using epipolar geometry based method. yielding spatially fused beatmup, Deformable convolutions then applied for temporal fusion. Finally, the resalting spatistemporally enriched heatmap is processed via triangulation to obtain accurate 3D keypoints.
heatmaps. These spatially fused
2 PROBLEM FORMULATION
Sparse Interleaved Multi-view Images. this study, we introduce the concept of "Sparse In- terleaved Multi-View Images" describe a sequence of sparsely sampled images from multiple viewpoints in multi-view setting. Specifically, we define them as a collection of sparsely sampled images from multiple viewpoints, represented as:
D = (1) L
(I)
1=
where M is the number of viewpoints V {V1 V22 22.... ... VMJ: h. N is the total number of frames, and i is the index of the input group. Each group I, contains one image from each viewpoint, ranging from the {M (i 1) 13 frame of viewpoint V1 to the {M -i--th frame of viewpoint VM. Here, the i-th group is denoted as:
I 1/
M-(i--)+1
(-(1-1)+2
MM
where r ( E #x represents the image from viewpoint Vg with frame number M (i 1) j. This structure ensures each time frame contains only one image from a specific view, creating an interleaved sparse input.
Taking M 4 as an example, the sparse interleaved inputs for the first two frame groups are I = - 22, T ] and I2 = [T i 122 17 }.
Target for Modeling. Traditional multi-view 3D human pose estimation tasks rely on densely synchronized multi-vien view images input. In contrast, our approach leverages sparse interleaved multi-view image sequences D.
The objective of our model is to predict a set of 3D) skeletons $ = (S1, S SN3 E Nx333 where each S, E / represents the 3D pose coordinates at the i-th frame for keypoints. We aim to achieve pose estimation performance comparable to dense full-frame input. Formally, we define the mapping as follows:
T: ()) S
where & denotes the 3D pose estimation model designed for sparse interleaved inputs, & represents the camera parameters, S E Nxxx3 is the resulting set of 3D skeletons, and J represents the number keypoints.
For each specific interleaved input group L, such I II H, ). the model produces corresponding set of 3D skeletons [S1, S2, S3, S1}. where each skeleton S, corresponds to the pose at the n-th frame. This can be expressed mathematically as:
T(T
(HTTM- M-(i-DD 1= o; =
(4)
where N. (1-11 E HxWx33 the image from view V, at the frame number M (i - 1) + j. and SM-(i-1)+5 represents the predicted 3D pose coordinates for that time frame.
Sliding Window Mechanism. In conventional methods, new input group I, must wait for all views to complete sampling. For instance, with M 4, the second group 12 (19, 15 17, 15 cannot be processed until 1 is available.
3

---

Published as a conference paper at ICLR 2026
Uncorrected
Heatmap
Spatially Corrected
V.
Heatmap
(a) Illustration of the Point-I.ine Correspondence Two Views
(b) Epipolar Geometry-bused Spatial Heatmap Fusion
Figure 3: Epipolar guometry based spatial bealmup fusion architecture. Geometric interpretation of point line relationship for keypoints aeross different views; (h) the pipeline for spatial heatmup fasion based on epipolar geometry. For an inaccurate heatmap point we use points from other views identify the maximum respouse along the line associated with and add these values to the original response at g in its beatmap. This process yields spatially corrected beatmap. processed according to this method.
First, compute the corresponding epipolar lines in the other
hearmaps. Then,
ligure, non-diagonal heatmaps masking represent target heatmaps
all
The sliding window method allows immediate processing once any view finishes sampling by reusing previously computed heatmaps. For example, after V1 samples IV. new input can be formed as:
r2 = ( R Tv. P.
where 1211 1, 19, are already processed and cached.
This mechanism enables real-time incremental processing of sparse interleaved inputs without wait- ing for all views. A caching mechanism simultancously allows for the reuse of computed heatmaps, thereby reducing latency and computational cost while maintaining estimation accuracy.
DENSEWARPER
Figure illustrates overall network structure, where our core module, Dense Warper, consists of two main components. First, the sparse interleaved heatmaps are input into a spatial fusion module based on epipolar geometry, producing an initially rectified dense 2D heatmap. Next, these initially rectified heatmaps are processed through temporal correction module, the warping module, which integrates pose information from time frames generate dense heatmap with enhanced spatio- temporal consistency. Finally, utilize the Triangulation method (Iskakov al.) 2019b] Remelli cet al., 2020b) to spatially reconstruct rectified heatmaps, thereby obtaining the 3D human pose. Section 3.1 introduces the concepts related epipolar geometry and multi-view spatial heatmap fusion; Section 3.2 discusses the temporal fusion process using Warper.
3.1
HEATMAP FUSION WITH EPIPOLAR GEOMETRY
3. BASIC CONCEPT OF EPIPOLAR GEOMETRY
Epipolar Geometry. It describes the geometric relationship between the corresponding points that two cameras observe in 3D scene. Readers can refer to (Hartley & Zisserman| (003). This rela- tionship is established by the epipolar constraint and the fundamental matrix (He et al. 2020 Xu & Zhang 1998), which are widely used in tasks such multi-view 3D reconstruction.
Zhang 2013
Let 3D point Q E R.4, with its fourth dimension set to 1, be projected onto the image planes of two cameras Vi and V2, with 2D projections q I23 and q' R respectively, where the third dimension of them is The projection matrices and P' map Q onto each image plane can be denoted as:
PQ, q'=
(5)
According to epipolar geometry, the corresponding points q and q' satisfy the epipolar constraint, defined by fundamental matrix F E 33x3 as follows:
Fq 0.

---

Del, Cony. Published as a conference paper at ICLR 2026
Offsets
20 Conv. (dillation = 33
Conv. (dilation 3)
2D Cony. (dilation = 6)
Def. Conv. (ditation
Difference
Residual Blocks
20 Conv. (dilution 12)
20 Cony. (dilation 18)
Def. Cony. (dilation - 18)
Warper Feature Map
210 Cony. (dilation 24)
Def. Conv. (dilation 24)
Figure structure temporal fusion module (Warper). perform temporal correction obtained from multi view spatial fusion. For each heaimup in larget time frame (ix. diagonal heatmaps the ligure), compute difference with the corresponding accurate heatmaa in the same view (the diagonal heatmup) and apply temporal pose feature learning module correct the heatmaps along the temporal dimension further. We food the computed differences
initial corrected heatmaps
stack
residual blocks,
followed by five 3 convolutional layers with dilation rates d 6, 6, 12, 18, 21). Each convolutional layer predicts a set of live offsets ofd) (p.)) for each pixel location pa. which are used to rewarp pose beatmap B. TTee live rewarped beatmaps are then summed, and the resulting temsor used predict target heatmaa..
This constraint indicates that, given point q in the first view, its corresponding point q' must lie on the epipolar line I' Y Fq in the second view, thus reducing the correspondence search from a 2D region ID line.
Application in Hulti-View Fusion. In the Warper model, we aim to use the Sampson dis- tance to correct the heatmaps of missing time frames within each view, thereby fully leveraging the information across different views. The appendix explains framework's epipolar geometry principles and Sampson distance metric.
3.1.2 HEATMAP FUSION
As shown in Figure we can obtain the corresponding heatmaps based on the RGB image input
Gu 2022 Zhang et all 2021 Li et al. 2020). We define the first set of sparse interleaved heatmaps as ( (x), H (z), ..... H }. For ease of discussion, subsequent formulations will be based on the first heatmap set, where Hi; (x) represents the heatmap for view V, the n-th frame at location x, where j E M.. To supplement the missing information, we first replicate each heatmap Hi, (x)) across the disappeared frames for each view, resulting in expanded set H of replicated heatmaps.
Thus, the expanded set H of heatmaps is formulated
H ((H), (x), H (z). .... H (x)) ...,, (H V (z)).
H (a)), (H (I)). Hi (z), H Var (x), ..... H (x)))
where each original heatmap H V (I)) is replicated M times to fill the blank in view Vn.
Due to incomplete alignment, the replicated heatmaps introduce spatial-temporal offsets, which can be corrected using epipolar geometry. This approach aligns each replicated heatmap with precise spatial information from other views. Since features at low-probability locations along the epipolar line contribute minimally to cross-view fusion, exact correspondences between views essary. Instead, we select the maximum probability point along the epipolar line as the matching point, which is reasonable simplification, response. For instance, as shown in Figure 3 for each position æ, we compute the epipolar lines in the other views and fuse the maximum responses along these lines with the response at æ.
unnec-
accurate correspondences typically yield the highest
Specifically, for each replicated heatmap H in view v. we use the corresponding heatmap from another view u (u o v) provide the correct spatial information. The correction process for each heatmap H (z) in view u at the n-th frame is defined by:
(1 A) M
(x) = (x) +
Σ
max H²),',, (z)
(8)
M

---

Published as a conference paper at ICLR 2026
Table 1: MPIPE Comparison with state-of-art art pose estimation methods on Human'3 6M (mmn) using ground-truth and detected Dposes. Ingut
types: Singlo-view (Single), Multi-viow full-frame (Pull), Multi-view interpolated (Interp). Best bold.
ACTIONS
Dr:
Disc
Ex.
Grazt
Phote.
Photo.
Piac.
Par.
Si.
Sub.
Smnkk.
Whi.
Wale.
WalLD.
WHIKT.
Goo-GGewdd Tark (G3)
(T=243)
Singl:
Full
12.9
23.6
23.7
21.6
24.0
23.1
22.8
26.0
12.3
23.9
24.4
23.3
22.3
22.4
Adatue SLERP
22.7
22.4
23.6
21.4
22.9
22.5
Adhiow
27.
21.4
25.6
36.3
26.3
24.1
25.0
21.3
34-4
22.7
35.0
25.3
26.6
36.4
26.0
25.4
EVT Ma [809]
Full
23.2
26.3
22.0
22.9
25.2
23.1
23.8
29.5
31.2
25.2
27.4
23.6
26.5
23.4
25.0
25.2
PPT -X
2002 5
Inter
23.5
27.5
21.5
21.9
24.7
29.0
23.0
23.3
30.6
34.1
26.6
22.2
22.4
27.6
24.0
25.5
PPT SLEEE
2002
Intep
23.0
26.0
21.6
21.6
24.9
23.1
22.8
23.3
26.2
32.4
24.9
22.1
23.1
26.2
24.7
24.3
Spure:
24.4
17.1
22.8
34.6
75.8
54.7
75.6
26.3
11.1
24.8
18.2
25.3
78.4
17.4
75.7
26.4
Spuree
232
23.5
21:31
31.9
30.5
20.2
20.5
22.8
21.2
19.7
21.4
20.5
22.1
21.8
20.7
21.3
GLA-GEN (T=263)
Singl: 41.4
44.4
40.8
41.8
46.0
54.1
42.1
43.5
$1.0
62.9
45.1
42.6
20.3
45.9
20.0
44.4
KTP-Fomer (Tm243)
Singl:
37.7
39.7
35.9
37.7
42.1
45.0
36.7
39.2
52.5
56.2
41.3
40.0
26.8
39.6
27.6
40.2
FiscPose (T=245) Sa 2004a
Single
40.2
Adduse
Full
35.0
37.1
52.2
34.0
35.2
36.5
33.0
34.5
40.5
41.3
37.2
35.3
33.8
37.4
33.1
35.8
A + MCC
Intep
$1.9
16.9
10.1
32.8
15.06
52.0
12.0
11.4
31.4
08.8
33.0
15.7
59.4
16.5
51.6
54.4
A SLEPP
Intep
54.4
16.6
51.9
34.0
14.6
55.7
12.5
54.1
40.3
41.0
16.7
34.8
53.4
16.8
52.4
55.3
Athdum:
Spare:
55.9
37.2
53.5
36.2
36.7
57.2
35.1
36.9
41.8
41.0
37.6
35.6
$0.0
16.2
55.1
56.9
Stradommr Zhang 2004
Full
35.4
Ohrs
Sparse
32.00
35.2
20111
32.4
35.11
34.4
36.3
32.3
16.9
01.1
15.5
13.9
31.41
36.0
30.4
33.6
GLA-GUN (T=343)
Singl:
41.1
42.9
40.5
30.3
44.2
52.8
42.5
40.9
54.1
60.5
44.5
40.4
52.3
44.8
35.2
43.7
KTP-Remer (T:243)
Single
14.4
16.9
16.5
14.7
19.6
61.4
16.
35.0
47.5
57.4
19.6
15.4
15.7
75.4
16.1
PizzPosc (7:349)
Singke
$1.7
13.3
28.7
39.7
15.6
54.0
79.1
24.3
40.9
40.9
12.9
11.4
23.1
31.6
22.A
51.4
Adbdus:
Full
28.3
20.0
25.3
29.5
36.9
26.A
27.0
28.1
28:7
52.1
22.8
28.8
26.7
29.6
25.5
25.1
A MCC
Inter
27.6
30.0
25.1
29.4
26.5
26.0
26.4
27.9
29-2
32.3
22.0
25.5
26.4
29.0
25.A
25.0
Adatus SLERP
Interp
28.3
30.0
25.3
31.5
26.9
26.4
26.9
26.1
26.7
32.1
27.7
28.8
20.8
29.6
25.5
26.1
Addiio
Sparnn
29.9
30.5
26.1
11.2
32.9
27.4
27.7
79.7
37.3
18.5
79.8
30.8
31.0
75.0
Alytene Movons dtal 3019
Full
19.8
40.9
31.9
21-3
31.4
58.6
2300
24.1
Wounei::
Full
18.8
21.7
19.6
31.2
21.0
11.8
57.1
11.5
48.4
23.5
21.7
23.5
Sprafomee
Pull
34.5
24.9
26.5
32.1
21.6
Ouss
Spane
21.2
380
1488
21.6
25.9
31.3
31.6
353
21.7
19.8
Note: Complete version with all baseline comparisons. Gray rows highlight our method. Action abbreviations: Directions (Dir), Discussion (Disc), Sitting Down (SitD), Walking Dog (WalkD), Walking Together (WalkT), Time frames (T=243) are shown where applicable. For the 2D pose estimation, we utilize ground truth, CPN (Cascaded Pyramid Network), and SimpleBaseline to obtain the corresponding 2D pose sequences. T rep- resents the number of imput time frames. MCC (Motion Consistency and Continuity) and SLERR (Spherical Linear Interpolation) are keypoint interpolation methods. MCC is neural network-based interpolation method, while SLERP is traditional interpolation technique.
Table P-MPJPE Comparison state-of-ar pose estimation methods on Human3.6M 3.6M (mm) using ground detected 2D poses. Input types: Single-view (Single), Multi-view full-frame (Full), Multi-view interpolated (Interp). Best in hold.
METHOD
INPUT
ACTIONS
AVG
Dr:
Dic.
Eai.
Greei
Phols,
Post
Psr.
SiD..
Smake:
Wat.
Wal:
WILLD.
WolKT.
GLA-GON (T-243 af 1053
Single
32.1
13.3
13:0
35.6
405
53.1
33.6
50.0
35.5
32.5
25.5
17.1
27.1
35.3
SIP-Somer (T~243))
Single
23.6
36.9
32.7
34.9
20.2
41.3
33.5
22.4
31.4
Single
Adalse
21.0
20.9
20.8
23.2
233
19.4
Adafuas 73dmar
20.5
22.6
183
31.0
20.5
21.0
19.1
19.4
12.6
23.3
21.6
18.1
35/7
Adllse SLER
Idtp
20.9
22.4
19.0
22.3
20.8
252
19.3
19.9
22.2
23.0
21.2
20.1
19.3
22:2
18.1
23.7
Adaluae 2har
p00g
Spene
22.5
22.6
193
22.5
21.5
2038
2000
20.
22.5
23.4
21.7
20.6
25.5
25.4
20.9
21.3
Sanfimer
Full
19.9
20.1
16.1
18.1
2018
2033
17.1
12.2
25.0
26.1
2199
19.6
17.7
20.7
17.9
199
Oin
Spane
203
17.8
183
17.9
19/1
15.6
17.
27.
18.1
18.3
19.01
30.5
14.6
19.4
where H (z) represents the corrected heatmap at location x in view u, À is balancing parameter between the current and other views, and pzzz denotes the epipolar line of x in view u. M the number of camera views. The term max Hor""" represents the maximum response along the x'Ep"
epipolar line in view u.
Following spatial correction, this process ensures that the expanded set H of heatmaps accurately captures spatial information across multiple views. By compensating for missing frames, this ap- proach enhances the model's 's robustness and improves the quality of 3D pose estimation in sparse, interleaved multi-view scenarios. Consequently, it enables the generation of a spatially rectified, dense, full-frame heatmap input derived from the initial sparse, interleaved heatmaps.
3.2
WARPER: TEMPORAL POSE AGGREGATION
After performing epipolar geometry- based heatmap fusion described in Section 3.1.2 obtain set of partly corrected heatmaps H;, (3r) that integrate spatial information across multiple views

---

Published as a conference paper at ICLR 2026
V (Vi. V2..... .... V. We introduce Warper module designed to capture temporal pose infor- mation from adjacent frames, further enhancing the temporal consistency of the sparse heatmaps Pöppel 1994 Zhang al. 2019).
For each view Vj at a target frame n, when n + M- E [M MM [[M],)], we compute the difference between the corrected heatmap H (x) and the heatmap from the sparse interleaved input H M (x) to capture temporal changes between frames, which can be denoted as:
-
11 (z) = H (2) H M+++
(9)
This difference is then passed through a series of 3 residual blocks extract temporal features. Following the residual blocks, apply five 3 3 convolutional layers (Yu al. 2017) with dilation rates d E {3, 6, 12, 18, 24), allowing the model to predict
of offset maps {o (x) d=1 for each pixel æ in the heatmap. These offsets are used to warp the target heatmap deformably (Dai et al. 2017) aligning it with temporal features. The details of the Warper module are shown in Figure Thus, for each spatially corrected heatmap H we generate five temporally aligned warped heatmaps by applying the offsets. These warped heatmaps are then aggregated by summation:
H Warper Di o (z)),
(10)
d=1
Ov (z)) represents the warping operation that uses the offset 0y, (r) to align each pixel in the heatmap. Notably, we trained a distinct warper model for each temporal mode.
where Warper(-: (d)
(d)
This warping and aggregation process is applied all interleaved frames in the sparse input set, ensuring that each refined heatmap Hy, integrates both spatial and temporal information. The re- sulting tensor captures comprehensive pose information across views and time frames, improving the robustness and accuracy of 3D pose estimation under sparse interleaved multi-view settings.
EXPERIMENT
4.1 DATASET
Human3.6M. Human3.6M is large-scale benchmark dataset widely used for 3D human pose estimation in controlled indoor environments. It consists of 3.6 million frames recorded from four synchronized high-resolution cameras capturing 11 professional actors (6 male, female) performing 15 distinct activities, including walking, sitting, and object interactions. This dataset includes accurate 3D skeletal annotations obtained via motion capture system and full-body 3D scans of each actor, enabling precise analysis of both 2D and 3D poses.
MPI-INF-3DHIP The MPI-INF-DDHP dataset a comprehensive resource for multi-view 3D human pose estimation, featuring annotated frames from indoor and everyday settings. It includes 8 actors (4 male, 4 female), each per- forming 8 activity sets, such as walking, sit- ting, complex exercises, and dynamic actions. With diverse scenarios and multi-view record- ings, the dataset enables robust evaluation of models under varying conditions.
Table 3: Recenstruction Enror (MPJPE in mm) on the MPI-INF-3DHP Dataset, Input pose sequences are obtained using SimpleBuseline detector: T denotes the number of input frames. Best results are high. lighted in hold.
MPJPP (Simpleliseeinee
Input Method
MPJPE.
GLA- (T=233)
Single
75.00
KTP-Former (T-243)
Single
67.59
Adafuse
Full
78.57
Adafuse MCC
Interpolation
Adafuse + SLERP
Interpolation
83.37
PPT
Full 106.30
Interpolaticn
PPT SLERP
Interpolation
110.44
Ours
spurse Interleaved
65.89
It is worth noting that the MPI-INF-3DHD dataset has not been extensively trained with a 2D detector. We are the first to process and align this dataset, and we will open-source all models and codes in the experiments.

---

Published as a conference paper at ICLR 2026
Table 4: Model Parameter Count and Performance Elficiency. Performance Eliiiencyy (MP))E/mm per MB) is calculated as the rañio of MPIPE (in mm) model size MB). Smaller values of this metric indicate better trade-offs between performance (MPJPE) model size (MB), with more efficient models achieving lower MPJPE while maintaining smaller parameter sizes. The average lalency specifically refers to the computational time of single model inference (in millisecomis).
Method
Para.M)
Flops.
Average Lafency. (ms)
Performance per MB (MPJPEEmmm per .MB)
GLA.GCN (T=243)
69.39
24.10
0.624
KTP-Hormer (T=243)
103.85
51.64
24.11
0.367
FinePose (T=243)
269.23
287.32
82.24
0.117
Adafuse (T=1)
69.66
204.26
96.028
0.403
Adafuse SLERP
69.66
96.03
0.403
Adafuse MCC
72.25
204
96.028
0,385
Sgraformer
81.23
204.28
99.19
0.299
Ouzs
76.51
111 136
44.51
0.291
4.2
EVALUATION METRICS.
The MPJPE measures 3D pose accuracy via mean Euclidean distance between predicted (P
.i......9))) and ground truth (P = (pi...., pJ
J
MPJPE
J
Σ
11 - PEll2
(1I)
where || [l2 is Euclidean norm.
4.3 EXPERIMENTAL RESULTS
Results on the Human3.6M Dataset. We conducted extensive experiments on the Human3.6M dataset using different 2D pose detectors evaluate the effectiveness of our sparse interleaved ap- proach. As shown in Table our method consistently achieves new state-of-the-art performance across various settings.
First, with ground truth (GT) 2D poses, our model achieves minimum average MPJPE of 21.33 21.3mm. This result is not only significantly better than all single-view methods (e.g., an improvement of 25.2% over GLA-GCN's 28. 28.5m)) but also outperforms multi-view approaches like Adafuse (23.7mm), yielding a performance gain of approximately 10.1%. Notably, our method achieves the best performance on 12 out of 15 action categories, with a particularly low MPJPE of 19.7 19.7mm on the challenging "SitDown" action, which is notably better than Adafuse's 's 21.6mm.
Second, we used CPN and SimpleBaseline as 2D pose detectors validate the model's robustness in real-world scenarios. With CPN, our method still achieves the best average MPJPE of 33. 6mm, rep- resenting substantial 24.3% performance improvement over the single- view GLA-GCN (44.4mm) Similarly, with SimpleBaseline, we achieve an impressive 22.3mm MPJPE, a significant improve- ment of approximately 20.6% over Adafuse (28.1mm) Furthermore, our method achieves the best result in 14 out of 15 action categories with this detector.
We also evaluated our model using the P- P-MPJPE metric, shown in Table which assesses the ac- curacy of relative joint positions. With SimpleBaseline 2D inputs, our model's average P-MPJPE is only19.4mm, representing 2.5% and 6.3% improvement over Sgraformer (19.9mm) and Adafuse (20.7mm), respectively. Our method achieves the best P-MPJPE in 13 out of 15 action categories, for instance, remarkable 15.6mm on the "Pose" action, which is significantly better than Sgraformer's 17.1mm. This result provides strong evidence that our sparse interleaved approach not only pre- cisely reconstructs the absolute 3D pose but also accurately captures the intricate internal structure and relative relationships of the human joints.
Results on MPI-INF-3DHP. As shown in Table Our method demonstrates generalization ca- pabilities on the more challenging MPI-INF-3DHP dataset, which features diverse outdoor settings and complex motions. With SimpleBaseline, we achieve 65.89 55.89mm MPJPE, substantially outper- forming both single-view methods (GLA-GCN) 75.00mm KTP-Former: 67.59mml and multi-view approaches (Adafuse: 78.57mm, PPT: (06.30mm). This advantage extends to comparisons with enerpolation-based methods (Adafuse+SLERP- 83.37mm, PPT+SLERP: 110.34mm), highlighting the effectiveness of our sparse interleaved input strategy in challenging real-world scenarios.
Model Efficiency Analysis. As shown in Table 4 our method exhibits significant advantages across multiple performance indicators. First, with parameter count of only 76.51M, achieves an effective balance between model capacity and performance, being considerably more lightweight
8

---

Published as a conference paper at ICLR 2026
and efficient than (FinePose) (269.23M), This demonstrates that model attains high accuracy with substantially fewer parameters. In terms of performance efficiency (MPJPE/mm per MB), our method achieves impressive value of 0.291, notably surpassing (GLA-GCN) (0.624) and (KTP- Former) (0.367). This highlights the model's ability to provide higher accuracy per unit of model capacity. Although (FinePose) attains a slightly lower MPJPE, its excessive parameter count and computational demand result in less favorable efficiincy. Furthermore, our method achieves the lowest average latency of 44.51 ms, which is significantly faster than (Adafuse) (96.03 ms) and (FinePose) (82.24 ms), ensuring real-time responsiveness. In addition, our approach reaches pro- cessing speed four times that of the input FPS (4f), further confirming its potential for real-time pose estimation applications. Overall, our method is highly suitable for scenarios that require both high performance and low latency.
Ablation Analysis. To validate the effectiveness of our key components, we conduct ablation studies on both datasets using SimpleBaseline as the 21) detector. On Human3.6M1 starting from a base- line MPJPE of 36.06mm, the addition of spatial heatmap fusion improves performance to 31.54mm, demonstrating the effectiveness of our epipolar geometry-based fusion strategy. Further, incorporat- ing the Warper module reduces the MPJPE to 22.28 mm, yielding a 38.2% improvement compared to the Spatial Fusion only baseline. The detailed results are presented in Table
On the MPI-INF-3DHP dataset, our spatial fusion module reduces the enror from 94.46mm to 88.63mm, while the complete model achieves a significantly lower error of 65.89mm, yielding an overall improvement of 30.25%. These quantitative results substantiate the efficacy of both temporal and spatial fusion strategies across diverse datasets. The detailed experimental setup and specifies are provided in the Appendix.
5 CONCLUSION
In this paper, we addressed fundamental limitation of conventional 3D human pose estimation: the reliance dense, synchronized multi-view inputs. To this end, pio- neered sparse interleaved input paradigm that leverages the rich spatio-temporal dependencies often overlooked by traditional methods. Our work makes two key contributions to the field. First, we introduced a new task formulation by demonstrating that a high-frequency pose output can be accurately reconstructed from sparse, temporally-inee -nntellll-nteeeeaeeed inputs. Second, we designed the Dense module, an efficient end-to-end framework capable transforming these sparse inputs into dense outputs with high ppatio-temporal coherence. Our extensive experiments the Human3.6M and MPI-INF- 3DHP datasets rigorously validate that our approach surpasses conventional dense input methods and achieves state-of-the-art performance. This breakthrough highlights our method's ability to challenge the long-held assumption of dense synchronized inputs fundamentally. Ultimately, our research provides a compelling proof-of-coneept for the future of resource-efficient,, real-time 3D perception, paving the way for applications in domains requiring high temporal resolution, such as robotics and VR/AR.
Taable 5: Ablation study results. We conducted abiation studies on the Human3 6M and MPI- INF INF- datasets validate the effectiveness of the pruposed space fusion module based on epipolar geometry and the temporal fusion module Warper. We use SimpleRoseline as 2D baseline model. We have bolded the best results.
Method
Spatial Heatmap Fusion
Warper
Avg. 1
36.06
Ours (Human3.6M)
31.54 X
V
22.28
X
X
94.46
Ours (MPI-INT-3DHP)
X
88. 63
65.89
Limitation. Our method has not been explored under non-unifon uniformfm intervals or extremely low- frame-rate camera sampling. For cases with extensive inter-camen ceerraa sampling intervals, as shown in Figure 5 in the appendix, our method may struggle to extract and recover spatio-temporal informa- tion effectively. Consequently, our approach is, to a certain extent, dependent on the density of the multi-view input stream, particularly in the temporal dimension.
Future Work. Our work opens up several promising directions for future research. First, we plan to extend our sparse interleaved input paradigm to other multi-view 3D tasks, such object detec- tion, to explore its generalizability and provide new insights into these fields. Second, we will delve deeper into the theoretical underpinnings of novel input method, examining it from the perspec- tive of interpretability. Future work will also focus on enhancing the model's 's robustness under more challenging conditions, including sparse and irregular sampling.

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was supported in part by Beijing Municipal Science Technology Commission No. 7251 100000000625009 by the National Science Foundation of China (NSFC) under Grant No. 62176134, and by a grant from the Assisted Medical Consultation Project Based on DeepSeek.
ETHICS STATEMENT
Our research aims to advance 3D human pose estimation by proposing a sparse interleaved in- put paradigm that addresses the insufficient spatio-temporal information utilization and frame rate limitations of traditional methods. All data used in this paper is from publicly available bench- mark datasets (Human3.6k and MPI-INF-DDHP) where personal identifying information was anonymized prior to release. We made modifications to these datasets to re-identify any in- dividual. We confirm that this study did not involve the direct participation of human subjects. We believe proposed technology, which includes the sparse interleaved input paradigm and the Dense Warper module, is designed solely to enhance computational efficiency and poses no risk of harmful applications, bias, or privacy infringement. We declare that the research and writing of this paper have adhered to the principles of academic integrity, with all citations and references clearly acknowledged.
REPRODUCIBILITY STATEMENT
To ensure the full reproducibility of our work, we provide all necessary information within this paper and its supplementary materials. The source code for our proposed DenseWarper framework, including the model implementation, training configurations, and evaluation scripts, will be made available via an anonymous link in the supplementary materials. Our experiments are based on the publicly available Human3.66 M. and MPI-INF-3DHP datasets, with all data processing steps also detailed in the supplementary materials. These measures ensure our work can be fully verified and built upon by the research community.
THE USE OF LARGE LANGUAGE MODELS (LLMs)
This paper utilized large language model (LLM) as the writing. The LLM's role was strictly limited to improving the clarity, grammar, and style of the text. It was not used for research ideation, data analysis, or the generation of core scientific content. The authors take full responsibility for the content, originality, and accuracy of the paper.
assist tool to aid or polish
REFERENCES
Tobias Baumgartner and Stefanie Klatt. Monocular 3D Human Pose Estimation for Sports Broad- casts Using Partial Sports Field Registration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 5109-5115 2023.
Lewis Bridgeman, Marco Volino, Jean- Yves Guillemaut, and Adrian Hilton. Multi-Person 3D Pose Estimation and Tracking in Sports. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 0-0, 2019.
Yujun Cai, Liuhao Ge, Jun Liu, Jianfei Cai, Tat-Jen Cham, Junsong Yuan, and Nadia Magnenat Thalmann. Exploiting Spatial-Tempone Relationships for 3D Pose Estimation via Graph Con- volutional Networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2272-2281, 2019.
Long Chen, Haizhou Ai, Rui Chen, Zijie Zhuang, and Shuang Liu. Cross-Vice View Tracking for Multi- Human 3D Pose Estimation at Over 100 FPS. In Proceedings of the IEEESCVF Conference on Computer Vision and Pattern Recognition, pp. 3279-3288, 2020.
Tianlang Chen, Chen Fang, Xiaohui Shen, Yiheng Zhu, Zhili Chen, and Jiebo Luo. Anatomy- Aware 3D Human Pose Estimation With Bone- Based Pose Decomposition. JEEE Transactions Circuits and Systems for Video Technology, 32: 198-209, 2021.
10

---

Published as a conference paper at ICLR 2026
Yilun Chen, Zhicheng Wang, Yuxiang Peng, Zhiqiang Zhang, Gang Yu, and Jian Sun. Cascaded Pyramid Network for Multi-Person Pose Estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7103-7112, 2018.
Ziyi Chen, Akihiro Sugimoto, and Chang-Hong Lai. Learning Monocular 3D Human Pose Estima- tion With Skeletal Interpolation. In ICASSP, 2022.
Jifeng Dai, Haozhi Qi, Yuwen Xiong, Li, Guodong Zhang, Han Hu, and Yichen Wei. Deformable Convolutional Networks. In Proceedings of the IEEE International Conference Computer Vision, pp. 764-773, 2017.
Maria de Lurdes Calisto and Soumodip Sarkar. Systematic Review of Virtual Reality in Tourism and Hospitality: The Known and The Paths to Follow. International Journal of Hospitality Man- agement, 1 16: 103623, 2024.
Jia Gong, Lin Geng Foo, Zhipeng Fan, Qiuhong Ke, Hossein Rahmani, and Jun Liu. DiffPose: Toward More Reliable 3D) Pose Estimation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 13041-13051, 2023.
Benjamin Graham, Martin Engelcke, and Laurens van der Maaten. 3D Semantic Segmentation with Submanifold Sparse Convolutional Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9224-9232, 2018
Zuguang Gu. Complex Heatmap Visualization. Imeta, 1:e43, 2022.
Dehao Han, Shijie Yang. Ping Zhao, Xiaoming Chen, Chen Wang, and Vera Chung. Enhancing Scene Understanding in VR for Visually Impaired Individuals with High-Frame Videos and Event Overlays. Proceedings of the IEEE International Conference on Consumer Electronics, pp. 1- 5, 2024.
Richard Hartley and Andrew Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press, 2:1-655, 2003.
Yihui He, Rui Yan, Katerina Fragkiadaki, and Shoou-I Yu. Epipolar Transformer for Multi-View Human Pose Estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 1036-1037, 2020.
Mir Rayat Imtiaz Hossain and James J Little. Exploiting Temporal Information for 3D Human Pose Estimation. In Proceedings of the European Conference on Computer Vision, pp. 68-84, 2018.
Wenbo Hu, Changgong Zhang, Fangneng Zhan, Lei Zhang, and Tien-Tsin Wong Conditional Di- rected Graph Convolution for 3d Human Pose Estimation. Proceedings of the ACM Interna- tional Conference on Multimedia, pp. 602-611,2022
Pawel Hyla. Multi-Cames Triggering and Synchronization Issue: Case Study, Journal of KONES, 23:193-200, 2016.
Catalin Ionescu, Dragos Papava, Vlad Olaru, and Cristian Sminchisescu. Human3.6.. 3.6:: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments. IEEE Trans- actions on Pattern Anulysis and Machine Intelligence, 36:1325-1339, 2013.
Karim Iskakov, Egor Burkov, Victor Lempitsky, and Yury Malkov. Learnable triangulation of human pose. Proceedings the IEEE/CVE International Conference on Computer Vision (ICCV), pp. 7718-7727, 2019a.
Karim Iskakov, Egor Burkov, Victor Lempitsky, and Yury Malkov. Learnable Triangulation of Human Pose. In Proceedings of the IEÉE/CVF International Conference on Computer Vision, pp. 7718-7727, 2019b.
Boyuan Jiang, Lei Hu, and Shihong Xia. Probabilistic Triangulation for Uncalibrated Multi-View 3D Human Pose Estimation. Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 14850-14860, 2023.
11

---

Published as a conference paper at ICLR 2026
Misha Karim, Shah Khalid, Aliya Aleryani, Jawad Khan, Irfan Ullah, and Zafar Ali. Human Action Recognition Systems: A Review of the Trends and Sfate-of-the-Ae IEEE Access, 12:36372- 36390, 2024.
Georgios Lampropoulos and Kinshuk. Virtual reality and gamification in education: a systematic review. Educational Technology Research and Development, 72:1691-1785, 2024.
Chuan-Kang Li, Hong-Xin Zhang, Jia-Xin Liu, Yuan-Qing Zhang, Shan-Chen Zou, and Yu-Tong Fang. Window Detection in Facades Using Heatmap Fusion. Journal of Computer Science and Technology, 35:900-999 9120 2020.
Han Li, Bowen Shi, Wenrui Dai, Hongwei Zheng, Botao Wang, Yu Sun, Min Guo, Chenglin Li, Junni Zou, and Hongkai Xiong. Pose-Oriented Transformer with Uncertainty -uideed Refinement for 2D-to-3D Human Pose Estimation. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 1296-1304, 2023.
Sijin Li and Antoni B Chan. 3D Human Pose Estimation from Monocular Images with Deep Convo- lutional Neural Network. In Proceedings of Asian Conference on Computer Vision, pp. 332-347, 2014.
Sijin Li and Antoni Chan. 3D Human Pose Estimation from Monocular Images with Deep Con- volutional Neural Network. In Proceedings of the Asian Conference Computer Vision, pp- 332-347, 2015.
Wenhao Li, Hong Liu, Hao Tang, Pichao Wang, and Luc Van Gool. MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13147-13156, 2022.
Kenkun Liu, Rongqi Ding, Zhiming Zou, Le Wang, and Wei Tang. A Comprehensive Study of Weight Sharing in Graph Networks for 3D Human Pose Estimation. In Proceedings of the Euro- pean Conference on Computer Vision, pp. 318-334, 2020a.
Ruixu Liu, Ju Shen, He Wang. Chen Chen, Sen-ching Cheung, and Vijayan Asari. Attention Mecha- nism Exploits Temporal Contexts: Real-Time 3D Human Pose Reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5064-5073, 2020b.
Diogo C. Luvizon, David Picard, and Hedi Tabia. 2D/3D Pose Estimation and Action Recognition Using Multitask Deep Learning. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 5137-5146, 2018
Diogo C Luvizon, Hedi Tabia, and David Picard. Human pose regression by combining indirect part detection and contextual information. Computers & Graphics, 85:15-22, 2019.
Haoyu Ma, Zhe Wang, Yifei Chen, Deying Kong, Liangjian Chen, Xingwei Liu, Xiangyi Yan, Hao Tang, and Xiaohui Xie. PPT: foken-Pruned Pose Transformer for Monocular and Multi-view Human Pose Estimation. In Proceedings of the European Conference on Computer Vision, pp. 424 442, 2022.
Nikoleta Manakitsa, George S Maraslidis, Lazaros Moysis, and George Fragulis. A review of ma- chine learning and deep learning for object detection, semantic segmentation, and human action recognition machine and robotic vision. Technologies, 12:15, 2024.
Julieta Martinez, Rayat Hossain, Javier Romero, and James Little. A Simple yet Effective Base- line for 3D Human Pose Estimation. In Proceedings of the IEEE International Conference Computer Vision, pp. 2640-2649, 2017.
Dushyant Mehta, Helge Rhodin, Dan Casas, Pascal Fua, Oleksandr Sotnychenko, Weipeng Xu, and Christian Theobalt. Monocular 3D Human Pose Estimation in the Wild Using Improved CNN Supervision. In Proceedings of the International Conference on 3D) Vision pp. 506-516, 2017.
Rahul Mitra, Nitesh B. Gundavarapu, Abhishek Sharma, and Arjun Jain. Semi-Supervised Learning for 3D Human Pose Estimation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 6907-6916. 2020.
12

---

Published as a conference paper at ICLR 2026
Gyeongsik Moon and Kyoung Mu Lee. 121-meshnet: Image-to-lixe prediction network for accurate 3d human pose and mesh estimation from single rgb image. In Proceedings of the European Conference Computer Vision, pp. 752-768, 2020.
Gyeongsik Moon, Ju Yong Chang, and Kyoung Mu Lee. Camera Distance- Top-Down Ap- proach for 3D Multi-Person Pose Estimation From Single RGB Image. In Proceedings of the IEEE/CVE international conference on computer vision, pp. 10133-10142, 2019.
Sungheon Park, Jihye Hwang, and Nojun Kwak. 3D Human Pose Estimation Using Convolutional Neural Networks with 2D Pose Information. In Proceedings of the European Conference Computer Vision, pp. 156-169, 2016.
Georgios Pavlakos, Xiaowei Zhou, and Kostas Danilidis. Ordinal Depth Supervision for 3D Hu- man Pose Estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7307-7316, 2018a.
Georgios Pavlakos, Luyang Zhu, Xiaowei Zhou, and Kostas Daniilidis. Learning to Estimate 3D Human Pose and Shape from a Single Color Image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 459-468, 2018b.
Pavlakos, Georgios and Zhou, Xiaowei and Derpanis, Konstantinos G. and Daniilidis, Kostas. Coarse-To-Fine Volumetric Prediction for Single- 3D Human Pose. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pp. 7025-7034, 2017.
Dario Pavllo, Christoph Feichtenhofer, David Grangier, and Michael Auli 3D Human Pose Estima- tion in Video With Temporal Convolutions and Semi-Supervised Training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7753-7762, 2019a.
Dario Pavllo, Christoph Feichtenhofer, David Grangier, and Michael Auli. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7753-7762, 2019b.
Jihua Peng, Yanghong Zhou, and PY Mok. KTPFormer: Kinematics and Trajectory Prior Ehowledge-Enhaneed Transformer for 3D Human Pose Estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1123-1132, 2024.
Ernst Pöppel. Temporal Mechanisms in Perception. International Review of Neurobiology, PD- 185-185, 1994.
Haibo Qiu, Chunyu Wang, Jingdong Wang, Naiyan Wang, and Wenjun Zeng. Cross View Fusion for 3D Human Pose Estimation. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 4342-4351, 2019a.
Haibo Qiu, Chunyu Wang, Jingdong Wang, Naiyan Wang, and Wenjun Zeng- Cross View Fusion for 3D Human Pose Estimation. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 4342 4351, 2019b.
Edoardo Remelli, Shangchen Han, Sina Honari, Pascal Fua, and Robert Wang. Lightweight Multi- View 3D Pose Estimation through Camera-Disentanglee Representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6040-6049, 2020a.
Edoardo Remelli, Shangchen Han, Sina Honari, Pascal Fua, and Robert Wang. Lightweight Multi- View 3D Pose Estimation through Camera-Disentanggle Representation. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6040-6049, 2020b.
Helge Rhodin, Jörg Spörri, Isinsu Katircioglu, Victor Constantin, Frédéric Meyer, Erich Müller, Mathieu Salzmann, and Pascal Fua. Learning Monocular 3D Human Pose Estimation From Multi- View Images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8437-8446, 2018
Cédric Rommel, Victor Letzelter, Nermin Samet, Renaud Marlet, Matthieu Cord, Patrick Pérez, and Eduardo Valle. ManiPose: In Proceedings of the Advances in Neural Information Processing Systems, pp. 107350-107378, 2024.
Multi-Hypothesis 3D Human Pose Estimation.
13

---

Published as a conference paper at ICLR 2026
Wenkang Shan, Zhenhua Liu, Xinfeng Zhang, Zhao Wang, Kai Han, Shanshe Wang, Siwei Ma, and Wen Gao. Diffusion Cefuusion-Based 3D Human Pose Estimation with Multi Ayytti-Hyptssesis Aggregation. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 14761-14771, 2023.
Fei Shen and Jinhui Tang. IMAGPose: A Unified Conditional Framework for Pose-Guided Person Generation. In Pruceedings of the Advances in Neural Information Processing Systems, pp. 6246 6266, 2024.
Hui Shuai, Lele Wu, and Qingshan Liu. Adaptive Multi- View and Temporal Fusing Transformer for 3D Human Pose Estimation. IEEE Trunsactions on Patlern Analysis and Machine Intelligence, 45:4122-4435, 2022.
Yukun Su, Guosheng Lin, and Qingyao Wu. Supervised 3D Skeleton Action Representation Learning With Motion Consistency and Continuity. In ICCV, 2021.
Xiao Sun, Bin Xiao, Fang Wei, Shuang Liang, and Yichen Wei. Integral Human Pose Regression. In Proceedings of the European Conference on Computer Vision, pp. 529-545, 2018.
Zhenhua Tang, Zhaofan Qiu, Yanbin Hao, Richang Hong, and Ting Yao. 3D Human Pose Estimation With Spatio-Temporal Criss-Cross Attention. In Proceedings of the IEEE/CVF Conference on 2023.
Computer Vision and Pattern Recognition, pp.
Bugra Tekin, Artem Rozantsev, Vincent Leppettt, and Pascal Fua. Direct Prediction 3D Body Poses From Motion Compensated Sequences. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 991-1000, 2016.
Tan Wang, Linjie Li, Kevin Lin, Yuanhao Zhai, Chung-Ching Lin, Zhengyuan Yang. Hanwang Zhang, Zicheng Liu, and Lijuan Wang. DisCo: Disentangled Control for Realistic Human Dance Generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, pp. 9326-9336, 2024a.
Zixuan Wang, Jia Jia, Shikun Sun, Haozhe Wu, Rong Han, Zhenyu Li, Di Tang, Jiaqing Zhou, and Jiebo Luo. DanceCamera3D: 3D Camera Movement Synthesis with Music and Dance. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7892- 7901, 2024b.
Tom Wehrbein, Marco Rudolph, Bodo Rosenhahn, and Bastian Wandt. Probabilistic Monocular 3D Human Pose Estimation With Normalizing Flows. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 11199 1208, 2021.
Bin Xiao, Haiping Wu, and Yichen Wei. Simple Baselines for Human Pose Estimation and Tracking. In Proceedings of the European Conference on Computer Vision, pp. 466-481, 2018.
Jinxia Xie, Bineng Zhong, Zhiyi Mo. Shengping Zhang, Liangtao Shi, Shuxiang Song, and Ron- grong Ji. Autoregressive Queries for Adaptive Tracking with Pantio-Temmpra/TTaaffoomeee In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 19300-19309, 2024.
Gang Xu and Zhengyou Zhang. Epipolar Geometry in Stereo, Motion, and Object Recognition: A Unified Approach, volume Springer Science & Business Media, 2013.
Jinglin Xu, Yijie Guo, and Yuxin Peng. FinePOSE: Line-Griined Prompt-Drin Driven 3D Human Pose Estimation via Diffusion Models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 561-570, 2024a.
Qi Xu, Xuanye Fang, Yaxin Li, Jiangrong Shen, De Ma, Yi Xu, and Gang Pan. RSNN: Recurrent Spiking Neural Networks for Dynamic Peatial-Tempore Information Processing. In Proceedings of the ACM International Conference on Multimedia, pp. 10602-10610, 2024b.
Tianhan Xu and Wataru Takano. Graph Stacked Hourglass Networks for 3D Human Pose Estima- tion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16105-161 14, 2021.
14

---

Published as a conference paper at ICLR 2026
Wenjie Yin, Xuejiao Zhao, Yi Yu, Hang Yin, Danica Kragic, and Marten Björkman. LM2D: Lyries- and Music-Driven Dance Synthesis. arXiv preprint SxXx:22403309407, 2024.
Bruce X.B. Yu, Zhi Zhang, Yongxu Liu, Sheng-hua Zhong, Yan Liu, and Chang Wen Chen. GLA- GCN: Global-local Adaptive Graph Convolutional Network for 3D Human Pose Estimation from Monocular Video. In Proceedings of the IEEE/CVF International Conference Computer Vi- sion, pp. 8818-8821 2023a.
Bruce XB Yu, Zhi Zhang, Yongxu Liu, Sheng-hus hua Zhong, Yan Liu, and Chang Wen Chen. GLA- GCN: Global-local Adaptive Graph Convolutional Network for 3D Human Pose Estimation from Monocular Video. In Proceedings of the IEEE/CVF International Conference Computer Vi- sion, pp. 8818-8829, 2023b.
Fisher Yu, Vladlen Koltun, and Thomas Funkhouser. Dilated Residual Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp- 472-480, 2017.
Haokui Zhang, Chunhua Shen, Ying Li, Yuanzhouhan Cao, Yu Liu, and Youliang Yan. Exploiting Temporal Consistency for Real-time Video Depth Estimation. In Proceedings of the IEEF/CVF International Conference Computer Vision, pp. 1725- 1734, 2019.
Jinlu Zhang, Zhigang Tu, Jianyu Yang. Yujin Chen, and Junsong Yuan. MixSTE: Seq2seq Mixed Temporll Encoder for 3D Human Pose Estimation in Video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13232-13242, 2022.
Kaihao Zhang, Wenhan Luo, Yiran Zhong, Lin Ma, Wei Liu, and Hongdong Li. Adversarial Spatio- Temporal Learning for Video Deblurring. IEEE Transactions on Image Processing, 29:9090- 9101, 2020.
Lijun Zhang, Kangkang Zhou, Feng Lu, Xiang-Dong Zhou, and Yu Shi. Deep Semantic Graph Transformer for Multi-View 3D Human Pose Estimation. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 1225-72144 2024.
Zhe Zhang, Chunyu Wang, Weichao Qiu, Wenhu Qin, and Wenjun Zeng. AdaFuse: Adaptive Multiview Fusion for Accurate Human Pose Estimation in the Wild. International Journal of Computer Vision, 129:703-718, 2021.
Zhengyou Zhang. Determining the Epipolar Geometry and its Uncertainty: A Review. International Journal of Computer Vision, 27:161-195, 1998.
Long Zhao, Xi Peng. Yu Tian, Mubbasir Kapadia, and Dimitris N. Metaxas. Semantic Graph Convo- lutional Networks for 3D Human Pose Regression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3425-3222 3335,, 2019a.
Long Zhao, Xi Peng, Yu Tian, Mubbasir Kapadia, and Dimitris N Metaxas. Semantic Graph Convo- lutional Networks for 3D Human Pose Regression. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 3425-3433 2019b.
Qitao Zhao and Shubham Tulsiani. Sparse-view view Pose Estimation and Reconstruction via Analysis by Generative Synthesis. In Proceedings of the Advances in Neural Information Processing Systems, pp. 111899-11 1992,, 2024.
Weixi Zhao, Weiqiang Wang, and Yunjie Tian. GraFormer: Graph-Oriented Transformer for 3D Pose Estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20437-0444, 2022.
Ce Zheng, Wentao Wu, Tianyu Yang, Shiwei Zhu, Chen Chen, Ronggang Liu, Jianfei Shen, Nasser Kehtarnavaz, and Mubarak Shah. Deep Learning- Basedd Human Pose Estimation: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43:2359-2385, 2020.
Ce Zheng, Shiwei Zhu, Matias Mendieta, Taojiannan Yang, Chen Chen, and Zhengming Ding. 3D Human Pose Estimation with Spatial and Temporal Transformers. In Proceedings of the IEFF/CVF International Conference Computer Vision, pp. 11656-11666 2021a.
15

---

Published as a conference paper at ICLR 2026
Ce Zheng, Sijie Zhu, Matias Mendieta, Taojiannan Yang. Chen Chen, and Zhengming Ding. 3D Hu- man Pose Estimation With Spatial and Temporal Transformers. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 11656-11665 2021b.
Kangkang Zhou, Lijun Zhang, Feng Lu, Xiang-Dong Zhou, and Yu Shi. Efficient Hierarchical Multi-view Fusion Transformer for 3D Human Pose Estimation. In Proceedings of the ACM International Conference on Multimedia, pp. 7512-7520, 2023.
Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable DETR: Deformable Transformers for End-to-End Object Detection. In Proceedings of the International Conference on Learning Representations, pp. 1-15, 2020.
Yiran Zhu, Xing Xu, Fumin Shen, Yanli Ji, Lianli Gao, and Heng Tao Shen. PoseGTAC: Graph Transformer Encoder- Decoder with Atrous Convolution for 3D Human Pose Estimation. In Pro- ceedings of the International Joint Conference on Artificial Intelligence, pp. 1359-1365, 2021.
Zhe Zou, Yanyu Huang, Yang Lu, and Feng Liu. Structure- 3D Hourglass Network for Hand Pose Estimation from Single Depth Image. In Proceedings of the British Machine Vision Confer- ence, pp. 1-13, 2020.
Zhiming Zou and Wei Tang. Modulated Graph Convolutional Network for 3D Human Pose Es- timation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 11477-11487, 2021.
16

---

Published as a conference paper at ICLR 2026
A RELATED WORK
A.1 3D HUMAN POSE ESTIMATION
Early approaches to 3D human pose estimation relied heavily on direct regression from 2D images
to 3D coordinates (Li & Chan 2014 2015: Moon et al. 2019 Park et al. 2016 Paylakos et al.
2018a Tekin et al. 2016 Wehrbein et al. [022] Luvizon et al., 2018: 2019. Moon & Lee 2020
Pavlakos, Georgios and Zhou, Xiaowei and Derpanis, Konstantinos G. and Daniilidis, Kostas
2017
Shen & Tang 2024] Rommel ct al.]| (024). With the advent of deep learning, two-stage methods be-
came prevalent, first detecting 2D keypoints and then lifting them 3D space (Martinez al. 2017
Chen et al.
Hossain Hossann & Little 2018 [iu et al. [202bb Pavllo et al.
2019a] 2019a]/Zheng et al. |02 lb
Li al. 2022| Zhang et al. al.] [002 lang et al. 2023
[Shan et el. 2023
[
Cai et al. 2019 Hu et al,
202 Liu et al. [2020a] Xu & Takano 2021 Yu et al.
2023a|/haoo et al. 2019a] Zou & Tang 2021
Gong et al. 2023 Li et al. 2023 Zhao al. 2022; Zhu et al. 2021). Recent methods have incor- porated additional constraints such as bone length consistency (Zhao et al. (019b) and anatomical
priors (Pavlakos et al. 2018b) to improve estimation accuracy. Furthermore, self-attention mecha-
nisms (Zheng et a (2200) and graph neural networks (Zou et al.) [2020] Zhao & Tulsiani (2024) have
been introduced to capture long-range dependencies and structural relationships in human poses.
A.2 MULTI-VIEW 3D POSE ESTIMATION
Multi-view approaches have demonstrated superior performance to single-view methods due to their ability resolve depth ambiguity. Traditional methods typically use triangulation-base tech- niques (Hartley & Zisserman 2003 Qiu et al. 2019a Zhang et al. 2021 Mitra et al. 2020 Shhai al. aall.12000) 2022 et 2l.3))20))) to reconstruct 3D poses from synchronized multi view 2D detections. Recent learning-based approaches have explored various ways to fuse multi-view information. Qiu et al. (Qiu et al. 2019b) proposed cross-view fusion using epipolar geometry, while Remelli et al. (Remelli et al., 2020a) introduced lightweight architecture for real-time multi-view pose esti- mation. Rhodin et al. (Rhodin et al., 2018) leveraged geometric consistency across views to improve unsupervised learning of 3D pose estimation. However, these methods generally rely dense, syn- chronized multi-view inputs, which can be computationally expensive and may not fully utilize temporal information.
A.3 TEMPORAL INFORMATION IN POSE ESTIMATION
Temporal information is crucial for robust pose estimation, particularly challenging scenarios with occlusions or motion blur. Previous work has explored various approaches to incorporate temporal information, including recurrent neural networks (Hossain & Little| (018)) and temporal convolu- tions (Pavllo et al. 2019b). Recent works like Zheng et al. (Zheng et al., 2021a) have proposed combining spatial and temporal attention mechanisms to capture motion dynamics better. However, most existing methods process temporal information within a single view, potentially missing valu- able cross-view temporal correlations. Approaches like (Sun et al.) 2018)) have attempted to bridge this gap by incorporating temporal consistency constraints in multi-view settings.
A.4 SPARSE AND EFFICIENT VISION METHODS
Recent trends in computer vision have shown increasing interest in efficient processing methods. Sparse convolutions (Graham et al., 2018) and attention mechanisms (Zhu et al.) (0200) have been proposed reduce computational overhead while maintaining performance. sparse view selection (Zhang et al.) 2020; Zhao & Tulsiani] 2024) and adaptive sampling strategies have been explored, though primarily for static scene reconstruction rather than dynamic human pose estimation. Notable works like (Tiang et al., 2023) have introduced probabilistic frameworks for efficient multi-view processing.
multi-view tasks,
A.5 GEOMETRIC FEATURE WARPING
Our work builds upon recent advances geometric feature warping, particularly in the context of multi-view feature fusion. Epipolar geometry has been extensively used for cross-view feature
17

---

Published as a conference paper at ICLR 2026
alignment (He et al. 2020). while deformable convolutions (Dai al. 2017) have shown promise
in handling dynamic spatial transformations. Recent works (Chen et al. 2020) have explored the
integration of geometric constraints with learning- carning-bbsed based feature warping. However, combining these
techniques for spatio-temporal feature warping in sparse multi-view scenarios remains largely un- explored.
B CODES AND MODELS
We have organized the code for our method in the codes directory and compiled the checkpoint
files used the experiments, as shown Table 6 Upon publication of the paper, we will re- lease the code and pretrained models. Detailed usage instructions for the code are provided in the " /cooes/ DenseW arper .....R..ADD..... wrper/READM...mm file for reference.
Model Name
Dataset
Performance (Metrie) MPJPE: 75.00
Checkpoint Name GLA-GUN CDCC.. KTP-Fommer 3DH..in
Source
CLA-CCNT-233)
3PHI
Reproduced Reproduced
KTP-Formen (4223)
MPLINE-3DHP
67. mm
Adafuse
MPLINT--DHIP
MPJPE: 78.57 mm
Adafuse 3DHIPphh.tr poh.
Reproduced
Adafuse (Zhang
2021 MCC Sa
2021
MPLINF-3DHP
MPJРE:
Reproduced
Adatuse SE EKF Chen
0024
MPL-INT-DDHP
MPIPE: 83.37 mm
Reproduced
PPT
MPL-INF-3DHP
MPJPE: 106, 30m3
PPT_3DHPppth MCC. MCC.
Reproduced
PPT
MPIPE: mm
Reproduced
PPT + SLERP Ours
MPL-NNF-DHH
MPIPE: 110.34 mm
Reproduced
MPLINF-3DHP
MPIPF: 65 mm
Ours 3DHPpth.tar
Reproducdd
GLA-CCN:122331
Humam 5M
МР]РЕ: 40.39 mm
GLA
Original
FinePose(T=243) (CPP)
Human3 6M
MPJPF: 40.20 mm
Repoddicdd
mm
Original
Adafuse (CPN)
Human3 6M
MPJPE: 35.81 mm
Adafuse pth
Original
Adafuse MCC (CPN)
Human3.60 5M
MPJPE: 34.42
MCC p3h
Original
Adafase + SLERP (CPN
Human3
MPJPE: 35.27 mm
Original
Sgraformer (CPN)
Human3 6M
MPJPE: 35.40 mm
Reproduced
Ours (CPN)
Human3
MPJРE:
mm
H366. Gl.A-GCN.bin
Reproduced
GLA-GON(T (SimpleRaseline)
Human3 6M
MPJPF: 43.74 mm
Repoduced
FinePose(T-245) (SimpleBuseline]
Human3.66 5M
MPJPE: 31 31.40mm
Reproduced
Human3
MPJPE:
mm
KTP-Former
Reproduced
Adafuse (Simplefluseline)
Human3 6M
MPJPE: 28.06 mm
Adafuse H.36..
Original
Adafuse MCC (SimpleBascinn)
Human3
MPJPE:
MCC_H36M,pth
Original
Adafise + SLERP (SimpleBaseline)
Human3. 6M
MPJPE: 28. 10 mm
Original
Sgraformer (SimpleBasclinc)
Human3 6M
MPJPE:
24.32 mm
Reproduced
Ours (SvepleBuseline)
MPJPE: mm
OursHH66,,ph for
Reproduced
Table Pretrained Models List. We conducted extensive experiments Human3.66 6M and MPI-INF-3DHP benchmark datases combination of open-source models and reproduced code. Here, "reproduced" means we reimplemented and retrained the model code. while "original" indicales we used open source models testing.
C EXPERIMENTAL DETAILS
C.I EXPERIMENTAL SETTINGS
In our experiments, we proposed the DenseWarper module for spatiotemporal fusion. All mod- els were trained using two RTX A100 GPUs. Table / provides comprehensive overview of the parameter settings used for training Dense Warper compared to other models.
Method
Loss
Epoch
Batch
Optimizer
GLA-GCN(T-2433
MPJPE WMPJPE-MPIVEE
0.01
200
512
Ranger
(TP--Frmee(T=243)
0.00008
200
1024
AdamW
temporal consistency loss
Adafuse Adafuse + MCC
MSE MSE
0.0001
50
Adam
0.0001
50
4
Adam
Adafuse + SLERP
MSF
0.0001
50
4
Adam
PPT
24SmoothLoss
0.001
200
32
Adarn
PPT + MCC
24SmoothLoss
0.001
200
Adam
PPT SLERP
24Smooth
0.001
200
Adam
Ours
2dSmoothLoss
0.001
50
Adam
Table 7: Experimental parameter setlings for compurative analysis with different models.
18

---

Published as a conference paper at ICLR 2026
C.2 OTHER EXPERIMENTAL DETAILS
Experimental Details on Human3.6M (lonescu et al. 2013).
In this dataset, we employed three types of 2D pose detection methods to evaluate the performance of our models: GT, CPN (Chen et al. 2018). and SimpleBaseline (Xiao et al. 2018).
Among them, GT and CPN provide only the 2D coordinates of each keypoint, while SimpleBaseline directly outputs heatmaps for each keypoint. To ensure consistency in data formats, we generated heatmaps for the 2D coordinates of GT and CPN using a Gaussian distribution. Conversely, for SimpleBaseline, we extracted the 2D coordinates by identifying the locations of the maximum values in its heatmaps. This preprocessing ensures that GT, CPN, and SimpleBaseline data all have two forms: 2D coordinates and heatmaps.
Regarding model inputs, KTP-Former (Peng et al., 2024) and GLA-GCN (Yu et al. 2023b) require 2D coordinates, while Adafuse and Dense Warper use heatmaps as input. In contrast, the PPT model adopts an end-to-end architecture, taking only the raw images as input without relying on 2D pose detection results.
This setup allows comprehensive evaluation of the models under different input formats.
Experimental Details on MPI-INF-3DHP (Mehta et al. 2017).
The processing method for this dataset consistent with that of the Human3.6k dataset. We lected the same 17 keypoints corresponding to those in Human3.66 66M and used four camera views, specifically views 0, 2, 7, and
For this dataset, we employed the SimpleBaseline method for 2D pose detection, using model trained in-house. SimpleRaseline generates heatmaps for each keypoint. facilitate further pro- cessing, extracted the 2D coordinates by identifying the locations of the maximum values in the heatmaps, thereby obtaining both 2D coordinates and heatmaps as input formats.
In terms of model inputs, KTP-Former and GLA-GCN take 2D coordinates input, while Adafuse and Dense Warper require heatmaps. The PPT (Ma et al.] 2022) model, contrast, employs an end-to-end architecture that uses only images as input without relying any 2D pose detection results.
To ensure fair comparison, all models were trained and tested using results derived from Simple- Baseline.
D EPIPOLAR GEOMETRY
In the main text, we introduced the fundamental principles of epipolar geometry and the line-of-sight methods that employ epipolar geometry for multi-view fusion. Here, we expand upon these topics and provide a more detailed exposition.
MATHEMATICAL FOUNDATION OF EPIPOLAR GEOMETRY
Epipolar geometry defines the geometric relationships between two camera views, essential for multi-view feature matching and reconstruction. Using the pinhole camera model, a 3D point X Y,YY Z, 1J" is projected onto the image plane as 2D point 1j²²,
(12) where P - K[R t is the 3 x projection matrix. The intrinsic matrix K defines focal lengths and principal points, while the extrinsic parameters [R| |t] describe the camera's 's rotation and translation in the world coordinate system.
PX,
Given two cameras with projection matrices and P', a 3D point is projected onto the two images q and q':
q= PX, q' = P'X.
(13)
Eliminating X leads to the epipolar constraint:
Eq 0,
(14)
19

---

Published as a conference paper at ICLR 2026
where F is the 3 3 fundamental matrix, encapsulating the geometric relationship between uncali- brated cameras. If intrinsic parameters are known, the essential matrix E can be derived:
R,
E = K" FK
(15)
Here, [t]x represents the skew-symmetric matrix of the translation vector t.
The epipolar geometry also defines epipoles and epipolar lines. The epipoles, e and e', are the projections of one camera's optical center onto the other's image plane, satisfying Fe 0 and F e' = 0. For point in the first image, the corresponding point q' in the second must lie on the epipolar line:
T=FF. 1= F= F
(16)
This reduces the search space for matching points from 2D to ID, significantly improving efficiency.
II EPIPOLAR GEOMETRY FOR MULTI-VIEW FUSION
Epipolar geometry crucial in multi-view fusion for feature matching, 3D reconstruction, and op- timization:
The epipolar constraint reduces the matching search space to epipolar lines, filtering mismatches by validating q' Fq O.
1. Feature Matching:
Triangulation: Matched points are used to estimate the 3D position X via linear triangulation:
q x (PX) = q x (P'X) = 0.
(17)
To refine accuracy, non-linear optimization minimizes the reprojection error:
min Σ 11 Tu (X)]I?
(18)
where (X) projects X onto the k-th camera.
Optimization: Incorporating the epipolar constraint into the optimization objective ensures geo- metric consistency:
min
||qx TA (X)I + + Σ F.rq))
(19)
&
This enhances robustness against noise and improves multi-view consistency.
Epipolar geometry provides the theoretical foundation for effective feature alignment, matching, and 3D reconstruction, enabling precise multi-view fusion in computer vision applications.
Reprojection Distance. In multi-view matching tasks, the lines V and V2q' may not intersect precisely at the 3D point Q. To obtain the optimal estimate of Q, we define the reprojection distance
dRepooj as the minimum sum of squared distances from the estimated 3D human keypoint Q to the projections and q': Roproo min (d PQ) + dd? (ol (q'.
(20)
where d(-) denotes the Euclidean distance, and dReproj represents the reprojection error between and q'.
Sampson Distance. To simplify the calculation, we adopt an approximation of the reprojection distance known the Sampson distance, defined as:
Fq (Fq) + (Fq)3 + (FT q0) + (FT
dsampson
(21)
where is the fundamental matrix, and subscripts and 2 refer to the first and second elements of vector, respectively. Sampson distance allows to measure the geometric error between two points without explicitly solving for the intermediate 3DD point Q.
20

---

Published as a conference paper at ICLR 2026
View
View
View 3
View
Frame Interval:
(b) Frame Interval: 6
(c) Frame Interval:
Figure 5: Results of spatiotemporal heatmap fusion and correction using different frame intervals on the Human3 6M dataset. The camera sampling interval the Human3. 6M dotasct 50ms. Panels (a), (b), and (c) represent the results spatial beatmap fusion frame intervals of frame. frames, and 12 frames, respectively.
Input
Ground Truth
$
f
ff
$
f
f
f
$
mf
Ours
Figure 6: Visualization results showing the effects during continuous motion.
Input
Ground Truth
$
D
F
D
$
K
$
Ours
Figure Visualization demonstrating effects during complex
21

---

Published as a conference paper at ICLR 2026
Camera Sampling Interval
Camera Sampling Interval M
View 1
View 2
View 3
View 4
Sparse Input
Dense Output
Figure 8: Illustraioon of framne rate enhancement through interleaved multi-view input. The camera frame rate f = / where At represents the camera sampling interval. With fixed frome rate, the input frame be effectively increased using interieaved multi-view inputs, reaching to Md x f. where M denotes the number of camera viewpoints.
E SUPPLEMENTARY EXPERIMENTAL RESULTS
In the experiments, we further investigated the relationship between our epipolar geometry-based spatial heatmap fusion module and the temporal frames of the cameras. Theoretically, as the tem- poral frame interval increases (i.e., the camera sampling frequency decreases), the displacement of the heatmap points to be calibrated becomes larger, making heatmap fusion more challenging. As shown in Figure 5) the displacement of heatmaps from different views also increases under these conditions. In our experiments, the camera frame rate set to 50 fps, which enabled effective heatmap fusion. This demonstrates that our method performs well under standard camera sampling conditions.
Additionally, we visualized the results of our method by comparing the 3D skeletons estimated from sparse interleaved inputs with the ground truth. The experimental results demonstrate that our method achieves accurate 3D skeleton estimation, effectively leveraging both the temporal and spatial information embedded in the sparse interleaved inputs. This validates the capability of our approach to transform sparse inputs into dense outputs. Tigures[6land [ respectively izations of 3D skeletons for continuous actions and complex, challenging actions.
present visual-
E. SPARSE INPUT FOR UPSAMPLING
As shown in Figure when the staggered intervals of cameras with different viewpoints control- lable, we can achieve data upsampling on the coefficient input mode, and this method is applicable all 3D multi-view tasks.
E.2 QUANTITATIVE ANALYSIS OF DIFFERENT SAMPLING INTERVALS
To investigate how the sampling interval affects performance, conducted a quantitative analysis, with the results presented in Table 8 It can be observed that with moderate time interval (c.g., when Frame Interval = 6), the input still has some degree density, and the interleaved viewpoints are able to provide effective spatial correction. As result, the model performance does not de- grade significantly. However, when the time interval large (i.e., when the camera sampling rate is low), the data density decreases, leading to sparsity. In such cases, the sparse interleaved sampling paradigm becomes ineffective, and no effective correction is provided between different viewpoints. Furthermore, due to the large time interval, the performance of the temporal fusion module is also challenged. This experimental result suggests that our method is suitable for high frame rate, dense input scenarios, which aligns with the motivation of our paper.
22

---

Published as a conference paper at ICLR 2026
E.3 MODEL PERFORMANCE UNDER
INTERVAL SCENARIOS
To address the concern regarding potential non-uniform intervals between different viewpoints, we systematically sampled from the existing dataset simulate input scenarios with irregular temporal spacing.
Unlike the original uniform interval input (with window size of Window Size 4), no longer select input frames from a sequential window of size 4. Instead, we set the sampling window size be greater than 4 (i.e., viewpoints appearing at the same time frame. Specifically, we randomly and non-uniformly select time window size of x. This successfully simulates the case
4) ensure that all four viewpoints are selected, with no two
four viewpoints from sequence where the time intervals between input viewpoints are non-uniform. We performed experiments with x 6, 10, 12, and the experimental results are shown in Table
The experimental results indicate that non-uniform intervals have a certain impact on model perfor- mance. As the maximum non-uniform interval increases, the model performance degrades more, which confirms our hypothesis that as the interval time increases, the temporal fusion module of the model will be challenged. This is due to the randomness caused by the non-uniform interval time paradigm. However, overall, even with larger maximum non-uniform intervals, model still achieves relatively good results.
Frame Interval
Frame Interval
Frame Interval
Frame Interval 12
Sampling Interval (ms) Performance (MPJPE, mm)
39,92
84
Table Quantitative Analysis of Different Sampling Intervals.
Window Size
10
12
Maximum Non Uniform Interval (ms) Performance (MPIPE, mm)
20 (Uniform Interval)
40
120
160
22.3
26.77
29.99
31.58
Table 9: Model Performance under Non Uniform Interval Scenarios, In the original paper, the input interval is uniform and 20ms..
23


Reviews:
[{"review_id": "DGmVMrK95O", "reviewer": "Reviewer_9N3P", "summary": "The paper introduces a novel input design for multi-view, temporal 3D human pose estimation, called sparse interleaved input, which reduces the computational cost compared to methods that use all frames across all camera views. The proposed method uses a diagonal approach to select the input frames (e.g., view 1 at frame 1, view 2 at frame 2, view 3 at frame 3, view 4 at frame 4) to predict the 3D pose of the sequence (3D pose at frame 1, 2, 3, and 4). The paper argues that doing so theoretically can lead to an increased frame rate of N, for N cameras in a system. To process this input, the paper introduces a model called DenseWarper, which first replicates the selected frames across all time-steps to create a dense input space. Then, it uses epipolar geometry to fuse 2D heatmaps across different views. Next, several parallel temporal fusion networks aggregate the information across time and output the 3D human pose. By evaluating the proposed method on two popular benchmarks, the paper shows the effectiveness of the method.", "questions": "1. I would appreciate if the authors can address the points I raised in the Weaknesses section (e.g., missing references & results discrepancies) \n2. The spatial fusion described in the paper bears a strong resemblance to the method in AdaFuse. Could you please clarify what the methodological novelty of the proposed approach is compared to AdaFuse?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The sparse interleaved input idea is a simple, yet effective and novel way to reduce the computational load of multi-view, temporal human pose estimation. The method is somewhat counterintuitive (e.g., using less information leads to better performance), but experimental results support its effectiveness.\n2. The computational cost of multi-view, temporal models is an ongoing problem in the pose estimation research, which this paper has addressed.\n3. The proposed architecture is sound, and the presentation is clear. The paper is also well-written and uses clear terms to convey its points.", "weaknesses": "1. The paper reports state-of-the-art performance, but the conclusion is based on a comparison with baseline results that have been replicated from the original works. As a result, the majority of the results in the tables do not match the original works. While this is understandable for methods that have been replicated, it contradicts the results that use the original model weights (e.g., AdaFuse). What is the reason for this discrepancy? Was a different evaluation protocol used in this paper?\n2. I assume that camera parameters have been used in this paper. In that case, some crucial and highly cited references (e.g., [1] and [2]) are missing from the results.\n3. While the core idea is interesting, the paper does not position its performance within the existing literature. \n\nReferences:\n1. Iskakov, Karim, et al. \"Learnable triangulation of human pose.\" Proceedings of the IEEE/CVF international conference on computer vision. 2019.\n2. Remelli, Edoardo, et al. \"Lightweight multi-view 3D pose estimation through camera-disentangled representation.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "X8Pd6Fm439", "reviewer": "Reviewer_Ufuf", "summary": "This paper tackles an interesting and practical problem, proposing a solution that is logically sound and clearly explained. The experimental results demonstrate the method's effectiveness and advantages.", "questions": "1.  **Sampling Interval ($\\delta$) Analysis:** Could you provide a detailed quantitative analysis of how the sampling interval $\\delta$ affects performance, based on the heatmap in Appendix Figure 5? What are the specific quantitative results?\n2.  **Handling Non-Uniform Intervals:** In practice, the intervals between different views might be non-uniform. Is the algorithm designed to adapt to this? How does its performance hold under non-uniform sampling, or what modifications would be necessary to handle it effectively?\n3.  **Clarification on Reporting Metrics:** Could you please also report the absolute model size alongside the performance efficiency metrics to provide a complete picture of the model's complexity?\n4.  **Writing and Coherence Clarification:**\n    *   Can the authors clarify the abrupt phrasing in lines 439 and 447, and ensure methodological names are cited consistently throughout the text?\n    *   Could the authors confirm the provenance and carefully review the \"Model Efficiency Analysis\" section for coherence and accuracy?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "*   **Problem Significance:** The problem addressed in this paper is very interesting and corresponds to a genuine practical need.\n*   **Methodology:** The proposed solution is overall logically reasonable and clearly articulated.\n*   **Experimental Validation:** The experimental results substantiate the method's effectiveness and advantages.", "weaknesses": "*   **Insufficient Experimental Details and Analysis:**\n    *   The impact of the sampling interval $\\delta$ on performance within the defined scenario is not thoroughly analyzed. Although a heatmap is mentioned in Appendix Figure 5, corresponding experimental results, particularly quantitative findings, are missing.\n*   **Inadequate Citations:**\n    *   The methodology sections (Sections 2 and 3) contain very few references. References should also be added to the main text of the experimental section to help readers understand related work.\n*   **Formatting and Presentation Issues:**\n    *   **Layout:** The placement of several tables does not correspond well with the relevant textual discussions, hindering readability and understanding.\n    *   **Table Formatting:** Specific formatting issues exist: Table 3 is missing a bottom horizontal line, and Table 4 has an extra vertical line on the far right.\n    *   **Nomenclature Consistency:** The notation for the heatmap (`H` or possibly `**H**`) is not used consistently throughout the text. The variable name `rH` is also somewhat unconventional.\n    *   **Writing Quality Suspicions:** Specific lines, such as \"the input fps f\" (line 439) appearing abruptly, inconsistent citation of method names within parentheses, and the phrase \"As shown in Table 5.\" (line 447) seeming erroneous or out of context, raise concerns. Based on my experience, the entire \"Model Efficiency Analysis\" section reads as if it might have been generated by an LLM, lacking the flow of human-written academic prose.\n*   **Potential Obfuscation in Reporting:**\n    *   Only performance efficiency (e.g., MPJPE/mm per MB) is reported, omitting the absolute model size. This gives the impression of potentially skewing the complexity presentation. Reporting the absolute model size is recommended for clearer understanding of the actual complexity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vmJK4PBcMm", "reviewer": "Reviewer_ieKm", "summary": "This paper introduces a novel input paradigm for 3D human pose estimation: sparse interleaved multi-view input.", "questions": "See Weaknesses.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "The \"sparse interleaving paradigm\" sounds okay.", "weaknesses": "1. I don’t quite understand the significance of this work. Is there still a need to study multi-view markerless human motion capture? In the past four years, there have been many works that directly use the results of multi-view markerless human motion capture as GT. Just like EasyMocap[1]. It can produce the 3D skeleton mentioned in this article, as well as the SMPL human skin template and very high-quality visualization. There is no research significance in this field anymore. Even if this work proposing DenseWarper is published, I don’t think I will use this algorithm. Just like the results in Tables 1 and 2, when MPJPE reaches 20 to 30, I think the visualization effects of these methods are no different to the naked eye.\n\n2. Continuing from the previous point. From the perspective of ICLR academic papers, the experiments also seem to lack a lot of content. First, in terms of visualization. The most important thing for a human pose estimation paper is the visualization presentation, but after reading the entire article, only Figures 6 and 7 have comparisons between the current method and GT. As a reviewer, I am very concerned about qualitative comparative experiments with other methods. But I am disappointed that there is no such content.\n\n3. Human3.6M and MPI-INF-3DHP are really too old. I think there is no need to study the datasets from more than ten years ago. A few years ago, the MPJPE of Human3.6M was already in the twenties[2]. After five years, it has dropped to single digits. Is it still necessary to conduct such experiments?\n\n-----\n\n[1] Dong J, Fang Q, Jiang W, et al. Fast and robust multi-person 3d pose estimation and tracking from multiple views[J]. IEEE transactions on pattern analysis and machine intelligence, 2021, 44(10): 6981-6992.\n\n[2] Zhang Z, Wang C, Qiu W, et al. Adafuse: Adaptive multiview fusion for accurate human pose estimation in the wild[J]. International Journal of Computer Vision, 2021, 129(3): 703-718.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aXLn76StC9", "reviewer": "Reviewer_FTR4", "summary": "The authors propose a novel 3d human pose estimation framework which leverages images captured from different camera views at various time points to capture rich spatio-temporal information and effectively boost performance. Their approach theorectically increase the output post frame rate by N times with N cameras and enhance the temperal resolution of the production. In addition, using a spare subset of available frames, their method can reduce data redundancy while simultaneously achieve better performance.\nThey also introduce DenseWarper model which leverages epipolor geometry for efficient spatio-temporal heatmap exchange. Extensive evaluations using Human3.6M and MPI-INF-3DHP show that their method outperforms SOTA methods.\nHowever, there are certain parts that need some clarifications and authors should discuss how their method can be extended to analyze videos involving multi-persons.", "questions": "•\tMotivate why CPN is used as 2D detector. It seems to perform poorer than simplebaselines\n•\tDiscuss how your method can be extended to handle multi-person videos.\n* Perhaps run one set of experiments using multi-person activity dataset such as CMU Panoptic to find out how your method performs compared to existing Multiview 3D pose estimation methods such as MV-SSM: Multi-View State Space Modeling for 3D Human Pose Estimation, a CVPR 2025 paper.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "•\tThey are the first that propose 3d pose estimation task based on sparse interleaved multi-view input\n•\tThey design DenseWarper to convert sparse interleaved inputs into dense pose outputs with high spatio-temporal consistency.\n•\tThey conduct rigorous experiments to demonstrate that their proposed technique achieves better performance.", "weaknesses": "•\tThere are some parts in current writeup that need clarification When the Spatial Fusion module (Eq. 8) runs on the expanded set H at n_th frame,  it is fusing the real heatmap H_n at view1 with the replicated heatmap H'_n at view 2(which is actually the heatmap from n+∆). This means the module is still applying epipolar constraints to heatmaps of a person at two different frames (n and n+∆). The epipolar geometry is invalid for a moving object at different times. It seems like this paper implicitly assumes that this step can still refine approximate feature correspondences, even if they are not geometrically perfect.\n•\tAuthors have only evaluated their method using datasets involving single person performing an action in each video. They should comment on how to extend their method to videos involving multiple persons. In this case, simply doing spatial fusion using their method will not work. There may be multiple heatmaps (one for each person) and if the people are close by, it may be hard to differentiate how to fuse nearby points correctly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)