Sparkle: A Robust and Versatile Representation for Point Cloud-based Human Motion Capture
Authors: Yiming Ren
Yujing Sun
Aoru Xue
Kwok-Yan Lam
Yuexin Ma
Venue: ICLR 2026 Poster (2026)
Keywords: Motion Capture
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Point cloud-based motion capture leverages rich spatial geometry and privacy-preserving sensing, but learning robust representations from noisy, unstructured point clouds remains challenging. Existing approaches face a struggle trade-off between point-based methods (geometrically detailed but noisy) and skeleton-based ones (robust but oversimplified). We address the fundamental challenge: how to construct an effective representation for human motion capture that can balance expressiveness and robustness.
In this paper, we propose Sparkle, a structured representation unifying skeletal joints and surface anchors with explicit kinematic-geometric factorization. Our framework, SparkleMotion, learns this representation through hierarchical modules embedding geometric continuity and kinematic constraints. By explicitly disentangling internal kinematic structure from external surface geometry, SparkleMotion achieves state-of-the-art performance not only in accuracy but crucially in robustness and generalization under severe domain shifts, noise, and occlusion. Extensive experiments demonstrate our superiority across diverse sensor types and challenging real-world scenarios.

Published as a conference paper at ICLR 2026
SPARKLE: A ROBUST AND VERSATILE REPRESENTA- TION FOR POINT CLOUD BASED
HUMAN MOTION CAPTURE
Yiming Ren 2:; Yujing Sun"; Aoru Xue', Kwok-Yan Lam"! Yuexin Ma-
1 "ShanghaiTech University, China
Digital Trust Centre, Nanyang Technological University, Singapore (renym2022, mayuexinnashagghaaee.. tec.. edu.cn
SparkleMation
(b) Multi-person(-11] Football Durk
(e) Evaluation SOTA datasets(L)
Point Cloud
Sparkle
Result
SparkleMootion(Ourr)
LiveHPS++
PointIPS
FreeCap
Saaise
Joints
LiveHPS
LiDARCap
Point Cloud
VotelIMR
SlopenlD
NotsaMootio
Prediation vat
OnA-Htmas
Occlusion
Results
Real Scene
(e) Heterogeneous Point Cloud Distributions(d)
Interaction of Basketball
Free Mettenn
Ch30 Interumnn sinde pnreina
iPhone
Point
Cloud
S as 25.2557. 25. o
Point Cloud
Results
Figure 1. We introduce (a) SparkleMotion, point- real-time motion capture solution based on novel human representation structure, Sparkle, benefiting both internal kinematics from skeletal joints and external geometry from surface anchors. is capable of handling challenging scenarios in the real world, such as (b) multi-penn culliippersnn foothall in dark, (c) heterogeneous point cloud distributions, and (d) close human interaction basketball. The raw point cloud input and our real-time results are illustrated in the corresponding scenarios. The real-time video is included in the supplementary materials. SparkleMotion's 's superiority to SOTA approaches and strong generalization across datasets showcased in (e), where methods don't work has area size 0, measured in Mesh Vertex Er)on(
ABSTRACT
Point cloud-based motion capture leverages rich spatial geometry and privacy- preserving sensing, but learning robust representations from noisy, unstructured point clouds remains challenging. Existing approaches face struggle trade-off between point-based methods (geometrically detailed but noisy) and skeleton- based ones (robust but oversimplified). We address the fundamental challenge: how to construct an effective representation for human motion capture that can balance expressiveness and robustness. In this paper. we propose Sparkle, a structured representation unifying skeletal joints joints and surface anchors with explicit invemattc-geomette factorization. Our framework, SparkleMotion. learns this representation through hierarchical modules embedding geometric continuity and kinematic constraints. By explicitly disentangling internal kinematic structure from external surface geometry, SparkleMotion achieves state-of-the-art perfor- mance not only accuracy but crucially in robustness and generalization under severe domain shifts, noise, and occlusion. Extensive experiments demonstrate our superiority across diverse sensor types and challenging weal-world scenarios.
"qqual contribution. Correspondence authors.

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Human motion capture (MoCap) (Horberry ct al. 2022 Ye et al. 2022 Xu ct al. 2020 2022 Jiang
ct al. 2023 Kaufmaan al., 2021 Yi al. [022 Charles al. 2016 Belagiannis & Zisserman 2017 Kim et al.) 2019 Yi et al. 2021 Andriluka et al. 2018) has emerged as fundamental tech- nology for wide range of namman-centr applications, encompassing sports performance analysis, healthcare, virtual reality, numan-robot interaction, etc. Although mainstream research on MoCap based on a wide range of data modalities, point cloud based motion capture using LiDAR (Ren er al. 2024b Dai et al. (022) or depth cameras (Cai et al.) (023) stands out by offering distinct advantages, notably accurate depth perception and stronger guarantees of data privacy, outweighing wearable sensor-based based (Yi et al. 2021 (2022) and camera-based (Shen et al. [2024] [Shin et al. (024) approaches. Furthermore, the rich geometric and spatial information inherent in point clouds enables accurate 3D localization and robust motion reconstruction, even under unconstrained conditions.
However, an everlasting challenge in point bosud MoCap is what intermediate representation should learn from unstructured, noisy, and often incomplete 3D point clouds. Existing approaches learn either (1) direct point-based representations (Qi et al. 2017: Vu et al.] 2022), which retain geometric detail but lack structural priors, making them susceptible to noise and occlusion, or (2)
skeleton-based abstractions (Ren et al.) 2024b a). which provide strong kinematic structural priors but discard the surface-level details essential for resolving rotational ambiguities and capturing nu- anced shape, both categories struggling to achieve expressiveness and robustness simultaneously.
A powerful representation should tightly couple the robust structural priors of internal kinematics, which are resistant to noise and occlusion, with external geometry to preserve ane-grained details. However, simple combination of existing skeletal and surface representations is insufficient, as each component itself suffers from inherent limitations: Skeleton-only methods lack surface con- straints for recovering fine-grained details, while point-only methods are unstable under sparse or noisy observations. To this end, we propose a unified intermediate representation, Sparkle, that in- tegrates internal kinematics with external surface geometry, thereby preserving skeletal coherence (robustness) while maintaining surface fidelity (expressiveness) across varying point cloud quali- ties and diverse scenarios. Crucially, the novelty of Sparkle lies not only in their union but also in the redesign of each constituent: (1) For the skeletal component, we introduce point-aligned es- timation process that explicitly models the spatial correspondence between the point cloud and the skeleton, moving beyond the common practice of direct regression, which often misaligns with the observed geometry. (2) For the surface component, we propose a set of semantically consistent an- chors that are dynamically refined through dxeleton-guided geometry-aware mechanism, offering a more stable and informative representation than unstructured points. This dual-novelty design en- sures that Sparkle is a fundamentally more expressive and robust representation, rather than a trivial assemblage of existing ones.
To be more specific, building on Sparkle, we further propose SparkleMotion, robust and effec- tive point based human motion capture approach that advances both representation learning and practical MoCap performance. Firstly, we construct the Sparkle Representation, which is the unification of an internal kinematics representation and an external geometry representation. Inter- nal Kinematics Representation consists of a set of accurately estimated skeletal joints. To obtain this, we design Toint-aligned Skeleton Tracker that learns predict precise skeletal joints and global body translation by explicitly modeling spatial correspondences between point cloud sur- faces and the target skeleton structure, while leveraging both local and global features through segmentation-aware design. Surface Geometric Representation, in contrast, consists of a set of precise surface anchors predicted by Skeleton Skeleton-guided guided Anchor Estimator. This estimator learns to fuse the structural prior with the observed geometric evidence, predicting surface anchors based on predicted skeletal joints and learning a non-linear correction to linear initial guess. To be clear, we first initialize the anchors by leveraging the structural relationships between internal joints and ex- ternal surface points. Then, by integrating both skeletal and surface features, the estimator enhances the stability and robustness of anchor predictions. As shown in Figure D Sparkle unifies skeletal joints and surface anchors and injects structured priors to guide learning from point clouds. Then design Sparkle-based SMPL Solver. The Solver decomposes axis-angt poses into swing-twist components (Li et 1022 analytically by aligning each template bone to the observed joint vector and calculating the in-bone twist via anchor alignment. This initialization exploits Sparkle's fac-
2

---

Published as a conference paper at ICLR 2026
torized kinematic-suffe surface structure, after which human pose and shape can be conveniently refined with an optimization network.
Extensive experiments across range of 11 challenging MoCap benchmarks demonstrate that SparkleMotion consistently outperform State-Of-The-Art point- MoCap methods in terms of accuracy and robustness, even under multi- multi-person large-scale scene, cross-sensor domain shifts, and occlusion and noise by close human interaction as shown in Figure Furthermore, SparkleMotion generalizes effectively to data from various LiDARs and depth cameras, like iPhones, well as multi-view capture setups, substantiating its utility as scalable representation for real-world de- ployments. Leveraging our pipeline, we also develop real-time MoCap system capable of stable and high-quality 3D human reconstruction in- in-the-wild, paving the way for broad-scale, privacy- aware, and interpretable human motion understanding. We summarize our contributions as follows:
We propose Sparkle, disentangled structured representation that unifies skeletal joints and surface anchors, explicitly designed to provide stronger structural prior for learning from point clouds, achieving promising balance between expressiveness and robustness.
We develop SparkleMotion, a robust and generalizable MoCap framework that hamnesses Sparkle representation for accurate, noise-resilient, and real-time motion capture.
We establish new Ofate-Of-The-Are performance on multiple public benchmarks, demon- strating SparkleMotion's strengths in accuracy, robustness, cross-domain generalization, and scalability for real-world applications through comprehensive evaluations.
2 LITERATURE REVIEW
2.1 POINT CLOUD-BASED MOTION CAPTURE
Human motion capture has been widely explored using wearable sensors and vision-basee meth-
ods. Wearable MoCap systems, such as inertial measurement unit-based (Ren et al.
2023 Yi et al.
2021 Huang al., 2018a) and marker-based approaches {Opti'Track] Loper et al.
2014 Xu al.
2019
offer high-quality motion reconstruction but require specialized hardware, limiting their flex-
ibility and ease of use. To achieve more non-intrusive solution, camera-based methods including
monocular Mool et al., 2014 Kanazawa et al.) 2018 Alldieck et al.] 2017: Shin et al. 2024
Wang et al. 2024] Sun al. 2023) multi-view (Dou al. [2077] Huang al.
2018b Malleson
et al. 2019). and RGB-D sensors (Zimmermann et al. 2018, Dang et al., 2023).
estimate human
motion from images or videos, leveraging deep learning to infer 3D human poses from visual cues.
However, these methods are limited to light conditions, global localization, and privacy concerns.
Point cloud-based MoCap methods mainly leverage LiDAR and depth cameras to reconstruct human motion without relying wearable markers or identity tracking. These approaches are particularly advantageous for privacy-sensitive applications and challenging environments. However, due to the distinct characteristies of LiDAR and depth camera point clouds, existing methods are often tai- lored to a specific sensor type, limiting their generalizability. LiDAR provides long-range depth perception but produces uneven and unstructured point distributions. LiDARRbased MoCap meth- ods (Ren et al. 2024b) 2023 2024a) are specifically designed for large-scale, sparse, and noisy LiDAR point clouds, where the data lacks rich geometric details. They extract skeletal structures from the limited point cloud input and focus primarily on optimizing the skeleton joints. Thus, when faced with the dense and detailed point clouds provided by depth cameras, they struggle to fully uti- lize the abundant geometric information, leading to suboptimal motion reconstruction. Conversely, depth camera- based MoCap (Cai et al., 2023) Liu et al. 202T) operates on dense, high-resolution point clouds, making it well-suited for indoor environments and detailed motion estimation. Depth sensors provide rich point clouds but suffer from limited range, occlusions, and noise. Methods designed for depth cameras focus on point cloud features to refine body part estimation but often lack robustness in outdoor settings or under sensor noise, making them ineffective when applied to sparse and irregular LiDAR data.
2.2 HUMAN MOTION REPRESENTATION
Traditional marker-based motion capture (Raskar et al., 2007; Loper et al. 2014) provides highly accurate, physically interpretable joint and marker trajectories, but requires elaborate hardware setup
3

---

Published as a conference paper at ICLR 2026
Ocolusion
Point-aligned Skeleton Tracker Skeleton-guidde Anchor Estimator
Sparkle-based SMPL Solver
Sequential Point Clouds &
Coneeuaninne Mar Thaining
Sporkie
Templar
PointNet-QRU
Jane
Object Noise
Environment Nolse
minimmior 11Mazad
Ag
L0 A
ge Nent
Diverse Point Pattors
Jay Mas
Ana
spartle
Ppace
Mooo
Pasee
Pa
Depth
Foot
Challange Scenaries
Diverse Setup
Peature Interaction
Viewa
Polnt-allgned Optimization
Close Interacton
View
Vieua
F
DJ,
AJ.
T
1g
Elematiic Seart
Feature Interaction
The pipeline of SparkleMotion. It can take point clouds of diverse patterns as input in different challenge scenarios, shown on the left. SparkleMotion consists of three primary modules, the foint-aligned Skeleton Tracker, and sseleton-guider Anchor Estimator construct the Sparkle Representation, and the Sparkle-based SMPL Solver for motion reconstruction.
Figure 2:
and calibration, restricting its usability controlled environments. IMU-based systems (Ren al. 2023 Yi et al., 2021; Huang et al., 2018a) enable more flexible deployment by reconstructing in- ternal joint positions from sparse sensor measurements. However, these approaches are sensitive to drift, misplacement, and rely on precise joint-to-sensor mappings. Vision-based algorithms have made sigmificant strides by regressing 2D 3D keypoints from single or multi-view images (Shin et al. 2024) Shen et al.) 2024), sometimes utilizing virtual markers (Ma et al., (023a) to encour- age structural consistency. Despite impressive progress, monocular methods remain fundamentally limited by depth ambiguities and occlusion, and errors in 2D detection may cascade through the 3D reconstruction pipeline. Within the domain of point cloud-based MoCap, two primary repre- sentation strategies exist. Some methods operate directly on raw, unstructured point sets (Qi et al., 2017 Fan et al. 2023), which preserves geometric detail but often suffers from noise, occlusion, and lack of spatial regularity. Others extract skeleton-basee abstractions (Ren et al., 2024b Xue ltt al. 2025), focusing on stable joint estimation, but sacrificing detailed surface information cru- cial for full-body mesh recovery and subtle pose nuances. These observations highlight key gap in current approaches: existing intermediate representations either prioritize structural consistency or surface fidelity. but rarely both. Addressing this, our work proposes novel unified represen- Shhion-Sparkl-tthiee jointly encodes internal skeletal joints and external surface keypoints. This structured formulation bridges the gap between kinetic stability and geometric richness, enabling more robust, generalizable, and expressive motion capture from diverse point cloud data.
METHODOLOGY
The rich and unconstrained 3D spatial information offered by point clouds makes point cloud-based human motion capture highly attractive for real-world applications. Nonetheless, their unstructured, sparse, and often noisy nature poses sigmificant challenges for learning consistent and robust human representations. To address this, we introduce novel structured human representation, Sparkle, explicitly designed to bridge the gap between internal kinematic structure and external surface ge- ometry, maintaining robustness and accuracy in diverse and challenging scenarios. As illustrated in Figure 2 Sparkle (Section 3. comprises an internal and an external representations: (1) Internal Kinematics representations, estimated by Point-aligned Skeleton Tracker (PST) and (2) External Geometry Representation, optimized via Skeleton-Guidee present Sparklemotion, a Sparkle-based SMPL Solver (SSS) (Section[3.2) that predicts human shape and pose by regressing precise SMPL parameters from the unified Sparkle representation through an efficient geometric initialization and iterative refinement strategy. Please refer to Appendix for details model architecture.
Anchor Estimator (SAE). Finally,
Problem Statement Let P. E 256x3 denote the input point cloud at frame t after the farthest point sampling (Qi et al. 2017) and normalization. Notice that for the sake of clarity, omit
4

---

Published as a conference paper at ICLR 2026
the symbol for time t below, but input and output all time series. Our goal is to estimate SMPL (Loper et al. 2015) parameters that define the predicted human mesh E R R including shape B E IR pose RT (in axis-angl angle form), and global translation T RR. Notice that
extract 24 skeletal joints J E R21x3 and 32 surface anchors E R32x3 from V. And each joint rotation be decomposed into swing R sw and twist R components R R R
3.1 SPARKLE REPRESENTATION
The design of the Sparkle representation motivated by the observation that internal skeletal joints provide a robust structural prior but form an incomplete description of human pose, as they lack the surface details required resolve rotational ambiguities. In contrast, external surface points preserve rich geometric information but are inherently vulnerable to noise and occlusion due to the absence of structural priors. reconcile these complementary properties, we introduce Sparkle, a unified representation that combines an Internal Kinematics Representation 3.1. 3.1.Twith Surface Ge- ometric Representation 3.1.2 thereby enhancing both expressiveness and robustness. In general, the Sparkle representation defined as S = Op' Aople which explicitly unifies 24 optimized skeletal joints Jop the Internal Kinematics Representation with 32 optimized anchors Aop the Sur- face Geometric Representation. Together, these components constitute comprehensive encoding of human pose, as elaborated in the following.
3.1.1 INTERNAL KINEMATICS REPRESENTATION
The Internal Kinematics Representation defined as a set of optimized skeletal joints. However, deriving consistent skeletal structure from highly variable point clouds remains challenging. To address this, we propose the Point-aligned Skeleton Tracker (PST), which establishes explicit, learn- able spatial correspondences between raw point clouds and skeletal joints through segmentation- aware design, enabling robust joint estimation across diverse and noisy observations.
Given a point cloud P, our PST module first employs PointNet backbone with a bidirectional GRU to predict initial joint positions Jinit and global translation Tinit. It then performs implicit semantic segmentation of into body parts via a part-based decomposition, yielding point-wise labels L, E 1, 24 that associate each point with its corresponding joint. This segmentation establishes critical spatial correspondences that guide subsequent refinement. Subsequently, the skeleton joints are optimized based correspondences between points and joints. To clear, we first center the entire point cloud using the predicted global translation, P Poentered P Tint Then, for each joint we extract the subset of points P3 = {p E Poemered Lp(p) = j} that share label j from the centered point cloud. Finally, we normalize these points relative to their corresponding predicted joint, P, = {p-- (p- Jmm.. P E P,}-
key insight: leveraging the predicted labels to establish
With above operations, the global regression problem is reformulated as set of local refinement tasks. We develop the Point-aligned Optimization, enables the use of a lightweight, shared PointNet to process each local point set P, and get the local point feature Hjintt then use the MLP decoder to predict a residual offset AJ,, Sharing weights across joints compels the network to learn generic refinement function that is invariant to specific body parts, thereby significantly enhancing gener- alization. To handle cases where a joint has too few associated points ()P)) < 3), we enhance the feature by the local neighboring feature. This allows the model to propagate confident predictions to uncertain regions, form of learned spatial reasoning.
After the refinement network predicts residual offsets AJ, for each joint E (0, 1,. 23]. the optimized joint positions and optimized translations (denoted Jop and Top) are computed by adding these residuals to the initial estimates:
Jap = Jini + AJ,, Top = Tin + AJo.
where AJ, represents the predicted residual offset for joint j. enabling precise alignment with the underlying point cloud structure.
Finally, joint positions Jop are further optimized together with global translation, and point-wise classification via loss LesT
LPST (J >2Ler 1ijg) + AgLMsEETT Tg))
5

---

Published as a conference paper at ICLR 2026
where LMSE is the Mean Squared Error loss, and Les is the cross-entropy loss for label classification, with A1 1.0, A2 = 0.5, and A3 1.0.
3.1.2 SURFACE GEOMETRIC REPRESENTATION
The surface geometry is represented by a set of surface anchors. Since directly predicting these an- chors from raw point clouds is challenging due their unstructured nature, we propose the Skeleton- guided Anchor Estimator (SAE). SAE adopts a prediction-vith-iiiiiiizzzi strong initial estimate is first obtained and subsequently refined by incorporating geometric context.
strategy, in which a
Structural Initialization. We derive a set of 32 canonical anchors Agt E R-2x3 from the SMPL mesh vertices via PCA (Ma et al., 2023b) maximize their representational power. linear map- ping between internal joints and external anchors is precomputed from ground-truth data via Least Squares:
M2A = (A Agu)
Agrgg gt:
gt
which provides a fixed, geometry-agnostie prior Ainit = MIA that roughly places the anchors based on the predicted skeletal pose alone.
Geometry-Aware Refinement. The initial estimate Ami lacks detail and vulnerable to errors in Jop- We refine it by learning non-lincar correction that attends to the actual geometric feature Fjoint- This is achieved through a cross-attention mechanism: joint features Fjoint from PST serve as queries (Q) that represent what expect the local surface geometry to look like given the current pose. In parallel, a point cloud transformer processes the initial anchors Ajnit to generate the anchor feature Fanchor keys (K) and values (V), which represent what the actual local geometry is. The cross-attention operation allows each joint query to selectively attend to and aggregate information from the most relevant geometric features in the point cloud. This enables the model to learn to compensate for both errors in the initial joint estimates and the limitations of the linear model, effectively fusing the structural prior with observed geometric details.
Morcover, instead of predicting explicit confidence scores for each anchor, we leverage the learned anchor-based segmentation La of the point cloud to define anchor reliability. By analyzing the segmentation quality and point distribution around each anchor, we implicitly derive confidence measures that allow the downstream solver to weight the reliability of this learned representation. The entire estimator is trained with a loss that balances anchor accuracy:
LSAB Aa Ag) + As (Lay Lan)
where Lag denotes the ground truth segmentation labels, A 1.0 and As 0.5.
3.1.3 JOINT REPRESENTATION
With the optimized skeleton joints Jop (Section incorporate a final kinematic optimization step Ren et al. 2024a) to enforce temporal consistency, yielding the final Sparkle representation S
3.1.1 and surface anchors Aop (Section 3.1.2p
[ Jop A for the following SMPL solver.
op.
3.2 SPARKLE--AASE SMPL SOLVER
At this point, we obtain the Sparkle representation S, which explicitly factorizes the human motion into two complementary spaces: a kinematic space of joint configurations J'sp and a geometric space of surface deformations A.ep This design incorporates powerful physical inductive bias, signifi- cantly reducing the learning complexity compared to unstructured representations. Each component specializes naturally: joints provide kinematic constraints for robust pose estimation, while anchors encode local surface details essential for resolving shape ambiguitics.
We then propose Sparkle-based SMPL Solver(SSS) to leverage this factorization for efficient es- timation of SMPL parameters of human shape and pose. Given the optimized representation S, recover body parameters (0, ,)) through two-stage process: geometric initialization that ploits the separate constraints of each space, followed by a learned refinement that integrates both information streams for final precision.

---

Published as a conference paper at ICLR 2026
3.2.1 GEOMETRY-DRIVER INITIALIZATION
key advantage of our structured representation is that if enables deterministic initialization through explicit geometric constraints, avoiding the biases inherent in learned initialization schemes.
This parameter-free procedure requires only: (1) bone vector composition J J2 J1: (2) trigono- metric operations on vectors (dot and cross products); and (3) Rodrigues' rotation formula.
For each bone b defined in SMPL, decompose axis-angle rotation into swing-twist compo- nents (Li et al., 2021) using associated joints and anchors (Jtem- J'pp Atems A'p) where Jeem and Atem means the joint positions and anchor positions in the zero pose((-PPsse) of SMPI..
Swing Rotation: The swing component D(a [x aligns the template bone direction Jeem to the predicted direction J'p using only skeletal information:
Jiem x op
Jeem J op
flw
Osw arccos
(4)
tem
x J opll
tem ||| op
Twist Rotation:
The twist component ((ott [Ttw) x)) then rotates around the bone axis Ntw
J opp ap//ll''on /Ill''l opll to align template anchors Atem with predicted anchors opp
Olew arctan 2 (llAtom A' o'opl Agem A oop).
(5)
The complete rotation R = 1 Rtw is computed via Rodrigues' formula, providing physically consistent initial pose estimate Omi without learned parameters. This demonstrates how Sparkle's explicit structure facilitates geometrically meaningful computation.
3.2.2 MARKLE-GUIIDER PARAMETRIC REFINEMENT
The geometrically initialized pose Ami provides strong, physically plausible starting point. How- ever, this deterministic solution has inherent theoretical limitations that affect its stability and uniqueness. First, the swing-twist decomposition is susceptible to kinematic singularities. Specifi- cally, the swing rotation becomes undefined when the template bone vector Jiem and the observed bone vector J are nearly aligned or anti-aligned, as their cross product approaches zero, mak- ing the rotation axis Vww numerically unstable. Second, the solution for the twist component lacks uniqueness under ambiguous geometric evidence. The twist angle aew is estimated from the align- ment of surface anchors, but this becomes ill-posed when anchors are occluded, noisy, lie close to the bone axis (becoming co-linear), leading to multiple valid twist angles that satisfy similar geomet- ric constraints. Consequently, the purely geometric formulation is sensitive noise in the estimated Sparkle representation and may converge to sub-optimal local minima, particularly in the presence of occlusions when the observed point cloud evidence is ambiguous. Furthermore, it cannot fully encapsulate the complex, non-linear correlations between body shape 3, pose 0, and the observed Sparkle features.
To bridge this gap between the analytical initialization and the final, refined mesh, we employ lightweight cross-attention network to learn non-linear correction. In this refinement stage, the initial pose parameters O are encoded to get pose feature Pr as queries. The Sparkle features Esparkte serve as keys and values, representing the observed geometry feature. The cross-attention mechanism thus learns adjust the initial pose by attending to the specific geometric constraints provided by the Sparkle, effectively performing learned, iterative refinement on the plausible poses Oap and shape B. The solver is trained with regression loss:
Lsss > (0p 0g) X7LMSE Bgx
where As = 1.0 and >; = 0.5 balance pose and shape parameter accuracy. This two-stage de- sign, analytical initialization followed by learned refinement, strikes optimal balance between efficiency and accuracy. It serves as the final validation that our learned Sparkle representation is not merely intermediate output, but powerful and efficient code for human pose and shape.
7

---

Published as a conference paper at ICLR 2026
Truth
GTA Ham-Hum Paint Hurlitumm Point
Frevtfation
SlopariD
LiDAR-based Motion Capture
Depth Motion Capture
FreeMotion-M/
LiveHPS++
Ground Truth
FreeCap
Ground Truth!
FreeCap
Ground Truth
Internamonn
View-1
CNND
View
A44
View
Close-interaction
Multi-view Human Motion Capture
Figure 3: Qualitative comparisons. Due to the natural spatial information of point clouds, we simul- tancously displayed both point clouds and global human mesh to reflect the accuracy. For multi-view MoCap, we present point clouds from three perspectives and unique fusion results from multiple per- spectives. The red rounded rectangle indicates where other methods did not work correctly.
4 EXPERIMENT
Implementation Details elaborated in the appendix.
Datasets 11 challenging benchmarks in total: I). datasets captured with LiDAR and depth
cameras involving severe occlusion and noise: FreeMotion(Ren al. 2024b). FreeMotion- Rbn et al.] 2024a). Sloper4D(Dai et al. 2023). Moin et al. 2024a). GTA-Human-
Point (Cai et 2024), and HuMMan-PPint (Cai al. 2022). II. multi-person datasets with
complex close interactions: InterHuman
et al., 2024), Chi3D (Fieraru et al. 2020) and
Hi4D (Yin et al.] (002) where point cloud data unavailable, generate point clouds follow- ing the LIP (Ren et al. (202)). III). 2 multi-view datasets: FreeMotion-MV (Ren et al.
2024b) and
HuMMan-MV (Cai al. (002)).
Baselines: 5 point cloud-based methods, including LiDARCap (i et al. 2022). LiveHPS (Ren al. 2024b), VoteHMR (Liu al.) (002). PointHPS (Cai al. 2023) and LiveHPS++ (Ren al.) 2024a), and multi-view MoCap method, FreeCap (Xue et al.
2025 Under multi-view setting.
Evaluation Metrics We adopt widely used metrics in motion capture to ensure compre-
hensive assessment of human motion accuracy, including local and global joint/vertex errors(I/V (m(()G))(mm 4) and angle errors(Ang Errr(gegre 4).
Method
Free Motion (Ren
Sloper4D)
Dai
FreeMMtion-OBi
20244
NNoisMMttion
20724
J/V EmrL)
EntG)
Ang Em
I/V Em(L)
J/V Exr(G
Ang Exc [/V EmtL)
J/V Emr()
Ang Er
J/V EmrL)
I/V Emt(G
Ang
LiDARCap
86.4/104.3 180.4/188.6
15.51
71.6/84.2
138.7/1/7.88
13.72
(64.1/100.6
181.8/189.3
16.61
52.6/61,7
400,7402.6
10.87
LiveHiS 74.7/90.8
4/144/141.1
1696
53.4/33.2
88.495.9
13.08
70.7/88.4
146.8/158.0
17.81
48.4/50.4
74.7/83.8
12.19
LiveHPS*
61.975.3
1/1204
42.7/50.6
77.081.7
58.172.55
128 326.60/136.94
341V42.8
58.5164.5
Ours
59.0/3.22 105.1/1139
9.66
41.4/50.5
70.977.1
10.48
50.8/62.7
104.1/110.5
8.49
27.8/36.3
38.8/45.8
7.57
Table 1: Evaluations on general scenarios with noisy and occlusion in 4 datasets. Notice our
substantial improvement noisy datasets, FreeMotion-OBI and NoiseMotion.
8

---

Published as a conference paper at ICLR 2026
4.1 COMPARISONS AND DISCUSSIONS
We evaluate SparkleMotion on the following four key capabilities:
Robustness to Noise and Occlusion in General Scenarios. conducted on 2 noisy datasets, FreeMotion-obj and NoiseMotion, where point clouds contain substantial sensor noise and occlu- sions, as well as two normal datasets FreeMotion and Sloper4D. As shown in Table I our method significantly outperforms existing approaches, particularly in global joint and vertex errors (J/V Err(G)), demonstrating its ability to handle incomplete or noisy inputs. Please refer to the LiDAR- based Motion Capture of Figure for visual comparison.
Effectiveness under Close-Interactions Scenarios. Due to the dense proximity between individuals, occlusion and noises more significant in such cases. In Ta- ble 2 we compare with LiveHPS++ on 3 clsse-interaction datasets. Our approach, significantly outperforms LiveHPS++ challenging, occluded scenarios. By inte- grating skeletal joints with surface keypoints, sparkle preserves fine motion details while ensuring structural consistency, enabling robust motion estimation. A visual comparison is shown in Figure
[LiveHPS++ (Ren et al 2024b
Ours
Dataset
p/y Emr() 1/V Emr(G)
Ang Ear Emr()) 1/V Err(C) Ang
Interhuman
41 81.7/62.8
56.0/3.8
18.47
30.4/39.9
40.4/48.4
6.75
Chi3D
39.8/2..8 47.5/71.9
55.074.4
23.28
33.7/46.4
47.3/57.8
12.09
67.9/8888
25.29
38.6/52.4
54.5/66.2
13.11
Table Evaluation on close-interaction datasets.
Cross-sensor generalization on hetero- geneous point distributions is evalu- ated on datasets "TD--Human-Poin and HuMMan- Point, which contain data with diverse point patterns, captured with mul- tiple point cloud-based sensors, including Ouster-1-128bean. Ouster-0-64beenn Kinect. As shown in Table [ and Table 3 method consistently achieves superior performance. LiveHPS++ struggles with domain shifts between sensor types, but our approach over- comes this limitation by capturing both global motion and fineggriieed details, improving robustness across diverse point patterns, refer to Figure
Method
[CTA-Human- Paint (Cai at
HaMMan-PPin
Frrt.) Emn()
Ang
Emril)
Em((C) Ang
VoteHMR
PowxHPP
929911700
20.18199
LiveHPS-4
77,590.4
127./137.
13.17
714924
95.6/13311
21.47
Ours
69.1882.2 122.7/1319
11.52
63.57664
87,597.6
12,00
and
Table 3: Evaluation on cross-sensor generalization.
Adaptability to Multi-view Setups. Our method is designed to seamlessly handle multi-view setups. In Table 4 we show our superiority on multi-view datasets FreeMotion-MV and HuMMan-MV. Un- like previous methods that rely on explicit multi-view fusion, our approach leverages
Datasets
FreeCap (Xee at al. 2025
Ours
J/V ErtL))
JV EnrG)
Ang Ert
J/V EmLL) J/V EmtG) Ang Err
FreeMotion
668 70/778885. 9100/102.38
12.24
55.1/68.2 618/7433 83.393.1
90.2/98.7
9.61
HuMMan--Ponn
22.73/87.00 60 88.07/98.06
13.12
12.14
Table 4: Evaluation multi view datasets.
sparkle-base based point cloud segmentation results, which predict the confidence levels of keypoints. This confidence allows us to dynamically select the most reliable combination of sparkle represen- tations across multiple viewpoints, significantly improving accuracy, as illustrated in Fig.
4.2 ABLATION STUDIES
Datneets
Netwoek Structure
Psint-aliahed Skeleton Tracker (Skdetos guided Ancher Eslimator (SAE)
(Spaelde-psssseeee SMPL Sulver Salver-(SS))
Ours
PST
weo
whu
CP
juto Inituization
Milc
(whu Initalicaioon
PrecMadion
149:21199
11600125.3
112.6/12.44
106.3/115
10K5/1179
106.2/1996
HuMMMas-Paitt Point
912.31123
95.310886
91,5110.
897700L 3
91.21 01.2/102.3
89.99..1
90.4/100.2
91 9L.010220
85,01001
$7.597.6
Interhaman
$8.486.1
$5.2860
43.29000
42.351
11.1111.3
42.64003
43.951
46.4831
48.44K.4
135.491428
W07100.5
95.4/0043
92.4V100.
92.0099
91.61002
02.6/0122
95391027
92.01007
96.27877
Table 5: Ablation studies for our network modules across multiple datasets.
We conduct extensive ablation studies 5 datasets FreeMotion, FreeMotion-MM. GTA-Human- Point, and InterHuman to validate the effectiveness of each component in sparklemotion (Table
Network Structure. Without PST, skeletal estimation degrades with increased global errors due to the lack of explicit point-joint correspondence modeling; without SAE, pose accuracy suffers as the model fails to leverage surface geometry for resolving rotational ambiguities; without SSS, SMPL.

---

Published as a conference paper at ICLR 2026
parameter regression becomes less accurate without exploiting the sinematic-geoment factoriza- tion of the Sparkle representation.
Component-level Analysis. PST: Replacing the offset-based refinement proves with direct pre- diction (w/o offset) increases joint error, as the residual learning paradigm simplifies the complex regression task. Removing the optimization (wlo op) further degrades global alignment, empha- sizing the importance of iterative joint-point cloud registration. SAE: Skipping linear initialization (w/o initialize) leads to unstable convergence, while omitting the non-linear refinement (w/o op) cause an increase in anchor error, demonstrating that the cross-attention based correction effectively fuses structural priors with geometric evidence. SSS: Removing the geometric initialization (w/o init) causes the network to converge to suboptimal poses, while skipping the refinement (w/o op) leaves errors from the analytical solution uncorrected, particularly for complex shapes.
Datasets
PCA Selection
Random Selection
Monaal Selection
Ours
16 Anchons 64 Anchors 96 Anchoes
16 Anchors
32 Anchors
64 Anchors 96 Anchors
[4] Anchors
50 Anchocs
60 Anchoes
FreeMotion
112.7/1100
107,4/115.3
110.1/11899
110.8/1187
117.2/126.3
116, 4/12500 111.2/118.71
104.8/11 14,1
105.//14.7
107. 3/1 14.9
105 1/113
HoMMannPPinn
03,12104.2
94.2010511
95.2/106.3
91/61016
94.6/0088
91.1/102.4
87.1/97.0
86.597.22
N5.3%%.4
87-5977
Interhuman
50.2/58.4
47.7/55.1
49.6/57.1
51,358.6
52,26000
49.3/56.4
$1,7/58.6
11,2/49.1
48.055..9
FreeMotion-MI
97.4/106.3
93,2/102.7
95.3/104.1
95.2/104.1
1026//1331
102.4/111.7 96,9/105.2
89.3/97.2
90.1/9880
92/4/100.1
90.2/98.7
Table 6: Ablation studies for our network in different selections and numbers of surface anchors.
Analysis of Anchor Design. A core component of our Sparkle representation is the set of surface anchors. To validate our design choices, we conduct a comprehensive ablation study on the num- ber and selection strategy of these anchors. We evaluate three strategies: PCA-based selection, which provides data-driven, compact representation of surface geometry; Random selection from SMPL vertices; and Manual selection based on the CMU (Carnegie Mellon University) marker set, representing an amatomically-infomee and report the performance on the FreeMotion dataset, with results summarized in Table 6 Our findings conclusively demonstrate the superiority of the PCA-based strategy with 32 anchors. The PCA-16 configuration suffers from insufficient coverage of surface details, leading to noticeable performance drop. Conversely, configurations with more anchors (PCA-64 and PCA-96) also ex- hibit degraded performance. Predicting an excessive number of anchors from noisy, sparse point clouds makes the regression task more challenging and prone to overfitting, as errors in estimating individual anchors accumulate. The random selection strategy yields unstable and suboptimal re- sults. Its performance fluctuates irregularly with the number of anchors, as the random distribution may over-cover certain body regions while neglecting others. For instance, the Random-64 config- uration performed worse than Random-16 due to its uneven coverage. While the manual selection strategy provides robust performance due to its comprehensive anatomical coverage (even slightly outperforming our default setup in some cases), the improvement is marginal. More importantly, requires manual selection and lacks the generalizability and automation of our data-driven PCA ap- proach. Therefore, the PCA-32 configuration strikes the optimal balance between representational capacity, robustness to noise, and computational efficiency. These experiments robustly validate that our anchor module, by augmenting skeletal joints with well-chosen set of surface anchors, instrumental in constructing a more expressive and robust representation for human motion capture.
prior. For each strategy, we vary the number of anchors
5 CONCLUSION
In conclusion, Sparkle novelly unifies internal kinematics and surface geometry to advance point cloud-based motion capture. SparkleMotion leverages this representation to achieve stable, aceu- rate motion estimation by dynamically balancing structural consistency and tneeggrined motion de- tails. Through extensive experiments across multiple datasets, we demonstrate that SparkleMotion achieves state-of-the-s performance in terms of robustness to noise and occlusion, effectiveness un- der intteriittaattoon scenarios, generalization across diverse point cloud patterns, and adaptability to different capture setups. Finally, the real-time deployment of SparkleMotion achieves high frame rates, making it well-suited for practical, large-scale motion capture in diverse real-world settings. The real-time demos in the supplementary material videos.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
This research focuses on motion capture from point clouds, which, by their nature, contain less identifiable information (e.g.. faces, skin texture) compared to RGB video, offering inherent pri- vacy benefit. We envision positive applications in virtual reality, human-robot interaction, and sports analysis.
Nonetheless, we acknowledge the potential for any perception technology to be misused. While the privacy risk is lower, detailed human motion data could potentially be used for malicious surveil- lance creating deepfakes. To mitigate these risks, the code and models we release are intended solely for research purposes, and the license will explicitly prohibit use for privacy infringement, creating misleading content, any harmful applications. We encourage the community to con- sider principles of fairness, accountability, and transparency when building upon this work and to establish appropriate governance frameworks to prevent misuse.
REPRODUCIBILLITY STATEMENT
To ensure the reproducibility of our work, we commit to the following:
Code and Models: We will open-source the complete codebase for training and inference, along with the pre-trained model weights for the best-performing checkpoint all benchmarks after paper acceptance.
Data: Our experiments conducted entirely publicly available datasets (e.g., Freemotion, HuMMan, SL.OPE44D). All sources are cited in the paper. For datasets involving point cloud generation (e.g., InterHuman), we have detailed the data pre-processing pipeline following public methods.
Implementation Details: The methodology section and the supplementary material A provide comprehensive descriptions of the network architectures, loss functions, and training hyperparame- ters (e.g., learning rate, batch size, optimizer). All critical parameters are explicitly listed.
Dependencies; The code repository will include detailed requirements.ox file and/or a Docker image specify the required software libraries (c.g.. PyTorch, CUDA) and their versions, ensuring consistent computational environment.
We are confident that these measures are sufficient for other researchers to replicate our results.
ACKNOWLEDGEMENTS
This work was supported by MoE Key Laboratory of Intelligent Perception and Human-Maching Collaboration HLMP-HHuMaCo), Shanghai Frontiers Science Center of Human-centered Artificial Intelligence (ShangHAI).
This research is supported by the National Research Foundation, Singapore and Infocomm Media Development Authority under Trust Tech Funding Initiative. Any opinions, findings and conclu- sions recommendations expressed in this material are those of the author(s) and do not reflect the views of National Research Foundation, Singapore and Infocomm Media Development Authority.
REFERENCES
Thiemo Alldieck, Mare Kassubeck, Bastian Wandt, Bodo Rosenhahn, and Marcus Magnor. Optical flow-based 3d human motion estimation from monocular video. In GCPR, pp. 347-360. Springer, 2017.
Mykhaylo Andriluka, Umar Iqbal, Eldar Insafutdinov, Leonid Pishchulin, Anton Milan, Juergen Gall, and Bernt Schiele. Posetrack: A benchmark for human pose estimation and tracking. In CVPR, pp. 5167-5176, 2018
Vasileios Belagiannis and Andrew Zisserman. Recurrent human pose estimation. In 2017 12th IEEE international conference on automatic face & gesture recognition (FG 2017), pp. 468-475. IEEE, 2017.
11

---

Published as a conference paper at ICLR 2026
Zhongang Cai, Daxuan Ren, Ailing Zeng. Zhengyu Lin, Tao Yu, Wenjia Wang, Xiangyu Fan, Yang Gao, Yifan Yu, Liang Pan, et al. Humman: Multi 4d human dataset for versatile sensing and modeling. In European Conference on Computer Vision, pp. 557-577. Springer, 2022.
Zhongang Cai, Liang Pan, Chen Wei, Wanqi Yin, Fangzhou Hong, Mingyuan Zhang, Chen Change Loy, Lei Yang, and Ziwei Liu. Pointhps: Cascaded 3d human pose and shape estimation from point clouds. arXiv preprint arXiv:2308. 14492, 2023.
Zhongang Cai, Mingyuan Zhang, Jiawei Ren, Chen Wei, Daxuan Ren, Zhengyu Lin, Haiyu Zhao, Lei Yang, Chen Change Loy, and Ziwei Liu. Playing for 3d human recovery. IEEE Transac- tions on Pattern Analysis and Machine Intelligence, pp. 1-12, 2024. doi: 10.1 2020 3450537.
Carnegie Mellon University. CMU MoCap Dataset. URL http: http://mocap..... cs. cmw.edu
James Charles, Tomas Pfister, Derek Magee, David Hogg, and Andrew Zisserman. Personalizing human video pose estimation. CVPR, pp. 3063-3072, 2016.
Yudi Dai, Yitai Lin, Chenglu Wen, Siqi Shen, Lan Xu, Jingyi Yu, Yuexin Ma, and Cheng Wang. Hsc4d: Human-centered 4d scene capture in large-scale indoor-outdoor space using wearable imus and lidar. In CVPR, pp. 6792-6802, 2022.
Yudi Dai, YiTai Lin, XiPing Lin, Chenglu Wen, Lan Xu, Hongwei Yi, Siqi Shen, Yuexin Ma, and Cheng Wang. Sloper4d: A sceme-aware dataset for global 4d human pose estimation in urban environments. In Proceedingg of the IEEE/CVF conference on computer vision and pattern recognition, pp. 682-692, 2023.
Bowen Dang. Xi Zhao, Bowen Zhang, and He Wang. Reconstructing 3d human pose from rgb-d data with occlusions, 2023. URL http::t/s//:tt..ccoc wtt.:oggaaabaooooooe 01228
Mingsong Dou, Philip Davidson, Sean Ryan Fanello, Sameh Khamis, Adarsh Kowdle, Christoph Rhemann, Vladimir Tankovich, and Shahram Izadi. Motion2fusion: Real-time volumetric per- formance capture. ACM Trans. Graph., (4::))):6::2246:11 November 2017. ISSN 0730-0301.
Bohao Fan, Wenzhao Zheng, Jianjiang Feng, and Jie Zhou. Lidar-hmr: 3d human mesh recovery from lidar, 2023. URL
htrgg//bb/233/ 11971
Mihai Fieraru, Mihai Zanfir, Elisabeta Oneata, Alin-Ionut Popa, Vlad Olaru, and Cristian Sminchis- escu. Three-dimensional reconstruction of human interactions. In The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
Tim Horberry, Christine Mulvihill, Michael Fitzharris, Brendan Lawrence, Mike Lenné, Jonny Kuo, and Darren Wood. Human centered design for an in-vehicle truck driver fatigue and distraction warning system. IEEE Transactions on Intelligen! Transportation Systems, 23)5:3350)5539 2022. doi: 10.1109/TTSS.02111 3.05309...
Human-centered
Yinghao Huang, Manuel Kaufmann, Emre Aksan, Michael Black, Otmar Hilliges, and Gerard Pons-Moll. Deep inertial poser: Learning to reconstruct human pose from sparse inertial mea- surements in real time, ACM Transactions on Graphics (TOG), 37(6))1-15, 2018a.
Zeng Huang, Tianye Li, Weikai Chen, Yajie Zhao, Jun Xing, Chloe LeGendre, Linjie Luo, Chongyang Ma, and Hao Li. Deep volumetric video from very sparse multi-view performance capture. ECCV, September 2018b.
Tao Jiang, Peng Lu, Zhang, Ningsheng Ma, Rui Han, Chengqi Lyu, Yining Li, and Kai Chen. Rtmpose: Real-time multi-person pose estimation based on mmpose. arXiv preprint arXiv:2303.07399, 2023.
Tianjian Jiang, Johsan Billingham, Scbastian Müksch, Juan Zarate, Nicolas Evans, Martin Oswald, Mare Pollefeys, Otmar Hilliges, Manuel Kaufmann, and Jie Song. Worldpose: A world cup dataset for global 3d human pose estimation, eccv, 2024.
Angjoo Kanazawa, Michael J. Black, David W. Jacobs, and Jitendra Malik. End-to-end recovery of human shape and pose. In CVPR, 2018.
12

---

Published as a conference paper at ICLR 2026
Manuel Kaufmann, Yi Zhao, Chengcheng Tang, Lingling Tao, Christopher Twigg. Jie Song, Robert Wang, and Otmar Hilliges. Em-pose: 3d human pose estimation from sparse electromagnetic trackers. In ICCV, pp. 11510-11520, 2021.
Wonhui Kim, Manikandasriram Srinivasan Ramanagopal, Charles Barto, Ming- Yuan Yu, Karl Rosaen, Nick Goumas, Ram Vasudevan, and Matthew Johnson- -oonesn.. Pedx: Benchmark dataset for metric 3-d pose estimation of pedestrians in complex urban intersections. IRAL, 4(2): 1940-1947, 2019.
Jialian Li, Jingyi Zhang, Zhiyong Wang, Siqi Shen, Chenglu Wen, Yuexin Ma, Lan Xu, Jingyi Yu, and Cheng Wang. Lidarcap: Long-range marker-less human motion capture with lidar point clouds. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 20502-20512, 2022.
Jiefeng Li, Chao Xu, Zhicun Chen, Siyuan Bian, Lixin Yang, and Cewu Lu. Hybrik: A hybrid analyt inverse kinematics solution for 3d human pose and shape estimation. In Pro- ceedings of the IEEE/CVE conference on computer vision and pattern recognition, pp. 3383- 3393, 2021.
Han Liang, Wenqian Zhang, Wenxuan Li, Jingyi Yu, and Lan Xu. Intergen: Diffusion-based multi- human motion generation under complex interactions. International Journal of Computer Vision, pp. 1-21, 2024.
Guanze Liu, Yu Rong, and Lu Sheng. Votehmr: Occlusion-aware voting network for robust 3d human mesh recovery from partial point clouds, 2021. URL https:://axxi..rrg//.
F110.08729
Matthew Loper, Naureen Mahmood, and Michael Black. MoSh: Motion and shape capture from sparse markers. 33(6):220:1-1 2014. ISSN 0730-0301. doi: 10. 29.11464/661222.661277
Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black. Smpl: A skimned multi-person linear model. ACM Trans. Graph., (201:)):2:::224:::11 October 2015. ISSN 0730-0301.
Xiaoxuan Ma, Jiajun Su, Chunyu Wang, Wentao Zhu, and Yizhou Wang. 3d human mesh estimation from virtual markers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patlern Recognition (CVPR), pp. 534-543, June 2023a.
Xiaoxuan Ma, Jiajun Su, Chunyu Wang, Wentao Zhu, and Yizhou Wang. 3d human mesh estimation from virtual markers. In Proceedings the IEEE/CVF conference on computer vision and pattern recognition, pp. 534-543, 2023b.
Charles Malleson, John Collomosse, and Adrian Hilton. Real-time multi-person motion capture from multi-view video and imus. IJCV, pp. 1-18, 2019.
OptiTrack. OptiTrack Motion Capture Systems. https: www. httpss//ww..ppritracc.... com/ 2009.
Gerard Pons-Moll, David Fleet, and Bodo Rosenhahn. Posebits for monocular human pose esti- mation. In CVPR, June 2014. doi: 10. 1011009VVR..201..3000
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In CVPR, pp. 652-660, 2017.
Ramesh Raskar, Hideaki Nii, Bert Dedecker, Yuki Hashimoto, Jay Summet, Dylan Moore, Yong Zhao, Jonathan Westhues, Paul Dietz, John Barnwell, et al. Prakash: lighting aware motion capture using photosensing markers and multiplexed illuminators. In TOG, volume 26, pp. 36. ACM, 2007.
Yiming Ren, Chengfeng Zhao, Yannan He, Peishan Cong, Han Liang, Jingyi Yu. Lan Xu, and Yuexin Ma. Lidar-aid inertial poser: Large-scale human motion capture by sparse inertial and lidar sensors. TVCG, 2023.
13

---

Published as a conference paper at ICLR 2026
Yiming Ren, Xiao Han, Yichen Yao. Xiaoxiao Long, Yujing Sun, and Yuexin Ma. Livehps++: Robust and coherent motion capture in dynamic free environment. In European Conference on Computer Vision, pp. 127-144. Springer, 2024a.
Yiming Ren, Xiao Han, Chengfeng Zhao, Jingya Wang, Lan Xu, Jingyi Yu, and Yuexin Ma. Livehps: Lidar-based scene-level human pose and shape estimation in free environment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1281 2024b.
Zehong Shen, Huaijin Pi, Yan Xia, Zhi Cen, Sida Peng, Zechen Hu, Hujun Bao, Ruizhen Hu, and Xiaowei Zhou. World-grounded human motion recovery via gravity-view coordinates. In SIGGRAPH Asia Conference Proceedings, 2024.
Soyong Shin, Juyong Kim, Eni Halilaj, and Michael J. Black. WHAM: Reconstructing world- grounded humans with accurate 3D motion. In IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), June 2024.
Yu Sun, Qian Bao, Wu Liu, Tao Mei, and Michael J. Black. TRACE: 5D) Temporal Regression of Avatars with Dynamic Cameras in 3D Environments. In IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2023.
Thang Vu, Kookhoi Kim, Tung M Luu, Thanh Nguyen, and Chang D Yoo. Softgroup for 3d instance segmentation point clouds. In Proceedings of the IEEE/CVF Conference Computer Viston and Puttern Recognition, pp. 2708-2717, 2022.
Yufu Wang, Ziyun Wang, Lingjie Liu, and Kostas Danilidis. Tram: Global trajectory and motion of 3d humans from in-the-wild videos. arXiv preprint arXiv:2403. 17346, 2024.
L. Xu, Z. Su, L. Han, T. Yu, Y. Liu, and L. FANG. Unstructuredfuuson: Realtime 4d geometry and texture reconstruction using commercialrgbd cameras. TPAMI, pp. 1-1, 2019. ISSN 1939-3539.
Lan Xu, Weipeng Xu, Vladislav Golyanik, Mare Habermann, Lu Fang, and Christian Theobalt. Eventcap: Monocular 3d capture of high-speed human motions using an event camera. In CVPR, June 2020.
Yufei Xu, Jing Zhang, Qiming Zhang, and Dacheng Tao. ViTPose: Simple vision transformer baselines for human pose estimation. In Advances Neural Information Processing Systems, 2022.
Aoru Xue, Yiming Ren, Zining Song, Mao Ye, Xinge Zhu, and Yuexin Ma. Freecap: Hybrid calibration-Fis motion capture in open environments. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 9032-9040, 2025.
Zijie Ye, Haozhe Wu, and Jia Jia. Human motion modeling with deep learning: A survey, Al Open, 3: 35-39, 2022. ISSN 2666-6510. doi: http://a....e/rrraa///////////// www. ici enceii rect. com
12.000 URL htttttts:
Xinyu Yi, Yuxiao Zhou, and Feng Xu. Transpose: Real-time 3d human translation and pose estima- tion with six inertial sensors. ACM Transactions on Graphics (TOG), 40(4):1-33 2021.
Xinyu Yi, Yuxiao Zhou, Marc Habermann, Soshi Shimada, Vladislav Golyanik, Christian Theobalt, and Feng Xu. Physical inertial poser (pip): Physies-aware real-time human motion tracking from sparse inertial sensors. In CVPR, June 2022.
Yifei Yin, Chen Guo, Manuel Kaufmann, Juan Zarate, Jie Song, and Otmar Hilliges. Hi4d: 4d instance segmentation of close human interaction. Computer Vision and Pattern Recognition (CVPR), 2023.
Yifu Zhang, Peize Sun, Yi Jiang, Dongdong Yu, Fucheng Weng, Zehuan Yuan, Ping Luo, Wenyu Liu, and Xinggang Wang. Bytetrack: Multi-object tracking by associating every detection box. 2022.
Christian Zimmermann, Tim Welschehold, Christian Dornhege, Wolfram Burgard, and Thomas Brox. 3d human pose estimation in rgbd images for robotic task learning. In 2018 IEEE In- fernational Conference on Robotics and Automation (ICRA), pp. 1986-1992, IEEE, 2018
14

---

Published as a conference paper at ICLR 2026
APPENDIX
A IMPLEMENTATION AND NETWORK DETAILS
A.1 IMPLEMENTATION DETAILS
Our method is implemented using PyTorch 1.8.1 and CUDA 11.1. We train the model for 200 epochs with batch size of 32 and sequence length of 32, using initial learning rate of 0.001. The training was conducted on server with an Intel(R) Xeon(R) Gold 5318Y CPU and 4 NVIDIA A40 GPUs. For LiDAR-base -bbsed datasets, we follow the settings from LiveHPS++ (Ren et al., 2024b).
The model is trained using the training datasets listed in the table and tested on their respective test sets. For Depth camera-bases datasets, we adopt the training setup from PointHPS (Cai al. 2023). training and testing each benchmark separately.
A.2 DETAILED ARCHITECTURE POINT-ALIGNED SKELETON TRACKER
Global Feature
Encoder
Decoder
T
init
T
T
centered
Time
GRU
g0,
J init
gp
Local Feature
Decoder
P centered
Time
Coarse Prediction
Point Number
Refinement
Joint-level Point Feature
Enhanced Point Feature
F joint
Parent-joint Feature
J op
Encoder
Decoder
Missing Feature
Time
MLP Fusion
A.J
Children-jolnt Feature
op
Joint Number
Figure 4: The details of Point-aligned Skeleton Tracker.
The Point-aligned Skeleton Tracker (PST) is implemented dual-branch network that jointly estimates initial skeletal joints J Jim global translation Timi. and point-wise semantic labels L, from the input point cloud sequence. Its architecture comprises two main components as shown in Fig
1. Coarse Prediction Network: This branch processes the spatio-tempores point cloud sequence. Given an input point cloud of shape (B, T,3, .) (Batch, Time, Channels, Points), PointNer backbone first extracts fer-frame features. A bidirectional GRU with hidden size of 512 (yielding a 1024-dimens output due to bidirectionality) models temporal dependencies. The network also ingests a trajectory signal. which is encoded by MLPs (3 - 64 128) and concatenated with the point cloud features. The branch outputs:
Initial joints J Regressed via an MLP (1024 + 512 - 256 - 24 x 3).
Initial translation Tim (optional): Regressed via a parallel MLP (1024 + 512 + 256 +
3).
15

---

Published as a conference paper at ICLR 2026
Point- wise part labels Ly: A segmentation head, consisting of 1D convolutional layers (2048 + 1024 + 512 - 256 + k), predicts a label for each of the N points, where k = 36 is the number of semantic parts (including background).
2. Refinement Network: This branch refines the initial estimates using local geometric context. It takes as input the raw point cloud P. the initial skeleton int. and the predicted part labels Ly. For each point, it constructs 6D feature by concatenating its spatial coordinates with the coordinates of its associated joint (as indicated by Ly). A PointNer processes this enriched input to extract local features. A key operation is a label-wise max-pooling: for each of the 24 joints, features from all points belonging to that joint part are pooled to form a robust, joint-centric feature descriptor. These descriptors then decoded by MLP (1024 512 256
to predict the residual joint offsets AJ, yielding the final refined joints Jop = Jin + AJ and translations Top mim + AJo-
The PST module effectively couples a global spatio-temport model for coarse estimation with local geometry-awant network for fine-grained refinement, establishing the explicit point-joint cor- respondences central our method.
A.3 DETAILED ARCHITECTURE OF SKELETON-GUIDED ANCHOR ESTIMATOR
A.3.1 STRUCTURAL INITIALIZATION VIA VERTEX-JOINT MATCHING
Prior to training, we precompute a linear map- ping between skeletal joints and surface an- chors to provide a strong structural initializa- tion. This process begins with Vertex-Joint Matching on the SMPL mesh, where each vertex is associated with its nearest skeletal joint, establishing Joint-level Segmentation of the mesh surface. From the full set of mesh vertices, select canonical set of 32 an- chors Agt via PCA to maximize their repre- sentational power. linear mapping matrix MJ2A E 224x32 is then computed in least- squares manner from ground-trute data:
Module Pre-processing
Posege
Target: M,zA
SMPL
Beta,
J minimize (M, Anدو
Translation, 5)
Module Training
F piin
Docodar
Interaction
Tnoker
Encoder
kinemaric oprimization
MJZA = (A)) A
=
[ Apm
Expandation Module
During inference, this matrix provides geometry-agnostic prior, yielding an initial an- chor estimate based solely on the predicted skeletal pose: Ainit = MJ2A
Vertex-Joiint Matching
Joint-level Segmentation
Matching
Vertex- Segmentation
L
Jonsinal legnonsten
Fine Grained Matching
Coarse-Graid Grained Matching
A.3.2 GEOMETRY-AWARE REFINEMENT NETWORK
Figure 5: The details Skeleton-guided Anchor Estimator.
The initial estimate Ajnit is refined by network that learns non-linear correction using geo- metric evidence from the predicted joint Jop-
The core of this network is Transformer- based decoder that performs cross-attention between anchor queries and geometric features.
Feature Encoding: A MLP backbone processes the predicted joints to extract local geometric fea- tures. Simultaneously, the initial anchors A iii are encoded into 1024-dimensional feature space via MLP (3 + 256 512 1024).
Cross-Attention Refinement: The refinement is formulated as a sequence-to-sequeeeee problem. The encoded initial anchors serve the query sequence (32 anchors, dimension 1024). The joint features from the MLP serve as the key and value sequence. A 2-layer Transformer decoder with attention heads performs cross-atts cross-attention, allowing each anchor query to attend to the most relevant joint-level geometric context. The attended features are then decoded by an MLP (1024 - 512 256 3) to predict the optimized anchors App
16

---

Published as a conference paper at ICLR 2026
After get the predicted joints and anchors, the Kinematic-awart Optimizer from LiveHPS++ Ren et al. (2024a) to optimize the Sparkle from the raw point cloud data and fully utilize the temporal information.
A.3.3 ANCHOR-LEVEL SEGMENTATION FOR EXPANDATION
The module also predicts an anchor-level segmentation of the point cloud, which crucial for downstream tasks like multi-view fusion. The gemeration of the ground-truth segmentation labels LA for training is a multi-stage process that ensures precise correspondence between points and anchors, illustrated in Figure
1. Vertex-Joint Matching on SMPL Mesh: The process begins on the canonical SMPL mesh. We compute the Euclidean distance from each mesh vertex to all skeletal joints. Each vertex is then assigned to its closest joint, establishing a fundamental Joint-level Segmentation at the mesh level. This defines a surjective mapping from the of vertices to the set of joints.
2. Point Cloud to Vertex Correspondence (Coarse-Graines Matching): To propagate the mesh-based segmentation to an unstructured point cloud P., we establish correspondence between points and mesh vertices. This is achieved via KNN Matching algorithm, where each point in Pe is matched to its & nearest neighbors in the set of SMPL mesh vertices. The joint label for point determined by a majority vote among the labels of its k nearest vertices. This step effectively projects the mesh's Joint-level Segmentation onto the point cloud, yielding the coarse-grained point-wise labels 1,,."
Anehor to Vertex Assignment: The surface anchors Age are derived from the mesh vertices via PCA. Crucially, each anchor is assigned to specific joint based the pre- computed Vertex-Joint Matching. Specifically, an anchor is assigned the joint label of the mesh vertex from which it was predominantly derived. This creates well-defined, many-to-one mapping from anchors to joints.
4. Point to Anchor Correspondence (Fine-Grained Matching): The final Anchor-level Segmentation LA,, obtained by performing a fine-grained matching within the con- fines of the coarse segmentation. For each point in Pr, its anchor label is determined by finding its nearest anchor from the subset of anchors that share the same joint label as the point itself (as determined by L, This joint-localized KNN is critical, it prevents roneous matches that could occur when point physically closer to an anchor belonging neighboring joint (e.g., point the wrist mistakenly matched leg anchor).
This hierarchical labeling strategy ensures that the anchor-level segmentation is both geometrically precise and anatomically consistent. The network learns to predict this segmentation through parallel segmentation head, composed of ID convolutional layers (2048 > 1024 - 512 + 256 k). The quality of the predicted segmentation L,, implicitly defines confidence measure for each predicted anchor, enabling downstream modules (e.g., the multi-view fusion logic) to dynamically weight the reliability of the Sparkle representation.
A.4 ROTATION DECOMPOSITION IN SPARKLE-BASED SMPL SOLVER
We detail the initialization of SMPL pose in Sparkel-based SMPL Solver. The rotation E 50(3) be decomposed into twist rotation Rtw and swing rotation R Given the start template body-part joint vector Jem- marker vector Agem target joint vector Jhpp and target marker vector A'y. the solution process for R can be formulated as:
R = D(Jum, Are, Sop A'o)) Dow (Jum- Pay)- nwN (0((1 (Pop Aum, Kip)
R 12
where Daw and Diw represent the closed-form solutions for the swing rotation and twist rotation, respectively. The swing rotation has axis Vow that perpendicular to both Jum and f It can be
17

---

Published as a conference paper at ICLR 2026
formulated as:
Jrem x J op
Now
11 x Sopll
The swing angle asw satisfies:
Jeem Jop
COS
|| |||
|| x op
Clyw
|| |||| Jap
Thus, the closed-form solution for the swing rotation R can be derived using the Rodrigues for- mula:
R Dow (Jums Top)
I + sin Oksw [Tsw] + (1 COS (ksw) [
where [/ is the skew-symmetric matrix of New, and I is the 3 3 identity matrix. The twist rotation has an axis niw defined as:
||
The twist angle Oww satisfies:
Abani
Arp
COS Cltw
||-4gem
||
||
||-A4op
IAproo
A
sin Cltw
A ||
A
where
Aproj and Approl
projections onto the plane perpendicular to the axis New
Aprij
Frem
Agem
Raw
Thps
|
AAop
Thew
Aproo
Arp
Thw
|/ew
Thus, the closed-form solution for the twist rotation R be derived using the Rodrigues formula:
ns 2 (Jop- Arem: Akp)) + sin Oew [Fo Ix + (1 coS atw) [iw
where [[xw]x is the skew-symmetric matrix of Tlw-
B DETAILS OF MULTI-VIEW MOCAP BY SPARKLEMOTION
Our method also can handle arbitrary-viewpoint point clouds as input and robustly fuse infor- mation from multiple perspectives. Given set of multi-view point clouds [P.) captured from different viewpoints, we aim to obtain globally consistent human motion representation while maintaining robustness to occlusions, sensor noise, and varying point distributions.
View-Independent Sparkle Prediction For each individual view P, we utilize our trained SparkleMotion model to predict the segmentation result, the corresponding Sparkle representation and global human rotation S;:
S, = FSparkleMotiion (Pt),
(7)
where S, consists of:
Segmented point cloud assignments from the input.
18

---

Published as a conference paper at ICLR 2026
Multi-view Setup
Multi-view Sparkle
SparkleMotion
Sparkle-based
SMPL Solver
Single-view Results
Result
Sparkles
Selector
PCs
A
Confidence
Figure 6: The pipeline of the multi-view SparkleMotion. Our method can take point clouds from any perspective as input to obtain optimized results, demonstrating the powerful scalability of Sparkle.
Root-relative sparkle capturing local motion structure.
Global rotation estimation align multiple views.
Each predicted sparkle is assigned confidence score w; based on segmentation quality, reflecting its reliability:
W; (.),,
where G represents confidence estimation function, which considers point density, occlusions, and consistency in spatial feature aggregation.
Multi- View alignment via global rotation estimation Before fusing the information across views, we first calibrate each predicted global rotation R, Each viewpoint provides an independent estimate of global orientation, and refine it by minimizing the misalignment across different views:
N
R = arg min
R
(9)
where R" is the optimized global rotation used for aligning all views.
Gonfidence-Guudeed Multi- Fusion Once the multi-view predictions (S,, w.lh are obtained, we perform confidence-based selection to filter out unreliable keypoints and obtain refined sparkle:
S"
W. S.
(10)
This ensures that keypoints predicted with higher confidence have greater influence, while uncer- tain keypoints from occluded or noisy views are suppressed.
Sparkle-Based SMPL Parameter Estimation Finally, the optimized sparkle S fed into our Sparkle-based SMPL Solver to predict human body parameters:
=(SS),
(11)
where ** includes body shape 3, pose parameters 0, and global translation t. Our solver ensures that the final motion estimation maintains structural consistency while fully utilizing the information across views.
Multi-view motion capture is essential for achieving robust and accurate human motion estimation, especially in real-world scenarios with occlusions and varying viewpoints. Traditional methods of- ten rely explicit camera calibration and heuristic fusion strategies, limiting their flexibility and
19

---

Published as a conference paper at ICLR 2026
adaptability. Our proposed framework, Sparklemotion, introduces segmentation-drive confi- dence mechanism and unified motion representation, allowing seamless integration of multiple views without requiring predefined camera setups. By dynamically filtering unreliable keypoints and leveraging hierarchical optimization, Sparklemotion ensures scalable, robust, and generalizable motion capture across diverse capture conditions, from sparse LiDAR data dense depth-camera inputs.
C ANALYSIS ON DISTANCE AND OCCLUSION
This section provides quantitative analysis of the system's performance under challenging con- ditions, specifically increasing capture distance and varying degrees of occlusion. These scenarios directly test the stability of the geometric solver, as they often lead sparser point clouds, noisier Sparkle estimates, and situations where joints anchors may become occluded or their observations may become numerically unstable (e.g., near-colinear anchors).
C.I PERFORMANCE IN DIFFERENT CAPTURE DISTANCE
We first evaluate the robustness of SparkleMotion the FreeMotion dataset by stratifying the test results based on the distance between the subject and the LiDAR sensor. Increased distance leads to sparser point clouds, making the estimation of both joints and anchors more challenging. The results are presented in Table
Distance Range
J/V Err(L.
J/V Err(G)
Ang Err
5m 10m
52.6/65.4
89.9/98.3
9.01
10m 20m
61.6/76.1
115.4/123.6
9.99
20m 30m
63.7/79.0
114.2/124.0
10.04
Table 7: Performance analysis under differentt capture distances on the FreeMotion dataset.
The results indicate graceful degradation in performance distance increases. The error metrics see the most significant jump from the 5-10m to the 10-20m range, which aligns with the quadratic fall-off point density with distance. Notably, the performance stabilizes between the 10-20m and 20-30m ranges, demonstrating the resilience of the Sparkle representation and the two-stage solver. The kinematic prior and the refinement network effectively prevent complete failure even with very sparse data.
C.2 PERFORMANCE IN DIFFERENT OCCLUSION RATIO
To directly analyze the stability under the nearly colinear or occluded conditions, we evaluate performance sequences with artificially generated occlusion. We simulate different occlusion levels by randomly removing a specified percentage of points from the human point cloud. The
results are summarized in Table
Occlusion Ratio
J/V Err(L)
J/V Em(()
Ang Err
0%
59.0/73.2
105.1/113.
9.66
30%
60.0/74.4
107.8/116.7
9.81
50%
61.7/76.5
110.8/119.8
10.07
70%
66.7/82.6
118.7/128.3
10.92
90%
97.4/121.2
163.3/179.0
14.18
Table 8: Performance analysis under different occlusion ratios. Metrics reported a held-out test set with simulated occlusion.
As shown Table the system maintains reasonable accuracy up to 70% occlusion. The degra- dation in performance is monotonic and controlled. The increase in AngErr under heavy occlusion (90%) reflects the increased ambiguity in estimating the twist component of the rotation when sur- face anchors missing. This empirically validates that two-stage design, where the refinement
20

---

Published as a conference paper at ICLR 2026
network corrects for failures of the geometric initializer, provides significant robustness against the instability the analytical solver under severe occlusion.
C.3 DETAILED ABLATION STUDIES ON PST AND SAE MODULE DESIGNS
This section provides more granular analysis of the design choices within the Point-aligned Skele- Tracker (PST) and the Skeleton-guide Anchor Estimator (SAE), extending the coarse-grained ablation studies in the main paper.
C.3.1 PST REFINEMENT STRATEGY ANALYSTS
We evaluated several refinement strategies for the PST module beyond the simple direct prediction baseline. The results on the FreeMotion dataset are summarized in Table 9
Refinement Strategy
J/V Err(L)
J/V Err(G)
Ang Err
Direct Prediction (w/o offset)
68.1/83.5
107.2/116.7
10.23
Iterative Refinement stages)
58.5/72.8
104.8/113.5
9.61
Attention-based Refinement
59.8/74.1
(06.2/114.9
9.80
Residual Offset (Ours)
59.0/73.2
105.1/113.9
9.66
Table 9: Detailed ablation of different refinement strategies for the PST module on the FreeMotion dataset.
The residual learning paradigm proves most effective, substantially outperforming direct predic- tion by simplifying the regression task, while both iterative and attention- refinements yield diminishing returns relative to their increased computational complexity.
C.3.2 SAE INITIALIZATION AND REFINEMENT ANALYSIS
We further dissect the SAE module by evaluating different initialization and refinement strategies. Results are reported on the FreeMotion dataset in Table T0
Method
J/V Err(L)
J/V Err(G)
A. Initialization Strategy Analysis
Random Initialization
95.4/115.2
155.7/168.3
MLP-based Initialization
65.3/80.1 59.0/73.2
112.4/121.0
Linear Mapping (Ours)
105.1/113.9
B. Resinement Strutegy Analysis (using Linear Mapping Init)
No Refinement (w/o op)
66.2/81.0 61.5/75.8 59.0/73.2
107.8/117.0
MLP Refinement
106.9/115.6
Cross-Attention Refinement (Ours)
105.1/113.9
Table 10: Detailed ablation of initialization and refinement strategies for the SAE module on the FreeMotion dataset.
Our linear mapping initialization establishes robust anatomical prior that significantly outperforms alternatives, while the cross-attention refinement proves essential for achieving precise, context- aware corrections beyond what standard MLP refinement can offer.
D ANALYSIS OF DATA EFFICIENCY AND GENERALIZATION
To rigorously evaluate both the generalization capability and data efficiency of the proposed representation, we conduct cross-dataset transfer learning experiment. A model is first pre- trained mixture of datasets excluding FreeMotion, and then fine-tuned on progressively
21

---

Published as a conference paper at ICLR 2026
larger fractions of the FreeMotion training set before evaluation the held-out FreeMo- tion test set. As summarized in Table [T] SparkleMotion demonstrates remarkable zero- shot generalization when no FreeMotion data is used (0%), significantly outperforming the baseline. Furthermore, it exhibits superior data efficiency, achieving performance compa- rable to the fully supervised baseline using only 50% of the target domain training data. Theoretically, these empirical advantages stem from the structured prior em- bedded in the Sparkle representation, which effectively reduces the hypoth- esis space and enhances identifiability. By explicitly factorizing the human state into internal kinematics (skeletal joints) and external geometry (surface
LiveHPS++
0%
84,0
65.2
25%
78.4
63.1
50%
68.9
61.5
anchors), we introduce strong inductive bias that disentangles pose from shape variations. This factorization mitigates the ill-posedness inherent estimating human pose from partial noisy point clouds, the kinematic structure provides robust constraints that are largely invariant to sensor type surface noise. From a learning theory perspective, this structural disentan- glement simplifies the mapping function that the network must learn, thereby lowering sample complexity and enabling more efficient knowledge transfer across domains. The consistent skeletal topology serves as domain-invariant feature, while the surface anchors capture sensor-specific geometries without corrupting the structural estimate. This result provides strong evidence that the structural prior embedded in the Sparkle representation is only robust to domain shifts but also drastically reduces the required amount of labeled data for achieving oftt-offthhe-rt performance, confirming its effectiveness as a powerful and efficient representation-leariin principle for 3D human motion capture.
75%
63.2
59.8
100%
61.9
59.0
Table 11: Cross-
dataset generaliza- tion and data effi- ciency.
E REAL-WORLD DEPLOYMENT AND BENCHMARKING
Ping-pong Show
Comparison between Sparkle with Optical Method
Front View
Sparkle
Taekwondo Show
Vertical View
Optical
Figure 7: More sports demo shows and comparison with optical-base MoCap System.
E. EVALUATION IN THE REAL WORLD
Our method, SparkleMotion, demonstrates exceptional robustness and generalizability across wide spectrum of challenging real-world scenarios, directly validating the core contribution of the Sparkle paresentation--alante
expressiveness and robustness. As illustrated Figure [ and Figure 7 the framework seamlessly handles inputs from diverse sensor modalities, including sparse, long- range LiDAR point clouds and denser data from consumer-gran graddd depth sensors like the iPhone. This sensor-agnostic performance stems from Sparkle's factorized design, which allows it to maintain structural coherence even when the surface geometry from the point cloud is incomplete or noisy. This capability is critically evaluated in demanding situations such as basketball games with intense occlusions and large-scale soccer matches featuring over 20 players at distances beyond 80 meters. In these environments, where point clouds highly fragmented, the explicit coupling of internal skeletal priors with external geometric evidence in Sparkle prevents error accumulation and ensures stable full-body motion estimation.
22

---

Published as a conference paper at ICLR 2026
E.2 DETAILS OF DEPLOYMENT
To validate the practical utility of SparkleMotion, we deployed as real-time system capable of operating in challenging. large-scale scenarios. Our complete network comprises 82.3 million pa- rameters. On consumer-grand grade NVIDIA GeForce RTX 4090 GPU, the system achieves stable end-to-end frame rate of 60 FPS in visualization by interpolation, the real frame rate is 10 FPS lim- ited in LiDAR throughput. This performance includes all stages of processing: point cloud prepro- cessing (e.g., Farthest Point Sampling), the core Sparkle estimation, and the final SMPL parameter regression. This efficiency demonstrates its suitability for real-world applications. Furthermore, the system exhibits strong scalability, qualitative results from large-scale scenes, such as soccer fields and basketball courts, are showcased in Figure [
E.3 PERFORMANCE COMPARISON WITH OPTICAL SYSTEM
To quantitatively benchmark its accuracy against a gold standard, we compared SparkleMotion to an industry-grade OptiTrack optical motion capture system in controlled laboratory setting. On a complex ballet sequence involving rapid spins and precise limb movements (Fig 77). our method achieved remarkable accuracy, with mean Global MPJPE of 34.27 mm. This result confirms that the geometric initialization and refinement enabled by the Sparkle representation can yield preci- sion comparable to marker-based systems for individual subjects. The significant advantage of our approach, however, lies in its scalability and accessibility. Unlike OptiTrack, which is confined to a calibrated volume, our system operates in large-scale, in-the-wild environments, requires wear- able markers or complex setup, and naturally supports an unrestricted number of subjects. These capabilities, showcased extensively the supplementary video, underscore the potential of Sparkle- Motion as versatile and practical solution for next-generation motion capture, bridging the gap between laboratory-gnnnee -rraee accuracy and real-world applicability.
E.4 MULTI-PERSOO TRACKING AND SYSTEM SCALABILITY
While core contribution is the Sparkle representation, we further demonstrate the practicality of the full system in large-scale, multi-person scenarios. Our end-to-end pipeline achieves a throughput of approximately 90 FPS a single NVIDIA RTX 4090 GPU in football scenes with 25 persons, far exceeding the data capture rate of typical LiDAR sensors (e.g., 10 Hz). This confirms that the system introduces negligible latency.
A key design advantage is the decoupling of person segmentation and pose estimation. The pose es- timation network processes each individual independently, ensuring that the accuracy of the motion capture is invariant to the number of people in the scene.
For robust person segmentation, we employ clustering based algorithm within a predefined activity zone, followed by task-specifie point cloud segmentation network. This offers superior general- ization across diverse environments compared to object detectors trained for narrow domains. We benchmark our tracking pipeline against the strong baseline ByteTrack (Zhang et al., (022) on the challenging FIFA soccer dataset (Jiang et al. (024). As shown in Table our method achieves superior performance across standard multi-object tracking metrics, demonstrating more stable and accurate tracking.
To comprehensively evaluate tracking performance, we adopt several standard metrics:
MOTA (Multiple Object Tracking Accuracy): An overall measure combining false posi- tives, false negatives, and identity switches. Higher is better, with 1 being the ideal value.
IDF1 (ID F1-Score): Measures the correctness of identity preservation. Higher is better.
IDs (Identity Switches): Counts how many times a tracked target loses its correct identity. Lower is better.
HOTA (Higher Order Tracking Accuracy): A unified metric that balances detection and association accuracy. Higher is better.
23

---

Published as a conference paper at ICLR 2026
Method
MOTA 1
IFF1 f
IDs 4
HOTA f
ByteTrack
0.9485
0.7652
33
0.8451
Ours
0.9883
0.8172
18
0.9975
Table 12: Multi-person tracking performance comparison on the FIFA dataset.
F LIMITATIONS AND FUTURE WORK
While SparkleMotion demonstrates robust performance, it is important to discuss its limitations, which primarily stem from the final stage of the pipeline, the mapping of the learned Sparkle rep- resentation onto the parametric SMPL model. This dependency inherently restricts the framework human subjects and bounds its expressivity within the topological and shape space of SMPL. Consequently, the model may struggle with highly unconventional poses, significant non-rigid de- formations caused by loose clothing (e.g., long skirts or robes), or the detailed articulation of hands, which are beyond the representation capacity of the standard SMPL model. Furthermore, the front- end estimation the Sparkle representation itself can be challenged under conditions of severe and persistent occlusion extreme point cloud sparsity, where establishing reliable spatial correspon- dences becomes difficult.
These limitations, however, not fundamental the Sparkle representation itself but rather to its current instantiation within a human-specific, parametric modeling paradigm. This distinction opens several promising research directions. The most compelling future work lies in generalizing the core principle of Sparkle, the factorization an object's ' state into internal structural keypoints and external surface anchors, beyond the human domain. We hypothesize that this representation serves as a powerful and universal inductive bias for learning from point clouds, applicable to wide range of articulated and non-rigid objects, such as animals or robotic arms. Decoupling the Sparkle representation from SMPL could involve learning category-agnostic structural priors or em- ploying an implicit surface decoder to reconstruct detailed geometry, thereby overcoming the current constraints on shape and deformation. Additional avenues include the development of more sophis- ticated temporal models to enhance long-term consistency and the exploration self-supervised learning frameworks to reduce the reliance on extensive 3D annotations. In summary, while the current system pragmatically limited by its parametric backbone, the underlying Sparkle repre- sentation offers versatile and promising foundation for future research in general 3D dynamic understanding from sparse sensing.
USAGE OF LARGE LANGUAGE MODELS
During the preparation of this work, the authors used LLM solely for the purpose improving language fluency and checking grammatical emrors in previously written drafts. After using this tool, the authors review and edit the content as needed and take full responsibility for the intellectual content of the publication.
24


Reviews:
[{"review_id": "YtZoGYC2QJ", "reviewer": "Reviewer_ZhY4", "summary": "The paper proposes a solution for 3D human motion capture from sparse and noisy point cloud data. The paper introduces a novel human representation, Sparkle, that augments the conventional 24 skeletal joints with 32 anchors. It then uses a point-aligned tracker that employs body-part segmentation to obtain point-joint segmentation. The features are then passed to an anchor estimator to obtain the Sparkle representation. In the last stage, the SMPL parameters are first estimated from the Sparkle representation analytically, and then further refined using a lightweight network. The authors present a rigorous set of experiments across 11 datasets, covering both LiDAR and depth sensors in various challenging scenarios. The results demonstrate consistent state-of-the-art performance, particularly in reducing rotational error (Ang Err), which strongly supports the paper's core claims.", "questions": "1. Have you experimented with different numbers of anchor points? How about different methods of choosing the anchor points instead of PCA (e.g., manual or random selection)? The cited paper (Ma et al., 2023b) chose 64 virtual markers. Is there a particular reason for choosing 32 markers in this paper?\n2. What hardware was used to obtain 60FPS? Is it the same as the training hardware (A40)? Is this reported after the LiDAR/depth preprocessing?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The paper explains the technical details very well, is well-written, and easy to follow. The supplementary materials also addressed my technical questions.\n- While the paper takes some inspiration from several prior works (e.g., the addition of anchor points or the swing & twist decomposition of rotations), it pushes the field one step forward through adaptive combination.\n- The experiments, including both ablations and comparisons, are thorough, and the performance improvements show the effectiveness of the proposed approach", "weaknesses": "- In Figure 1.e, the labels for HuMMan-Point are repeated twice with different results across methods. The images are also of low quality, making 1.d. unclear. The paper would benefit from these being corrected.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tLrLibKzFQ", "reviewer": "Reviewer_j8qZ", "summary": "This paper presents Sparkle, a structured representation for human motion from 3D point clouds that explicitly separates internal kinematics (24 skeletal joints) and external geometry (32 surface anchors). Built upon this representation, the authors introduce SparkleMotion, composed of three modules:\n> a Point-aligned Skeleton Tracker (PST) predicting joint positions and point-->joint correspondences,\n> a Skeleton-guided Anchor Estimator (SAE) refining surface anchors via joint-conditioned attention, and\n> a Sparkle-based SMPL Solver (SSS) that analytically initializes pose via a swing-twist decomposition before learned refinement.\nExperiments cover 11 datasets across diverse sensing conditions (LiDAR, depth cameras, multi-view), showing state-of-the-art robustness under occlusion, domain shift, and close human–human interactions, while running in real time (~60 FPS). Extensive ablations validate the contribution of each module.", "questions": "> Could the authors clarify training datasets used for cross-sensor generalization (Table 3)? Are any experiments zero-shot across unseen sensors or datasets?\n\n> How sensitive is performance to the number and placement of anchors?\n\n> Does Sparkle yield improved data efficiency (e.g., same accuracy with fewer labeled frames)?\n\n> Have the authors analyzed swing-twist stability—when anchors/joints are nearly colinear or occluded?\n\n> What is the runtime and accuracy trade-off as the number of persons increases?\n\n> Are there examples where the representation fails (e.g., loose clothing, extreme articulation)?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "+ Robust and elegant factorization: The decomposition into skeletal and surface anchors, coupled with analytic swing-twist initialization, forms a coherent and interpretable representation.\n\n+ Comprehensive evaluation: Results on 11 datasets cover varied conditions (noise, occlusion, cross-sensor, multi-view), clearly outperforming prior methods.\n\n+ Well-designed ablations: Each module’s effect is isolated and justified (Table 5).\n\n+ Strong practical relevance: Real-time operation, privacy-preserving sensing, and robustness under domain shift make the method appealing for real-world deployment.\n\n+ Clear writing and high-quality visuals: The presentation is professional and the pipeline is easy to follow.", "weaknesses": "- Representation framing not fully realized.\nThe core novelty (i.e., the Sparkle factorization) could be elevated from an engineering mechanism to a representation-learning principle by analyzing identifiability or data-efficiency properties. Currently, the paper stops short of showing why the representation generalizes better.\n\n- Ambiguous training protocols.\nCross-sensor generalization (Table 3) may rely on dataset-specific training (App. A.1). A leave-one-dataset-out experiment would clarify whether the robustness arises from representation bias or tuning.\n\n- Anchor design sensitivity.\nThe fixed 32 anchors (via PCA) are never compared to other counts or selection schemes. A sensitivity study (16/48/64 anchors) would test how geometric coverage affects stability and accuracy.\n\n- Scalability and identity consistency.\nMulti-person handling is demonstrated visually but not analyzed quantitatively for identity tracking, throughput, or robustness under dense scenes.\n\n- Limited theoretical insight.\nThe analytic swing–twist solver is promising but lacks discussion of conditions for stability or uniqueness—analysis that could give the paper conceptual weight.\n\n- Overreliance on generated data for interactions.\nSome interaction benchmarks use point clouds synthesized by LIP, which might bias evaluation toward the rendering assumptions of that pipeline.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gvYPLq15a4", "reviewer": "Reviewer_xLUJ", "summary": "This paper proposes a structured representation for point clouds, which unifies skeletal joints and anchors. It also proposes a point cloud human pose estimation framework called SparkleMotion. In order to solve the problem that current point cloud MoCap methods have a hard time balancing robustness and expressiveness. The paper also conducted extensive experiments on multiple datasets , and achieves SOTA results. And it is also able to run in real-time.", "questions": "1. Could the authors provide more results on Sloper4D, including performance-distance analysis showing that J/V error increases with distance between the subject and LiDAR, a complete ablation experiment, and more qualitative results, especially at greater distances?\n2. Please clarify the implementation details of the SAE cross-attention module. Which description is correct? If the description in the main body is correct, please explain in detail why the anchor features from Ainit represent the actual local geometry.\n3. If the initial Jop bias is too large, will it cause Ainit initialization to fail due to a large error?\n4. How does the evaluation metric in the paper handle the case of missing targets in the supplementary material video (4:26)? Could you discuss the recall rate of detection in occluded scenes (soccer, basketball)?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The author puts forward the sparkle framework, which can unify the internal structure and anchor points, and solve the problem between robustness and expressiveness in point cloud mocap.\nThe author's sparklemotion has achieved SOTA in multiple benchmarks, which shows the effect and generalization of the model.\nThe author designed a system that can run in real time with 60fps, which has high value in use.", "weaknesses": "1. First, I have questions about how PST solves the bias problem in the initial joint-related point sets. If the initial segmented point subsets are wrong, the points will be assigned to the incorrect joints, and this will affect the following local refinement.\n2. Second, because of the problem in point 1, the predicted joints $J_{op}$ from PST might have bias. This bias will be amplified in the SAE module. As the author said, the initial anchor $A_{init}$ is easily affected by the errors in $J_{op}$.\n3. Third, the definition of K (Keys) and V (Values) in the SAE module is not clear. In Section 3.1.2, the author claims to use joint features from PST as Q (Query), and $A_{init}$ from SAE generates anchor features to be K and V. The author also claims this anchor feature represents the \"actual local geometry\". However, the anchor feature comes from $A_{init}$, and $A_{init}$ comes from $J_{op}$ through a linear map. This forms a circular loop. The model is finally using the predicted $A_{init}$ (from $J_{op}$) to query the predicted $J_{op}$ (or its features), instead of querying the real point cloud geometry information. This contradicts what the author said in the Introduction. The author claimed Sparkle is a \"geometry-aware\" module, which should use the original point cloud data to ensure surface fidelity.\n4. Also, when I checked the Appendix for more details, I found that the description in Appendix A.3.2 is completely opposite to the description in Section 3.1.2. In A.3.2, the author describes another attention mechanism, using $A_{init}$ as Q, and joint-centric features extracted from the point cloud as K and V. This needs to be clarified.\n5. Fifth, the PST module fails under severe occlusion. The author claims robustness in multi-person soccer and close-interaction scenarios, but the supplementary video (especially at 4:26) exposes a key failure. When the crowd causes severe occlusion, the model completely ignores (misses) a visible person. What causes this?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "g2Id1QgQwM", "reviewer": "Reviewer_6jTA", "summary": "The paper proposes Sparkle, a unified intermediate representation for point cloud–based human motion capture that combines skeletal joints and surface anchors. Built into the SparkleMotion framework, it achieves state-of-the-art accuracy and robustness across 11 diverse datasets, with real-time performance.", "questions": "1. Could Sparkle be adapted to non-human articulated shapes (e.g., animals, robotic arms)? Is the representation intrinsically tied to SMPL’s structure, or could it be learned from scratch for new kinematic templates?\n\n2. In multi-view fusion, the confidence weighting seems heuristic—have the authors considered learning this weighting jointly during training for potentially better view integration?\n\n3. For noisy, sparse LiDAR cases, how sensitive is PST segmentation to the initial prediction errors? Would a joint iterative optimization of PST and SAE be beneficial?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The Sparkle representation presents a notable conceptual improvement in intermediate representations for point cloud–based motion capture, by combining structural priors with geometric detail.\n\nThe PST and SAE modules are thoughtfully designed to address limitations of purely skeletal or point-based approaches, and the SSS solver makes effective use of geometry-aware initialization.\n\nThe experimental evaluation is extensive, covering multiple sensors, diverse scene complexities, and multi-view setups, and demonstrates consistent gains over strong baseline methods.\n\nThe demonstrated real-time performance in practical, in-the-wild scenarios, together with its privacy-friendly nature, enhances the potential impact for real-world applications.", "weaknesses": "1. **SMPL model dependency limits generalization**: While Sparkle is conceptually general, its current instantiation and evaluation rely entirely on the SMPL human body model. This introduces constraints on skeleton topology and shape space, limiting direct applicability to non-human subjects or highly non-standard apparel and poses. Expanding experimental scope could strengthen claims about representation-level generality.\n\n2. **Ablation granularity is somewhat coarse**: Table 5 support component necessity, but do not fully isolate why specific designs outperform alternatives. For example, PST’s residual offset refinement is compared only to direct prediction rather than other refinement paradigms; similarly, SAE’s initialization and refinement contributions could be examined through more diverse baseline strategies.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)