ToF-IP: Time-of-Flight Enhanced Sparse Inertial Poser for Real-time Human Motion Capture
Authors: Yuan Yao
Shifan Jiang
Yangqing Hou
Chengxu Zuo
Xinrui Chen
Shihui Guo
Yipeng Qin
Venue: NeurIPS 2025 poster ()
Keywords: IMU
ToF
Motion Capture
Primary area: applications

Abstract: Sparse inertial measurement units (IMUs) provide a portable, low-cost solution for human motion tracking but struggle with error accumulation from drift and sensor noise when estimating joint position through time-based linear acceleration integration (i.e., indirect measurement). 
To address this, we propose ToF-IP, a novel 3D full-body pose estimation system that integrates Time-of-Flight (ToF) sensors with sparse IMUs. 
The distinct advantage of our approach is that ToF sensors provide direct distance measurements, effectively mitigating error accumulation without relying on indirect time-based integration. 
From a hardware perspective, we maintain the portability of existing solutions by attaching ToF sensors to selected IMUs with a negligible volume increase of just 3\%.
On the software side, we introduce two novel techniques to enhance multi-sensor integration: (i) a Node-Centric Data Integration strategy that leverages a Transformer encoder to explicitly model both intra-node and inter-node data integration by treating each sensing node as a token; and (ii) a Dynamic Spatial Positional Encoding scheme that encodes the continuously changing spatial positions of wearable nodes as motion-conditioned functions, enabling the model to better capture human body dynamics in the embedding space.Additionally, we contribute a 208-minute human motion dataset from 10 participants, including synchronized IMU-ToF measurements and ground-truth from optical tracking.
Extensive experiments demonstrate that our method outperforms state-of-the-art approaches such as PNP, achieving superior accuracy in tracking complex and slow motions like Tai Chi, which remains challenging for inertial-only methods.

ToF-IP: Time-of-Flight Enhanced Sparse Inertial Poser for Real-time Human Motion Capture
Yuan Yao!
Shifan Jiang'
Yangqing Hou'
Chengxu Zuo
Xinrui Chen
Shihui Guo
Yipeng Qin²
School of Informatics, Xiamen University, China School of Computer Science & Informatics, Cardiff University, UK
Abstract
Sparse inertial measurement units (IMUs) provide a portable, low-cost solution for human motion tracking but struggle with error accumulation from drift and sensor noise when estimating joint position through time-based linear acceleration integration (i.c., indirect measurement). To address this, we propose ToF-IP, novel 3D full-body pose estimation system that integrates Time-of-Flight (ToF) sensors with sparse IMUs. The distinct advantage of our approach is that ToF sensors provide direct distance measurements, effectively mitigating error accumulation without relying on indirect time-based integration. From a hardware perspective, we maintain the portability of existing solutions by attaching ToF sensors selected IMUs with a negligible volume increase of just 3%. On the software side, we introduce novel techniques enhance multi-sensor integration: (i) Node- Centric Data Integration strategy that leverages a Transformer encoder to explicitly model both intra-node and inter-node data integration by treating each sensing node as a token; and (ii) a Dynamic Spatial Positional Encoding scheme that encodes the continuously changing spatial positions of wearable nodes as motion-condiiioned functions, enabling the model to better capture human body dynamics in the embedding space. Additionally, we contribute a 208-minute human motion dataset from 10 participants, including synchronized IMU-ToF measurements and ground- truth from optical tracking. Extensive experiments demonstrate that our method outperforms state-of-the-an approaches such as PNP, achieving superior accuracy in tracking complex and slow motions like Tai Chi, which remains challenging for inertial-only methods.
Introduction
Sparse inertial measurement units (IMUs) have emerged as promising solution for human motion
tracking due to their portability, low cost, and camera-free nature [8] [ 20 42 However, despite their potential, sparse IMUs face inherent numerical challenges due to their indirect method of position estimation. Specifically, sparse IMUs estimate velocity and position by time-based linear acceleration integration, a process highly prone to error accumulation from drift and sensor noise. further amplified by the task's reliance human body forward kinematics, where positional inaccuracies intermediate joints propagate along the kinematic chain, leading to greater errors at terminal joints. Consequently, accurately tracking subtle positional changes key joints during low-velocity motions (where the motion-signal-to-nois ratio is low) and enhancing the long-term stability of sparse IMU systems remain persistent and critical challenges in this field [37
These
"Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

LEME
Taf Clii
Sidefiip
Loy cretioooion motion
Van Com paze with and wathour Tuu
Table TebieTennis
ToF-IP integrates distance maps from four Time-of of--Fi (ToF) sensors to overcome the drift and error accumulation inherent in inertial-only motion capture, enabling more accurate and stable motion tracking even for challenging motions such as slow, controlled sequences (e.g., Tai Chi) and rapid, complex actions (e.g., sideflips).
Figure I:
To date, most existing methods foeus on providing software-based solutions to the aforementioned challenge. For example, deep learning approaches, such as bidirectional RNNs, have been developed to regress IMU data into pose sequences [8]. Subsequent RNN-based methods further improved pose prediction accuracy and integrated global position estimation [36), with some incorporating more precise dynamic models 37 Beyond RNNs, alternative architectures have emerged, including attention-base based models for capturing physical motion during stationary phases [II] Nevertheless, software solutions are inherently limited, as they still rely on indirect pose estimation and can only mitigate, rather than fully resolve, the problem of error accumulation.
In this paper, we present ToF-IP, a novel 3D full-body pose estimation system that addresses the abovementioned challenge by integrating Time-of-Flight (ToF) distance sensors with sparse IMUs. The distinct advantage our approach is that ToF sensors provide direct distance measurements without relying indirect time-based integration, offering learnable solution space constraints for the position of limb-end joints. A key innovation of our ToF-IP lies in its hardware design, which maintains the lightweight and portable nature of existing sparse IMU systems. Specifically, without altering the standard 6-IMU layout, we incorporate highly integrated ToF sensors based on single- photon avalanche diodes (SPAD) (26] directly onto the IMU circuit boards equipped at left forearm, right forearm, left lower leg. and right lower leg of the human body, ensuring minimal impact on wearability. On the software side, we fully harness the potential of ToF-inertial sensing by proposing a unified Transformer- based framework with two key innovations. First, we introduce a Node-Centric Data Integration strategy that explicitly captures the hierarchical structure of multi-sensor data. Unlike prior methods that flatten all sensor inputs into a single vector-wise organization (discarding the spatial and structural semantics of node), our approach represents each sensing node as an independent token. This token is constructed through intra-node data integration of ToF depth, IMU acceleration and orientation data. These node tokens are then contextually integrated using the self-attention mechanism of a Transformer encoder, which naturally facilitates inter-node communication and dynamic weighting based on
dependencies. Second, we propose a Dynamic Spatial Positional Encoding (Dyn-PE) method tailored to the unique challenges of wearable sensing. Unlike traditional positional encodings NLP or vision tasks that assume static or grid-bases positions, the spatial configuration of wearable nodes evolves continuously with human motion. To capture this, Dyn-PE models each node's 's position as learnable function of global motion signals, generating time-varying encodings that reflect the node's 's physical displacement in space. This dynamic encoding enhances the model's spatial awareness, allowing it to better resolve ambiguous interactions and motion patterns across nodes. Extensive experimental results show that, compared to state-of-the-art (SOTA) methods, our approach significantly improves joint position estimation, achieving superior accuracy in tracking complex and slow movements like Tai Chi.
In summary, our contributions include:
We design an in-situ enhanced bimodal wearable sensing platform for 3D full-body tracking, retaining the conventional layout of 6 sensing nodes. The platform allows for the flexible use of either single IMU sensing IMU+ToF bimodal sensing, with only a 3% increase in volume.

---

We propose ToF-IP. novel Transformer based inertial motion capture framework that introduces two key innovations on the software side: (i) Node-Centric Data Integration strategy that preserves the structural semantics of multi-sensor data by treating each sensing node as token and hierarchically integrating intra- and inter-node information via self-attention; and (ii) Dynamic Spatial Positional Encoding scheme that models the continuously evolving spatial positions of wearable nodes as motion-conditioned functions, enhancing spatial awareness and robustness to body movement variations. Our approach demonstrates substantial improvements over state-of-the-an methods, delivering higher positional precision and more accurate joint angle estimation, particularly in the upper limbs and legs.
We propose ToF-IP-DB, a large dataset containing over 20 types of motion activities, 208 minutes (749,000 frames) collected from 10 participants (3 male, 7 female), including dynamic motions such as dances and aerobics, well as slow-paced movements like Tai Chi and Baduanjin. This dataset uniquely combines synchronized ToF distance maps, 6-DoF IMU signals, and SMPL reference poses, with GT motion data.
2 Related Work
2.1 Pose Estimation Using Inertial Sensors
With the rapid advancement of MEMS technology 91. IMUs (Inertial Measurement Units) have become smaller, more power-efficient, and affordable. This has led to numerous works leveraging IMUs for human pose estimation. Despite their independence from external environments, the working principle of IMUs-using accelerometers, gyroscopes, and magnetometers irientation-limn their accuracy. In the commercial market, motion capture systems employing 17-19 IMUs for human pose estimation exist [35] 23), but they require operation in uniform magnetic field environments and have limited usage durations, as accumulated integration errors can lead to model collapse.
compute
The pursuit of lightweight solutions has spurred research into using sparse IMUs- typically six sensors placed at limb extremities. The advent of the SMPL [18] and AMASS [19] datasets has enabled the creation of large-scale mocap//MMU--llgned datasets. Synthetic continuous acceleration and rotation data were generated by placing virtual IMUs on specific body parts in the AMASS dataset, facilitating pose estimation in both offline [3 and real-time settings. Deep learning methods, such bidirectional RNNs, were designed to regress IMU data to pose sequences [8]. Building on this, RNN-based approaches have improved pose prediction accuracy and incorporated global position estimation [36], with some integrating more precise dynamic models, such those in [37 Other methods explore alternative network architectures, including attention-based models for learning physical motion during stationary points [T]] and spatiotemporal modules for more accurate pose estimation (34). Efforts have also been made to enhance comfort and convenience. For example, some methods use VR headsets, smartphones, and other wearable devices estimate upper-body poses and predict lower-body movements. Zuo et al. integrated IMUs into loose clothing [42). effectively regressing upper-body poses while mitigating artifacts caused by fabric-induced jitter.
Despite such success, challenges remain in pose estimation using IMUs alone 22 10
28 These
include IMUs' inability directly measure velocity position, accumulation of drift errors [15] 13, and reliance on forward kinematics models 25 composed of bones and joints. Sparse IMU layouts- those at the extremities of limbs, such hands and legs-face difficulties due to the higher degrees of freedom and heavier prediction tasks borne by individual sensors.
2.2 Time-of-Flight Distance Sensors
To mitigate these issues, recent research has explored hybrid solutions that integrate additional sensor modalities 17]. One common strategy is incorporate global positioning or localization techniques to provide absolute positional constraints [7]. For instance, Zihajehzadeh et al. [4] combined IMUs with ultrawideband (UWB) localization eliminate yaw angle drift in lower-body tracking, leveraging UWB's absolute position measurements to correct inertial estimates. Similarly, Liu et al. [16] used micro-floo flow sensor to estimate motion velocity, enabling accurate extraction of gravitational acceleration from accelerometer data and improving posture tracking stability.
3

---

IMU Acceleration Noise
Acceleration Measuremcal
Velocity Measuremenl
Position Measurcmecl
10.0
IMU
10
GI
GT
GT
Auae Aonolonation (awar)
IMU Ace (mis")
Volucity (mo))
Velucity
Position (m)
Poviion
-10.0
-10.0
10.0
Time
(6)
Figure 2: Illustration of error accumulation in velocity and position. Even slight noise in IMU acceleration leads to significant integration errors velocity and position within just 4 seconds. Acceleration signals are taken from the TIC dataset 433. with ground-truth provided for comparison.
Another class of methods exploits vision- and depth sensors compensate for inertial drift 30 32 5 21 38). Depth sensors such LiDAR and structured-light cameras have been explored for markerless tracking. ToF sensors broadly refer to methods for precise distance measurement based on the time taken by light pulses continuous waves travel [6 [2] Unlike vision-based methods, ToF sensors are resistant to variations lighting and occlusions, providing robust depth measurements in challenging conditions [33 24). Some studies [27 4 have integrated ToF sensors into external environments for global pose estimation, but their application in wearable systems remains underexplored.
By carefully balancing factors like power consumption and heat dissipation, we augment the con- ventional SiX-IMU sparse layout with four low-resolution depth Time-of-Fligh (ToF) sensors. This hardware integration enhances inertial pose estimation by capturing inter-limb distances and contact points with the environment, providing real-time, in-situ constraints that mitigate positional drift while preserving the wearability and portability characteristic of traditional inertial tracking systems.
3 Inherent Limitation of Sparse Inertial-only Motion Capture
Under sparse IMU configuration, only subset of joint orientations can be directly measured. To compensate for missing measurements, acceleration data is commonly used as an additional input (8). it carries implicit cues about joint positions that can aid in inferring unobserved joint orientations. In principle, joint positions can be obtained by double-integrating the acceleration signals over time. However, in practice, real-worle acceleration measurements are prone to various sources of error, such as sensor noise and signal drift, inevitably resulting in significant error accumulation over time:
Proposition 3.1 (Error Accumulation Analysis). Following standard stutistical practice, assume that the acceleration meusurement error at any timestamp T E (0, 1) follows a normal distribution ea (T) N(, o²). Then, we have:
Distribution of joint velocity enror: (t(t) NNut, x2)
Distribution of joint position error: (t) N(sutt 20222)
Proof. The proof provided in Section 2 of the supplementary material.
Proposition 3.1 and Fig. 2 shows that the joint position error grows quadratically over time, highlight- ing an inherent limitation of sparse inertial-only solutions.
4 Method
To address the inherent limitation of sparse inertial-only solutions discussed in Sec. we propose integrating Time-of-Flight (ToF) sensors, which provide direct distance measurements mitigate error accumulation from tneebased integration.

---

ToF coF-lnertial Sensing Prototype
ToF-Inertin Motion Capture
Motion Estimator
Transtormr Encoder
fp
40 am
Orleatations
00Cymank:
Rootioall
(+)
Accelerations
s
6,,(1)
65
Ulreo Node Fuson
ToF
X,.1
Distance Mags
Figure 3: Illustration of Our ToF-inertial Sensing Prototype and Motion Capture Method. Left: Our ToF-Inertial sensing prototype comprises IMU-attached sensing nodes, with 4 of them (located on the left / right forearm and left right lower-leg) additionally integrated with ToF sensors. Middle: Our ToF-Inertial motion capture method comprises two novel techniques: (i) a node-centric data integration strategy based Transformer encoder, encompassing intra-node and inter-node data integration between ToF-Inertial sensor; (ii) a dynamic spatial positional encoding to adapt to the dynamic changes in the spatial positions of sensing nodes during motion capture; Right: Three sequentially connected LSTM networks (fv, fp. fa) serve as motion estimators to transform the integrated sensing node data into human body movements.
4.1 ToF-Inertial Sensing Prototype
As shown Fig. (left), maximize compatibility with existing inertial-only solutions and minimize impact on wearability, i) we adopt the standard 6-node layout used in prior works [36] 37 II 39 and place 6 IMUs on the left/right forearms, left/rihht lower legs, pelvis and head, respectively; ii) we integrate 4 ToF sensors into the IMU nodes on the left/right forearms and left/right lower legs. Specifically, we mount the ToF sensors on the inner wrists and rear ankles to capture distance measurements from distal joint endpoints to nearby body parts and the ground. Note that omit ToF sensors from the head and pelvis nodes, as these positions seldom observe relevant surfaces.
4.2 TnF-Inlrtial Motion Capture
Overview. As shown in Fig. (middle, right), our ToF- Inertial Motion Capture framework adopts Transformer-basee architecture that incorporates two key innovations for integrating IMU and ToF data: (i) Node-Centric Data Integration, which explicitly models intra- and inter-node interactions through tokenized node representations; and (ii) Dynamic Spatial Positional Encoding, which encodes the time-varying spatial positions of sensing nodes using motion motion-conditioned functions. Following we further employ cascade of three LSTM networks as motion estimators.
36 37
4.2.1
Node-Centric Data Integration
Conditions for Effective ToF Integration. Although the direct distance measurements provided by ToF establish a data foundation for improving joint position estimation, their effectiveness hinges on two key data integration conditions:
[ntra-node Integration] Each ToF sensor must integrate orientation and acceleration data from its co-located inertial sensor within the sensing node to determine the viewing direction and motion state information;
[Inter-node Integration] Since both the motion of the sensing node itself and the captured object can cause changes in the ToF distance map, integrating measurements from other sensing nodes is required distinguish absolute motion from relative motion.
I Limitation of Existing Data Integration Method. Existing methods typically use fully-connected networks (FCNs) for data integration. This approach directly flattens data from multiple nodes into single vector, losing the inherent data structure organized by sensing nodes and overlooking
5

---

intra-node integration. Formally, consider an embedded feature fi from an FCN layer, we have:
Flatten(x(1) ...., .())) W + bç
(1)
where (x(1)) ..... x! | are the data of the N nodes. In this formulation, the linear combination of x(0),) 1, ...., N inherently achieves inter-node integration, but overlooks the intra-node data integration within each x()).
Node-Centric Data Integration. To address this challenge, we propose replacing fully- funne-con networks used in previous methods with a Transformer Encoder. In this scheme, each sensing node is converted into an independent token, with all encoding performed at the token (node) level. Consider the encoding process for an arbitrary output token zy of node n, which unfolds two structured steps to explicitly perform intra-node and inter-node data integration:
1) Intra-node Data Integration via Tokenization. The multi-modal data of each node x(") [d(m), a(m), R(a)] is encoded into intra-node token (n) Ra follow:
(n) intra fo (x(")) PB(*)
(2)
where fr is a tokenize function, dmodel is size of token. d E R A EERR and R E R are ToF depth maps, IMU acceleration and orientation, respectively. PE is positional encoding of the n-th sensing node. For IMU-only sensing node (head and hip), is set to zeros.
2) Inter-node Data Integration via Self-Artention. The intra-node tokens { (N)) are then integrated via Transformer Encoder's 's self-attention mechanism, which computes inter-node interaction weights E R N applied to intra-node tokens:
inter
inttra
+
Σ
Aig A () (nitm W.)
where W. projects tokens into value space, and the inna is residual connection term. Then each ( will go through layer norm (LN) and feed-forward network (FFN) in the Transformer Encoder and concatenated to produce the final embedding vector Zeer RN (duaas))
4.2.2 Dynamic Spatial Positional Encoding
Static Positional Encoding. As Eq. 2 shows, positional encoding PE is a fundamental process to incorporating position information into tokens. Existing static positional encoding methods assign static positional values o to each token via an addition operation. For example, the static positional
encoding in the original Transformer 22 is as follows:
P 21 isin (wq: Opos) , cos (w: Opos)], Opos= =n, w, 10000 2i/dans
where is the index of the input token, dmuseel is the dimension of the token, Opos is the static positional value determined by the token index (sequential position). The n-th sensing node token processed by static positional encoding then be represented 2(n)) (x(()) PE(N), where (p:(m) is static positional encoding calculated by Eq.
Our Dynamic Spatial Positional Encoding, However, unlike the static and discrete positions in natural language processing and computer vision applications (such as word order or pixel grid coordinates), the positions of sensing nodes are dynamic and continuous spatial positions that change with human body movements. Considering these characteristic, we model the positions sensing nodes continuous functions of motion signals and propose dynamic spatial positional encoding (Dyn-PE) as follows:
(n...22 26+ (t) = jsin (wi - )n cos (w, on (n)l. dn(() = fx((xn()) wi = 10000 2i/dmaa dyn
(5)
where dh (1) are the dynamic positional value of sensing node at time (n = 1, 1,2, 6). Xmn d- (1), (1). R- (1) are data of all six sensing nodes, serving as human motion signal. The fg...., f are position estimation functions implemented with a 2-layer MLP. Then, our Dynamic (x)
Spatial Positional Encoding can be integrated into Eq.
as: Tinn (x(x()) PE(M) (t).

---

4.2.3 Motion Estimators
Following 36 37 39, we feed the sensor data embedding ANcI the shared input to three sequential
motion estimators for joint velocity R joint position p E IRJx3 and joint rotation yp IR. fx
vtt = fv (Zxci(t))
p(t) = fp (ZNci(t), v(t))
(t(t) = fup (t)
(6)
where J 18 is the total number of tracked joints, .v. fp. fip are RNN-based motion estimators.
These motion estimators are trained in supervised manner using the following motion loss:
mation
vit) v GT (t)ll + llp(t) -p ST (t)l + 11 - (t)l
(7)
where GT denotes the ground truth value.
4.2.4 Global Translation Tracking
The global translation tracking in this work is powered by velocity output of Motion Estimator and SMPL kinematic model. Specifically, we first compute the estimated velocity of the four joint endpoints (left and right forearms and lower legs) equipped with TnF-inertial nodes and convert into pseudo stationary label 9s:
if (10ll <e
(t)
1
if e ( < +.2
(8)
0.2
0
otherwise
Where denote endpoint index, the e is a cut-off threshold to filter small jitter in || (t)ll2 (we use 0.05m/s in this work). Subsequently, based yttt) provided by the Motion Estimator, calculate the root translation relative to the endpoints is stationary, denoted as 8PX (4), using the forward kinematics:
(1.2.3.4) (1) FK(p(tt- At)) - FK(((())
FK
Where A/ is time gap of continuous captures. Then we define FK-based translation follow:
SFK(1) Ei dh (t) (t)
(10)
Σi 9 (1)
Similar to previous works 36), we fusion the SFR (t) with root velocity provide by Motion Estimator to obtain the final translation estimation:
NNN (t) Uroot At) At
(11)
s st) (1 - 9m) SFK (t) + 9m (t)
Where On min (1) air, sliding), and 8 NNN denotes translation estimation base on neural network (Motion Estimators).
denotes the pseudo label of full-body moving de.g.jummiing on the
Experiment
5.1 Experimental Setup
Synthetic Dataset. We leverage the AMASS dataset [19] to synthesis a large-scale paired ToF-IML Motion data for motion estimators pre-training, which includes both IMU and ToF data simulation.
IMU Data Simulation: Similar to previous works g 36
the IMU orientation and acceleration are calculated based on the global joint orientation of the SMPL [8] model and the
trajectory of the selected mesh vertices.
ToF Data Simulation: ToF data simulation is implemented using Unity. We simulate ToF sensor with virtual depth cameras positioned the rendered SMPL. body, aligned with the hardware wearing setup. The original depth maps are then down-sampled into x fit the configuration of ToF. More detailed settings provided in the supplementary materials.
7

---

Table I: Comparison of methods DIP and ToF-IP-DB datasets across multiple metrics.
We evaluated our ToF-IP on DIP with additional ToF synthesis.
ToF-IP-DB
DIP (with synthesis ToF)
Method
SIP Err
Ang Eir
Pos Err
EndPos Err
Jitter
SIP Err
Ang Err
Pos Err
EndPos Eir
Jitter
Transpose
20.73
14.51
7.58
13.58
0.18
17.06
8.86
6.03
8.73
1.11
TIP
20.37
14.58
7.63
13.97
0.17
16.90
9.07
5.63
8.27
1.56
PIP
20.22
13.85
7.32
12.77
0.12
15.33
8.78
5.12
7.78
0.17
DynalP
19.04
13.33
7.26
13.05
0.16
13.78
7.07
4.98
7.44
0.18
PNP
18.52
13.23
6.86
12.39
13.71
8.75
7.49
0.17
ToF-IP(Ours)
17.26
12.09
6.31
11.41
0.12
13.62
6.75
4.59
6.65
0.17
ToF-IP-DB Dataset. We collected full-body motion capture dataset, containing over types of movements, 208 minutes (749,000 frames) from 10 participants (3 male and 7 female) with heights ranging from 170 em 185 em. All participants were informed about the purpose of the experiment and signed consent agreements. Participants were required to perform the following steps for data collection:
Simultaneously wears optical motion capture suit (with multiple optical markers attached) along with our 6-node Ine-iilrtial motion capture prototypes.
Performs a T-pose for IMU calibration.
During data collection, the participant is asked to perform diverse types of motion, e.g., dances, aerobics, and daily social activities. Sensor data and motion data are collected synchronously at 60Hz. The motion data is captured using the NOKOV marker-based based optical motion capture system, including full- bill-body pose and global translation. Each collection session lasts 6-10 minutes.
Training Settings. All our experiments run on a PC with Intel(R) Core(TM) i7- 1--33700KF CPU and an NVIDIA RTX 4080 GPU. The model implemented using PyTorch 1.12.1 with CUDA 11.3. We use the Adam optimizer with a learning rate of Ir = 1 x 10-3 and weight decay of Ir x 10-6 during n epochs training. The batch size was set to 512.
Metrics. We use the following five error metrics to evaluate the accuracy and quality of captured motion: I) Angular Error (9), which represents the global rotation error of all joints; 2) Positional Error (cm). which is the joint position eтror of all joints; 3) SIP Error (9), defined as the global rotation error hips and shoulders; 4) Endpoints Positional Error (cm), positional errors the four ToF sensor attached joints (the left and right wrist and ankle); 5) Jitter (km/s"), denoting the jerk (time derivative of acceleration) of all body joints in the global space.
5.2 Comparison with SOTAS
Quantitative Results. Table [ shows the evaluation results on the ToF-IP-DB and DIP g dataset. The results demonstrate that our method consistently outperforms existing approaches across all metrics, particularly in SIP angular егrоr and positional eтrоr. The reduction in SIP error signifies more accurate tracking of upper arm movements and knee lifts, which is attributed to the ToF-enabled improvement in joint position estimation as we expected.
Qualitative Results. As illustrated in Fig. we selected Tai Chi and Baduanjin movements from the ToF-IP-DB dataset, which are characterized by long durations and gentle velocitiss-conditoonn conditions where inertial-only position tracking inherently fails to measure accurate joint position. contrast, our method leverages ToF-derived direct distance measurements and introduce inter-joint distance constraints, leading marked improvements overall pose estimation accuracy.
5.3 Ablation Study
Effectiveness of ToF Integration. As shown in Table. 2 removing ToF leads to a noticeable decrease in all metrics, specifically terms of EndPos Error, this validates our core motivation that the direct distance measurements from ToF sensors can improve the estimation accuracy of joint endpoints, thereby enhancing the estimation of non-sensor-attact measured joints (lower SIP). Notably, introducing ToF without using the proposed Node-centric Data Integration (NCI)

---

DynalP
PNP
OURS
Ground Truth
Figure 4: Qualitative comparisons with the state-of oftte-of-the-rt methods on our ToF-IP-DB dataset. We highlight the joint-to-joint distances in the ToF's line-of-sight direction.
w/o ToF
w/ ToF
GT
DA X
D
Figure 5: Qualitative comparison of results with and without ToF direct distance measurements, The pose estimation samples are from Case and Case 4 in Table
fails to achieve the desired improvement (Case vs. Case 2), demonstrating the necessity of the proposed NCI for our ToF-Inertial motion capture framework. Qualitative results in Fig, 5 demonstrate how ToF integration significantly enhances inertial-only motion capture. The direct distance measurements provided by ToF effectively reduce the estimation errors of joint positions, thereby ensuring continuous and accurate pose estimation.
Table 2: Ablation study results on DIP and our ToF-IP-DB datasets (ToF integration).
ToF-IP-DB
DIP (with synthesis ToF)
Case
ToF
NCI
SIP Frr
Ang Eir
Pos Fir
EndPos Fir
Jitter
SIP Frr
Ang Eir
Pos Er
EndPos Frr
Jitter
18.87
13.14
6,88
12.27
0.07
16.34
7.64
5.80
8.49
0.13
18.95
6.92
12.39
0.13
16.10
5.47
8.08
0.17
17.90
13.18
6.81
12.06
0.13
15.52
7.30
5.30
7.79
0.17
17.26
12.09
6.31
11.41
0.12
13.62
6.75
4,59
6.65
0.17
Effectiveness of Dynamic Spatial Positional Encoding. As shown in Table 3 our proposed Dynamic Spatial Positional Encoding consistently outperforms both the traditional static encoding and its learnalle variant (where parameters are optimized during training but remain fixed during inference) 2 demonstrating the effectiveness of modeling position in positional encoding for ToF-Incrtial motion capture, Fig further supporting the dynamic nature of our encoding scheme.

---

Left Witss
Root LefiAable
0101
Righe Aride
Frame:50
Frame:214
Frame:424
Frume:618
Frame
Figure 6: Quantitative visualization of the temporal dynamics of (t) across different motion types.
Each curve represents the thaase-based positional encoding of a specific sensing node over time.
Table 3: Ablation study results on DIP and our ToF-IP-DB datasets (positional encoding).
ToF- JuF-IP-DB
DIP
Positional Encoding
SIP Er
Ang Err
Pos Err
EndPos Err
Jitter
SIP Err
Ang Err
Pos Err
EndPos Err
Jitter
Static
17.76
12.52
7.60
11.98
0.13
14.20
6.87
4.77
6.95
0.17
Static (Leamable)
17.56
12:20
7.36
11.56
0.14
13,95
6.85
4.62
6.73
0.18
Dynamic Spatial (Ours)
17.26
12.09
6.31
11.41
0.12
13.62
6.75
4.59
6.65
0.17
Limitations
Despite the superiority our approach, several limitations highlight avenues for future research. The performance of our method is contingent on the availability and reliability of ToF--based distance measurements, which may be compromised scenarios with occlusions, or limited field of view. Additionally, the inherent noise in ToF sensors introduces jitter in pose estimation, reflected in our results. Furthermore, the current evaluation is conducted in controlled environments, and the generalization of our approach to more diverse and dynamic real-world scenarios remains to be validated. Future work could focus on improving robustness to ToF sensor limitations through hybrid models, reducing noise with advanced filtering techniques.
7 Conclusion
We introduced ToF-IP, a novel ToF-inertial motion capture system that overcomes the inherent limita- tions of sparse IMUs by integrating direct distance measurements from lightweight, body-mounted ToF sensors. Through a hardware-efficient design and unified Transformer-baat ToF-IP achieves accurate joint position estimation while preserving the portability and wearability of existing IMU systems. Our software contributions, including Code-Centric Data Integration and Dynamic Spatial Positional Encoding, enable structured multi-sensor integration and dynamic spatial awareness, which are critical for handling complex, low-velocity, and non-linear human motions. Extensive experiments validate ToF-IP's effectiveness across diverse movement scenarios, setting a new standard for hybrid sensor-based human motion tracking.
framework,
8 Acknowledgments
The authors would like to thank Qiannan Cao, Yingqi Yang. Hui Zou, Lishuang Zhan, Jiabao Gan, Yile Pan, Ziyi Shan, Ran Li, Bingling Liu, Wenjing Wu, Jiaqi Li, Yuenan Ji, Ziqian Huang and Shuyang Xing for their help on live demos and dataset collection. This work was supported by the National Natural Science Foundation of China (Nos. 62472364, 62072383), the Public Technology Service Platform Project of Xiamen City (No. 3502720231043) the Xiaomi Young Talents Program Xiaomi Foundation, and the Fundamental Research Funds for the Central Universities (No. 20720240058, "Young Eagle Plan" Top Talents of Fujian Province). This work was also supported by the National Key R&D Program of China (No. 2023FFC330560).
10

---

References
[1] Yiming Bao, Xu Zhao, and Dahong Qian. Fusepose: Imu-vision sensor fusion in kinematic space for parametric human pose estimation. IEEE Transactions Multimedia, 2022.
[2] Jacob Devlin, Ming- Meng-WWiioe Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidi- rectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume flong and short papers), pages 4171-4186, 2019.
[3] Jiawei Fang, Ruonan Zheng, Xiaoxia Gao, Chengxu Zuo, Shihui Guo, Yiyue Luo, et al. Fip: Endowing robust motion capture daily garment by fusing flex and inertial sensors. arXiv preprini arXiv:2502.15056 2025.
[4] Varun Ganapathi, Christian Plagemann, Daphne Koller, and Sebastian Thrun. Real time motion capture using single time-of offghht-fight camera. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 755-762. IEEE, 2010.
[5] Andrew Gilbert, Matthew Trumble, Charles Malleson, Adrian Hilton, and John Collomosse. Fusing visual and inertial sensors with semanties for 3d human pose estimation. International Journal of Computer Vision, 127:381-397, 2019.
[6] Miles Hansard, Seungkyu Lee, Ouk Choi, and Radu Patrice Horaud. Time-of-fight cameras: principles. methods and applications. Springer Science & Business Media, 2012.
[7] Hashim Hashim, AbdeIrahman EE Eltoukhy, and Kyriakos G Vamvoudakis. Uwb ranging and imu data fusion: Overview and nonlinear stochastic filter for inertial navigation. IEEE Transactions on Intelligent Transportation Systems, 25(1 251))359-366, 2023.
[8] Yinghao Huang, Manuel Kaufmann, Emre Aksan, Michael J Black, Otmar Hilliges, and Gerard Pons-Moll Deep incrtial poser: Learning to reconstruct human pose from sparse inertial measurements in real time. ACM Transactions on Graphics (TOG), 1-,5) 2018.
[9] Yunhan Huang, Arvind Sai Sarathi Vasan, Ravi Doraiswami, Michael Osterman, and Michael Pecht. Mems reliability review. IEEE Transactions on Device and Materials Reliability, 12(2):482 493, 2012.
[10] Yonatan Hutabarat, Dai Owaki, and Mitsuhiro Hayashibe. Quantitative gait assessment with feature-rich diversity using imu sensors. IEEE Transactions on Medical Roboticy and Bionics, 2(4):633-664, 2020. doi: 10.1 109/TRBBB 2020.3021132.
[11] Yifeng Jiang, Yuting Ye, Deepak Gopinath, Jungdam Won, Alexander W Winkler, and C Karen Liu. Transformer inertial poser: Real-time human motion reconstruction from sparse imus with simultaneous terrain generation. In SIGGRAPH Asia 2022 Conference Papers, pages 1-9, 2022.
[12] Andreas Kolb, Erhardt Barth, Reinhard Koch, and Rasmus Larsen. Time-of-flight cameras in computer graphics. In Computer Graphics Forum, volume 29, pages 141-159. Wiley Online Library, 2010.
[13] Jung Keun Lee, Edward Park, and Stephen N Robinovitch. Estimation attitude and external ac- celeration using inertial sensor mcasurement during various dynamic conditions. IEEE transactions on instrumentation and measwrement, 61(8)2266 2273, 2012.
[14] Li, Xiaofeng Liu. Zhelong Wang, Xu Zhou, and Ziyang Wang. Sensor combination selection for human gait phase segmentation based on lower limb motion capture with body sensor network. IEEE Transactions on Instrumentation and Measurement, 71:1-14, 2022. doi: 10.1 109/TIM.2022.320199
[15] Gabriele Ligorio and Angelo M Sabatini. A novel kalman filter for human motion tracking with an inertial-based dynamic inclinometer. IEEE Transactions on Biomedical Engineering, 62(8):2033.2043, 2015.
[16] Shi Qiang Liu, Jun Chang Zhang, and Rong Zhu. A wearable human motion tracking device using micro flow sensor incorporating a micro accelerometer. IEEE Transactions on Biomedical Engineering, 67(4): 940-948, 2020. doi: 10. 10911009...01111 29294999.
[17] Shiqiang Liu, Junchang Zhang, Yuzhong Zhang, and Rong Zhu. A wearable motion capture device able detect dynamic motion of human limbs. Nature communications, 11(1):5615, 2020.
[18] Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black. Smpl: A skinned multi-person linear model. ACM Trans. Graph., 34(6), oct 2015. ISSN 0730-0301. doi:
hottpa:ta otwpio.og/1... 1145/2816795. 2818013
[19] Naureen Mahmood, Nima Ghorbani, Nikolaus F Troje, Gerard Pons-Moll, and Michael J Black. Amass: Archive of motion capture surface shapes. In Proceedings of the IEEE/CVF international conference on computer vision, pages 5442-5451, 2019.
[20] Vimal Mollyn, Riku Arakawa, Mayank Goel, Chris Harrison, and Karan Ahuja. Imuposer: Full-body pose estimation using imus in phones, watches, and earbuds. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1-12, 2023.

---

[21] Md Moniruzzaman, Zhaozheng Yin, Md Sanzid Bin Hossain, Hwan Choi, and Zhishan Guo. Wearable motion capture: Reconstructing and predicting 3d human poses from wearable sensors. IEEE Journal of Biomedical and Health Informatics, 2023.
[22] Cecilia Monoli, Juan Francisco Fuentez-Pérez, Nicola Cau, Paolo Capodaglio, Manuela Galli, and Jeffrey A. Tuhtan. Land and underwater gait analysis using wearahle imu. IEEP Sensors Journal, 21()):1119221 1202, 2021. doi: 10.1 109/SSN, 2021.366112
[23] Noitom, Perception neuron series. pettps:/wwyy noitom. wwoiomm.com...... n.d. Accessed [2025-02-07].
[24] Simone Pasinetti, M Muneeb Hassan, Jörg Eberhardt, Matteo Lancini, Franco Docchio, and Giovanna Sansoni. Performance analysis of the pmd camboard picoflexx time-of-fight camera for markerless motion capture applications. IEEE transactions on instrumentation and measurement, 68(1 1):446644771 2019.
[25] Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani. Timo Bolkart, Ahmed AA Osman. Dimitrios Tzionas, and Michael J Black. Expressive body capture: 3d hands, face, and body from single image. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10975-10985 2019.
[26] François Piron, Daniel Morrison, Mehmet Rasit Yuce, and Jean-Michel Redouté. A review of single-photon avalanche diode time-of-ffight imaging sensor arrays. IEEE Sensors Journal, 21(11): (161)):1654-12666 2020.
[27] Alice Ruget, Max Tyler. Germán Mora Martín, Stirlng Scholes, Feng Zhu, Istvan Gyongy, Brent Hearn, Steve Malaauhlin, Abderrahim Halimi, and Jonathan Leach. Real-time, low-cost multi-person 3d pose estimation. arXiv preprint orXiv:21 10. 1414, 2021.
[28] Laura Susana Vargas-Valencia, Valencia, Felipe B. A. Schneider, Arnaldo G. Leal-Junior, Pablo Caicedo Rodriguuz. Wilson A. Sierra Arévaloo Luis E. Rodríguez Cheu, Teodiano Bastos- Filho, and Anselmo Frizera-Neto Sleeve for knee angle monitoring: An imu-pof sensor fusion system. IEEE Journal of Biomedical and Health Informatics, 25(2):465-4 474, 2021. doi: 10. 1091////BHIIIC (009//BLL2000029999 2986.29..360.
[29] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[30] Daniel Vlasic, Rolf Adelsberger, Giovanni Vannucci, John Barnwell, Markus Gross, Wojeiech Matusik. and Jovan Popović. Practical motion capture in everyday surroundings. ACM transactions on graphics (TOG), 26(3):35-es, 2007.
[31] Timo Von Mareard, Bodo Rosenhahn, Michael Black, and Gerard Pons-Moll. Sparse inertial poser: Automatic 3d human pose estimation from sparse imus. In Computer gruphics forum, volume 36, pages
349-360. Wiley Online Library, 2017.
[32] Timo Marcard, Roberto Hensehel, Michael J. Black, Bodo Rosenhahn, and Gerard Pons-Moll Recovering accurate 3d human pose in the wild using imus and moving camera. In Proceedings of the Furopean Conference on Computer Vision (ECCV), September 2018.
[33] Stefanie Walz, Mario Bijelic, Andrea Ramazzina. Amanpreet Walia, Fahim Mannan, and Felix Heide. Gated stereo: Joint depth estimation from gated and wide-baseline active cues. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13252-13262. 2023.
[34] Yinghao Wu, chaoran wang, Lu Yin, Shihui Guo, and Yipeng Qin. Accurate and steady inertial pose estimation through sequence structure learning and modulation. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomezak, and C. Zhang, editors, Advances in Neural Information Processing Systems, volume 37, pages 42468 42493. Curran Associates, Inc., 2024.
[35] Xsens. Xsens 3d motion tracking. https: //www xsens. com/ n.d. Accessed [2025-03-01].
[36] Xinyu Yi, Yuxiao Zhou, and Feng Xu. Transpose: Real-time 3d human translation and pose estimation with six inertial sensors. ACM Transactions on Graphics (TOG), 40(4):1-13, 2021.
[37] Xinyu Yi, Yuxiao Zhou, Mare Habermann, Soshi Shimada, Vladislav Golyanik, Christian Theobalt, and Feng Xu. Physical inertial poser (pip): Shysics-aware real-time human motion tracking from sparse inertial sensors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
13167-13178, 2022.
[38] Xinyu Yi, Yuxiao Zhou, Marc Habermann, Vladislav Golyanik, Shaohua Pan, Christian Theobalt, and Feng Xu. Egolocate: Real-time motion capture. localization, and mapping with sparse mody-mounted sensors. ACM Trans. Graph., 42(4). 2023.
[39] Xinyu Yi, Yuxiao Zhou, and Feng Xu. Physical non-inertial poser (рпр): Modeling non-inertial effects in sparse-inertial human motion capture. In SIGGRAPH 2024 Conference Papers, 2024.
[40] Yu Zhang, Songpengcheng Xia, Lei Chu, Jiarui Yang, Qi Wu, and Ling Pei. Dynamic inertial poser (dynaip): bast- motion dynamics learning for enhanced human pose estimation with sparse inertial sensors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1889-1899, 2024.
12

---

[41] Shaghayegh Zihajehzadeh and Edward J. Park. A novel biomechanical model-aided imu/uwb fusion for magnetometer-free lower body motion capture. IEEE Trunsactions Systems, Man, and Cybernetics: Systems, 47(6)9227-938, 2017. doi: 10.1 1C1T0T/MM22200222222222.
[42] Chengxu Zuo, Yiming Wang, Lishuang Zhan, Shihui Guo, Xinyu Yi, Feng Xu, and Yipeng Qin. Loose inertial poser: Motion capture with imu- attached loose-weat jacket. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2209-2219, 2024.
[43] Chengxu Zuo, Jiawei Huang, Xiao Jiang, Yuan Yao, Xiangren Shi, Rui Cao. Xinyu Yi, Feng Xu, Shihui Guo, and Yipeng Qin. Transformer imu calibrator: Dynamic on-body imu calibration for inertial motion capture. ACM Transactions on Graphics, 44(4), 2025.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as it is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Yes, discussed them in Sec.
6
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
14

---

While we provide the core assumptions and sketch key derivation steps for our theoretical results, we not include full formal proofs.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: Our paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to hosted model (e.g., in the case of a large language model), releasing of model checkpoint, other means that are appropriate to the research performed.
general. releasing code and data often
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results way reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g.. s. to registered users), but it should be possible for other researchers to have some path reproducing verifying the results.
5. Open access to data and code
15

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: Yes, we will release the code/data later.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//nips. ctapps.cc/pubiic/uidde. httplliigggui//eeeooo /Coeeeubso/ ssi siinnoliccy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Yes, we specified all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: No, we did not do that.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
16

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Yes, For each experiment, we provided sufficient information on the computer resources (type of compute workers, memory, time of execution) needed reproduce the experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't 'tt make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics huttps://neurre hwtp:ippppppppppppppppeeeptee hwtp///www...tttihcccueeee......
Answer: [Yes]
Justification: Yes, the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https///wwrrrr..cpppp/pu/ccccaaa
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consider- ation due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our work does not have direct potential positive societal impacts and negative socictal impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
17

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e-g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper does not have such issues.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification; Yes, all the creators or original owners of assets (e.g., code, data, models), used in the paper, are properly credited and are the license and terms of use explicitly mentioned and properly respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
18

---

For scraped data from particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/dataseta has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do not have such issues in this paper.
Guidelines:
The answer NA means that the paper does not release assets. Researchers should communicate the details of the dataset/cod//model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [Yes]
Justification: Yes, we included relevant details.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [Yes]
Justification: This research involving human participants was reviewed and approved by the Institutional Review Board (IRB). All participants provided informed consent after being fully informed of the potential risks and procedures.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
19

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may required for any human subjects research. you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: This work does not involve the use of large language models (LLMs) as an important, original, non-standard component of the core methodology.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy for what should or should not be described.
20


Reviews:
[{"review_id": "pOwI4Fworu", "reviewer": "Reviewer_K3tx", "summary": "The paper introduces ToF-IP, a system for human motion capture that fuses sparse Inertial Measurement Units (IMUs) with body-mounted Time-of-Flight (ToF) sensors. The authors claim that direct distance measurements from ToF sensors can mitigate the inherent drift accumulation in IMU-only systems, which is a well-documented and critical challenge in the field. The authors present three main contributions: a wearable hardware prototype, a new motion capture dataset (ToF-IP-DB), and a Transformer-based software framework featuring Node-Centric Integration and Dynamic Spatial Positional Encoding. The authors also contribute a new 208-minute motion dataset with synchronized IMU-ToF measurements and demonstrate that their method achieves state-of-the-art accuracy, especially for complex and slow motions.", "questions": "**-** Could you provide a more robust statistical analysis of your key results, particularly for the ablation studies (Table 2) and SOTA comparison (Table 1)?\n\nActionable Guidance: We suggest running the experiments multiple times (e.g., 3-5 times with different random seeds) and reporting the mean and standard deviation for key metrics like Positional Error and SIP Error. This would provide crucial insight into the stability and reliability of your method's improvements.\n\n\n**-** How does the ToF-IP system behave when one or more ToF sensors are temporarily occluded? Does performance degrade gracefully, or does it fail catastrophically?\n\nActionable Guidance: For the rebuttal, I would find it very helpful to see an analysis of this failure mode. This could be a quantitative experiment where you synthetically block the ToF signal for a portion of a test sequence, or a qualitative analysis of the system's output during self-occlusion events. A discussion of whether the Transformer's self-attention mechanism learns to down-weight unreliable (occluded) sensor inputs would be particularly insightful.\n\n\n**-** Could you elaborate on this trade-off between improved positional accuracy and output jitter? Is the jitter a direct result of the raw ToF sensor noise, and have you explored post-processing or filtering techniques to mitigate it?\n\nActionable Guidance: We encourage a brief discussion on the source of this jitter. Furthermore, demonstrating the effect of a simple filtering technique (e.g., a Savitzky-Golay filter or a Kalman filter) on the output poses could be very effective. A comparison of the smoothness of ToF-IP's output with and without filtering, and against an inertial-only baseline, would be highly illustrative.\n\n\n**-** Could you provide more detail on the domain gap between your simulated ToF data and the real data from your hardware prototype?\n\nActionable Guidance: To help with this question, please provide a side-by-side qualitative comparison of a real ToF depth map  and a simulated one for a similar body pose. Please also clarify if your Unity-based ToF simulation  models common real-world noise sources, such as multi-path interference or absorption by certain clothing materials.", "limitations": "In Section 6, the authors acknowledge several limitations: the system's performance is contingent on ToF sensor reliability, it may be compromised by occlusions, ToF sensor noise introduces jitter, and the approach needs validation in more diverse real-world scenarios. While these are valid points, the discussion is brief and lacks depth. Several other limitations are not acknowledged by the authors:\n\n**-** The performance of optical ToF sensors is known to be highly dependent on the environment. They can struggle with highly reflective surfaces (e.g., mirrors, polished floors) or highly absorbent surfaces (e.g., black fabrics), which can corrupt distance measurements. These are common conditions that are not addressed.\n\n**-** The proposed Transformer-based architecture is computationally intensive. The authors report running experiments on a high-end NVIDIA RTX 4080 GPU. While they claim \"real-time\" performance, they provide no concrete metrics like frames-per-second (FPS) or processing latency in milliseconds. Without these metrics, the claim is unsubstantiated and the practicality of deploying this model on a portable or embedded device remains unclear.   \n\n**-** The paper mentions that the collected data is \"synchronized\" but provides no details on the hardware or software mechanisms used to achieve this synchronization between the IMUs and ToF sensors. For high-quality sensor fusion, precise, low-latency time-stamping is critical, and the lack of detail on this implementation aspect is a limitation.", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths\n\n**+** The work addresses a fundamental and persistent problem in wearable motion capture: the quadratic error accumulation resulting from the double integration of noisy accelerometer data in IMU-only systems.   \n\n**+**  The design of a lightweight, wearable prototype that integrates ToF sensors onto a standard 6-IMU layout with minimal volume increase is a practical and valuable engineering contribution.   \n\n**+** The creation and planned release of the ToF-IP-DB dataset, which includes synchronized IMU, body-worn ToF, and optical ground-truth data, is a significant service to the research community, enabling future work and benchmarking in this specific subfield.\n\nAreas for improvement\n\n**-** The claims of superior performance are not supported by a detailed statistical analysis. The absence of error bars, confidence intervals, or significance testing makes it impossible to determine if the reported improvements are meaningful or the result of random variance.   \n\n**-** The results indicate that while ToF integration may reduce positional error, it may not improve or even worsen, pose jitter. While ToF-IP shows improvements in positional and angular errors, it performs poorly on the Jitter metric. In Table 1, the jitter of ToF-IP on the DIP dataset (0.17) is no better than that of PIP (0.17) or PNP (0.17). The ablation study in Table 2 shows that adding ToF data without the proposed NCI module (Case 2) increases jitter from 0.07 to 0.13 on the ToF-IP-DB dataset compared to an IMU-only baseline (Case 1). This suggests that while ToF's direct distance measurements may help correct for drift, the inherent noise in ToF sensors introduces high-frequency errors, or jitter, into the final pose estimation. The paper fails to discuss this drift-jitter trade-off, presenting a potentially incomplete and overly optimistic picture of the proposed method's performance.\n\n**-** The contribution of Dyn-PE requires more explanation. The authors argue that static positional encodings, common in NLP, are ill-suited for wearable sensors whose spatial relationships are constantly changing. Their proposed solution is to make the positional value for each node a learned function of the entire sensor input at that timestep, i.e., PE_dyn (t) is a function of X_all (t). This approach raises a fundamental question: if the positional encoding already depends on the full input, what new relational information is the Transformer's self-attention mechanism expected to learn? The core purpose of self-attention is precisely to compute dynamic, input-dependent relationships between tokens. Dyn-PE appears to be pre-computing a simplified version of this relationship and adding it to the input, which feels redundant. While the ablation study in Table 3 shows a marginal improvement in some metrics (e.g., Positional Error on ToF-IP-DB drops from 7.36 to 6.31), this result is presented without any statistical validation. It is therefore impossible to know whether this small gain is a genuine contribution of the architecture or simply a product of random initialization or training dynamics. The claim that this technique \"enhances spatial awareness\" requires more rigorous proof to be considered a significant contribution.", "was_revised": true, "final_justification": "Authors have not justified their choices, nor have they attempted to address the concerns raised in my initial reviews. I am not satisfied with the justification and problematic ethical concerns around human subjects. I am not satisfied.\n\nI wish the authors good luck with the improvement of the paper.", "rebuttal": "Thank you for your constructive feedback! We carefully address each point below, grouped by the original comments. For brevity, we refer to Weaknesses, Questions and Limitations as W#, Q# and L# respectively. All major clarifications and revisions will be incorporated into the final manuscript.\n\nW1, Q1: We trained our model 10 times with different random seeds and recorded the mean, std and the 95% confidence intervals for each metric. \nThe results for the ablation study are as follows:\n| DIP     | SIP Global Error (°)        | Angular Global Error (°)    | Positional Error (cm)    |Endpoint Error (cm) | Jitter (1000 m/s³) | \n|----------|-----------------------------|-----------------------------|--------------------------|---------------------------|-------------------------------|\n| Case 1   | 18.94 ± 0.13 (18.84, 19.03) | 13.18 ± 0.17 (13.06, 13.30) | 6.86 ± 0.07 (6.80, 6.91) |  12.31 ± 0.11 (12.23, 12.39)   |0.07 ± 0.00 (0.07, 0.07) |\n| Case 2   | 18.98 ± 0.18 (18.86, 19.11) | 12.85 ± 0.11 (12.77, 12.93) | 6.94 ± 0.06 (6.90, 6.98) |  12.37 ± 0.10 (12.29, 12.44)   |0.13 ± 0.01 (0.13, 0.14)|\n| Case 3   | 17.91 ± 0.12 (17.83, 18.00) | 13.19 ± 0.08 (13.13, 13.25) | 6.81 ± 0.04 (6.78, 6.84) |   12.06 ± 0.08 (12.01, 12.12)   |0.13 ± 0.00 (0.13, 0.13)|\n| Case 4   | 17.21 ± 0.08 (17.16, 17.27) | 11.93 ± 0.06 (11.89, 11.97) | 6.31 ± 0.03 (6.29, 6.33) |  11.47 ± 0.06 (11.42, 11.52)   |0.12 ± 0.00 (0.12, 0.13) |\n\n\n| ToF-IP-DB  | SIP Global Error (°)         | Angular Global Error (°)      | Positional Error (cm)         |    Endpoint Error (cm)  |  Jitter (1000 m/s³)        |\n|--------|-------------------------------|-------------------------------|-------------------------------|-------------------------------|-------------------------------|\n| Case 1 | 16.34 ± 0.02 (16.33, 16.36)   | 7.68 ± 0.04 (7.65, 7.71)      | 5.79 ± 0.06 (5.75, 5.84)      |    8.44 ± 0.09 (8.37, 8.51)      | 0.13 ± 0.01 (0.12, 0.13)  |\n| Case 2 | 16.13 ± 0.08 (16.07, 16.19)   | 7.53 ± 0.07 (7.48, 7.57)      | 5.49 ± 0.04 (5.46, 5.51)      |    8.10 ± 0.06 (8.05, 8.14)      |0.18 ± 0.01 (0.17, 0.18) |\n| Case 3 | 15.49 ± 0.11 (15.41, 15.56)   | 7.29 ± 0.07 (7.25, 7.34)      | 5.30 ± 0.03 (5.28, 5.32)      |    7.79 ± 0.07 (7.74, 7.83)      |0.17 ± 0.01 (0.16, 0.17) |\n| Case 4 | 13.62 ± 0.42 (13.32, 13.92)   | 6.82 ± 0.16 (6.71, 6.93)      | 4.59 ± 0.09 (4.52, 4.65)      |    6.67 ± 0.13 (6.58, 6.76)      |0.17 ± 0.01 (0.16, 0.17)  |\n\n\nDue to the time constraints of the rebuttal period, we can only provide the results for the comparison with the closest SOTA:\n|  DIP  | SIP Global Error (°)         | Angular Global Error (°)      | Positional Error (cm)         |Endpoint Error (cm)           | Jitter (1000 m/s³)       |\n|-----------|-------------------------------|-------------------------------|-------------------------------|-------------------------------|-------------------------------|\n| DynaIP| 13.85 ± 0.46 (13.49, 14.21)   | 7.00 ± 0.18 (6.87, 7.13)   | 5.01 ± 0.11(4.93, 5.09)      | 6.82 ± 0.12(6.78, 6.90)      | 0.12 ± 0.00 (0.12, 0.13)      | \n| OURS  | 13.62 ± 0.42 (13.32, 13.92)   | 6.82 ± 0.16 (6.71, 6.93)      | 4.59 ± 0.09 (4.52, 4.65)      | 6.67 ± 0.13 (6.58, 6.76)     | 0.17 ± 0.01 (0.16, 0.17)      | \n\n| TOF-IP-DB   | SIP Global Error (°)         | Angular Global Error (°)      | Positional Error (cm)         | Endpoint Error (cm)           | Jitter (1000 m/s³)       |\n|-----------|-------------------------------|-------------------------------|-------------------------------|-------------------------------|-------------------------------|\n| DynaIP  | 19.00 ± 0.10 (18.93, 19.07)  | 13.38 ± 0.07 (13.33, 13.44)  |  7.22 ± 0.04 (7.19, 7.26)     | 13.02 ± 0.08 (12.97, 13.08)     |   0.16 ± 0.01 (0.15, 0.16)    | \n| OURS    | 17.21 ± 0.08 (17.16, 17.27)   | 11.93 ± 0.06 (11.89, 11.97)      | 6.31 ± 0.03 (6.29, 6.33)      |11.47 ± 0.06 (11.42, 11.52)      | 0.12 ± 0.00 (0.12, 0.13)     |\n\nWe will provide a full statistic analysis of other SOTAs in revision.\n\nW2: To clarify, the slight increase in jitter introduced by ToF is not due to the inherent sensor noise, but rather the numerical uncertainty of distance map pixels when they correspond to the edge regions of the observed object. In fact, jitter caused by sensor data noise is a common challenge in wearable motion capture systems. For jitter reduction, a practical solution is to apply motion optimization to post-process the raw motion output. As shown below, after applying motion optimization in PNP, the jitter decreases significantly—from 0.12 m/s³ to 0.08 m/s³—ensuring the quality of motion capture in practical applications.\n\nW3: We would like to highlight the crucial role of Dyn-PE to the value matrix V in self-attention: it injects dynamic positional information into token values, which cannot be achieved with predefined or learnable static positional encodings.\nAlso, as detailed in our response to W1, Q1, the statistical analyses of Case 2 and Case 4 (ablation study) across 10 random seeds show that Dyn-PE consistently improves performance. \n\nQ2: Due to space constraints, please refer to our response to Reviewer 2RCo, W2 for details. The results show that our model’s performance degrades gradually, without catastrophic failure, demonstrating its robustness to partial ToF sensor loss.\n\nQ3: This trade-off has already been addressed in detail in W2 of our response.\n\nQ4: As shown in the table below, we compared depth data collected from the real ToF hardware prototype and simulated ToF data generated under the same pose. Specifically, we selected representative frames from the left hand and left foot, and extracted the corresponding 4×4 pixel regions from both the real and simulated distance maps. The simulated ToF distance values were generated based on the same skeletal motion used during real data capture.\n\nLeft Ankle                 \n| Simulated Distance（mm） | Real Distance（mm） |\n| ------------- | ------------- |\n| 2000          | 2000          |\n| 2000          | 2000          |\n| 2000          | 2000          |\n| 2000          | 2000          |\n| 2000          | 2000          |\n| 2000          | 2000          |\n| 2000          | 2000          |\n| 2000          | 2000          |\n| 376           | 369           |\n| 337           | 361           |\n| 337           | 369           |\n| 353           | 337           |\n| 235           | 235           |\n| 235           | 235           |\n| 227           | 227           |\n| 227           | 219           |\n\n\nLeft Wrist                 \n| Simulated Distance（mm） | Real Distance（mm） |\n| ------------- | ------------- |\n| 227        | 125        |\n| 243        | 133        |\n| 243        | 133        |\n| 282        | 251        |\n| 243        | 133        |\n| 243        | 125        |\n| 243        | 125        |\n| 204        | 188        |\n| 2000       | 2000       |\n| 243        | 133        |\n| 227        | 133        |\n| 133        | 149        |\n| 1969       | 2000       |\n| 2000       | 2000       |\n| 243        | 133        |\n| 125        | 133        |\n\n\nBased on this quantitative comparison, the average absolute difference across the 16 pixels is 7.2 mm, this result indicates that the simulated ToF data closely approximates real sensor outputs under similar motion conditions.\n\nL1: We agree that optical ToF sensors may degrade under extreme conditions. However, in our tests, our system performed reliably on typical indoor and outdoor surfaces (e.g., tile floors, dark carpets, and asphalt), where ToF sensors returned stable and valid measurements at a true distance of 1000 mm across these scenarios.\n\n| Scene                          | Mean Distance(mm) | Std(mm) |\n|-------------------------------|--------------|-------------|\n| Office(Ceramic tile)        | 991.36       | 17.67       |\n| Office(Dark Carpet)         | 999.63       | 36.82       |\n| Outdoor(Sunny Grass)         | 1005.00      | 51.63       |\n| Outdoor(Shaded Grass)         | 999.34       | 23.59       |\n| Outdoor(Sunny Asphalt Road)     | 1004.37      | 80.91       |\n| Outdoor(Shaded Asphalt Road)      | 998.37       | 15.81       |\n\n\nIn addition, we hope to clarify that the core contribution of our work is to demonstrate that integrating absolute distance measurements help IMU-based motion capture. ToF sensing is currently the most practical and accessible means to achieve this. As emerging MEMS-based or alternative distance sensors mature, they can be seamlessly integrated into our framework to address other challenging scenarios, with little or no changes to our algorithm.\n\nL2: We exported the model to ONNX and tested it on an Intel i7-13700K CPU. It achieved 0.65 ± 0.69 ms per frame (311.73 ± 78.34 FPS) over 18,000 frames. To ensure temporal consistency with time-differentiated acceleration, we cap the runtime at 60 FPS, under which latency remains 1.10 ± 1.05 ms and measured FPS is 60.44 ± 4.57.\nThese results confirm real-time performance on a standard CPU, supporting deployment on portable or embedded systems without a GPU.\n\nL3: Our system samples IMUs at 100 Hz and ToF sensors at 60 Hz, using an MCU-based hardware timer to trigger data acquisition: 10 ms for IMUs and 16.66 ms for ToF. Hardware-level triggering minimizes jitter and ensures stable sampling intervals.\nFor post-processing, we apply linear interpolation to align IMU, ToF, and motion capture ground truth to a unified 60 Hz timeline, ensuring consistent timestamps and low-latency alignment across modalities."}, {"review_id": "8PfBGIrl3i", "reviewer": "Reviewer_zeff", "summary": "This paper introduces a new framework to capture egocentric 3D human motion using the integrated Time-of-Flight (ToF) and Inertial-Measurement-Unit (IMU) sensing system. The authors propose a unified Transformer-based framework featuring a Node-Centric Data Integration strategy and a Dynamic Spatial Positional Encoding scheme to explicitly model intra-node and inter-node sensor interactions. They also contribute a new ToF-IP-DB dataset containing more than 3 hours of human motion data captured by their proposed sensing system paired with optical motion capture. Authors conducted extensive evaluation against state-of-the-art methods (although none of them shares the same sensor input) and ablation studies to support their argument.", "questions": "– I wonder how much cost and battery inefficiency we need to consider by adding the sparse resolution ToF sensors. Since the authors mentioned that they carefully balanced between the power consumption and the performance, I would like the authors to provide this information in the main paper more comprehensively.\n\n– From their supplementary video, it seems like their system runs in real time. Since this is an important factor for the wearable motion tracking system, I would recommend the authors to state their time and computational complexity (FPS, FLOPs) in the main paper.", "limitations": "I do not have any other concern than the practicality of the new sensor addition with insufficient accuracy gain.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "**Stengths**\n– The authors design and build a wearable motion capture sensing system by integrating 4 ToF sensors into a standard 6 IMU setup, and they validate the hardware end-to-end, greatly boosting the paper’s practical credibility.\n– They propose a novel Transformer-RNN architecture with Node-Centric Data Integration and Dynamic Spatial Positional Encoding, enabling principled fusion of sparse depth maps and IMU streams to recover full 3D kinematics.\n– The authors collected the real data of human movement captured from their proposed sensing system paired with optical motion capture, providing both training and evaluation data to the community.\n\n**Weaknesses**\n– The measured accuracy gains over IMU-only approaches are modest relative to the added hardware complexity, raising questions about the cost–benefit trade-off in practical deployments. Considering the fact that ToF sensor may not be generalized to the real-world scenario including uneven terrain, this accuracy gain seems insufficient.", "was_revised": true, "final_justification": "The authors responded to my main concerns and questions. I will keep my positive rating.", "rebuttal": "Thank you for your constructive feedback! We address each point in detail below, organized by the original comments. For clarity and brevity, we denote Weaknesses, Questions, and Limitations as W#, Q#, and L#. All key clarifications will be reflected in the revised manuscript.\n\nW1: We appreciate the reviewer’s concern about the cost–benefit balance of adding ToF sensors. To address this directly, we provide a quantitative comparison with a SOTA method: UIP (SIGGRAPH 2024), which integrates Ultra-Wideband (UWB) sensing for positional correction.\n\nIn UIP, the authors report using six UWB modules, each costing approximately USD 20.19, leading to a total added hardware cost of USD 121.14. Their system achieves an 11.51% accuracy improvement over an IMU-only baseline. In contrast, our system integrates only four sparse ToF sensors, each priced at approximately USD 9, with a total added cost of USD 36, while achieving a 18.47% improvement. This yields a significantly higher accuracy-per-dollar ratio, demonstrating that our method offers superior cost-effectiveness for lightweight and wearable motion capture.\n\nQ1:Power Consumption: In our system, the ST VL53L8CX ToF sensor operates in continuous ranging mode, with a measured power consumption of approximately 215 mW. If needed, the sensor also supports autonomous ranging mode, which significantly reduces power consumption at the cost of frame rate.\nOur inertial sensing uses the Xsens MTi-3 IMU, with a typical power consumption of 44 mW @ 3.0 V, which is comparable to alternatives such as the BNO085 (35–45 mW depending on sampling rate).\nWe conducted practical power measurements on a complete sensing node (including MCU, IMU, ToF, and wireless module) powered by a 1000 mAh 3.7 V Li-ion battery. The total system power consumption during wireless transmission and continuous ToF operation is 0.97–1.00 W, yielding a measured runtime of ~4.38 hours per charge, which is comparable to the battery life of commercial wireless IMU-based motion capture systems (typically around 5 hours). This confirms that our design maintains practical battery life while incorporating sparse ToF data.\n\nCost: In terms of hardware cost, the integration of the sparse-resolution ToF sensor introduces minimal overhead:\n- VL53L8CX ToF module: USD 8.72\n- 1.8 V LDO regulator (e.g., TLV73318): USD 0.23\nThis brings the total ToF-related hardware cost per node to under USD 9 at prototype scale. In large-scale manufacturing, we expect further reductions in both component and integration costs.\n\nQ2: We exported the model to ONNX and tested it on an Intel i7-13700K CPU. It achieved 0.65 ± 0.69 ms per frame (311.73 ± 78.34 FPS) over 18,000 frames. To ensure temporal consistency with time-differentiated acceleration, we cap the runtime at 60 FPS, under which latency remains 1.10 ± 1.05 ms and measured FPS is 60.44 ± 4.57.\nIn terms of theoretical complexity, we further computed the total number of floating-point operations (FLOPs) required for a single frame of inference. The complete model requires only 0.31 MFLOPs, making it highly efficient and suitable for deployment on portable or embedded platforms.  \nThese results confirm real-time performance on a standard CPU, supporting deployment on portable or embedded systems without a GPU."}, {"review_id": "bAoON11jvV", "reviewer": "Reviewer_2RCo", "summary": "The paper proposes a hardware and algorithmic pipeline that augments the standard 6‑node sparse‑IMU mocap setup with four on‑board time‑of‑flight (ToF) sensors mounted on the wrists and ankles.\n\nOn the hardware side, the authors claim only a 3 % volume increase while keeping the canonical 6‑IMU attachment pattern, so the system remains lightweight and untethered. \n\nOn the algorithmic side, they introduce\n\nNode‑Centric Data Integration (NCI): Each sensing node forms a token that first fuses its own ToF depth patch, IMU acceleration, and orientation (“intra‑node”), then exchanges information through a Transformer encoder (“inter‑node”). \n\nDynamic Spatial Positional Encoding (Dyn‑PE): A motion‑conditioned positional encoding that replaces the static index‑based encodings typically used in Transformers, letting token positions evolve with the body’s motion across time. \n\nThree RNNs predict joint velocities, positions, and rotations from the Transformer embeddings.\n \nA new 208-minute paired ToF‑IMU‑optical dataset (10 subjects, 20+ movements) is introduced, and the authors also provide a synthetic ToF extension of the public DIP‑IMU dataset.", "questions": "**i) Performance & Power:**\n\nWhat is the inference throughput (FPS) of the full pipeline on your target hardware?\n\n  - What are the per‑module latencies for the Transformer and LSTM components?\n\n  - What is the typical battery life or power draw of a single IMU–ToF node under continuous operation? (if possible to obtain)\n\n\n**ii) Dynamic Positional MLP (L. 221):**\n\nIs the MLP that computes each node’s dynamic positional encoding shared across all sensing nodes, or is there a separate MLP per node?\n\n**iii) Supplementary Video Lag:**\n\nIn the supplementary video (timestamps 2:10–2:13), why is there a visible lag between the actor’s movement and the estimated pose during fast actions?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "## **Strengths**\n\ni) The paper is well written and easy to follow.\n\nii) The problems of the existing purely IMU‑based systems (integration and drift errors) are clearly explained, and the motivation for the proposed approach is compelling.\n\nii) The architecture is well-suited for the task.\n\niii) Experiments and ablations are thorough and justified.\n\niv) The limitations of the proposed method are discussed in detail.\n\nv) The proposed dataset, ToF‑IP‑DB, that pairs body‑mounted ToF with IMU and optical ground truth, will be a valuable resource for the community.\n\n## **Weaknesses**\n**i) Unclear Runtime & Power Metrics:**\n- The claim of “real‑time” performance is unsupported. No FPS, per‑module latency (Transformer vs. LSTM), or end‑to‑end throughput is reported.\n- Power consumption of the IMU–ToF nodes is not provided.\n\n**ii) Occlusion Robustness:**\n\nTime‑of‑flight ranging can fail when body parts occlude one another; no discussion about this is present.\n\n**iii) Figure Legibility:**\n\nFig. 4 is too small or low‑contrast to read clearly in print.\n\n**iv) Missing Units:**\n\nBenchmark results lack metric units (e.g., mm, cm), making quantitative comparisons ambiguous.\n\n**v) Typographical Errors:**\n- Table 1 headers contain typos.\n- The caption of Fig. 5 mentions Case 1 and Case 4; is that correct?", "was_revised": true, "final_justification": "The authors have addressed my primary concerns and questions, and I will maintain my positive rating. I am not an expert in IMU-based pose estimation, but in my opinion, the initial submission and rebuttal provide sufficient ablation studies and comparisons to show that the addition of the TOF sensor improves pose estimation results. Furthermore, the proposed dataset, which combines IMU and TOF data, could be a valuable contribution to the community for future developments in this domain. That said, reviewers with deeper expertise in this area should provide a more precise assessment of the paper’s contribution and technical validity.", "rebuttal": "Thank you for your constructive feedback! We address each point in detail below, organized by the original comments. For clarity and brevity, we refer to Weaknesses, Questions, and Limitations as W#, Q#, and L#, respectively. All major clarifications and revisions will be incorporated into the revised manuscript.\n\nW1: Inference Speed and Real-Time Performance: \nWe exported the model to ONNX format and tested on an Intel i7-13700K CPU. It achieved process speed of 311.73 ± 78.34 FPS (0.65 ± 0.69 ms/frame) over 18,000 frames. We also lock the run time fps to 60 to simulate the real-time capture situation, get latency of 1.10 ± 1.05 ms and FPS of 60.44 ± 4.57.\nModel Complexity (FLOPs)\nWe further computed the total number of floating-point operations (FLOPs) required for a single frame of inference. The complete model requires only 0.31 MFLOPs, making it highly efficient and suitable for deployment on portable or embedded platforms.  \nPower Consumption and Battery Runtime: \nWe conducted practical power measurements on a complete sensing node (including MCU, IMU, ToF, and wireless module) powered by a 1000 mAh 3.7 V Li-ion battery. The total system power consumption during wireless transmission and continuous ToF operation is 0.97–1.00 W, yielding a measured runtime of ~4.38 hours per charge. This confirms that our design maintains practical battery life while incorporating sparse ToF data.\n\nW2: To evaluate the system’s robustness under partial occlusion, we simulated long-term ToF sensor blockage by setting the input of 1 to 4 sensors to zero in 25% of the frames on the TOF-IP-DB dataset.\nThe results show that the model’s performance degrades gradually, without catastrophic failure, demonstrating its robustness to partial ToF sensor loss.\n| ToF Sensor Dropout     | SIP Global Error (°) | Angular Error (°) | Positional Error (cm) |  Endpoint Error (cm) |Jitter (1000 m/s³) |\n|------------------------|----------------------|--------------------|------------------------|---------------------------|----------------------|\n| 1 ToF Sensor    | 18.04                | 12.60              | 6.65                   |  12.03                |0.12                      |\n| 2 ToF Sensors  | 18.42                | 12.86              | 6.81                   | 12.32                |0.12                      | \n| 3 ToF Sensors   | 18.76                | 13.09              | 6.97                   |  12.60                |0.12                      |\n| 4 ToF Sensors   | 19.02                | 13.25              | 7.08                   |  12.86                |0.12                      |\n\n\n\nW3: We will revise the figure by increasing its contrast and adjusting its size to ensure better clarity in both digital and printed formats.\n\nW4: For clarity, we specify the units used for each evaluation metric:\n- Angular Error is reported in degrees (°);\n- Positional Error, SIP Error, and Endpoints Positional Error are measured in millimeters (mm);\n- Jitter is quantified in millimeters per second cubed (km/s³), representing the global jerk magnitude.\n\nW5: The header in Table 1 mistakenly lists “FIP-DB” and should be corrected to “ToF-IP-DB”; we will revise this accordingly.\nAs for Fig. 5, it indeed compares Case 1 and Case 4 and we agree that the legend may cause confusion. We will update the figure legend to clearly reflect the comparison and eliminate any ambiguity.\n\nQ1: Inference Throughput (FPS): For deployment, we convert the model to ONNX format and run inference on a consumer-grade Intel i7-13700K CPU. During a real-time test over 18,000 frames, the system achieved an average inference latency of 0.65 ± 0.69 ms per frame, corresponding to 311.73 ± 78.34 FPS. Since acceleration is computed using time-differentiation, we cap the system to 60 FPS to ensure stability. Under this setting, the model achieved 1.10 ± 1.05 ms latency and 60.44 ± 4.57 FPS, confirming real-time capability even without GPU acceleration.\n\nModule-wise Latency (measured on GPU):\nThe average inference times and standard deviations for each network component are as follows: Transformer Encoder – 1.20 ± 0.57 ms, LSTM for Velocity Estimation – 3.96 ± 4.23 ms, LSTM for Joint Estimation – 3.82 ± 0.55 ms, and LSTM for Pose Refinement – 3.80 ± 0.51 ms.\n\nPower Consumption and Battery Life:\nWe conducted practical power measurements on a complete sensing node (including MCU, IMU, ToF, and wireless module) powered by a 1000 mAh 3.7 V Li-ion battery. The total system power consumption during wireless transmission and continuous ToF operation is 0.97–1.00 W, yielding a measured runtime of ~4.38 hours per charge. This confirms that our design maintains practical battery life while incorporating sparse ToF data.\n\nQ2: We clarify that the MLP computing the dynamic positional encoding (phi_estimator) is shared across all sensing nodes. It takes as input a concatenation of the features from all endpoint nodes, and outputs the full set of structural encodings (phi) for all target joints. Hence, it is not a per-node MLP, but rather a global shared module that models the positional structure in a joint fashion.\nJoint encoding encourages the model to learn a unified representation of the human body's structure, rather than overfitting to specific nodes. This improves robustness to unseen motion patterns or slightly different sensor configurations. Since the encoding depends on all available endpoints, it is less sensitive to noise or dropout in any individual sensor. \n\nQ3: The lag observed in the supplementary video is not caused by the model’s inference latency, but rather by real-time rendering delays in Unity. During video recording, we used a laptop equipped with an RTX 2060 Laptop GPU and connected it to a 32-inch 4K monitor to simultaneously display the actor's physical movements and the predicted human model animation. The high resolution imposed a significant rendering load on the laptop, especially during fast movements, leading to slight delays in Unity visualization. However, we have verified that the model itself performs inference in real time. We will consider using higher-performance hardware in future demonstrations to avoid potential misinterpretation caused by such visualization artifacts."}, {"review_id": "KorL5qdYll", "reviewer": "Reviewer_8u8c", "summary": "The paper introduced Time-of-flight sparse inertial pose estimation.\n\nBy introducing four cameras at the limbs, the paper obtains absolute distance measurements between the body and the environment, which is supposed to stabilize body pose estimation.\n\nThe main contributions are:\n1) the use of ToF cameras at the limbs\n2) a 200min dataset for pose estimation from ToF sensors \n3) a method that improves pose estimation slightly", "questions": "Abstract:\n\nWill the training code be released as well?\n\n27: the sentence sounds like there is only one way of doing that. IMUs measure local accelerations and local angular velocities. One naive way of integrating that is through linear acceleration. Typically more complex state estimation with calibration schemes, e.g. Kalman Filters, are required. I recommend a rephrasing.\n\n28-30: This is not an amplification of error that is related to IMU, but an error source due to human modeling. I recommend to rephrase it slightly. \n\n44: Please clarify what kind of distances are measured, between sensors, between environments?\n\n48-50: why are those locations chosen for the ToF sensors? \n\n53: how is multisensor data hierarchical? Please elaborate. Multisensor data from different limbs does not follow a clear hierarchy in my opinion.\n\n56: unclear what is meant by intra-node? Inbeween sensors? Or just one sensor? Maybe define it prior or name it differently, e.g. per-sensor individual integration?\n\n56: What do you integrate? Is it numerical integration? Why do you integrate orientations? How do you obtain orientations? IMU measure angular velocities. If you like previous methods assume orientations as input, please state so clearly :)\n\n60: The dynamic spatial positional encoding - sounds simply like learnable positional encodings of 3D positions in that part. Later in the method section, it becomes a bit clearer. Can you clarify the motivation on why your positional encoding is superior to learning an encoding based on sensor position?\n\n119: maybe cite the very closely related baseline ultra inertial poser here. \n\n141: how do you obtain the original joint position which is needed for integration? How do you obtain the original joint velocity which is needed when double integrating accelerations?\n\n147: Do you mean $\\tau$? t seems to be overloaded as the limit of the interval and the current timesamp\n\nSuppl Eq 1 - seems to ignore starting conditions, if the original velocity estimation is not 0, the velocity error should have a constant bias. Please add starting conditions as an assumption, i.e. the original position is always (0,0,0) and the starting velocity is always 0. In the same regard, your theoretical derivation seems irrelevant if you discard your obtained results (cubic variance) in favour of a quadratic relationship between variance and time. Also the explanation and reasoning why the relationship is actually quadratic and not cubic as you derived does not become clear to me. Can you clarify where and how you use your theoretical findings, why other works and practical findings come up with σ^2 t^2 instead of your theoretical findings and why going from continuous to discrete time sampling drops a factor of t in the variance (after equation 6 in supplementary)\n\nMethod 3: I am not sure what insight you generate from section 3, and how it is related to your contribution\n\n158: you do not use the standard layout used in previous work but instead place the sensors traditionally placed at the knees, at the feet. While there is no inherent disadvantage of this setup, you dont discuss the change of layout and also other methods. Additionally, a comparison to other methods becomes questionable when a different layout is used in your evaluation.\n\n158: what is the energy consumption of your hardware? How much runtime would a wearable device have with just ACC, ACC+GYRO and ACC+GYRO+TOF\n\n160: what is the orientation of your TOF sensors, how are they pointing? Can you also indicate that in the figure 3\n\n170: What is the reason for partially using transformers and partially LSTMs, what is your finding on the suitability of each architecture. What if you only use LSTMs in you model. What if you only use transformers in your architecture?\n\n175: Why does orientation need to be integrated?\n\n179: “Integrating measurements from other sensing nodes”. How are you integrating distances? Are you talking about the mathematical integration?\n\n186-187: This argument does not become clear to me for once all the values from one node, can interact with all other nodes through the MLP, but also all measurements from a single node can interact with all measurements from that same sensor node. Additionally, along the temporal axis previous works employ LSTMs, and thus both the inter sensor node (spatial) interaction and single node interaction (temporally) is possible. Your claimed limitation of previous work does not become apparent.\n\n218: can you clarify on your positional encoding. The encoding seems to come purely from orientation, acceleration and distances. How would you distinguish sensors when their orientations, accelerations and distances are the same? What is the reason to force your learned sensor embedding $\\phi_n$ thorough the cosine embedding again? Do you have results that show that this additional passing though the sinusoidal encoding is needed?\n\n231: what subsets of AMASS do you use?\n\n233: do you assume shaped input or mean shape?\n\n240: what are the 20 types of motion? Can you provide more examples of your dataset. Both your dataset and the one from DIP are rather slow paced. Does your dataset contain fast sequences? How does your method perform on fast paced data?\n240: Can you report results on TotalCaptureReal dataset? The DIP dataset does not contain translation, has inaccurate ground truth, very noisy sensor data and \n\n251: How do you obtain orientations from sensor data? What is the drift of your sensor data over time?\n252: does your method work window based, or frame based? How do you do inference?\n\n256: can you report the units of your metrics? Can your report translation errors?\n\nTable 1: How do you train the other methods, considering that you have different sensor layout of your method?\n\nTable 1: Do you ever finetune on the FIP-DB dataset or is it used purely for evaluation?\n\nTable 1: Can you train the closest baselines with additional ToF input to investigate their benefit of ToF data. On DIP your architecture seems to be much worse (15.52 sip error vs 13.71 sip error of PNP) when not using ToF data.\n\n4.2.3: Do you predict smpl angles or do you represent only the joint positions directly? What is the velocity output and angle output used for? Do you evaluate those outputs?\n\nSupp 1.2) How does the ToF sensor work in the wild. How is the noise? Do you have an analysis of the noise of the ToF sensor. You just highlight the sensors characteristic within one sequence. How much base noise does the sensor have? What is the mean distance error?\n\nSupp 4.1) do you add noise in the simulation to model real world sensor noise?\n\nSupp 4.5) Why do you not report global translation error, if your method is capable of global translation tracking? Also \n\nVideo Qual results) the results of your method look surprisingly jittery (especially in the first part of the video), why is your jitter metric lower than other methods jitter?\n\nThanks for your work. Despite my many critique points I am happy to adjust my score depending on how the questions are answered during the rebuttal.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n- Novel sensors for pose estimation\n- a 200min dataset of ToF and inertial sensors for pose estimation\n- good results, the new sensors together with the architecture deliver state of the art results\n\nWeakness:\n- the method does not seem very novel, despite showing slightly improved results\n- the method uses slightly different sensor layout, making comparisons hard\n- the paper compares to baselines that do not have access to ToF data, making comparisons less fair (comparing with baselines that get the tof data as an additional input would allow a fairer comparison of the method)\n- the sensor hardware and noise could be discussed in more dteail", "was_revised": true, "final_justification": "The paper presents an additional ToF sensor for inertial pose estimation. \n\nThe authors show that the incorporation of ToF sensor data is marginally better for most metrics and the Ang. Error improves most. \nExperiments regarding the proposed architectures and their modules exist, but could be extended to bring additional insights on why the proposed method is only slightly better than pure Inertial Methods and how well the method would be using just TOF inputs.\n\nThe authors clarified concerns about a different sensor layout, compared to baseline methods, in the rebuttal.\n\nOverall, the proposed dataset, together with sota results, makes me believe that this submission is above the acceptance threshold, but the evaluation of the architecture and comparisons with baseline methods and the effect of ToF sensors could be studied further for a stronger submission.", "rebuttal": "Thank you for your constructive feedback!  \nW1: Our key innovation is the first exploration of integrating distance map sensing and inertial sensing to improve mocap accuracy. According to our ablation results, this can not be naively achieved by simply adding ToF data as input, and the proposed Node-centric data embedding successfully addresses the challenge in ToF-Inertial data fusion and brings expected improvements.\nWe believe our exploration and real-world dataset reveal new possibilities and directions for the community in developing consumer-grade mocap systems.\n\nW2, Q14, Q27: To ensure fair comparison, we matched sensor layouts across all evaluations. On DIP-IMU, we used the standard configuration and simulated ToF data at the same positions. On ToF-IP-DB and AMASS, all methods were trained and tested with our ToF-augmented layout.\n\nW3: We clarify that the goal of our comparisons is to investigate whether introducing distance maps from ToF can bring improvement to a basic backbone (LSTM) and outperform well-designed IMU-Only SOTAs. Thus, it is essential to maintain their original, optimized form without ToF input.\nW4: According to the official ST VL53L5CX Datasheet, the ToF sensor we use has ±15 mm noise at 1 m and ±50–60 mm noise at 3 m. We will add more details in the revision.\n\nAbstract: Yes, the training code will be released publicly upon acceptance.\n\nQ1&2&9: Thanks! We will revise the text as suggested.\n\nQ3: Our ToF measurements capture the distance to the first surface hit by emitted IR light, which may be the body or environment depending on pose and sensor direction.\n\nQ4: The ToF sensors are all placed at / near the ends of body skeleton chain (e.g., wrists and ankles) as:\n(1) This is a common practice in sparse mocap systems, e.g., PNP, TIP, etc., as measurements from end joints are essential for performing inverse kinematics to determine the full body pose; \n(2) For the feet, we ensure that the ground surface falls within the field of view (FoV) of the ToF sensors, enabling the system to better capture foot–ground distance, which helps to determine foot height when performing challenging slow motion, e.g., Taichi.\n\nQ5: By “hierarchical structure of multi-sensor data”, we refer not to sensor hardware hierarchy, but to the skeletal hierarchy of the human body (e.g., wrist and ankle are downstream of the shoulder and hip in the kinematic chain). \n\nQ6: By “intra-node data integration”, we mean fusing multiple modalities (IMU acceleration, orientation, ToF distance) collected at the same joint-level node. Each node hosts co-located IMU and ToF sensors. The term distinguishes local, per-node fusion from the later inter-node attention performed by the Transformer.\n\nQ7, Q18, Q19: To clarify, “integration” refers to feature-level fusion of IMU and ToF data at each node, not numerical integration. Like prior works (e.g., DIP, SIP), we use IMU-provided orientations.\n\nQ8: Dyn-PE is motivated by the fact that wearable sensor positions change during mocap, while conventional learnable positional encodings remain static and fail to model this dynamic spatial change. Our approach adapts positional encoding to sensor movement, and ablation studies show Dyn-PE significantly outperforms both static and absent positional encodings, validating its effectiveness.\n\nQ10: To clarify, our error analysis assumes zero original joint velocity and position—a common practice in IMU-based tracking without global initialization. We will make it explicit in the revision.\n\nQ11:\n**Proposition (Error Accumulation Analysis):**  \nFollowing standard statistical assumptions, we consider the acceleration measurement error at any timestamp $\\tau \\in (0, t)$ to follow a normal distribution:  \n$$\n\\epsilon_a(\\tau) \\sim \\mathcal{N}(\\mu, \\sigma^2)\n$$\n\nAssuming zero initial velocity and position, i.e., $\\mathbf{v}_0 = 0$, $\\mathbf{s}_0 = 0$, we derive the following:\n\n- **Joint velocity error distribution:**\n  $$\n  \\epsilon_v(t) \\sim \\mathcal{N}(\\mu t,\\ \\sigma^2 t)\n  $$\n\n- **Joint position error distribution:**\n  $$\n  \\epsilon_s(t) \\sim \\mathcal{N} \\left( \\frac{1}{2} \\mu t^2,\\ \\frac{1}{2} \\sigma^2 t^2 \\right)\n  $$\n\nQ12: Regarding the variance order, the cubic term arises from continuous integration under white noise, but in discrete sampling, noise terms are not fully accumulated due to zero-order hold and filtering effects, leading to an empirical quadratic variance growth (σ²t²), as commonly observed in prior works. \n\nQ13: Sec.3 highlights a core limitation of IMU-only motion capture: integrating acceleration leads to cumulative errors, with velocity and position errors growing linearly and quadratically over time. This motivates our use of ToF distance measurements to provide direct spatial constraints and underscores the need to augment IMU data with ToF.\n\nQ15: Our accelerometer and gyroscope share a single IMU module and are sampled together without added power cost, so we compare power usage between IMU-only and IMU+ToF setups.\nAs the table below shows, measured power usage includes both the MCU and the wireless transmission module.\n| Configuration         | Power Wireless Tx  | Power Standby|\n|-----------------------|---------|---------|\n| IMU-only board        | 0.76 W  | 0.24 W |\n| ToF-only board        | 0.93 W  | 0.40 W |\n| IMU+ToF integrated    | 0.97 W  | 0.45 W |\n\nAt the node level, the IMU consumes ~0.04–0.05 W (3.3 V), and the ToF sensor adds ~0.20–0.22 W (3.3 V + 1.8 V). Powered by a 1000 mAh, 3.7 V battery (3.7 Wh), the integrated board runs for up to ~4.3 hours under continuous wireless operation.\n\nQ16: All ToF sensors are rigidly co-mounted with IMUs on the same PCB, with optical axes aligned to the IMU’s local Z-axis to ensure consistent spatial reference. We will add the ToF pointing direction in Fig. 3 for clarity.\n\nQ17: We use a hybrid design: the Transformer models spatial dependencies across nodes, while LSTMs capture temporal dynamics. Though we haven't tested pure alternatives, this modular setup aligns with our goal of combining spatial and temporal cues effectively.\n\nQ20: Prior works often flatten sensor inputs, losing node-level structure. Our Node-Centric design treats each IMU-ToF pair as a token, and uses a Transformer to model inter-node relationships aligned with body topology, enabling structured, scalable fusion of spatial and temporal cues.\n\nQ21: The Dyn-PE can still provide distinguished position values to each sensor because each position \\phi is calculated by a DNN model and has different bias terms as follows:\nTable: Bias value of 6 sensors.\n\n| Joint         | Value（$\\phi$）   |\n|---------------|-------------|\n| Left Wrist    | 3.317   |\n| Right Wrist   | 2.885   |\n| Head          | -2.269  |\n| Root          | 2.084   |\n| Left Ankle    | -2.356  |\n| Right Ankle   | -2.158   |\n\nCos/sin embedding: We use both cos and sin to encode the position \\phi, which is the same as traditional Positional Encoding apart from that the position \\phi is no longer a static predefined value but a dynamic value calculated from real-time sensor data.\n\nQ22: We remove the TotalCapture Dataset from the original AMASS and use the rest subsets.\n\nQ23: We use shaped SMPL meshes in ToF data synthesis to make our model generalize better to different users.\n\nQ24: Our dataset includes 20 motion types such as walking, running, jumping and does contain fast-paced sequences. We will provide more examples in the revision. When excluding TotalCapture from the training set, we obtained a preliminary positional error of 4.52 cm. We will include the remaining results in the revision.\n\nQ25: We use orientation estimates from IMU hardware. Our method is window-based and performs frame-wise inference within each window.\n\nQ26: Please refer to our response to Reviewer 8u8c W4 for metric units. Since our work focuses on improving pose estimation, the analysis of global translation error is out of our scope.\n\nQ28: To ensure fairness, all methods (including both baseline IMU-only models and our ToF-IP) were fine-tuned on the ToF-IP-DB dataset under the same training pipeline and sensor layout.\n\nQ29: Due to time constraints, we conducted a targeted test by directly concatenating ToF data into the Dyna-IP (the closest SOTA method) input. The results below show no performance gain, suggesting that simple fusion is insufficient. This supports our claim that our node-centric learning (NCL) design is critical in utilizing ToF data.\n| Input Setting | SIP Error (°) | Angular Error (°) | Positional Error (cm) | Endpoint Error (cm) | Jitter (1000 m/s³) |\n| ------------- | ------------- | ----------------- | --------------------- | ------------------- | ------ |\n| DynaIP with ToF      | 20.34         | 13.42             | 7.35                  | 13.43               | 0.16   |\n| DynaIP without ToF   | 19.04         | 13.33             | 7.32                  | 13.05               | 0.16   |\n\nQ30: Our framework outputs at three stages: S1 predicts joint velocity, S2 predicts joint position, and S3 predicts SMPL joint angles (for final evaluation). While S1 and S2 are not directly evaluated, they are essential—velocity (S1) aids global position (as in PIP), and position (S2) captures spatial structure for angle prediction.\n\nQ31: Due to space constraints, please refer to our response to Reviewer K3tx, W2, Q4 and L1 for details on ToF noise, where the table in L1 shows that our method performs reliably in-the-wild. We will include additional demo videos in our revision.\n\nQ32: No, we do not explicitly add noise during simulation. Due to space constraints, please refer to our response to W2, Q4, where we discuss noise in more detail.\n\nQ33: The demo includes complex, free-form actions not seen in training, making minor jitter more noticeable and highlighting real-world generalization challenges. Our jitter metric is based on smoother benchmark data, so the visual–metric gap reflects motion complexity, not evaluation inconsistency."}]

Author rebuttal:

Meta-review: This paper received 2 Borderline Accept, 1 Accept and 1 Borderline Reject. 
The paper shows how the integration of ToF sensors and IMU improves pose tracking, in particular it addresses the typical drift of double integration in IMU trackers. 
The main concern of the negative reviewer was a lack of ethics form, but authors confirmed subjects signed and ethics form and are compliant with ethics guidelines in their institution but did not upload the ethics form due to anonymity. 
Reviewers overall acknowledge the novelty, the dataset introduced and the quality of results overall. The remaining concerns such as lack of results and ablations are not grounds for rejection. 
The AC recommends acceptance.
Decision: Accept (poster)