ImDy: Human Inverse Dynamics from Imitated Observations
Authors: Xinpeng Liu
Junxuan Liang
Zili Lin
Haowen Hou
Yong-Lu Li
Cewu Lu
Venue: ICLR 2025 Poster (2025)
Keywords: Motion Understanding
Inverse Dynamics
Biomechanics
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Inverse dynamics (ID), which aims at reproducing the driven torques from human kinematic observations, has been a critical tool for gait analysis. However, it is hindered from wider application to general motion due to its limited scalability. Conventional optimization-based ID requires expensive laboratory setups, restricting its availability. To alleviate this problem, we propose to exploit the recently progressive human motion imitation algorithms to learn human inverse dynamics in a data-driven manner. The key insight is that the human ID knowledge is implicitly possessed by motion imitators, though not directly applicable. In light of this, we devise an efficient data collection pipeline with state-of-the-art motion imitation algorithms and physics simulators, resulting in a large-scale human inverse dynamics benchmark as Imitated Dynamics (ImDy). ImDy contains over 150 hours of motion with joint torque and full-body ground reaction force data. With ImDy, we train a data-driven human inverse dynamics solver ImDyS(olver) in a fully supervised manner, which conducts ID and ground reaction force estimation simultaneously. Experiments on ImDy and real-world data demonstrate the impressive competency of ImDyS in human inverse dynamics and ground reaction force estimation. Moreover, the potential of ImDy(-S) as a fundamental motion analysis tool is exhibited with downstream applications. The project page is https://foruck.github.io/ImDy.

Published as a conference paper at ICLR 2025
IMDY: HUMAN INVERSE DYNAMICS FROM IMITATED OBSERVATIONS
Xinpeng Liu Junxuan Liang', Zili Lin', Haowen Hou?, Yong-Lu Li Cewu Lu Shannhai Jiao Tong University, SShanghai Innovation Institute, *Soochow University
ximpengliu09070gmai com, {whitefork, ( 1inzi1i111666 edu. on haowenhousout look. com, {yonglu.ii, Lucewu]esjtu..duu cn
ABSTRACT
Inverse dynamics (ID), which aims at reproducing the driven torques from human kinematic observations, has been a critical tool for human motion analysis. How- ever, is hindered from wider application to general motion due to its limited scal- ability. Conventional optimization-base ID requires expensive laboratory setups, restricting its availability. To alleviate this problem, we propose to exploit the recently progressive human motion imitation algorithms to lear human inverse dynamics data-driven manner. The key insight is that the human ID knowl- edge is implicitly possessed by motion imitators, though not directly applicable. In light of this, we devise an efficient data collection pipeline with state-of-the- motion imitation algorithms and physics simulators, resulting in large-scale human inverse dynamics benchmark as Imitated Dynamics (ImDy), ImDy con- tains over 150 hours of motion with joint torque and full-body ground reaction force data. With ImDy, we train data-driven human inverse dynamics solver ImDyS(olver) in fully supervised manner, which conducts and ground reac- tion force estimation simultaneously. Experiments ImDy and real-world data demonstrate the impressive competency of ImDyS in human inverse dynamics and ground reaction force estimation. Moreover, the potential of ImDy(-S) as fundamental motion analysis tool is exhibited with downstream applications. The project page is hntps://oouuk.gihuubbo///// github.io/lmDy
INTRODUCTION
The rapid progress in human motion capture based on computer vision has made an enormous amount of human motion data available to the research community (Mahmood et al. 2019 Mandery jtt al. 2016). The accumulation of human motion manages to push motion understanding forward in various tasks, including behavior understanding (Punnakkal et al. [221] Shahroudy et al. 2016 and character animation (Guo et al.] 2022; Tevet et al., 2023] Liu et al.] (025a]b). However, given the vision-based nature, most current efforts focus only visible kinematics information. The invisible factors, especially the dynamic factors, which could carry deeper insights into the underlying pro- duction mechanism of human motion, are typically overlooked, such as driven torques and ground reaction forces.
forces. This limits the current motion understanding algorithms from wider applications to domains where physical constraints must be seriously considered, such robotics (Figueredo al. 2020 Teramae et al. 2017), healthcare (Yao et al., 2018), and sports training (Caruntu & Moreno, To alleviate this, we focus on identifying the driven torques and ground reaction forces for human motion from pure kinematics MoCap data, known human inverse dynamics (ID).
2019
Human inverse dynamics, a basic step toward physical motion modeling, has been extensively dis- cussed by the biomechanics community for applications like gait analysis. A fundamental obstacle that it could not measured non-intrusively. Therefore, computationally expensive optimization- based methods are widely adopted and mature software is developed (Delp et al.) 2007 Damsgaard et al., 2006 Werling et al. 2021). However, accurately measured ground reaction forces are re- quired to ensure a determinate solution, which could be expensive and applicable only in restricted
"Corresoonding authors.

---

Published as a conference paper at ICLR 2025
laboratory settings. Also, the optimization process could be sensitive to small disturbances in ei- ther motion capture noises or subject variances. These make it hard to scale up for wider appli- cations to general motion. Given the success achieved by data-driven methods in CV and NLP, deep-learning-basee methods are proposed (Zell & Rosenhahn 2015 Zell et al. 2017 Ly et al.) 2016), aiming scalable human inverse dynamies with only kinematic observations as inputs. Unfortunately, data acquisition becomes ma- jor bottleneck since laboratory setups are still required for ground-truth acquisition.
Imitated Dynamics
Motion (in SMPL)
Joint Torque (Full body)
Given this, we project our sights on the recent progress of Imitation Learning (IL) (Luo et al. 2021 2023), which replicates recorded human motion through fully simulated humanoids with physical control signals, namely, joint torques. A key insight is that with the goal of kinematics phenomenon imitation, IL might also implicitly imitate the dynamics production mechanism, known as ID. However, IL is not directly appli- cable to ID. Despite the visual resemblances be- tween the recorded and simulated motion, kine- matic errors still exist. These errors could be neglected for kinematic analyses, however, for dynamic analysis, they could be amplified dras- tically (Uchida & Seth 2022). Moreover, exist- ing successful IL algorithms are typically based on joint-actuated SMPL (Loper et al.) (0055) avatars, whose physical properties and topol-
(Full Body)
Kick
Upper Body
Kneel
Down
Squat
Figure 1: ImDy pairs diverse SMPL motion data with dynamics including full-body torques and ground reaction forces (GRF) like the right knee GRF for kneeling, which could be hard to achieve under conventional laboratory setups.
ogy differ from real humans. To this end, extracting ID knowledge from IL becomes critical. Here, we adopt the state-of-the art motion IL algorithm (Luo et al. (203) and physics simulator (Makoviy- chuk al. 2021) to imitate recorded motions, extracting the observed kinematic states, joint torques, and the ground reaction forces, resulting in a large-scale human inverse dynamics database named Imitated Dynamics (ImDy) with more than 150-hour human motion. There
major merits of ImDy. First, it is scalable. Multiple samples could be concurrently collected in the simulator with- expensive laboratory setups, extending the border of ID data acquisition. As shown in Fig. [ we could even pair some rather complex motions with ID data, which is hard to achieve in laboratories. Second, is holistic. Beyond the ground reaction force and ID typically recorded in laboratories for previous efforts (Zell et al.) 2020: Mourot et al. 2022 Han et al.) (023). the physics simulator enables us to access the GRFs and joint torques of all human body segments, as shown in Fig. D
With the accumulated data, we could address the human inverse dynamics in fully supervised manner. Given the observed kinematics states that describe a motion transition in a certain period, train data-driven solver as ImDyy(olvve) to estimate the ground reaction forces and the internal dynamics to drive the transition. We also devise losses to regulate ImDyS with forward dynamics awareness and motion plausibility constraints.
We demonstrate the efficacy of ImDyS through a wide span of experiments. First, we evaluate our method on ImDy for basic performance illustration with simulated ImDy. Then ImDyS is evaluated on GroundLink (Han et al. 2023), which contains real-world ground reaction force. Fur- thermore, we demonstrate the efficacy of ImDy the recent real-world human dynamics dataset AddBiomechanies (Werling et al. 20255)
Our contribution could be summarized as: (1) We propose novel pipeline for human inverse dy- namics data collection, introducing a large-scale benchmark as ImDy. (2) Based on ImDy, a data- driven ID solver is instantiated as ImDyS. (3) Extensive experiments are conducted with analyses of the proposed data-driven methodology, demonstrating the feasibility of ImDyS.
BACKGROUND
Conventional Inverse Dynamics. Inverse dynamics, known as inferring forces/moments from kine- matic observations, have been discussed for long in the biomechanics community. In this literature,
2

---

Published as a conference paper at ICLR 2025
it is formulated optimization problem: given representative model of subject, the joint kinematics over time w.r.t the subject model, and the external forces, find the driving torques that produce the motion (Cchida & Delp 2021). The Newtonian dynamic equations are involved
+ + + G(g) +X+T,
where (q) is the generalized human inertia matrix w.r.t. generalized coordinate q. Cg q) is the the Jacobian matrix mapping external forces À to the generalized coordinates. Thus, the driven torques could be obtained by minimizing the difference between the left and right terms of Eq. [1] Mature software based on this has been de- veloped like OpenSim (Delp et al., 2007), AnyBody (Damsgaard al., 2006), and Nimble (Werling ctt al. 202T). addition, many efforts are made for clinical motion analysis (Fukuchi et al.) 2018 Schreiber & Moissenet 2019). However, these efforts are not as extensively recognized by the com- puter vision and computer graphics community as expected due to the scalability issue. Despite the elegant formulation, the efficacy of optimization-base heavily relies on the quality of external force (like GRF) measurement, whose cost could be non-trivial. Therefore, most of them focused on limited motion in laboratory settings. Some resort to wearable devices (Latella al.) 2016 2019) to partially mitigate the limitation. In addition, fitting the raw captured kinematic observations to specific human model for joint kinematics could be time-consuming and unstable, even with recent progress on it (Keller et al. 2023 Werling et al. (023).
Coriolis and centrifugal forces, G(q) represents gravity,
Learning-based Inverse Dynamics. With the progress in deep learning, there have been efforts to adopt neural networks to address the human ID problem. Many efforts focus on lower-bod body- only (Johnson & Ballard] 2014, Xiong et al., 2019) or upper-body-only (Manukian et al. 2023) inverse dynamics. More recently, Lvv et al. (2016) collected over hour of motion with an optical MoCap system, four force plates, and a pair of pressure insoles. The ground truth was obtained through optimization and Gaussian mixture framework was devised. Zell & Rosenhahn (2015);
Zell et al. (0177); Zell & Rosenhahn (017)) introduced a predictive dynamics-baas ing for the acquisition of ground truth. Hundreds of motions were collected and different data-driven techniques were adopted for joint torque regression. Zell et al. (2020) proposed a weakly supervised method based only on motion for gait analysis. These efforts were constrained by costly data quisition in real-world scenarios, resulting in limited data scale. Very recently, Werling et al. (2025) aggregated multiple existing biomechanics datasets, considerably boosting the data scale. How- ever, most of the collected sequences contained only regular exercise motion with limited diversity.
human model-
Some efforts focused on ground reaction forces such as (Rempe et al., 2020, Scott et al., 2020).
UnderPressure (Mourot et al. 2022), and GroundLink (Han et al. 2023)). Some recent works in- corporated inverse dynamics into vision- markerless MoCap systems. [Shimada al. (202T)
and Li et al. (2022) simultaneously captured motion and joint torques with customized fully dif- ferentiated pipelines. A series of works (Yi et al., 2022] Gartner et al.] 2022 Gärtner et al. 2022 Huang et al.) 2022] Wang al. 2023) imitated the captured motion in physical simulators with PD controllers and obtained the torques. However, an inherent problem is the amplification effect from kinematic errors dynamic errors. As measured by Uchida & Seth 2002). only a 2-cm uncertainty of marker placement in a marker-based MoCap system could result in a peak ankle plantarflexion moment of 26.6 N m. Considering the precision of current markerless MoCap algorithms, the accuracy of the accompanied inverse dynamics could be questionable. Also, among all these efforts for learning-based inverse dynamics, only a few (Zell et al. 2017 Zell & Rosenhahn, 2017 Zell et al. 2020) were quantitatively evaluated with limited locomotion data. A scalable benchmark for learningbaased inverse dynamics is still not available.
Motion Imitation. IL for human motion replicates recorded human motion sequences with phys- ically controlled simulated characters, which could be inherently close to ID. Most early efforts focus on specified usages with limited generalizability (Bergamin et al.] 2019 Peng et al. 2021 Won et al., 2021 2022 Peng et al., 2022). With residual force control (Yuan & Kitani 2019). which imposed supernatural forces at the root joint of the humanoid, Luo et al. (2021) generalized to 97% sequences in AMASS (Mahmood et al., 2019). Luo et al. (2023) eliminated the supernatural root force and achieved 98.9% success rate AMASS with fall-state recovery. The progress in human motion IL makes it possible to collect human like motions with full dynamics, shedding new light on the scalable human ID data collection.
3

---

Published as a conference paper at ICLR 2025
From large-scale motion Data
Simulated in Gym
Sg Observed state
Imitation Policy
À Contact Force
Action a -
kP (a*
q²)
Joint Torque
State imitate Sj
Imitated Dynamics
Figure 2: ImDy construction. We first train a motion imitation policy following Luo et al. 2023) Then, the policy is adopted to imitate arbitrary motions, with the imitated states recorded as ImDy.
CONSTRUCTING IMITATED DYNAMICS
ImDy aims to exploit the inherent closeness of inverse dynamics and imitation learning. Generally, the inverse dynamies (ID) and imitation algorithms (IL) could be abstracted as
(s ) 5 IL(S st+)),
with driven torque т, timestamp observed kinematic states 8o, and the state to imitate 8,, Both ID and IL learn the dynamic production mechanism of human motion. However, IL algorithms are not directly applicable to due to the non-equivalence between s*+1 and 8 The errors in kinematics could be magnified in dynamics (Uchida & Seth 2022). This also makes ID algorithms that are deeply coupled with markerless MoCap less reliable. However, it is possible to extract knowledge from II. for ID. In this section, introduce simple but effective ID data collection pipeline with IL algorithms. First, the adopted IL algorithm (Luo et al. 2023) is brietly covered in Sec. 3.1 Then, the data collection pipeline is introduced in Sec. 3.2 An overview is given in Fig.
3.1
IMITATION LEARNING BASICS
A motion imitator x (a |sc, st) is trained following Luo et al. (2023) to solve the Markov Decision Process M = S, R, 7)- The transition dynamics T and states S are governed by the physics simulator. For each timestamp t, the policy produces action a' E A and the reward R. based on
state S. The training goal is maximizing the reward expectation E EEE (
Transition. IsaacGym Makoviychuk et al. 2021) is adopted for simulation. 24-joint humanoid with SMPL (Loper al. (015) kinematics and physical properties following Luo et al. [2021 2023)
adopted with variable shape parameter 3 E IR 10 Thus, a human pose at timestamp could be defined as q¹ {0t p²), where 0' R the joint rotation in the 6d representation (Zhou al.
2019) and p² E J33 is the 3D joint position.
State. At timestamp s² contains the observed s5 and s!+1 to imitate. *6 is defined in simulator as st (gt, qr, (3) with 3D body pose gr. velocity ée. and body shape B. 5/+1 defined similarly except that it is the reference motion with finite-differentiated velocities.
Action. All joints but the pelvis are actuated with proportional derivative (PD) controllers, with a as the PD target. The torque applied could be calculated as
+
k. (a' g¹) kd o
Reward. The reward composed of four terms: motion imitation reward for minimizing the dif- ference between the imitated states and the expected states, fail-state recovery reward (Luo et al.) 2023). AMP reward (Peng et al., 2021), and energy reward to reduce jittering.
Training. Following PHC, three primitive policies are progressively trained with hard negative mining, two for pure motion imitation, and one for fail-state recovery. Then, a composer learns to combine the primitives dynamically. PPO (Schulman et al. 2017) is adopted to train the policies.
3.2 IMITATED DATA ACQUISITION
With the imitator म, we pursue to extract its inherent ID knowledge. As in Eq. though the imitator- produced is not accurate for sf, s since 4 is not guaranteed to reach, is accurate for
4

---

Published as a conference paper at ICLR 2025
Table 1: ImDy compared to related human dynamics datasets, Zell et (2020) recorded full-body
data but simplified the upper body with single torso segment. All previous efforts contain only GRF for feet (indicated with *), while we include full body GRF
Dataset
#Subj
Duration (h)
Dynamics
Body Repr.
Style
Zell et al.
2020
22
0.07
GRT & Torques
Partial Skeleton"
Real
Scott et at
2020
10
7.6
vertical GRF
Skeleton
Real
UnderPresstrre (Mournl
2022
5.5
verticul GRFs
Skeleton
GroundLink (Han
7
1.5
GRT
SMPL
Real
AddBiomechanies Werting ct al. 2025
273
57.6
GRF & Torques
Rajagopal al. 2016
Real
ImDy
435
152.3
GRF & Torques
SMILL
Simulated
Model Architecture
Sim2real two-stage training
Internal Dynamics
1
Motion State sf-we+ur:
Stage ImDy
Stage 2: AddBiomechanics
Contact Force 2
ImD Encader
ImD
ImDyS
Head
Encoder
Encoder
ImD2ys
ImDyy
Teaas Eeas 1:z
Head
Head
Tam norm
norm
norm
Prior
FD
1am
A
Lyrior
LED
Leser Lip Liprtor
Lpp. Leprior
Figure 3: ImDyS overview. Taking a motion transition, ImDyS predicts the internal dynamics and ground reaction forces. Moreover, a prior discriminator trained with the feature from ImDyS. A two-stage sim2real training curriculum is further designed.
o6 + Thus, the idea could be simple as using imitate arbitrary motions in the simulator, then collecting all the observed states Su. the applied torques т, and the full-body GRF X.
We adopt AMASS (Mahmood al. 2019) and KIT (Krebs al.] 2021) two major data sources. Sequences involving humans interacting with objects other than the ground are excluded, resulting over 50 hours of motion. All the sequences are re-sampled gravity axis. Then, the sequences are imitated three times by the two primitive policies and the multiplicative policy with a simulation frequency of 60Hz, resulting over 150 hours of human motion data with dynamics. States including q. Ú, 3 are recorded in synchronous with the torque т, all restored in the format of SMPL (Loper al. 2015) if possible. Moreover, GRFs for the whole body are also recorded, resulting in ImDy, large- scale human motion dynamics dataset.
30FPS, with the z-axis as the
Detailed statistics of ImDy are demonstrated in Tab. There are three major advantages. First, a considerably larger data scale is 100x compared to previous efforts with full-body dynamics data, covering a wide span of human motion, which could be hard to acquire laboratory setups. Second, thanks to the advanced simulator (Makoviychuk et al. (021), could include ground reaction forces for the whole body instead of the two feet only like in previous efforts. Finally, we represent humans with SMPL (Loper al. 2015), increasing availability.
4 LEARNING IMDYS
With the collected ImDy, we could address the human inverse dynamics a full-supervised manner with data-driven solver ImDyS. In Sec. 4.1 first introduce the formulation of data-driven inverse dynamics. Then, the proposed data-driven solver is introduced in Sec. 4.2 The overall pipeline of ImDyS is illustrated in Fig.
4.1 FORMULATION
Recall the abstraction of ID in Eq. which we rewrite as
x TmDysS(S w+ttw1+
5

---

Published as a conference paper at ICLR 2025
Given the kinematics states from timestamp t - to + w 1, ImDyS is required to estimate the internal dynamics 7 for the transition from s' to sh+ and the ground reaction forces that the subject bears in timestamp t and ++1. 1.
Motion States s could be represented by either SMPL parameters, joint angles, joint coordinates, marker coordinates. However, due to the topology divergence, the conversion among SMPL parameters, joint angles, and joint coordinates is non-trivial with limited performances. To guarantee that ImDyS could be seamlessly adopted to both ImDy and real-world biomechanics data, we adopt marker coordinates as motion state representation for ImDyS. The state s' - (m', rin') is composed of marker coordinates m² and differeeiifre velocities rn'' at timestamp t, which are easy to obtain for both ImDy and AddBiomechanies (Werling et al. (2025). Two temporal windows before and after the transition with a length of are included for contextual information. Notice that human physical properties like height and weight could also be implicitly represented by the markers. The states are canonicalized w.r.t. the heading direction of s'
Internal Dynamics т. For ImDy, the imposed angular momentum Tam is adopted for dynamics rep- resentation. Notice that Sec. 3.2 the original sequences are in 30FPS, while the simulation runs at 60FPS. This means for each motion transition (s', ((1), two torques were applied sequentially, each for 60 Predicting both torques is a plausible design choice. However, the second torque is based on the un-recorded mid-state between s", s Predicting it involves the forward dynamics from sf to the mid-state, with increased complexity. To this end, instead of predicting instantaneous torques, we switch to predicting the imposed angular momentum Tam E ( the time-accumulation effect of torque, for each motion transition. Thus, the modeling could stay consistent with proper complexity, only needing to sum the two torques up for s s and then multiply it with the delta time. For AddBiomechanics, joint torque Tjt is adopted for dynamics representation.
Ground Reaction Forces A. Different from previous efforts (Mourot al. 2022 Han et al. 2023 xx
Werling et al. 2025 with foot GRFs only, we predict full-body GRF À E R
as in Fig.
4.2 DATA-DRIVEN IMDYS
Model architecture. With the enormous data scale of ImDy, would like to keep ImDyS simple. An encoder-head structure is adopted. st-wit+++1 E Mx2x is first flattened as 8 1 E IR XMMxxx114441 (12m+112 with window size w and M markers. Then, transformer encoder converts & into ID feature Sin E R4, where dl is the feature dimension. For prediction, we de- compose Tam and À into magnitudes |rāml, | [ and direction vectors Fam Xta++1 and predict each of them with linear head. r/. is predicted with another linear head. The final predictions are Af = I am: p = || [xte++ and T/e-
Loss terms. L1 loss, cosine loss, and L2 loss are adopted to optimized the predicted magnitudes [Tām]] |, direction vectors F. Xt:++1 and joint torques Tye as Lmag- Leoe: LL respectively. Besides. a forward dynamics (FD) loss Lya is proposed with auxiliary FD model to inform the learning with the ID-FD cycle. The FD model takes gt-wir 7 = =(ram (i) T/.), X' as input, predicts the next-frame joint angles. The FD loss is thus computed with eycle consistency as
LFD = 1+ (D(ss- wet **, (.)
(5)
Finally, we devise a loss term similar to Peng et al. (2021) h, which encourages the ImDy feature fID to model physically plausible motion transitions. A linear discriminator takes fID and outputs logit indicating whether the motion transition is plausible. To train the discriminator, besides the positive samples from ImDy and AMASS (Mahmood et al. 2019). we propose two negative sample generation strategies. First, s-- w- is randomly permuted along the temporal axis. Second, random Gaussian noises are added on sf t-w:t++++1 Binary cross- cross-entropy loss is adopted as Lole-
Sim2Real training curriculum is devised in simple two-stage manner. In the first stage, ImDyS is trained on ImDy, with the overall loss as L,1 = a Lmag + +a + +agLFD + a Lagee In the second stage, freeze the encoder and train the linear head for joint torques Tja. The loss is calculated as L.2 = agLen + agLas + L12- Results show that ImDy pre-trained encoder converges fast on AddBiomechanies, indicating that it holds useful knowledge real-world human dynamics.

---

Published as a conference paper at ICLR 2025
PHC
Imays
|
x|
Cos.eeaeeeoo Knne
Laft Shouder
Figure 4: Qualitative results on ImDy. indicates a low-pass filter at 14Hz is applied. A typical gait sample and an arm man-wviing sample are visualized.
5
EXPERIMENTS
5.1
IMPLEMENTATION DETAILS
PHC (Loo 2023) adopted the position- conttrll mode implemented by IsaacGym (Makoviychuk et al. 2021). where the imposed torque is calculated differently from the naive PD controller and inaccessible. Therefore, we re-trained the PHC on AMASS (Mahmood et al. 2019) with the effort- control mode, and naive PD controller was adopted. Training the PHC took approximately 10 days, with a success rate on AMASS 91.3%. The window size w is set as to keep a short- term motion modeling, which is proven helpful in Sec. 5.3 The encoder of ImDyS is three- layer transformer with a dimension 64, ReLU activation, and LayerNorm. The loss weights are set 01 03 0.01, 02 44 05 = to maintain all terms similar numerical scales for training stability. ImDyS, the prior discriminator, and the FD model are all trained using the AdamW optimizer with batch size of 2,400 for 140 epochs on ImDy for the first stage. For the second stage, ImDyS is further tuned on AddBiomechanics for only 10 epochs with the same hyper- parameters. When generating negative samples for the prior discriminator, the two strategies are randomly adopted with a positive-negativy ratio of 1:1. We split ImDy into a training set of 27,501 sequences and conducted on single NVIDIA RTX3090 GPU.
of 3,055 sequences. All the data collection processes and experiments
5.2 EVALUATION ON IMDY
Metric. We calculate the mPJE (mean Per Joint Error) for т and À as
1 4yl2, mPJE
J -
mPJE,
(6)
where J is the number of joints. The result is further normalized by body weight to align different subjects, with units of N m -s/ky and N/kg. Specifically, the mPJE for the GRF on both feet mP mPJ Ex., is also reported.
Baseline. Few efforts except algorithms are feasible as baselines. this end, we introduce PHC a baseline, where the sequences in ImDy are re-imitated by the re-trained PHC. The imposed angular momentums and the GRF obtained via the re-imitation process are adopted as the baseline predictions. With this baseline, we demonstrate the amplification effect from the kinematics error to the dynamics error, thus validating the performance of directly adopting IL for ID.
Results. Quantitative results shown in Tab. 2 PHC produces mPJPE of 56.13 mm, which admirable for kinematics but results in high dynamics errors. ImDyS demonstrates considerably better performance. We further visualize two qualitative samples in Fig. Since the raw data could

---

Published as a conference paper at ICLR 2025
GT PHC
Table 2: Quantitative results on ImDy.
ImDy5
mPJE is normalized by the body weight.
Groundl inkNet
Methods
PHC
ImDy8
mn MP
E. (N (iss/kg)
0.055
0.021
2
MW
in MPJ.R (N/kg) 4
0.409
0.289
m mP.. (N/kg) L
3.034
1.843
Left Foot
mP.. (N/kg)
2.866
1.842
[2 1
Table Ground reaction force predic- tion results on Groundl.ink.
Methods
Gnn ImDyS
mJ (N7kg) mPJEx, (N/kg)
2.362
5.423
0.986
Right Foot
1.149
[2 21
Table 4: Quantitative results on Ad-
dBiomechanics.
Figure 5: Qualitative results on GroundLink including PHC. GroNndLinkNet, and ImDyS. The GRF for both feet are shown. Surprisingly, ImDyS provides better con- sistency with the ground truth.
Baseline (150 epochs)
ImDy5
Methods
(10 epochs)
mPJE, E2 (Nm/kg))
0.1699
0.1626
mPEEx (Nm//g)
1.0876
1,6333 18.1
be jittering, we also filter the predictions with low-pass filter 14Hz, denoted as +, A, which helps reveal the general trend of the predictions. For the gait sample at the left, the imposed angular momentum т the left hip and the left knee are plotted, along with the GRF at the left toe. We also plot the error between the predicted values and GT values. ImDyS manages to faithfully reconstruct for the left knee and hip with minor errors. Meanwhile, PHC typically produces higher errors due to phase mismatch. As shown, it tends to lag behind the input motion. For GRF, ImDyS also produces reasonable predictions. Besides a typical gait analysis sample, we also demonstrate the performance of ImDyS with an arm-waving motion. The at directly related body segments including the left thorax and shoulder is visualized. ImDyS reproduces the dynamic status with better alignment to GT compared to PHC. Generally, ImDyS produces reasonable ID predictions. A potential issue is the jittering prediction, which is a consequence of the jittering observations in ImDy. However, show that ImDyS could handle real-worke smooth observations well even when trained only on jittering ImDyS. More demonstrations are available in the supplementary video.
5.3 EVALUATION ON GROUNDLINK
Metrics. GroundLink (Han et al. 2023) provides 1.5-hour motion from 7 subjects with GRF. We adopt subject 7 for evaluation. EPJ both feet normalized by body weight is reported.
We also report the performance of BroundL.inkNer (Han et al. 2023). PHC and ImDyS are not exposed to GroundLin Link during training. resulting in a zero-shot eva evaluation for ImDyS and the PHC baseline. Also, Groundl on 250FPS motion, while ImDyS and the PHC re-imitation baseline only operate on 30FPS motion. Finally, GroundLinkNer predicts GRF for both feet, while ImDyS and PHC could decouple feet into ankles and toes, and predict GRF separately for each part. We add up the ankle GRF and the toe GRF as the foot GRF. All predictions are re-sampled to 30FPS.
Baselines. PHC is evaluated similarly to Sec. 5.2
operates
Quantitative results are illustrated in Tab. 3 Surprisingly. both ImDyS and PHC manage to outperform the specifically trained GroundLinkNet. We attribute this to the enormous scale of AMASS and ImDy, which much larger than Groundl.ink. Moreover, even though ImDyS is trained on simulated ImDy only, it generalizes to real-world data with competitive performance. We visualize the results Fig. 5 7 The PHC re-imitation baseline produces jittering predictions similar to Fig. 4 GroundLinkNet. though specifically trained on GroundLink, fails to capture the rapid GRF changes in this jumping jack motion, resulting in relatively flat output. In contrast, ImDyS surprisingly presents good consistency with GT, and even faithfully reproduces the intense peak GRFs for the left foot for the jumping jack. Besides, the prediction is not as jittering as in Fig. 4 indicating ImDyS could handle real-world smooth data well.
Results.
8

---

Published as a conference paper at ICLR 2025
ET
Table 5: Ablation study ImDy.
Baseline
mPJE,
mPJEs
mPJEE
mPJEx.
Methods
(Nms/kg)
(N/kg)
(N/kgN)
(N/kg)
Subratrr Angle
TmiDyS
11.021
012889
1.843
1.842
ImDys-SMPL
0.011
0.272
1.746
1.764
InDyS-Joimt
0.014
0.273
1.755
1.775
Angle
w/o LED
0.023
0.302
1.962
1.976
w/o Lets
0.022
0.294
1.884
1.891
Libre Angle
Table 6: Ablation study AddBiomechanics.
1-1p Adduction
ImDyS
ImDyS
ImDyS
Methods
ar=2
wse1
ur3
mJEE, (Nm/kg) 4
0.1626
0.1690
0.1720
Figure 6: Joint torque predictions Ad- dBiomechanics.
MPJE, E (Nm/kg)
1.0633
1.0990
1.1030
5.4 EVALUATION ON ADDBIOMECHANICS
Metries. AddBiomechanics ( (Werling et al. 2025 is recently proposed with over 50 hours of human dynamics data from 273 subjects. We adopt armless part of this dataset. We follow the train/test split in Addbiomechanics and report mPJE for the joint torque normalized by body weight.
Results. baseline model trained only on AddBiomechanics for 150 epochs with the same archi- tecture as ImDyS is reported to showcase the generalization from ImDy to real-world dynamies. All data are re-sampled to FPS. Quantitative results are illustrated in Tab. 4 ImDyS outperforms the baseline with faster convergence, indicating the efficacy of Imdys in pre-training and mitigating the sim2real gap. Qualitative results are shown in Fig. 6 where ImDyS shows better alignment with GT and more precise magnitude predictions. More analyses on the relationship between per- formance, data distribution, and quality are in the appendix.
5.5 ABLATION STUDIES
Different Motion Representations are evaluated on ImDy in Tab. Though SMPL and joint-based based representations perform better, we adopt marker-based representation for generality.
Different Loss Terms are evaluated in Tab. LFD is proven to contribute more than Lete.
Different Window Sizes w are evaluated AddBiomechanics in Tab. ImDyS achieves the best balance between rich contexts and conciseness with w 2.
DISCUSSION
Given the fully simulated nature of ImDy, a reasonable question is the sim2real problem. ImDy could be unnaturally jittering as in Fig. 4 Also, the physical properties of the simulated humanoid differ from those of real humans. Empirically, experiments show that ImDyS generalizes well to real-world data, partially mitigating this gap. The reason could be threefold. First, the jitters are unnatural but still physically plausible given that ImDy faithfully preserves consistent information for the simulated physics phenomena. Second, the small window size of ImDyS prevents it from relying on long-term contexts, where jitters are more salient. Finally, the enormous scale of ImDy is helpful for generalization. To further mitigate the sim2real gap with ImDy is meaningful goal to pursue. Besides, ImDyS is designed as a first-step bascline to demonstrate the efficacy of ImDy. Introducing more sophisticated designs regulate the behavior of ImDyS would be preferable. Moreover, ImDy only considers GRF, while other external forces are not involved. Also, interaction with other entities is absent. Exploration of these would be interesting for future works.
CONCLUSION
Leveraging the inherent resemblance between inverse dynamics and imitation learning, we proposed novel human dynamics dataset ImDy, which contained over 150 hours of human motion paired

---

Published as a conference paper at ICLR 2025
GT
ImDyS
GroundLinkNet
Tennis Serve
Ballet Small Jump
Chair
Hap
Tennis Ground Stroke
Jumpinn Jack
RF
Figure 7: Extensive visualization on GroundLint Link. ImDyS shows superior alignment with GT for various motions compared specifically trained Croundl.inkNer showcasing the efficacy of ImDy. Especially, the intense peaks are also reproduced by ImDyS.
GT
Baseline
ImDyS
L-Hip Flexion
Lumbar Rotation
R-Htip Flexton
R-Hip
Rotation
R-Knee
Angle
Flexion
Figure 8: Extensive visualization on AddBiomechanies. ImDyS demonstrates superior performance to the baseline, indicating ImDy's generalization ability.
with full-body driven torques and GRFs from well-developed simulator and imitation algorithms. Based on ImDy, data-driven human inverse dynamics solver ImDyS is devised to reconstruct the driven angular momentum and contact forces from kinematic observations. ImDyS demonstrated impressive performance on both simulated and real-world data. As a first step toward scalable and easily accessible human inverse dynamics, we hope ImDy can shed new light on the data-driven physical analysis of human motion.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENTS
This work is supported in part by the National Natural Science Foundation China under Grant No.62306175, CCF-Tencent Rhino-Bird Open Research Fund.
REFERENCES
Kevin Bergamin, Simon Clavet, Daniel Holden, and James Richard Forbes. Drecon: data-driven responsive control of physics-based characters. ACM Transactions On Graphics (TOG), 38(6):
2019.
Dumitru I Caruntu and Ricardo Moreno. Human knee inverse dynamics model of vertical jump exercise. Journal of Compulational and Nonlinear Dynamics, 14(10):10105, 2019.
Michael Damsgaard, John Rasmussen, Søren Tørholm Christensen, Egidijus Surma, and Mark De Zee. Analysis of musculoskeletal systems in the anybody modeling system. Simulation Mod- elling Practice and Theory, 14(8): 11111100111111111111111111111111 2006.
Scott L Delp, Frank C Anderson, Allison S Amold, Peter Loan, Ayman Habib, Chand T John, Eran Guendelman, and Darryl G Thelen. Opensim: open-source software to create and analyze dynamic simulations of movement. IEEE transactions on biomedical engineering, 54(1 1):1940- 1950, 2007.
Luis FC Figueredo, Rafael Castro Aguiar, Lipeng Chen, Samit Chakrabarty, Mehmet R Dogar, and Anthony G Cohn. Human comfortability: Integrating ergonomics and infoorer-informmed metrics for manipulability analysis during human-robot collaboration. IEEE Robotics and Automation Letters, 6(2):351-358 2020.
Claudiane A Fukuchi, Reginaldo Fukuchi, and Marcos Duarte. A public dataset overground and treadmill walking kinematics and kinetics in healthy individuals. PeerJ, 6:e4640, 2018.
E. Gartner, M. Andriluka, E. Coumans, and C. Sminchisescu. Differentiable dynamics for ar- ticulated 3d human motion reconstruction. In 2022 IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), pp. 13180-13 190. Los Alamitos, CA, USA. jun 2022. IEEE Computer Society. doi: 10.1 SD09CCVPRS2288.20222200 reeeconputersoceety.. .rt.iy..rrg/i 100 1109/
URL https::/doi.
1111//101992222 http///VR5522288 2022. 01284
Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng, Generating
diverse and natural 3d human motions from text. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5142-5151, Los Alamitos, CA, USA, 2022. IEEE Computer Society, doi: 10. 10/.0//RRRP2888.... 2022.00509.
Erik Gärtner, Mykhaylo Andriluka, Hongyi Xu, and Cristian Sminchisescu. Trajectory optimization for physics-basel reconstruction of 3d human pose from monocular vidco. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13096-13105, Los Alami- tos, CA, USA, 2022. IEEE Computer Society. doi: 10. 10/11099/V8552688.... 2ttpC//VPPR5268222220177777
Xingjian Han, Ben Senderling, Stanley To, Deepak Kumar, Emily Whiting, and Jun Saito. Groundlink: A dataset unifying human body movement and ground reaction dynamics. In SIGGRAPH Asia 2023 Conference Papers. SA '23, New York, NY, USA, 2023. Association for Computing Machinery. ISBN 9798400703157. doi: 10.1145/3610548.6118
URL
B. Huang, L. Pan, Yang. J. Ju, and Y. Wang. Neural mocon: Neural motion control for physically
plausible human motion capture. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6407-6416, Los Alamitos, CA, USA, jun 2022. IEEE Computer Society. URL https: //o::///oo... eeecoopptersoccctyy
doi: N0.1109/CVPR52688 htw.:/PP55668822222.000 202200062...2.2
org 1.c0/00..10/////R2222533 2ww.::255555222222.222222.2222 00631
Leif Johnson and Dana H. Ballard. Efficient codes for inverse dynamics during walking. In Carla E. Brodley and Peter Stone (eds.), Proceedings of the
AAAI Conference Artificial Intelligence, pp. 343-349, Québec City, Québec, Canada, 2014. AAAI Press, doi: 10.16.9/AAAL. V2811.8747, URL htt.hhtts//./ pttttss:/ddoi.orgg./t http://100.1111188111 609/aaai v28i1 .8777

---

Published as a conference paper at ICLR 2025
Marilyn Keller, Keenon Werling, Soyong Shin, Scott Delp, Sergi Pujades, C Karen Liu, and Michael J Black. From skin to skeleton: Towards biomechanically accurate 3d digital humans. ACM Transactions on Graphics (TOG), 42(6):1-12, 2023.
Franziska Krebs, Andre Meixner, Isabel Patzer, and Tamim Asfour. The kit bimanual manipulation dataset. In IEEE/RAS International Conference on Humanoid Robots (Humanoids), pp. 499-506, 2021.
Claudia Latella, Naveen Kuppuswamy, Francesco Romano, Silvio Traversaro, and Francesco Nori. Whole-body human inverse dynamics with distributed cerro-accelerometers, gyros and force sensing. Sensors, 16(5). 2016. ISSN 1424-8220. doi: 10.3390/s16050777 URL https:
www.mddp... com 1424
Claudia Latella, Silvio Traversaro, Diego Ferigo, Yeshasvi Tirupachuri, Lorenzo Rapetti, Fran- cisco Javier Andrade Chavez, Francesco Nori, and Daniele Pucei. Simultaneous floating-base estimation of human kinematics and joint torques. Sensors, 19(1)), 2019. ISSN 1424-8220. doi: 10.3390/s19122799 URLpttts::///
www mdpi. com 1424 8200 19/12/2919///
Jiefeng Li, Siyuan Bian, Chao Xu, Gang Liu, Gang Yu, and Cewu Lu. D &d: Learning human dynamics from dynamic camera. In Computer Vision - ECCV 2022: 17th European Conference, Berlin, Heidelherg, 28. URL
Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part V, pp.
2022. Springer- Verlag. ISBN 978-3-031-10 -00---001-099, 1007 / 978 3 031 20065- 6_
doi: 10.1007/978
https:///bi.orgg
Xinpeng Liu, Haowen Hou, Yanchao Yang, Yong-Lu Li, and Cewu Lu. Revisit human-scene inter- action via space occupancy. In European Conference Computer Vision, pp. 1-19. Springer, 2025a.
Xinpeng Liu, Yong-Lu Li, Ailing Zeng, Zizheng Zhou, Yang You, and Cewu Lu. Bridging the gap between human motion and action semantics via kinematic phrases. In European Conference Computer Vision, pp. 223-240. Springer, 2025b.
Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Mole and Michael J. Black. Smpl: skinned multi person linear model. In ACM Transactions Graphics, volume 34, New York, NY, USA, oct 2015. Association for Computing Machinery. doi: 10.1145/2816799528180000 URL https:::p:..... /doi .org/ 10. 1145/ 281 6795. 281 28188113111
Z. Luo, J. Cao, A. Winkler, K. Kitani, and W. Xu. Perpetual humanoid control for real- time simulated avatars. In 2023 IEEF/CVE International Conference on Computer Vision (ICCV), pp. 10861-10870, Los Alamitos, CA, USA, oct 2023. IEEE Computer Society. doi: URL
10. 10.1109/CCC551000.222...00 10. 10.1109/ http//CCCV510000 210.0...223..2.0 01000
Zhengyi Luo, Ryo Hachiuma, Ye Yuan, and Kris Kitani. Dynamics-regulated kinematic policy for egocentric pose estimation. Advances in Neural Information Processing Systems, 34:25019- 25032, 2021.
Xiaolei Lv. Jinxiang Chai, and Shihong Xia. Data-driven inverse dynamics for human motion. ACM Transactions on Graphics (TOG), 1-16 2016.
N. Mahmood, N. Ghorbani, N. F. Troje, G. Pons-Moll, and M. Black. Amass: Archive of motion capture as surface shapes. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 1441154550, Los Alamitos, CA, USA, nov 2019. IEEE Computer Society. doi: 10. 1109/CCV.2099 005544. htth:tts::/ http://ee....eeerooooooooe.ooooo ICCV. 2019. 00554
Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, and Gavriel State. Isaac gym: High performance GPU based physics simulation for robot learning. In Joaquin Vanschoren and Sii-Kit Yeung (eds.), Proceedings of the Neural Information Pro- cessing Systems Track Datasets and Benchmarks, Virtual, 2021. Curran Associates Inc. sts.e owtpp///eeeeaaae......oooo/o//// p007//aar/2ddde33eee//54444fff62
html
12

---

Published as a conference paper at ICLR 2025
Christian Mandery, Ömer Terlemez, Martin Do, Nikolaus Vahrenkamp, and Tamim Asfour. Uni- fying representations and large-scale whole-body motion databases for studying human motion. IEEE Transactions Robotics, 32(4):796-80) 2016.
Mykhailo Manukian, Serhii Bahdasariants, and Sergiy Yakovenko. Artificial physics engine for real-time inverse dynamics of arm and hand movement. Plos one, 18(12):c0255750, 2023.
Lucas Mourot, Ludovic Hoyet, François Le Clerc, and Pierre Hellier. Underpressure: Deep learning for foot contact detection, ground reaction force estimation and footskate cleanup. Computer Graphics Forum, 4 (1)8)
2022. doi: o/on1 ne aebbbrryy ry. wi ley- com www.o//oo///oooooooooooo///ooooo /tad////b///// 1tt1:/0..1111//101 cgf.
14635.
URL https:
Xue Bin Peng, Ze Ma, Pieter Abbeel, Sergey Levine, and Angjoo Kanazawa. Amp: Adversarial motion priors for stylized physics- based character control. ACM Transactions on Graphics (ToG), -00, 2021.
Xue Bin Peng, Yunrong Guo, Lina Halper, Sergey Levine, and Sanja Fidler. Ase: Large-scale reusable adversarial skill embeddings for physically simulated characters. ACM Transactions On Graphics (TOG), 41 141: 2022.
A. R. Punnakkal, A. Chandrasekaran, N. Athanasiou, A. Quiros- Quiros-Ramirez and M. J. Black. Ba- bel: Bodies, action and behavior with english labels. In 2021 IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pp. 722-731, Los Alamitos, CA, USA, jun 2021. IEEE Computer Society. doi: 10.1109/CVPR446477.2000 000088. URL https: C0078
//doi //doi.ieeccompeeeeooo
Apoorva Rajagopal, Christopher L Dembia, Matthew S DeMers, Denny D Delp. Jennifer L Hicks, and Scott L Delp. Full-body musculoskeletal model for muscle-driven simulation of human gait. IEEE transactions on biomedical engineering, 63(10)22068 2079, 2016.
Davis Rempe, Leonidas J. Guibas, Aaron Hertzmann, Bryan Russell, Ruben Villegas, and Jimei Yang. Contact and human dynamics from monocular video. In Computer Vision - ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28 2020, Proceedings, Part V. pp. 71-87, Berlin, Heidelberg, 2020. Peringer-Verlag. ISBN 978-3-030-58557-00 978-3-030-58558-77
doi: 10.1007/
httpstt:::/// http///o..o......oo//oo//o/o.
978 0-000//97788--00555-------
Céline Schreiber and Florent Moissenet. A multimodal dataset of human gait at different walking speeds established on injury-free adult participants. Scientific data, 6(1):111, 2019,
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. preprint on webpage at ar TtV.:1707.06344 2017.
Jesse Scott, Bharadwaj Ravichandran, Christopher Funk, Robert T. Collins, and Yanxi Liu. From image to stability: Learning dynamics from human pose. In Computer Vision - ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXIII, pp. 536-554, Berlin, Heidelberg, 2020. Springer-Verlat ISBN 978-3-003-5555114, doi: 1007/
978-3-030-58592-11 1.32. URL https:/ hupps://dol. http://to..o...../////o 00--:00007770------000 32
A. Shahroudy, Liu, T. Ng. and G. Wang. Ntu rgb+d: A large scale dataset for 3d hu- man activity analysis. In 2016 IEEE Conference on Computer Vision and Pattern Recogni- tion (CVPR), pp. 1010-1019, Los Alamitos, CA, USA, jun 2016. IEEE Computer Society. doi: 00.1109//VPR.201..11. URL hitts:://aaaa wwt.bb......ccppttsssccctyyy.ooo 2109/CVPR.2016.111
Soshi Shimada, Vladislav Golyanik, Weipeng Xu, Patrick Pérez, and Christian Theobalt. Neural monocular 3d human motion capture with physical awareness. ACM Transactions on Graphics
(ToG), 4055):1-15, 2021.
Tatsuya Teramae, Tomoyuki Noda, and Jun Morimoto. Emg-based based model predictive control for physical human robot interaction: Application for assist-as- control. IEEE Robotics and Automation Letters, 3(1):210-217, 2017.
13

---

Published as a conference paper at ICLR 2025
Guy Tevet, Sigal Raab, Brian Gordon, Yonatan Shafir, Daniel Cohen- and Amit Haim Bermano. Human motion diffusion model. In The Eleventh International Conference on Learning Represen- tations ICLR 2023, Kigali, Rwanda, 2023. OpenReview.net. htth:t/:: /ttp://///ppeeeeeee.../
net/pdff? d kSy02jw
Thomas K Uchida and Scott L Delp. Biomechanics of movement: the science of sports, robotics, and rehabilitation. Mit Press, 2021.
Thomas K Uchida and Ajay Seth. Conclusion or illusion: Quantifying uncertainty in inverse anal- yses from marker-based motion capture due errors in marker registration and model scaling. Frontiers in Bioengineering and Biotechnology, 10:874725, 2022.
J. Wang, Yuan, Z. Luo, K. Xie, D. Lin, U. Iqbal, S. Fidler, and S. Khamis. Learning human dynamics in autonomous driving scenarios. In 2023 IEEE/CVF International Con- ference on Computer Vision (ICCV), pp. 20739-20749, Los Alamitos, CA, USA, oct 2023. IEEE Computer Society. doi: 10.10990CCV5100002233000000 seecomputersociety. 1tt.////1/11111/1111111000000000 2023. 01901
URL https://doi,
Keenon Werling, Dalton Omens, Jeongseok Lee, Ioannis Exarchos, and C. Karen Liu. Fast and Feature-Complee Differentiable Physics Engine for Articulated Rigid Bodies with Contact Con- straints. In Robotics: Science and Systems XVII, Virtual Event, July 12-16, 2021, Virtual, July 2021. RSS Foundation. doi: 10.15607/RSS.2022 2021.XVII.0344 URL https:: //doi oorg/10.
15607/RSS 2021.0022 XWII.VIII... 034
Keenon Werling, Nicholas A Bianco, Michael Raitor, Jon Stingel, Jennifer L Hicks, Steven H Collins, Scott L Delp, and C Karen Liu. Addbiomechanies: Automating model scaling, inverse kinematics, and inverse dynamics from human motion data through sequential optimization. Plos one, 18(11):a025115, 2023.
Keenon Werling, Janelle Kaneda, Tian Tan, Rishi Agarwal, Six Skov, Tom Van Wouwe, Scott Uhlrich, Nicholas Bianco, Carmichael Ong, Antoine Falisse, al. Addbiomechanics dataset: Capturing the physics of human motion at scale. In European Conference on Computer Vision, pp. 490-508. Springer, 2025.
Jungdam Won, Deepak Gopinath, and Jessica Hodgins. Control strategies for physically simulated characters performing two-player competitive sports. ACM Transactions on Graphics (TOG), 40 (4):1-11, 2021.
Jungdam Won, Deepak Gopinath, and Jessica Hodgins. Physics-based character controllers using conditional vaes. ACM Transactions on Graphics (TOG), 41(4):1-22, 2022.
Baoping Xiong, Nianyin Zeng, Han Li, Yuan Yang, Yurong Li, Meilan Huang, Wuxiang Shi, Min Du, and Yudong Zhang. Intelligent prediction of human lower extremity joint moment: an artifi- cial neural network approach. leee Access, 7:29973 299970, 2019.
Shaowei Yao, Yu Zhuang, Zhijun Li, and Rong Song. Adaptive admittance control for an ankle exoskeleton using an emg-driven musculoskeletal model. Frontiers in neurorobotics, 12:16, 2018.
X. Yi, Y. Zhou, M. Habermann, S. Shimada, V. Golyanik, C. Theobalt, and F. Xu. Phys- ical inertial poser (pip): Physics-aware real-time human motion tracking from sparse iner- tial sensors. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13157-13168, Los Alamitos, CA. USA, jun 2022. IEEE Computer Society. doi: 10.1109/CVPRS26883 2022. 1009 CttCC//RR55268888 2022. 01282
URL https:: /wi. http::://o...eeeeeeeeeeeoo www.oppttttt...ss.c.............
Y. Yuan and K. Kitani. Ego-pose estimation and forecasting real-time pd control. In 2019 JEFF/CVF International Conference on Computer Vision (ICCV), pp. 10081-10091, Los Alami- tos, CA, USA, 2019. IEEE Computer Society. doi: 10.1 109/1CCC.2019.000008 URL
wtt.:/ICVV.2221..00
P. Zell and B. Rosenhahn. Learning- based inverse dynamics of human motion. In 2017 IEEE International Conference on Computer Vision Workshop (ICCVW), pp. 842-850, Los Alamitos,
CA, USA, oct 2017. IEEE Computer Society. doi: 10.1 10.1100//....217.... 104. URL https: TCCVM 2017. 2017..0..1111
7/doi. weecompptersooccttyyyyoooo
14

---

Published as a conference paper at ICLR 2025
P. Zell, B. Wandt, and B. Rosenhahn. Joint 3d human motion capture and physical analysis from monocular videos. 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 17-26, Los Alamitos, CA, USA, jul 2017. IEEE Computer Society. URI. hhttp:://do.
doi: 111100CCPRW.2017..
11C9/CVPRM. 2017.5
Petrissa Zell and Bodo Rosenhahn. A physics-based statistical model for human gait analysis. In Juergen Gall, Peter Gehler, and Bastian Leibe (eds.), Pattern Recognition, pp. 169-180, Cham, 2015. Springer International Publishing. ISBN 978-3-319-24947-6
Petrissa Zell, Bodo Rosenhahn, and Bastian Wandt. Weakly-superrised learning of human dynam- ics. In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (eds.). Computer Vision ECCV 2020, pp. 68-84, Cham, 2020. Springer International Publishing. ISBN 978-3- 030-58574-7.
Y. Zhou, C. Barnes, J. Lu, Yang, and H. Li. On the continuity of rotation representations in neural networks. In 2019 JEEE/CVF Conference on Computer Vision and Pattern Recog- nition (CVPR), pp. 5738-5746, Los Alamitos, CA, USA, jun 2019. IEEE Computer Society. doi: N..119/CCVP.200190000
URL. https: //doi. eeccompttersocietyyoo
10. 1109/VVRR 2019.0058.
15

---

Published as a conference paper at ICLR 2025
APPENDIX
A APPLICATIONS OF IMDYS
In this section, we demonstrate some downstream applications of ImDyS.
Human Work Analysis. With the predicted T, we could calculate the work conducted at each joint. Visualizations are in Fig. 9 reasonably revealing the energy flow during human motion.
Left Ankle Work (J)
Left Knee Work (J)
Left Hip Work (J)
Figure 9: Human work visualization with ImDyS prediction. Green indicates positive work and red indicates negative work.
Motion Assessment. Another interesting application of ImDyS is based on the discriminator in- troduced in Sec. 4.2 Besides facilitating ImDyS learning, could also assess whether motion
transition is physically plausible as in Fig. I0 Specifically, adopt ImDyS to assess the motion
generated from MDM Tevet et al. (2023). ImDyS reasonably tells when the motion starts to deviate
from realism.
Indicator
Figure 10: Motion assessment visualization. The motion artifacts are annotated with red with a low indicator value from ImDyS. As shown, ImDyS manages to identify implausible transitions in kicking motion generated by MDM.
16

---

Published as a conference paper at ICLR 2025
GT
Baseline ImDys
L-Ankle Angle
R-Ankle Angle
L-Knee
Angle
lumbar
Rotation
Figure 11:
Visualization of failed joint torque prediction case on AddBiomechanies. For the "jumping" motion, the baseline and ImDyS both perform sub-optimally. Neither of them correctly predicts the joint torques.
AddBiomechanics
ImDy
stand stand
raape
kamp
large
Data distribution of AddBiomechanics and ImDy. Among all activities, walking and running account for over 75% of AddBiomechanics. In comparison, according to the annotations from BABEL (Punnakkal et al.) 2021), ImDyS is less imbalanced with better diversity.
Figure 12:
B ADDBIOMECHANICS RESULTS ANALYSIS
We visualize a failure case on the AddBiomechanics dataset in Fig. II As shown, neither the baseline nor ImDyS manages to faithfully predict the joint torques for the jumping motion. In the following, we discuss the reasons for the failure.
Data distribution. Fig. 12 shows the data distribution of AddBiomechanies (Werling et al. 2025 As shown, over 75% of the data are either walking, running, or standing, which are extremely limited. Though ImDyS is empowered with the diverse ImDy shown in Fig. T2 it still requires
17

---

Published as a conference paper at ICLR 2025
DmDyS better
Baseline better
250
0333
0833
4ni AMP [(Ninkg)
#5005
200
#seqs
0,33
AmPJE
0333
2013
0-0.1
0.1-0.2
0.2-0.3
0.3-0.4
0.4-0.5
0.5-0.6
0.6-0.7
0.7-0.8
0.8-0.9
Residual Tarque/NM//aa
Antertor polvic th:
Lotekle
LoAnble
I-dnblle
Angle
Angee
Angle
RAAekla
RAAriiia
Angle
Angl
Angle
Figure 13: Relationship between data quality and model performance differences. Higher residual torque indicates lower data quality with lower reliability of the optimized GT torques. AmPJE is the difference between the mPJE of ImDyS and the baseline. #seqs the number of sequences. With the residual torque increasing, the baseline provides lower mPJE than ImDyS, indicating the baseline overfits low-quality data. Instead, ImDyS, with the knowledge inherited from ImDy, shows less overfitting for these cases.
data to learn the mapping between simulated torques and real torques for non-gait data. These result in ImDyS' poor performance when processing non-gait data. Further mitigating the limited data issue for non-gait motions would be a meaningful goal to pursue.
Data quality. Besides the distribution, the quality is also limited in AddBiomechanics, As shown Fig. IT joint torques for some joints (like the lumbar) suffer from unstable optimization with classified with
jittering results. According to Werling al. (2025) 21.2% of AddBiomechnics clinical- grade high quality (residual torque 0.1 * body weight height). There exists a 1.6829 Nm/kg average root residual torque of the optimized GTss in AddBiomechanies, which consider- ably higher than the mPJE of ImDyS (0.1626 Nm/kg). We further analyze the relationship between the data quality and the model performances. We adopt residual torques as an indicator of the data quality and calculate torques. Notice that higher residual torque indicates lower data quality with lower reliability of the optimized GT torques. Results are shown in Fig. 13 As shown, some samples could suffer from bad kinematics fitting (like the unnatural anterior pelvic tilt Fig. 13). resulting less reliable GT opti- mized joint torques. An interesting phenomenon is that the lower the residual torques are, the better ImDyS performs, which means ImDyS performs better for high-quality samples. This indicates the baseline might overfit low-quality data with high residual torques. Instead, ImDyS, with the knowl- edge inherited from the large-scale diverse ImDy, manages to resist the negative influences from low-quality samples. We also show how the mPJE of ImDyS changes with data quality in Fig. 14 As shown, the performance of ImDyS degenerates synchronously with data quality.
of sequences with different residual
Per-Joint Performance Analysis. It is also noticeable in Fig, that the gap between GT and prediction differs for different joints. To this end, we further analyze the per-joint performance of ImDyS. The per-joint mPJE of ImDyS and the per-joint mPJE of ImDyS in each frame for samples
18

---

Published as a conference paper at ICLR 2025
0.1750
0.1725
0.1700
ImDys mPIE (Nm/kg)
0.1675
0.1650
0.1625
0,1600
0.1575
0.1550
0.00
0.25
1.00
1.25
1.50
Residual Torque (Nm/kg)
Figure 14: Relationship between data quality and ImDyS performance. Higher residual torque indicates lower data quality with lower reliability of the optimized GT torques. The performance of ImDyS degenerates synchronously with data quality.
Table 7: Per-Joint mPJE of ImDyS for samples with clinical-grade quality.
Right mPJE, 4
Left mPJE, +
Joint Name
ImDyS
Baseline
ImDyS
Baseline
Hip Flexion
0.267
0.270
0.273
0.274
Hip Adduction
0.196
0.212
0.198
0.199
Hip Rotation
0.087
0.109
0.083
0.098
Knee
0.193
0.188
0.195
0.209
Ankle
0.197
0.195
0.202
0.199
Subtalar
0.069
0.071
0.079
0.084
MTP
0.0005
0.0006
0.0007
0.0008
Lumbar Extension
0.328
0.339
Lumbar Bending
0.255
0.255
Lumbar Rotation
0.113
0.113
with clinical-grade quality (residual torque < 0.1 body weight * height) is demonstrated in Tab. 7 ImDyS manages to improve the performance on most joints compared to the baseline without ImDy, especially for the hips. An interesting phenomenon is that ImDyS performs slightly better the right half the body.
C SIM2REAL ANALYSIS
We further analyze the Sim2Real effect of ImDy(S) via Fig. 15 An interesting question is the perfor- mance of ImDyS without any fine-tuning on AddBiomechanies. Though this could be inapplicable for most joints due to the human model definition discrepancy between Rajagopal's 's model in Ad- dBiomehanies and SMPL in ImDy, the knee joints the two models could roughly correspond to each other. Therefore, we visualize the knee torque magnitudes of ImDyS and ImDyS w/o Sim2Real finetuning on AddBiomechanics Fig. 15 Even without fine-tuning, ImDyS could reproduce the trends of knee torque magnitudes. However, artifacts could also be observed in two aspects. First, ImDyS w/o Sim2Real tends to produce much larger torques. Second, ImDyS w/o Sim2Real could be over-active compared to real humans and ImDyS like in the red circles. The reason could be two-fold. First, the simulation parameters used by ImDy, like mass and inertia, are different from real humans. Second, though the knee joints could roughly correspond, the knee in SMPL has more
19

---

Published as a conference paper at ICLR 2025
GT
ImDyS
ImDyS Sim2Real
Left Knee
Right Knee
Figure 15: Knee torque magnitude visualization of ImDyS and ImDyS w/o Sim2Real fine-tuning on AddBiomechanies. ImDyS w/o Sim2Real produces larger magnitudes and over-active torques w/o Sim2Real fine-tuning as circled in red.
Sample from Imby:/adBiiomeehanii/AAAAA
Lprior
LFD
Motion State
st+1
Negative
Sample
Generation
S st-wit
Temporal
ImDyS Encoder
Prior
x
Permutation
ImDyS
st+1
FD
Encoder
ImDyS
Head f
Random Noïse
Figure 16: Details of Lprior and LFD-
DoFs than Rajagopal's model, which might require larger torques to produce similar motions. With the simple Sim2Real fine-tuning of ImDyS, the issues could be alleviated. Further exploration for better Sim2Real performance would be meaningful future work.
D DETAILS ON DATA FLOW
Details of the adopted Lprior and LFD are illustrated in Fig. 16
For Lprior, the input motion state is treated as the positive case, and we generate corresponding negative cases by either temporal permutation or adding random noises. The samples are fed to the encoder, and the prior discriminator predicts whether the sample is positive.
For LFD. we first feed ImDyS with motion state w obtaining т, A. Then, T, A. st --wwst wet are
fed into the FD model, outputing g++1, The FD loss is computed L | g/+11.
20

---

Published as a conference paper at ICLR 2025
Table Extended results on the FD model on Addbiomechanics.
Methods RMSE
Baseline
ImDyS 0.0194
Nimble
0.0302
0.0186
GT
Original PHC
Re-trained PHC
Im DyS
LF
LF
RF
RF
LF
LF
RF
RF
Figure 17: Original PHC on GroundLink.
E ANALYSIS ON FD MODEL
We report the marker RMSE of the FD model on the AddBiomechanics test set Tab. g ImDyS noticeably outperforms the baseline trained on AddBiomechanies only, indicating the importance of ImDy pre-training. Moreover, ImDyS is competitive even compared to the differentiable simulator Nimble.
F COMPARISON WITH ORIGINAL PHC
Due to the inaccessible torques, we did not include the original PHC as a baseline for ImDy, How- it is noticeable that the original PHC can also conduct GRF prediction. To this end, also evaluate the original PHC on GroundLink. It provides a left-foot mPJE of 1.559 and a right-foot mPJE of 3.518, which comparable re-trained PHC and worse than proposed ImDyS. Some visualizations are included in Fig. 17 Even without the naive PD controller, the original PHC could suffer from jittering predictions, which could result from the non con-perfect contact simulation. In contrast, ImDyS could produce smoother predictions with higher precision.
21


Reviews:
[{"review_id": "7AoSW7nwSY", "reviewer": "Reviewer_qrg1", "summary": "This paper focuses on human inverse dynamics, aiming to estimate the driving torques of body joints from motion observations. While previous methods have primarily targeted lab settings, this paper employs imitation learning within physics simulators. Building on this approach, the authors created a dataset by importing the AMASS and KIT datasets into Isaac Gym, obtaining contact forces and joint torques based on the previous PHC method. Experimental results also demonstrate improved performance compared to the AddBiomechanics benchmark after fine-tuning.", "questions": "1. KIT is also part of the AMASS dataset, but the paper mentions using the AMAS and KIT datasets. Is there any difference?\n\n2. ImDyS is tuned on AddBiomechanics for 10 epochs. How is the performance without fintuning?  \n\n3. I am also curious about whether a better-estimated dynamics is helpful for downstream tasks like human pose estimation.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Estimating human dynamics from motion observation is an interesting and meaningful direction.\n\n2. The paper is well-written and easy to follow.\n\n3. The proposed strategy is to make human dynamics look effective.\n\n4. Experiments show better performance than previous methods.", "weaknesses": "While the proposed method looks promising, I am concerned that its technical contributions may be limited for an ICLR paper. Numerous previous works on physics-aware pose estimation have utilized human dynamics through either non-differentiable physics simulators with reinforcement learning or differentiable dynamics solvers, both of which seem to be more challenging approaches. In these prior works, dynamics estimation often served as an intermediate task, either implicitly or explicitly. In this paper, it seems that only part of the previous methods were adopted, with an increased focus on dynamics estimation. Furthermore, the method employed is also similar to as that of PHC.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qqcrfsE4EH", "reviewer": "Reviewer_qgUn", "summary": "The paper presents a dataset ImDy (Imitaed Dynamics) with 150 hours of physically simulated humanoid characters imitating kinematic human motions with joint torques and full-body ground reaction forces. The paper claims that ImDy data is unique in its scale (100x more than previous ones), includes more general physics parameters such as the ground reaction forces (GRFs) for the entire body, and is represented in SMPL. Using this ImDy, the paper proposes ImDyS (Imitaed Dynamics Solver), a data-driven humanoid inverse dynamics solver. ImDyS is a simple encoder-head design where a transformer encoder maps the pose state to the ID (inverse dynamics) feature, which is used to predict the dynamics using linear heads. The notable loss designs are the use of the FD (forward dynamics) cycle consistency loss with an FD model trained together to predict the pose state from the dynamics, and a discriminator loss on the ID features, similar to AMP [Peng et al. 2021]. To overcome the sim2real issues, the model is trained in two stages, first with the ImDy data, and then with the data from AddBiomechanics while freezing the encoder and training only the linear head for joint torques.", "questions": "Have the authors studied the error in the learned FD model? Is it possible to replace this with a differentiable FD simulation?\n\nI would appreciate it if authors could share visual ablations with various motions, with and without the AddBiomechanics so I can confirm the effect of the sim2real, especially for motions out of distribution from the AddBiomechanics data. It is hard to tell how much jittering still exists just by looking at the mPJE.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The proposal is a great data and baseline model to advance the data-driven approach to understanding dynamics in human motions. While the baseline model ImDyS is simple, I see enough novel, especially in the loss design with the cycle consistency using the learned FD model and a discriminator loss on the ID features.", "weaknesses": "The paper is straightforward and looks good. I appreciate some clarifications in the questions", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xIuzXZw3P2", "reviewer": "Reviewer_xT3W", "summary": "In this paper, authors introduce data-driven inverse dynamics solver (ImDyS) that could estimate dynamic states of the person (including joints torque and ground reaction forces) from the observed kinematics. To train ImDyS in a fully-supervised manner, they first propose the data acquisition to collect scalable human dynamics data following the recently proposed imitation learning methods. \n\nThey benchmarked their method on both the imitated data (ImDy) and real-world data, AddBiomechanics, which has the measured ground reaction forces and 3D kinematics labels. The results showed that the proposed method is more accurate to estimate human dynamics from both the imitated and real datasets. \n\nSince the authors borrow the concept of imitation learning from the previous work (PHC), I do not see significant novelty in their data acquisition pipeline. However, given their contribution to processing and sharing of the dataset, and the performance of the supervised inverse dynamics module, I think the paper can contribute to the community.", "questions": "1) Connected to the weakness that I pointed above, why does authors' imitation on AMASS have lower success rate than PHC? I would suggest the authors to provide quantitative/qualitative comparisons of two imitation learning schemes (their and PHC), and justify their purpose of the proposed method.\n\n2) It is not clear what can ImDyS bring over the imitation learning. Let's assume we collect kinematics data and want to extract dynamics from it. We can either run ImDyS to estimate dynamics or ImDy policy to imitate the kinematics (where the dynamics will come together). What is the major benefit of using ImDyS in this case? I guess the inference speed but wonder authors' perspective. I would suggest authors to provide comparisons between the proposed IL and ImDyS to highlight the unique values of ImDyS.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The paper's strengths can be summarized as follows:\n\n1) They train an imitation policy that can imitate the large-scale human motion capture dataset (AMASS and KIT) which provides the pairs of human kinematics and dynamics with the larger scale than any other publicly available datasets.\n\n2) They train an inverse dynamics solver on the processed data, resulting in the state-of-the-art performance in estimating ground reaction force and dynamics state of the human (on both imitated and real data).\n\n3) Good visual presentation is appreciated", "weaknesses": "Data acquisition part of this paper does not seem to include significant novelty since most of the part were borrowed from the previous work (PHC, Luo et al.,). I would like authors to elaborate what their imitation learning can offer other than PHC. Can we run PHC to collect kinetic/kinematics pairs of AMASS and KIT data? If so, why do we need to use the proposed method for the dynamics data acquisitions?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ynAeUbZuwM", "reviewer": "Reviewer_yxxH", "summary": "This paper proposes ImDy, a human inverse dynamics benchmark & dataset that contains 150 hours of motion with joint torque and full-body ground reaction forces obtained from humanoid motion tracking. ImDy provides a scalable approach to obtain more joint-level actuation for human motion as well as ground reaction force grounded in physics simulation. Experiments show that data obtained from ImDy could support learning an inverse dynamics model that has competitive performance on real-world data, showcasing the advantage of having large-scale synthetic dynamics data obtained from simulation.", "questions": "Figure 11 visualizes a failed joint prediction case; the plot shows that ground truth torques are quite smooth while the predicted are jittery (before both the ground truth and predicted are jittery). Based on the provided analysis, it is possible the data quality of AddBioMechanics could be the cause here?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The proposed ImDy and ImDyS, as far as I know, is a novel attempt at using simulation and humanoid motion tracking to obtain large datasets for inverse dynamics analysis. The results show that the large-scale dynamics data obtained from simulation could be of use in learning models (ImDyS) that perform well on real human performance data.\n- A comprehensive analysis of the proposed ImDyS and the baselines is provided. Since this is a relatively new topic, not many prior arts exist. Nonetheless, ImDyS outperforms the baselines and generalizes to real-world data with competitive performance.\n- I appreciate the analysis of failure cases, which analyzed the advantages of ImDyS and plausible causes for failure in predicting the joint torques on the jumping motion.", "weaknesses": "- Since a dataset is proposed, an analysis of the data composition (like in Figure 12) would be beneficial. Since AMASS also contains many locomotion data, and the PHC with a naive PD controller has a lower success rate, it would be good to see what motions are tracked and included in the dataset.\n- ImDys has many details that seem to be omitted. A data flow chart that includes the different losses in the appendix would be helpful. What is the motivation for including the discriminator loss if it does not contribute much to accuracy (as shown in Table 5)? \n\n\nMinor:\n\n- I would recommend changing the term \"imitation learning\" to \"motion tracking\" as \"imitation learning\" is quite overloaded nowadays.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)