HOI-Dyn: Learning Interaction Dynamics for Human-Object Motion Diffusion
Authors: Lin Wu
Zhixiang Chen
Jianglin Lan
Venue: NeurIPS 2025 poster ()
Keywords: Human-object motion generation
Interaction dynamics
Causal consistency
Primary area: deep_learning
TL;DR: We propose HOI-Dyn, a controllable generative framework for synthesizing physically consistent human–object interactions via a dynamic, causal Driver–Responder model.

Abstract: Generating realistic 3D human-object interactions (HOIs) remains a challenging task due to the difficulty of modeling detailed interaction dynamics. Existing methods treat human and object motions independently, resulting in physically implausible and causally inconsistent behaviors. In this work, we present HOI-Dyn, a novel framework that formulates HOI generation as a driver-responder system, where human actions drive object responses. At the core of our method is a lightweight transformer-based interaction dynamics model that explicitly predicts how objects should react to human motion. To further enforce consistency, we introduce a residual-based dynamics loss that mitigates the impact of dynamics prediction errors and prevents misleading optimization signals. The dynamics model is used only during training, preserving inference efficiency. Through extensive qualitative and quantitative experiments, we demonstrate that our approach not only enhances the quality of HOI generation but also establishes a feasible metric for evaluating the quality of generated interactions. Project website: https://wulin97.github.io/hoi-dyn

HOI-Dyn: Learning Interaction Dynamics for Human-Object Motion Diffusion
Lin Wu
Zhixiang Chen
James Watt School of Engineering University of Glasgow
School of Computer Science
University of Sheffield
1. wu, 10reeearrhhh..com wlw.ga.............o...o...oo ac.
zhixiang. chen@sheff weweefffell...o ac. uk
Jianglin Lan'
James Watt School of Engineering
University of Glasgow
j i jianglin... wotiggi...nnn.p.... langglasgou ac uk
Abstract
Generating realistic 3D human-object interactions (HOIs) remains a challenging task due the difficulty of modeling detailed interaction dynamics. Existing methods treat human and object motions independently, resulting in physically implausible and causally inconsistent behaviors. this work, we present HOI- Dyn, novel framework that formulates HOI generation as driver-responder system, where human actions drive object responses. At the core of our method is lightweight transformer- based interaction dynamics model that explicitly predicts how objects should react to human motion. To further enforce consistency, we introduce residual-based dynamics loss that mitigates the impact of dynamics prediction errors and prevents misleading optimization signals. The dynamics model is used only during training, preserving inference efficiency. Through extensive qualitative and quantitative experiments, we demonstrate that our ap- proach not only enhances the quality of HOI generation but also establishes a feasible metric for evaluating the quality of generated interactions. Project web- site/hhtpss//wwlinn77.. htt.:/aiiiin.giguu..ooooii/////
Introduction
Synthesizing complex and realistic 3D human-object interactions (HOIs) is essential for progress in VR/AR, computer animation, and robotics 44, yet remains significant challenge. Compared to human motion generation whhtther for single or multiple people [5 HOI is significantly more difficult. Human motion generation typically involves relatively free movement, making it easier to generate plausible sequences [ 8) However, HOI requires capturing intricate interaction dynamics, such as stable contact, forces, and action-response relationships. Simply applying human motion generation frameworks to HOI often produces independent motion of the human and object, leading to physically unrealistic and causally inconsistent behaviors.
Previous works on HOI generation mainly focused on interactions between humans and static objects scenes 12H such sitting on sofa. Recent advances have shifted toward controllable synthesis of dynamic HOI, where both the human and object move synchronously [13] [4] For example, given an initial state and textual instruction, the system can generate motion sequence where a human
Corresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

picks up an object, such as bench, and places it elsewhere. This approach enables more flexible
motion generation and a wide range of applications.
However, existing methods often fail to capture the core interaction dynamics between humans and objects. These approaches typically focus on modeling either object affordances contact points [] 4 15], or simply integrating human and object motions through diffusion-based models [2] [3]. However, they do not fully address how objects should respond human actions, often leading to physical and causal inconsistencies.
In this work, propose new perspective: framing HOI generation driver-responder sys- tem [16], where human actions serve as the driver and objects respond accordingly. At the heart of this approach is the modeling of interaction dynamics, which describes how objects should naturally react to human motions. This view offers several advantages:
Contact is implicitly governed by the dynamics nno need explicitly model it. there is no contact, there is no response; if contact occurs, the object's 's response is naturally determined by the interaction dynamics.
Object motion is not independent-een step of their movement is driven by the human' 's actions and controlled through specific instructions or context, ensuring coherent and physically plausible interaction.
Building on this perspective. we design a new HOI generation framework that explicitly incorporates interaction dynamics into the motion synthesis process, yielding statc-of-the art performance on chal- lenging HOI benchmarks and offering a physically grounded solution to HOI generation. Specifically, our contributions are as follows:
We introduce a novel driver-responder formulation for HOI generation from a synchro- nized control perspective, modeling the causal dependencies between human actions and object responses in a dynamic and physically consistent manner.
We propose a lightweight transformer- based interaction dynamics model that answers how objects should react dynamically to human actions, taking into account the context of human motion and specific contact situations.
We introduce residual-basee based interaction dynamics loss that serves HOI motion diffusion, compensating for prediction noise in the dynamics model. This loss helps prevent misleading optimization gradients and ensures the focus remains core generative inconsistencies, thereby improving the quality of the generated motion sequences.
We demonstrate the effectiveness of our approach through extensive qualitative and quantita- tive experiments, highlighting that the proposed model not only improves HOI generation but also serves as a reliable evaluation metric for assessing the quality of generated interactions.
Related Work
2.1 Object-Guided HOI Generation
subset HOI generation methods leverages object trajectories or waypoints to guide human motion. OMOMO D1 takes full sequence of object states input and generates the corresponding human poses, whereas CHOIS [3] and Wu et al. 2 rely on sparse object waypoints (e.g., roughly waypoint every 30 frames), leaving the object's detailed responses be implicitly determined by the model. These methods typically add auxiliary supervision or constraints to encourage plausible human-object contact. However, such guidance mainly steers the diffusion process toward ensuring that contact occurs, rather than modeling the underlying interaction itself. While this strategy can yield globally coherent sequences, it remains at high level: the model is not trained to capture how objects physically react to human actions in a fine-grained and causally consistent manner, leaving the central challenge of realistic HOI underexplored.
2.2 Joint Human-Objes Motion Generation
Another line of work generates HOIs jointly without conditioning on future object states. HOI- Diff [17] relies affordance prediction and estimated contact points to guide interactions, while

---

leverages contact fields on the human mesh as strong priors. THOR [4] models relational cues for coordinated motions. HIMO [8] employs dual-branch conditional diffusion with mutual interaction module for cross-modal fusion, and ChainHOI [9] adopts a spatiotemporal graph architecture with a kinematics-awar module capture joint- and chain-level dependencies.
CG-HOI [5
Despite these advances, common challenges remain. First, predicting accurate contact points or modeling interactions precisely at contact regions inherently difficult. Second, although high-level constraints or interaction modules encourage consistency, they often fail to capture the causal and fine-gre une-grained dynamics of object responses human actions. As result, object behaviors may appear temporally inconsistent or physically implausible, even when global plausibility is achieved. This highlights missing perspective: explicitly treating human actions as the driver and object motions as their causal responses. Such formulation is crucial for achieving physically realistic HOI generation.
2.3 Driver-Responde Synchronization
Driver-Respondrr Synchronization, also referred to as master-slave synchronization, has been widely observed in coupled systems across biological and physical domains (2) In these systems, controllers achieve synchronization through adaptive and feedback strategies [23 24]. Drawing inspiration from these systems, conceptualize HOI generation
Driver-RRsponder System, where human actions serve as the driver, controlling the object's response. Unlike prior methods that treat human and object motions independently [ 25 26]. this perspective captures the causal relationship between human movements and object reactions, ensuring more coherent and physically consistent interactions. By introducing an internal control mechanism, the Driver-Responder formu- lation provides principled framework for generating HOIs with refined temporal coherence and realistic object dynamies, addressing the limitations of previous high-level constrained approaches.
Methodology
Our goal is to synthesize synchronized HOIs while maintaining internal causal consistency, by leveraging controllable signals such as textual descriptions and object geometry. We propose the HOI-Dyn framework (see Fig. [ that explicitly models interaction dynamics, enabling the generation of more plausible and coherent motion sequences. The framework consists of two key components: Motion Diffusion and Interaction Dynamies. The Motion Diffusion component, based on Transformer-eet based conditional diffusion model, jointly encodes the human, object, and interaction context into unified representation. The Interaction Dynamics component provides auxiliary supervision to reinforce fine- tie-grained causal consistency during motion generation.
hiti object
Obyez RPS
Join mocion anooder
Otject conditor mooeder
Noiry maciion
Tedi
Jevel
MC.PP
S Postice crbolling
CCLP
MLP
Cuzcot Lincar
Transforner
Motion Diffusion
Interaction Dynamies
Powiies
wnbodiing
Diffusion Model (Motion Transformer)
abe A.
Rot Trans Split
Svn
48, 26,,.. 2dous
Aii
Object Ri... VOCG
Object R
Clean motion
Cousality Aligument
Figure 1: Overview of the proposed HOI-Dyn framework. (a) Conditional Motion Diffusiio synthesizes human-object interactions No { Ô, X} using a Transformer-base diffusion model, where f {f= and Ô = (o- (b) The full framework integrates motion generation with interaction dynamics supervision. (c) Interaction Dynamics models object responses Aor based on human relative motion Ah, object pose ô,, and interaction context &
3

---

3.1 Interaction Dynamics
In diffusion- based motion generation, the output typically reflects clean motion rather than noise. The denoising process allows the model to internalize buman-specing dynamics, resulting in physically plausible free movement. contrast, object motion externally driven and inherently constrained by physical laws-it cannot occur autonomously. To reflect this asymmetry, we decouple their roles as follows: the human follows internally guided dynamics, while the object responds to external control. This interaction is formalized as
h((+1)= 110 + AL. P((11))
Driver (Human):
U/ -gh(h(o) ol++1) ol0) + At Fo(oo, 8(0), (0,,(0)
(I)
Responder (Object):
No (o(((())
where (0) and ((t) denote the latent states of the human and the object at time step t, respectively. The functions (-) and F-(-) describe their respective internal dynamics, while Jh and go project these latent states to the observable outputs v/ and yo At is the sampling time step. The control signal (+) is determined based on the error feedback e(yh yo ) which reflects the discrepancy between the human's 's intent and the object's behavior. The term (()) denotes the interaction context, which includes factors such contact state, object geometry, and other environment- task-specific conditions influencing the object's 's motion.
In practice, we observe that existing diffusion models often fail to generate causally consistent interactions due to the absence of precise control signal (((), which is essential for aligning the object's 's motion with human intent [13 14. To address this limitation, we propose the interaction dynamics as supervisory signal to implicitly optimize it in the diffusion process. Based on (I. we can derive the object's relative motion as
() (t+1) ( = At At- At-F, (()) (1) so (ts(t), " ;p)))
(2)
where (h ((+1)) - (t)) denotes the human's 's relative motion, and D(-) is learnable function
parameterized by 0p. This formulation emphasizes that object dynamics are governed not only by their internal state but also by human-induced interactions and the context.
To enhance sensitivity to varying interaction magnitudes, we extend the HOI prediction horizon from to k, with & being selected randomly from [1, K]. As shown in Dc). the model takes as input the current object state o(*), interaction context ((), and the cumulative human motion over the horizon Afg- = (t+k) httt,, and predicts the corresponding object motion:
A D(s(t), ()) Ahc
The above predicted motion is further represented rigid-body transformation comprising rotation ((++++))) E SO(3) and translation ((++++) E IR3 i.e., Ao [R[7] applied set of object's points p(t) to yield the predicted future configuration follows:
p((+H) = (())))))(()) (1-+2+1)
(4)
To ensure that (t++++++) is a valid rotation matrix, we apply singular value decomposition (SVD)- based projection to the raw network output R E 3x333 such that R = UEVT and R UVT We then define the following cost for object dynamics to quantify the error between the transformed keypoints:
Φb O(Dor-tth; Aoi = [|p((++))_ p(t++))
The overall loss function is defined as the expected value over time steps t and & sampled from (1, K) as follows:
C= E= h ((Dor-tth ( Ao5
66)
This loss function guides the model to capture both the motion magnitude and the causal structure of interactions, yielding more realistic and consistent object motion in HOI.

---

3.2 Conditional Motion Diffusion
Having introduced the interaction dynamics, we now proceed to the motion generation stage. Here, we propose conditional diffusion model that provides internal forces to synthesize temporally aligned motions for both the human and the object.
The joint human-object trajectory is denoted as r O, with the human motion H, object motion O, and interaction context X (e.g., hand-object and foot-object contact annotations). The context provides coarse guidance for synthesizing plausible interactions. We represent human motion using the SMPL-X parametric model [27], and represent each frame of the object motion using 3D translation and relative rotation. In the condition representation c, as shown in Fig. [ we follow the CHOIS framework [ integrate contextual cues such as text prompts and the Basis Point Set (BPS) 28 of the object. More details can be found in Appendix
The complete conditional HOI Diffusion comprises both forward and reverse processes 29 The forward process is modeled as a Markov chain over N steps as follows:
N
(TINNITO) = II ((mm//n-1); ((Ta = -N (Naii VI - Bn Tn 1, B,1),,
(7)
where Bn is predefined noise schedule. The reverse process defined as
Pe Pe(Tn-1| (nn c) 1i c), En), where 2n, is fixed variance, and μe On 11- Bn. The reverse process is modeled by a neural network 0g with the training loss:
(8)
(1 Tn
yun-1da
to. with an = II ag and
1-aa an
1-aa-
Choi Er..m Il c; 0g) - which minimizes the reconstruction error between the predicted and ground-truit trajectories.
Although the standard loss function ( produces plausible motion sequences, physical inconsistencies persist, particularly in object trajectories. To address this, we introduce auxiliary Interaction Dynamics Loss based on a shared dynamics model D. This loss penalizes discrepancies in object motion between generated and ground ground-truth truth sequences, enforcing a causal alignment between human actions and object responses, as follows:
Layn EE, (AO; Δo:) - D(>o;, 1o:)
(10)
where [Ao 1-11 D(A and [Ao = D(AT)) are the object motions predicted by D given the relative motion from the generated and ground-truth HOI io and respectively. The function ((:,)) computes the motion errors as described in (5). The per-frame residuals in Ldyn are
åD.gen 4(Ao,, Ao.), 0D (Doo; Ao:).
(11)
Ideally, 'D perfectly captures the interaction dynamics, then ōD.gt = 0 and the loss reverts to the direct supervision of åD..em However, in practice, prediction errors are unavoidable due to imperfections in and inaccuracies in the training data.
To mitigate the influence of these errors, we assume that D is locally smooth and time- homogeneous-iit bias depends on the state but not on time t. This assumption is reasonable because D predicts object dynamics from current states without explicit time dependence, and local smoothness reflects the continuity of physical motion (see Appendix Oppendix/E.2]oor a detailed discussion of these assumptions). Under this assumption, when the generative and ground-trune truth trajectories exhibit similar state distributions, the error residuals tend to cancel out expectation:
(12) This residual formulation provides a robust supervisory signal: even if D imperfect, its bias is removed through subtraction, allowing the learning algorithm to focus on the true inconsistencies in the generated motion. For a more detailed discussion, see Appendix C With this loss in place, each step of the generation process is guided to yield more accurate control signal U4 resulting in physically grounded and temporally coherent HOI sequences. We therefore extend standard HOI training objective Lhoi 99) by incorporating both Layn (10) and an object-level reconstruction loss Loy = E, (o,, oz).. )). All these losses are equally weighted to simplify the overall training process and avoid the need for complex hyperparameter tuning.
E, [.g Ep..a ~ 0.
5

---

4 Experiments
Dataset. We train and evaluate HOI generation using two datasets: (i) FullBodyManipulatioo, which provides 10 hours of high-quality paired object and human motion data involving 15 different objects M. We apply the OMOMO partitioning to this dataset: 15 subjects for training and 2 for testing; and (ii) 3D-FUTURE, which consists of 3D models of various furniture items [30]. To assess the generalization capability, we select 17 objects and pair them with motion sequences from the FullBodyManipulattoo testing set.
Metrics. We evaluate HOI from four aspects, as defined in CHOIS 13 (i) Condition Matching, which measures the alignment of the predicted object trajectory with input waypoints using Euclidean distance at the start, end, and intermediate points (cm). (ii) Human Motion Quality, which is evaluated with the foot sliding score (FS), foot height feee, and Fréchet Inception Distance (FID) KI that captures distributional differences between the generated and real motions. (ii)) Interaction Quality: Based on the frame-wise contact labels, we compute the contact percentage C%,, Fl score CF., and hand-object penetration score Paand to assess physical plausibility. (iv) Ground Truth Difference. which includes the mean per-joint position error (MPJPE), root translation error T Trosss object position error Taty. and orientation error Rubj-
and
Implementation Details. We train the interaction dynamics using FullBodyMannpulltion with K as the maximum prediction horizon. The network has 0.5M parameters. We use the Adam optimizer [3 with learning rate of 1 x 10-3, adjusted via Cosine/Annealing WarmRestarts (3 for 150 epochs with a batch size of 32. The model is then transferred to the HOI motion diffusion task, where fine-grained driver-responder relationships are encouraged during denoising, as in (T0). Training is done from scratch with a learning rate of x 10-4, batch size 32, and 100,000 steps. All experiments are conducted on a single NVIDIA RTX A4500 GPU, with total training time around 10 hours.
Baselines. We compare our method HOI-Dyn with the tfate-of-the-ar (SOTA) approach, CHOIS 13 whose results are reproduced under identical training conditions, including the same number of training steps. We also consider other baselines including the adapted versions of Inter- Diff 34) MDM [35], and OMOMO [.] whose results are borrowed from [13] for comparison. More specifically, InterDiff modified to accept additional inputs, including text and sparse waypoints. MDM is updated to incorporate our object geometry representation and sparse waypoints, extending to predict object motion. We introduce three variants of OMOMO: (i) Pred-OMOMO, which combines our object motion module with OMOMO; (ii) GT-OMOMO, which uses the ground truth object motion as input; and (iii) Lin-O OMOMO, which incorporates a linear interpolation strategy to generate object motion trajectories and ensure consistent object rotation throughout the sequence.
4.1 Qualitative Results
We compare HOI-Dyn with the SOTA HOI model CHOIS from two complementary perspectives in a synchronized manner: (i) Action-interaction-RRepsssss which evaluates the physical and semantic plausibility of object motion before and after human actions; and (ii) Sequence-level Alignment, which assesses the consistency of the generated HOI sequence with the input conditions and its overall causal coherence. Together, these perspectives capture both the micro-level physical causality and macro-level semantic consistency.
As shown in Figs. 2 (a) and (b), CHOIS, which lacks interaction dynamics modeling, often causes premature object motion. The objects tend to move toward expected contact points (c.g., the hand) spontaneously before human actions begin, leading to implausible behaviors such as bouncing, wobbling, or sliding. In contrast, HOI-Dyn reduces such artifacts, promoting more natural interactions.
Fig. 2 (c) further highlights the difference in object responses after contact: HOI-Dyn generates subtle, physically plausible reactions (with slight sliding after gentle kick), while CHOIS often produces exaggerated effects like flying or floating. Fig. 2 (d) shows the keyframes for sequence-level comparison. Both methods generally respect input constraints, demonstrating the effectiveness of motion diffusion. However, CHOIS often exhibits disjoint human-object motion and weak causality, while HOI-Dyn ensures consistent object motion and maintains contact, resulting in more coherent and plausible HOI sequences. More visualization results can be found in AppendixD

---

(a) Kick the plasticbox
(b) Pull the trashean
(c) Kick the whitechair
CHOIS
W.B.
Move before action
Move before action
exaggerated motion after action
HOI -yn
L.
(d) Put your hand on the back of the woodchair, pull the woodchair, and set it back down
CHOIS
The object unnaturally drifts to the hand
Move without contact
HOI-Dyn
Figure 2: Comparison of HOI-Dyn and CHOIS physical plausibility and sequence-level coherence. (a-b) CHOIS produces premature object motion lacking causal timing: (c) HOI-Dyn generates more realistic post-contact responses; (d) HOI-Dyn maintains consistent human-object interaction across the full sequence. Green markers indicate object initial state and sparse waypoints.
4.2 Quantitative Results
As shown in Table 1. HOI-Dyn consistently outperforms existing methods across all evaluation metrics. achieves the lowest T, and T. in condition matching, indicating better alignment with input constraints. For human motion, it yields the best foot sliding score (FS) and FID, demonstrating a more realistic motion synthesis. terms of interaction quality, HOI-Dyn attains the highest CF1 and C%, while maintaining comparable hand penetration score Phannd It also achieves the lowest MPJPE, root translation Teon and object motion errors (Toty Roaj). demonstrating improved accuracy in modeling human-objes dynamics.
Table 1: Comparison of methods across different metrics. Arrows indicate whether lower (L) or higher (f) is better, and the same notation applies hereafter.
Method
Condition Matching
Human Motion
Interaction C% 1
Difference
T,
T.
T.p
Hiss
FS
FID
CF1 1
Pasd
MPJE
Tea
Tai
Rai 4
Interdiff
0.00
158.84
72.72
0.90
0.42
208.0
0.33
0.27
0.55
25.91
63.44
88.35
1.65
19.42
0.43
Lin OMOMO
0.00
0.00
0.00
7.21
0.41
15.33
0.57
0.54
0.51
21.73
36.62
17.12
1.21
7.08
CTOOMOMO
0.00
0.00
0.00
7.10
0.41
5.69
0.67
0.59
0.55
15.82
24.75
0.00
0.00
CHOIS
24.33
1111 Dyn (Ours)
1.75
5.58
3.26
3.07
0.37
0.48
0.71
0.60
0.64
15.60
23,90
12.47
0.90

---

To assess the generalizability of our model to novel objects, we also perform experiments on the 3D- FUTURE dataset. As shown in T although not trained on this dataset, our model significantly improves the quality of human motion and, more importantly, enhances the fidelity of human-object contact, with only a slight increase in penetration.
Table 2: Interaction synthesis results the 3D-FUTURE dataset 30
Method
Condition Matching
Human Motion
Inferaction
7, 4
T,
Tix
Msa 4
FS L
FID
C% 1 Phan
InterDiff
0.00
161.26
26
72.77
-0.26
0.42
207.3
0.24
0.11
MDM
12.58
40.55
28.72
7.02
0.49
8.50
0.34
0.26
Lin-OMOMO
0.00
0.00
6.32
0.42
23.17
0.44
0.11
Pred-OMOMO
4.15
9.03
3.89
6,08
0.40
3.74
0,50
0.18
CHOIS
3.23
6.21
2,99
2.95
0.42
1.67
0.47
0.19
HOI HOI-Dyn (Ours)
2.95
2.56
0.54
0.26
4.3 Application in 3D Scene
We demonstrate the practical applicability of our method by synthesizing dynamic HOI within realistic 3D environments, conditioned on text descriptions. Specifically, we use 3D scenes from the Replica dataset (6) and manually define instructions such as "pull the floor lamp and move next to the sofa". The pipeline involves three steps: (i) parsing the textual description to identify the intended interaction and associated object(s), (ii) specifying the target object's initial placement within the scene, and (iii) planning collision-free navigation trajectory using Habitat. The resulting path is then provided to our HOI generation model, which produces coherent and physically plausible motion sequences aligned with the instruction.
As illustrated in Fig. our model successfully generates realistic agent behaviors in complex environments. In (a), the agent interacts with floor lamp and repositions it near a sofa, while in (b), it moves large box across the room. Both examples highlight the model's ability to generate environment-conscinn and interaction-conssstrr motions, with potential applications in animation, virtual reality, and robotics.
(a) Interaction with a floor lamp
(b) Interaction with a large box
Figure 3: HOI generation in realistic 3D scenes. The virtual agent interacts with different objects
while maintaining physical plausibility and environmental consistency.
4.4 Further Discussion
Effect of Interaction Dynamics Variants. We analyze the effects of prediction horizon K, network design, and model complexity on the interaction dynamics performance. As shown in Fig. g a small K limits large motion capture, while a large K weakens subtle interaction modeling. Empirically, K=2 or K=3 yield the best results. For network design, we compare the decoupled and coupled motion strategies. Inspired by [37]39], the decoupled approach predicts object rotation and translation separately. Our coupled design instead models the unified influence of human motion and contact, yielding a better performance under similar parameter and FL.OP constraints, highlighting the inherent coupling in HOI, as shown Table 3 We also evaluate the model efficiency by varying the Transformer depth (D), feature dimension (F),, and head count (H). The results in Table 3 show
8

---

that a lightweight model with 0.5M parameters and 0.2 GFLOPs suffices to capture high-quality interaction dynamics. For a more detailed discussion, see Appendix
0.5
0.463 mm
Point Cloud Loss
0.329
Polnt Cloud Loss
0.325
0.4
0.314 4
0.313
0.320
0.321
0.3
0,316
0,313
0.310
0.313
0.1
8
K Value used in Training
0.0
CHOIS
HOI-Dyn
Figure 4: Effect of Horizon K.
Figure 5: Object Loss via Dynamics.
Table Effect of Design Variants. Our lightweight coupled model captures high-quality interaction dynamics, outperforming the decoupled variants under similar constraints.
Architecture
Network Configuration
Object Point Cloud Loss
Params (M)
Flops (G)
K=1
K=2
K=3
K=l
K=5
K 10
Coupled (K+1)
0.316
0,51g=bree2
0.7634000
1.080-3004
1.44400ee
3.42240.009 0090
Coupled (K=2)
0.313
0:16240000
0.622
0.791
0,982
2.279e0.000
Coupled (K-3)
D4.F64-B8
0.313-1000
0
0.603=oom
0
0.920-mma:
1.900m1000
Coupled (=5)
0
0.45ga00002
0,50gatinss
0.736
0.87gatuXe
L63351000
Coupled (K-
0.32g
0
0.604momm
0
0.871 -o00th
1.50 2009
Couplad (K=2)
D1-P64-HB
0.34944000
0, 0.700mmaa
0.909438625
1.1 1.14ppa
2665441000
0-432
0.057
Coupled (K=2)
134-P64-H8
0.313=0000
O 0,46200000
0.62240omr
0.791
0. 0.9824uuns
2.27ge0.000
0,483
0.201
(K=2)
0.31840000
0.71000000
0.63340001
5.815420044
1.035
2.507402115
0.550
0.394
Coupled (K=2)
DS-F128-H18
0.563
0.845+0ree
1.136m991
1
1.791
3.630m9
0.994
1.552
Decoupled (K-2)
(DI-P64.H88)
0.35gat000
0.53240000
0,714
0.919
2.61390.009
0.463
0,108
Decoupled (K-2)
(D2-P64.H8)x2
0.34
0.50350000s
0.6767000n
1.000-0mm
2.537mn ceay
0.496
Decuppad (K=2)
(DI-F64H85xx2
0,33730006
0.503400004
0.67640004
0.866-21022
1.062400m7
2.409412040
0.564
0.385
Effect of Different Guidance. Classifier guidance is often employed during inference in generative models. Following [13, we apply two types of guidance: feet-floor and hand-object. The feet-floor term encourages physical plausibility during locomotion and standing still by penalizing unnatural foot height above the ground, while the hand-ohject term enforces physically consistent contact and temporal coherence for hand-object interactions. For more detailed description, see Appendix]] [ As shown in Table even without guidance, our method surpasses CHOIS, especially in the interaction quality. The feet-floor term enhances foot realism and reduces FID, while the hand-object term boosts contact accuracy but slightly harms motion quality. Combining both yields consistent gains across all metrics, achieving new SOTA.
Table Effect Different Guidance. Our method outperforms CHOIS even without guidance. The feet-floor term improves physical realism, while the hand-object term enhances contact accuracy. Combining both achieves SOTA performance across all metrics.
Method
Condition Matching
Human Motion
Interaction C%
GT Difference
T.
T.
Tzy
F
Cer
Taa
Oay 1
CHOIS (wlo gui)
186
5.79
3.05
3.39
0.57
3.63
0.54
0.42
0.61
16.05
25.34
12.36
0.99
gui)
14.29
HOI-Dyn (wlo gai)
1.56
5.72
3.03
5.56
0.37
3.49
0.60
0.61
15.56
24.61
11.67
0.91
HOI Dyn (fect-Boor door guij
L57
5.56
3.03
3.25
0.37
0.66
0.60
0.46
0.62
15.48
23.57
11.28
0.89
HOI (hand-ob) obj gui)
1,77
5.74
3.27
5.21
0.37
2.55
0.71
0.60
0.64
15.73
24.56
13.03
0.91
HOO-DDy (w/ gui)
Effect of Removing Object Waypoints. To evaluate the intrinsic capability of our interaction dynamics model, we consider a without waypoint (w/o WP) setting. removing predefined object waypoints both the Baseline (CHOIS) and HOI-Dyn models. In this setting, the GT difference metric is no longer meaningful, so we focus on metrics reflecting physical plausibility and interactive realism: foot sliding, penetration (any unrealistic interpenetration involving human, object, or floor), contact Fl and accuracy, FID, and diversity. Classifier-ban guidance is not applied during inference.

---

ensuring that results reflect the generator's learned dynamics. As shown in Table 5 HOI-Dyn demonstrates improved foot and contact quality, better FID and diversity, and comparable penetration. highlighting the reasoning and generalizability of the model.
Table 5: Performance under the without waypoint setting. HOI-Dyn improves foot and contact quality, FID, and diversity, while preserving penetration, showing its intrinsic interaction dynamics.
Method
FS 4.
Penetration
Cp1 1
Os.
FID
Diversity 1
CHOIS w/o WP
0.401 0.376
0.581
0.573
0.648
5.36
7.90
HOI-Dyn w/o WP
0.582
0.592
0.670
4.81
8.09
Dynamics as Causality Metric. We evaluate the interaction dynamics modeling as proxy for causal consistency. Specifically, we compute the point cloud loss [ Δo]] in (5), where Âδ, is the object's observed motion and Δo; is the predicted response from the dynamics model. As shown in Fig. [5] despite some noise, HOl-Dyn closely aligns with the ground truth, while CHOIS shows a notable deviation, indicating a better causal modeling by HOI-Dyn. Furthermore, we visualize the frame-wise dynamics loss in Fig- 6 where the loss spikes when objects behave unrealistically. The results show that the loss remains Tow for HOI-Dyn, indicating that our model more effectively captures physically consistent interactions. These findings support the use of dynamics-basse based losses as metric for causal evaluation in HOI synthesis.
w/e Dyn
DOI-Dyn
win Dyn
HOI-Dyn
w/l Dyn
HOI-Dyn
w/o Dyn
HOI Dyn
Activall houmcee
inka by kick
Flleting
Altechaa the ground
Decoupee fices
Devcee by hannen nuticn
No
Kezp contact
off grosal
Point Lona
0.015
Uym
Potnt Cloud Lus
0.015
wits Lyn. pill-Da
0.000
0.010
0.005
0.005
0
20
201
1000
120
0.000
20
60
80
100
Framne
Tiome
Figure Prame-wise object point cloud loss. Our method HOI-Dyn (green line), consistently achieves lower loss than the baseline without dynamics supervision (blue line). Notably, high loss peaks correspond to physically implausible HOI cases, which are effectively identified by our dynamics model.
Conclusion and Limitation
In this work, emphasize the role of interaction dynamics in ensuring physical and causal consis- tency HOI generation. As an early exploration, we propose a novel driver-responder framework that explicitly models these dynamics and integrates with existing HOI motion diffusion techniques to achieve more realistic human-object interactions. While alternative approaches based on physics simulators might seem capable of modeling object dynamics more directly [40], their black-box box nature, lack diffcrentiability, and strong reliance on precise physical properties make them difficult to incorporate into generative frameworks. A detailed discussion is provided in Appendix C..4
Our framework currently assumes rigid objects and relies on the SMPL.-X human model, which provides only coarse hand representations. Consequently, minor inaccuracies may appear in hand- object interactions, particularly in rotations (see Appendix [E.3.. Looking ahead, we aim to incorporate richer object attributes and enhance human modeling better capture fine-grained interactions [4]]. We also plan to explore tighter integration of HOI dependencies into generative models to further improve interaction realism [42). and extend our framework to handle multi human and multi-object scenarios to assess scalability and practical applicability.
10

---

Acknowledgments and Disclosure of Funding
Lin Wu was supported by a Graduate School PhD Scholarship from the College of Science and Engineering, University of Glasgow, Jianglin Lan supported by Leverhulme Trust Early Career Fellowship (Award No. ECF-2021-517), Zhixiang Chen acknowledges support from the N8 Research Partnership and the EPSRC (Grant No. EP/T022167/I), which provided access to the N8 Centre of Excellence in Computationally Intensive Research (N8 CIR). We also acknowledge financial support from the Division of Autonomous Systems and Connectivity within the James Watt School of Engineering, University of Glasgow.
References
[1] Li, J. Wu, and C. K. Liu, "Object motion guided human motion synthesis," ACM Transactions on Graphics (TOG), vol. 42, 6, pp. 1-11, 2023.
[2] Z. Wu, J. Li,
and C. K. Liu, "Human "Human-object interaction from human-level instructions." arXiv preprint Xiv:24406 1 17840, 2024.
[3] H. Pi, S. Peng, M. Yang. X. Zhou, and H. Bao, "Hierarchical generation of human-object interactions with diffusion probabilistic models," in Proceedings of the IEEE/CVE International Conference Computer Vision, 2023, pp. 073.
Z. Wang, Y. Chen, B. Jia, Li, J. Zhang, Zhang, T. Liu, Y. Zhu, W. Liang, and Huang, "Move as you say interact as you can: Language-guides human motion generation with scene affordance," " Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 433-444.
[5] C. Guo, S. Zou, X. Zuo, S. Wang, W. Ji, X. Li, and L. Cheng, "Generating diverse and natural 3d human motions from text," in Proceedings of the IEEE/CVF conference computer vision and pattern recognition, 2022, pp. 5152-5161.
[6] Y. Wang. Wang. Zhang, K. Fan, J. Wu, Jiang. and Y. Liu, "Temporal and interactive modeling for efficient human human-human human motion generation." arXiv preprint arXiv:2408. 17135, 2024.
[7] M. Zhang, Z. Cai, L. Pan, F. Hong, X. Guo, L. Yang, and Z. Liu, "Motiondiffuse Text-driven human motion generation with diffusion model," IEEE transactions on puttern andlysis and machine intelligence. vol. 46, no. pp. 4115-4128, 2024.
[8] B. Huang, with proxemics and physies-guided adaption," in JEEF/CVF Conference on Computer Vision and Pattern Recognition, 2024.
Xu, Pan, Y. Wang, and H. Lee, "Closely interactive human reconstruction
[9] S. Huang, Z. Wang. P. Li. B. Jia, T. Liu, Y. Zhu, W. Liang, and S.-C. Zhu, "Diffusion-based generation, optimization, and planning in 3d scenes," in Proceedings of the IEEE/CVF Conference Computer Vision and Pottern Recognition, 2023, pp. 16750-15 660016761. 761.
[10] Z. Cen, H. Pi, S. Peng, Z. Shen, M. Yang, S. Zhu, H. Bao, and X. Zhou, "Generating human motion in 3d scenes from text descriptions," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp.
[11] Z. Wang, Y. Chen, T. Liu, Y. 7hu, W. Liang, and S. Huang. "Humanise: Language-connitionee human motion generation 3d scenes," Advances Neural Information Processing Systems, vol. 35, pp. 14 959- 14971, 2022.
[12] K. Zhao, Y. Zhang, S. Wang. T. Beeler, and S. Tang, "Synthesizing diverse human motions in 3d indoor scenes," in Proceedings of the IEEE/CVF international conference on computer vision, 2023, pp. 14 738 14749.
[13] J. Li, A. Clegg, R. Mottaghi, J. Wu, X. Puig, and C. K. Liu, "Controllable human-object interaction synthesis." " in European Conference on Computer Vision. Springer, 2024, pp. 54-72.
[14] Q. Wu, Y. Shi, Huang. Yu, L. Xu, and Wang, "Thor: Text to human-object interaction diffusson via relation intervention," arXiv preprint 2rXiv:2403. 1..40...2208 1208,, 2024.
[15] C. Diller and A. Dai, "Ci-hoi: Contact-guided 3d human-objed interaction generation." in Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2024, pp. 19 888 19981.
11

---

[16] J. P. Ramirez, E. Garcia. and J. Alvarez. "Master-slave synchronization via dynamic control," Communica- tions in Nonlinear Science and Numerical Simulation, vol. 80, p. 104977, 2020.
[17] X. Peng. Y. Xie, Z. Wu, V. Jampani, D. Sun, and H. Jiang, "Hoi-diff: Text-driven synthesis of 3d human-object interactions using diffusion models," arXiv preprint arXiv:2312.0655 2023.
[18] X. Lv, L. Xu, Y. Yan, X. Jin, C. Xu, S. Wu, Y. Liu, L. Li, M. Bi, W. Zeng et al., "Himo: A new benchmark for full-body human interacting with multiple objects," in European Conference on Computer Vision Springer, 2024, pp. 300-318.
[19] L.-A. Zeng, G. Huang, Y.-L. Wei, S. Gu, Y.-M. Tang. J. Meng, and W.-S. Zheng, "Chainhoi: Joint-based kinematic chain modeling for human-object interaction generation," " in Proceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 358-111 12.69.....
[20] N. Luchetti, S. Filippi, and A. Loppini, "Multilevel synchronization of human f-cells networks," Frontiers in Network Physiology, vol. 3. p. 1264395, 2023.
[21] A. Kazemy, R. Saravanakumar, and Lam, "Master-slave synchronization of neural networks subject to mixed-lype communication attacks," Information Sciences, vol. 560, pp. 20-34, 2021.
[22] S. Saini, M. F. Orlando, and P. M. Pathak, "Intelligent control of master-slave based robotic surgical system," " Journal of Intelligent & Robotic Syxtems, vol. 105, no. 4, p. 94, 2022.
[23] F. Lanotte, Z. McKinney, L. Grazi, B. Chen, S. Crea, and N. Vitiello, "Adaptive control method for dynamic synchromization of wearable robotic assistance to discrete movements: Validation for use case of lifting tasks," IEEE Transactions on Robotics, vol. 37, no. 6, pp. 2193-2209, 2021.
[24] N. Kumari and S. Dwivedi, "Fundamental concepts of synchronization: An introduction: From classical to modern," Resomance, vol. 25, no. 4, pp. 539-565, 2020.
[25] J. Cha, J. Kim, S. Yoon, and S. Back, "Text2hoi: Text-guided 3d motion generation for hand-object interaction," in Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, 2024, pp. 1577-1585.
[26] W. He, Y. Liu, R. Liu, and L. Yi, "Syncdifff Synchronized motion diffusion for multi-body human-object interaction synthesis," arXiv preprint arXiv:2412.20104, 2024.
[27] G. Pavlakos, V. Choutas, N. Ghorbani, T. Bolkart, A. A. Osman, D. Tzionas, and M. J. Black, "Expressive body capture: 3d hands, face, and body from single image." in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 10975-10985.
[28] S. Prokudin, C. Lassner, and J. Romero, "Efficient learning on point clouds with hasis point sets," in Proceedings of the IEFE/CVF international conference on computer vision, 2019, pp. 4332-4341.
[29] Ho, A. Jain, and P. Abbeel, "Denoising diffiusion probabilistic models," Advances in neural information processing systems, vol. 33, pp. 6840-6851, 2020.
[30] H. Fu, Jia, L. Gao, M. Gong, B. Zhao, Maybank, and D. Tao, "3d-future: 3d furniture shape with texture," International Journal of Computer Vision, vol. 129, pp. 3313-3337, 2021.
[31] Heusel, H. Ramsauer, T. Unterthiner, Nessler, and Hochreiter, "Gans trained by a two time-scale update rule converge to a local nash equilibrium," Advances in neural information processing systems, vol. 30, 2017.
[32] D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," " 2017. [Online]. Available: https://anivv.orgaa///111
[33] I. Loshchilov and F. Hutter, "Sgdr: Stochastic gradient descent with warm restarts," in International Conference on Learning Representations, 2022.
[34] S. Xu, Z. Li, Y.-.. Wang, and L.-Y. Gui, "Interdiff: Generating 3d human-object interactions with physics- informed diffusionn" in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 14 940--4940.
[35] G. Tevet, S. Raab, B. Gordon, Y. Shafir, D. Cohen-or, and A. H. Bermano, "Human motion diffusion model," in The Eleventh International Conference on Learning Representations (ICLR 2023), feb 2023.
[36] J. Straub, T. Whelan, L. Ma, Y. Chen, E. Wijmans, S. Green, J. Engel, R. Mutalr C. Ren, S. Verma et al., "The replica dataset: A digital replica of indoor spaces," arXiv preprint 196.97190 2019.
12

---

[37] A. Avery and A. Savakis, "Deeprm: Deep recurrent matching for 6d pose refinement," in Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2023, pp. 6206-6214.
[38] T. An, K. Dai, and R. Li, "Dif-net: A dual attention transformer-based fusion network for 6d object pose estimation." in 2024 16th International Conference Intelligent Human-Machine Systems and Cybernetics (JHMSC). IEEE, 2024, pp. 104-107.
[39] J.-X. Hong, H.-B. Zhang, J.-H. Liu, Q. Lei, L.J. Yang. and J.-X. Du, "A transformer-based multi-modal fusion network for 6d pose estimation." " Information Fusion, vol. 105, p. 102227, 2024.
[40] Y. Wang, J. Lin, Zeng, Z. Luo, J. Zhang, and L. Zhang, "Physhoi: Physics-based imitation of dynamic human-object interaction." " arXiv preprint arXiv:2312.04393, 2023.
[41] X. Liu and L. Yi, "Geneoh diffuusio:: Towards generalizable hand-ohject interaction denoising via denoising diffusionn" in The Twelfth International Conference on Learning Representations.
[42] Z. Geng, Z. Hayder, W. Liu, and A. S. Mian, "Auto-regressive diffusion for generating 3d human-object interactions," Proceedings of AAAI Conference on Artificial Intelligence, vol. 39, no. 3, 2025, pp. 3131-3139.
[43] S. Xu, Y. Ling, Y.X. Wang, and L.-Y. Gui, "Intermimic: Towards universal whole-body control for physics-based human-object interactions," in Proceedingy of the Computer Vision and Puttern Recognition Conference. 2025, pp. 12 222266122777 277
[44] Y. Wang, Q. Zhao, R. Yu, A. Zeng, Lin, Z. Luo, H. W. Tsui, J. Yu, X. Li, Q. Chen et al., "Skillmimic: Learning reusable basketball skills from demonstrations, arXiv e-prints, pp. arXiv arXiv-2408, 2024.
[45] X. Zhang, B. L. Bhatnagar, S. Starke, I. A. Petrov, V. Guzov, H. Dhamo, E. Pérez Pellitero, and G. Pons- Moll, "Force: Dataset and method for intuitive physics guided human-object interaction," in International Conference on 3D Vision. IEEE, 2025.
13

---

A Linking HOI Dynamics to Classical Synchronization
A.1 Classical Synchronization in Control Theory
To motivate our approach, we revisit the classical driver-responder (master-slave) synchronization framework from control theory, which serves the theoretical foundation for modeling interaction dynamics in HOI generation:
{
ën = F(Fm)
Driver
(13)
Um
CTm
ig = F(xs) + Bus
Responder
(14)
Cas
where Im, W. E IR" are the state vectors of the Driver and Responder systems, respectively, yi E R = m, are the outputs, F()) Lipschitz continuous nonlinear function, B Raxr is the constant input matrix, and C E RRxx is the output matrix. The control input u, E IR" is given by
Ug (y ys) K2C xx), KE
(15)
where K is the coupling strength. The synchronization objective is defined as
lim Jm (t) - xt)ll 0.
(16)
This framework illustrates how a Responder can track the Driver through an error-correcting feedback signal, providing formal basis for causal coordination.
A.2 Interaction Dynamics Loss as Implicit Error Feedback
In HOI generation, human motion serves as the Driver, while object motion is the Responder. To enforce causal and physically plausible object responses, we introduce Interaction Dynamics Loss fayn. which measures discrepancies between generated object motion and ground-truth trajectories using shared dynamics model 'D. Conceptually, this loss functions guiding the Responder's state toward aligmment with the Driver.
error-feedback signal
During training, the diffusion model iteratively updates object states at each denoising step to minimize this error, implicitly internalizing an crror-feedbace control mechanism that captures causal human-object dependencies. At inference, even without access ground-truth object motion, the model generates synchronized and physically consistent object responses by leveraging the learned driver-responder coordination. This establishes principled connection between classical synchronization theory and modern generative HOI modeling.
B Details of HOI Motion and Condition Representation
Human Motion. To represent the human motion H, we employ the SMPL-X parametric model, which reconstructs the 3D human mesh from pose and shape parameters. The pose is represented as a 204-dimensional vector, consisting of the following components: (i) 3-dimensional translation vector that defines the joint position of the body, contributing total of 24 3 dimensions, and (ii) the rotations of 22 body joints, contributing 22 x 6 dimensions to the pose vector. In addition to the pose parameters, the model includes shape parameter vector 3 E 16 which controls the individual body shape.
Object Motion. Object motion characterized by two main components: the global 3D position and the relative rotation. The global position is represented by the centroid of the object, while the relative rotation, denoted as Ree((!), describes the rotation at frame relative to the object's geometry P. The object vertices at frame / are given by Pt Rmm(()) Therefore, the object motion is described by O x where each frame consists of translation relative to the centroid and corresponding relative rotation.
14

---

Condition. In our condition representation, we integrate contextual cues following the CHOIS framework. The text embedding extracted using a pretrained CLIP text encoder. The object geometry is encoded via a MLP applied to its Basis Point Set (BPS), and the resulting features are broadcast all frames. These BPS features are then concatenated with the initial states and waypoints (provided every 30 frames, where only the object's and y-coordinates are available, with the z-axis omitted) and further combined with the text embedding to form the condition c. In addition, due to the fact that we include a 4-dimensional binary contact indicator that specifies hand and feet contact states HOI data representation, we therefore construct a masked motion representation to represent the initial states and waypoint constraints m E (12+201))))1) where the initial state includes both the human pose and object pose at the first frame.
C Theoretical and Empirical Justification of Interaction Dynamics Loss
C.1 Theoretical Justification via Residual Dynamics
Definition. We define a learned dynamics model D that predicts the object's relative motion based on the current scene and human motion:
-o D(s(t), ( Ahe-s+ki 0D),
(17)
where (9) is the scene context at time I, (1)) the object state (e.g., pose), Aly-stRk is the future human motion, and 0D contains the model parameters.
To promote consistent physical behavior by the generated trajectories, we introduce an auxiliary loss that compares the residual dynamics behavior of generated and ground-truth sequences, as follows:
Layn |lôD.gen &D.
(18)
with the residuals defined
0D.gen (D(8(t), 600, A Abe
(19)
6p..t (D(s(t), 00)) Ahe-stk); Sol-stk):
where ( denotes a point-wise distance metric (e.g., the l distance between the point clouds decoded from object poses).
Bias Cancellation. Although may have approximation error, we show that comparing residuals (rather than raw predictions) reduces the effect of model bias. For theoretical clarity, we analyze in the prediction space without applying 4, and only consider 4 Lipschitz continuous final metric layer.
Let the true (unknown) dynamics function be F, and define the model bias
bxx) = D(x:) F(r).
(20)
Let Zgen = (s(t), 869, Ah) and Zgtt (s(1), (t)) Ah. Then the prediction space residu- als can be expressed as
8D D(Igen) - Ad.--44k
(21)
åD.gi D(Irg)) A0
Subsequently, the residual difference becomes
öD.gen 8D (D(r ((xr))) +
(al))+(Ooxo))) (Aor-1tk - Ad.
(22)
= [b((gee)= - b(arg))) + [F(#ge) F(xg)) (Do- Ao).
Assuming that both F and b are Lipschitz continuous with the Lipsethiz constants Ly and L,, and that ||=geen - Zgill S e, ||A Aoll < 6. then we obtain
|lōD.gen dD.gill S Lye L +
(23)
This implies that
| 6p. + as +
(24)
meaning the residual discrepancy vanishes if the input conditions and object motion predictions are sufficiently close. Thus, the residual-based loss is less sensitive to the absolute accuracy of D, and instead emphasizes consistency in behavior under the same model.
15

---

Connection to Training Loss. While our analysis is carried out in the model output space, the actual training loss applied using the decoded point cloud metric given by
(x. v) = (z) - 1
(25)
where f.)()) denotes deterministic decoder from object pose to point cloud. Since both (.e(.) and I1-111 are Lipschitz continuous, the residual error behavior remains consistent under this transformation.
C.2
Empirical Evidence from Motion Decomposition
Setup. To validate the approximation Ez(ôp.gen dD..t se O, we compute the residual difference
A (n) (1)
6 gen (1)
D.gen
((m) (1)
(26)
for each sequence f1.2..... ... and time step (1.2...., T). statistics over all NT samples, as follows:
We then compute the aggregate
N T
N T
Σ (t),
var(
ΣΣ
(n)
(t) - -
(27)
NT 1=
NT n=
These metries reflect the average residual discrepancy and its variability across all sequences and frames.
Table 6: Comparison of Statistical Measures.
Statistic
Mean
Variance
Ground Truth (gt)
0,3130
0,1190
Generated (gen)
0.3147
0.0950
Difference (gen gt)
0.0016
0.1561
Mean Values Comparison (Generated vs Ground Truth)
Variance Values Comparison (Generated vs Ground Truth)
Veriance of trror
Mean Error
0.25
0.2
diff
0.00
0.1
gen
-0.25
0.0
40
60
80
100
120
20
40
60
80
100
120
Step
Step
Actual Difference Between Generated and Ground Truth
Distribution of Ground Truth (gt)
Difference
Maan: 0.31 variance:
40
60
80
100
120
2
3
Step
Distribution of Generated (gen)
Distribution of Difference (gen gt)
Difference gti
Vahance
-2
Figure 7: Statistical and distributional comparison of interaction dynamics emrors for generated and ground-truth object motion. The mean and variance of residuals are nearly identical for gen and gt, while their difference is centered near with low variance. Probability density plots further confirm the alignment of gen and gt distributions, and support the theoretical claim of modeling bias cancellation.
16

---

Results. As shown in 6 the mean of the residual difference is nearly zero (0.0016), confirming that the residual signal has no significant bias across time steps or sequences, consistent with our theoretical approximation ge dD 2 0.
We further visualize the typical temporal patterns in Fig. where the residual differences fluctuate randomly around zero, with no observable drift or systematic deviation. These empirical observations validate the core assumption that residual-based comparison cancels out the model bias when D> is applied consistently to both generated and ground-truth trajectories.
Together with our theoretical insights, these results support the following: (i) The residual-based loss introduces natural bias cancellation, making the learning robust to approximation errors in D. (ii) This enables the use of D an auxiliary evaluator without requiring high-fidelity absolute predictions. (iii) The formulation enforces consistency in human-object dynamics, thereby improving the realism and coherence of generated motion. This bias-invariant design is key strength of our auxiliary loss and contributes to the stability, generalizability, and interpretability of the learned policy.
C.3 Residual-based Loss and Generated HOI Quality
To further evaluate the impact of our residual-basee interaction dynamics loss on generated HOI quality, we conducted ablation studies comparing different formulations: (i) Without Interaction Dynamics Loss, i.e., baseline CHOIS without any dynamics supervision. (ii) Residual Dynamics Loss, i.e our proposed formulation (10) that cancels noise and imperfections in the pretrained dynamics model. (iii) Ron-Residual Dynamics Loss, which directly penalizes the difference between the predicted object motion Δô, and the dynamics model output A without bias compensation:
Lyn
res
E, (A67, Aō:) [[]]
(28)
We report results without classifier based guidance to isolate pure generation capability. As summa- rized in Table 7 incorporating the interaction dynamics loss consistently improves generation quality over the baseline. In particular, the residual-base based formulation produces more stable, physically realistic, and contact-accurate HOIs, whereas the non-residual variant allows errors in the dynamics model to propagate, potentially degrading motion quality. These findings demonstrate that the residual formulation is essential for robust HOI generation and support the design choice of our auxiliary interaction dynamics loss.
Table 7: Ablation of interaction dynamics loss formulations. Residual dynamics consistently improves physical realism and contact accuracy.
Method
Try 4
F$
Cr1
Phand
MPJPE
Tex 4
Tan 4
Oskj
Without Interaction Dynamics
3.05
3.63
0.54
0.61
16.05
25.34
12.36
0.99
HOI-Dyn (Non-Residual Dynamics)
3.03
0.39
0.59
0.71
15.63
25.10
11.86
0.89
HOI-Dyn (Residual Dynamics)
3.03
0.37
0.60
0.61
15.56
24.61
11.67
0.91
C.4 Learned Object Motion Reactor as Differentiable Surrogate for Physies Simulators
Physics-basee based approaches are widely adopted to ensure physically plausible HOI. Representative methods such as InterMimic [3), PhysHO1 [40], and SkillMimic [44) rely black-box physics simulators in combination with reinforcement learning, typically learning one policy per skill. These methods primarily focus controlling human motion and do not explicitly model the object's reactive behavior. Moreover, their non-differentiable nature and dependence on detailed physical parameters (e.g., mass, friction, compliance) limit applicability fully differentiable, end-to-end generative frameworks, particularly for long-horizon or complex interactions.
To overcome these limitations, we introduce learned object motion reactor, differentiable, data- driven surrogate for object dynamics. Conceptually, it functions as a conditional world model, producing temporally coherent and physically plausible behavior without enforcing explicit physical laws. Its advantages can be summarized in three key aspects:
(i) Seamless integration with generative pipelines: The reactor's differentiable design allows direct embedding into diffusion-based HOI generation frameworks, enabling end-to-end training and gradient-based optimization.
17

---

(ii) Generalization without requiring physical parameters: Unlike physics simulators that need detailed mass, friction, and compliance values, the reactor learns directly from motion patterns in data, making it robust to missing or uncertain physical parameters.
(iii) Learning respect physical constraints: Although learned, the reactor captures essential behaviors such as responding appropriately to human motion and remaining stationary when not contacted, ensuring physically plausible and temporally consistent object dynamics.
By explicitly incorporating contact states, our approach further strengthens human-object coupling, extending prior work such CHOIS [13]. Hybrid paradigms combining learned dynamics with priors or simulators remain promising direction; incorporating measurable physi- cal properties, explored in FORCE 45). could enhance realism while preserving differentiability. Nevertheless, the learned object motion reactor provides a practical, scalable, and fully differentiable solution for generative HOI modeling, effectively bridging the gap between physically grounded simulation and data-driven generation.
D Additional Qualitative Results of HOI Generation
We provide additional qualitative comparisons to further evaluate the effects of modeling interaction dynamics.
the largetable
Hold the cloth stand
the Push the plasticbox
CHOIS
HOI-Dya
Kick the plastichox..
Push largetable
CHOIS
HOI-Dyn
{
f1
Figure 8: Qualitative comparison in Oetion-Interrction Response.
Interaction-Responsse.
In Fig. we assess the fitiio--nnteractinnRReppupee loop. The results show that incorporating interaction dynamics constraints enables the model to better understand how objects should respond to both human actions and the contact context. This suggests that modeling such dynamics improves physical plausibility and contextual coherence in human-object interactions.
In Fig. 9 we further investigate sequence-lewe alignment. We observe that explicitly enforcing interaction dynamics not only enhances the consistency of naman-object contact but also improves the overall motion quality. These findings align with our claim:
"Contact is implicitly governed by the dynamics- no need to explicitly model it. If there is contact, there no response; if contact occurs, the object's 's response naturally determined by the interaction dynamics."
18

---

Kick the largebox, lift the largebox, and move the largebox..
CHOIS
Move downward simultancuusy
Exaggerated motion
Floating object
HOI-Dyn
floorlamp
Cloth stand
Push largebox, and back down.
CHOIS
Feet sliding
COI-DyA
Figure 9: Qualitative comparison in Sequence-level alignment.
Moreover, by restricting the object from moving in physically implausible ways, our method encour- ages the human to take more active and reasonable actions. This results in motion that is not only more coordinated but also better aligned with the intended conditions or text compps--addressinng key limitation in prior works where human and object often move independently without dynamic coherence.
E
Network and Additional Results
of Interaction Dynamics
E.1
Model Architecture
Our dynamics model is a lightweight predictor designed to infer object motion induced by human actions and interaction context. It takes input the object-centric BPS point cloud, human joint relative motions, and contact information, and outputs the relative object motion (rotation + translation). The architecture consists of three main components:
(i) BPS Encoder. two-layer multilayer perceptron (MLP) encodes the input object BPS (1024 x 3) into a compact feature vector. This captures the object's shape and spatial context.
(ii) Human Motion Module. Mini Transformer models the temporal interaction dynamics between the human and the object. It first encodes the joint-level motion features (3D relative positions and rotations) through MLP, then injects condition
object motion, contact state, and BPS feature- via conditional encoding. A shallow transformer with learnable positional embeddings is applied, followed by attention pooling obtain a global human motion feature.
19

---

(iii) Motion Decoder. A MLP maps the fused interaction feature to the target object motion vector (3 + 9 dimensions), representing translation and flattened rotation. The output can be optionally projected to S3(3) to ensure a valid rotation.
E.2 Local Smoothness and Temporal Homogeneity
Building the design of our dynamics network, we examine how its architecture and formulation inherently enforces local smoothness and temporal homogeneity.
E.2.1 Temporal Homogeneity
Temporal homogeneity means that the mapping from human motion to object motion is invariant to absolute time indices. Since our dynamics network is designed for step-wise interaction modeling, only concerns the immediate effect of human motion object motion, analogous to how applying certain force induces a proportional response regardless of when it is applied. Intuitively, pushing with the same strength should produce comparable object displacement whether it occurs early or late in an episode. This property is directly regularized through our loss function:
1
L = E,, k-U(1
. ((Aoo-tth; Ao
(29)
where both the starting step t and horizon ht are uniformly sampled. The normalization factor k decouples error from horizon length, ensuring that supervision is distributed evenly across different positions and scales. As result, the model learns event-agnostic dynamics, capturing only the causal effect of human motion on object motion without bias toward specific temporal phases.
E.2.2 Local Smoothness
Theoretical Motivation. Object motions in buman-object interactions evolve continuously under smooth forces and contacts. Since the input human joint motion (0() and object state o()) are temporally continuous, the predicted object motion
(t D(s"), ol0) Ah(0;
(30)
expected to vary smoothly, forming the theoretical basis for local continuity.
Network Design Guarantees. The architecture. which includes differentiable MLPs, Mini- Transformer, and attention pooling, is inherently smooth. No discontinuous operations (e.g. quan- tization, and hard thresholds) are introduced. Training on continuous trajectories further promotes smooth transitions in the learned mapping.
Lipschitz Continuity and Empirical Test. We formalize local smoothness via Lipschitz continuity: function ff is Lipschitz continuous with constant L if [lf()1)) f(x2)l < D| x:2 for all Ii, x2. To test this empirically, we perturb human motion inputs by adding random unit-vector noise scaled by
Ah ( +e llalla'
7
" ~(0,1..
(31)
We then measure the output deviation defined as
AD [D)s(0) A DDs(()) (0) Ah
(32)
If AD grows approximately linearly with the perturbation scale e, the model exhibits locally Lipschitz behavior. Tables g summarize the results, confirming smooth and proportional responses across perturbation magnitudes.
E.3 One-Step and Auto-Regressive Prediction Results
We evaluate the predictive ability of our interaction dynamics model from two complementary perspectives: one-step prediction and auto-regressive prediction.
In the one-step setting, the model is given the current scene context and ground-truth human motion to predict the object state at the next frame (T = 1). This setting directly evaluates the local dynamics
20

---

Table 8; Effect of unit-vector perturbations scaled by e on predicted object motion.
Perturbation
Difference Norm AD
Combined (12D)
Translation (3D)
Rotation (9D)
1 1 10-5
2.6e-5
5.7e-6
2.5e-5
1 10
1.2e-4
2.4e-5
1.le-4
10
1.le-3
1.97e-4
1.07e-3
1 1x 10-2
1.le-2
1.99e-3
1.09e-2
1x 10-1
1.0c-1
1.96c-2
1.00c-1
Table 9: Relative magnitude of input perturbations (as percentage of the average input norm).
Perturbation scales correspond to 10-5,
10-1, 10-9,10-1 10-30011110
1003,10-2 10-20-2 10-1.
Component
Average Norm
Perturbation Levels (%)
Translation (3D)
4.42 10-4
2.3, 23, 23, 230, 2300, 23000 0.35, 3.5, 35, 348, 3480
Rotation (9D)
2.83 10 10-3
Combined (12D)
2.87 10-3
0.35, 3.5, 35, 348, 3480
response. As shown Fig. T0 our model achieves high accuracy in object translation, with very small deviations from ground truth. However, the rotation prediction still shows room for improvement. We attribute this the dataset lacking hand pose details, which limits the inference of fine-grained object rotations from body joints alone. Nevertheless, the predicted rotations remain broadly consistent with ground truth.
In the auto-regressive setting, we recursively feed the predicted object state back into the model generate long-term future motion. This setting reveals how prediction errors accumulate. As illustrated in Fig. []] even though our model is only trained with one- and two-step supervision, it demonstrates impressive generalization the auto-regressive regime. The predicted object transla- tions stay closely aligned with the ground truth over time. Although rotation errors do increase due to compounding uncertainty, they remain stable and plausible.
Importantly, our method relies solely on the one-step supervision during training, which avoids the inefficiency and instability of full auto-regressive rollout training. This design enables fast training while already delivering strong performance. The model's ability to generalize well in the auto-regressive case is bonus, not requirement. Hence, the proposed design achieves excellent balance between efficiency and long-horizon performance - a highly cost-effective formulation.
E.4 Comparison on Interaction Dynamics
We further analyze the quality of interaction dynamics generated by CHOIS and our proposed HOI-Dyn. As shown in Fig. 12 and Fig. we employ the predicted human-object sequences and compare their induced object motion against the reference dynamics. Here, the reference ground truth in the test set but the object's 's relative motion in the synthetic sequences used as initial conditions for prediction. Since motion synthesis is inherently multimodal, we focus on whether the predicted motion is physically and contextually plausible.
the
We observe that HOI-Dyn produces significantly more consistent and coherent object motion. In contrast, CHOIS often yields noisy or unstable object transitions, with noticeable discontinuities or implausible drifts over time.This comparison highlights the advantage of our synchronized control formulation and the dynamics-guided residual loss.
E.5 Joint-Level Attention Visualization
In our proposed driver-responder synchronized control framework, we posit that once contact is established, the object's is motion should predominantly follow the human motion. Based on the SMPL representation, human motion is determined by the transitions and rotations of body joints. We thus interpret object motion as a result of the aggregated influence of all human joints.
21

---

Falday T
200
033
Swiddd
-36
-03
Grord Kdl
Grond Tut
Figure 10: One-step performance.
Figure 11: Auto-regressive performance.
0006
0098
Sreval futh
5232
300
Grand
Sround Kuth
-0.0
10
Figure 12: CHOIS.
Figure 13: HOI-Dyn.
Our MiniTransformer module explicitly models joint-wise contributions to object dynamics by applying attention over all J 24 joints Specifically, given joint features Xj I2d for joint j E JJ....,J. and a learned attention weight wj. we compute the global representation as:
Fglobul
Σ Wi Xj. 1=1
(33)
where the attention weights wy [0, 1] are computed via a softmax layer and satisfy Σ, wy This formulation allows the model to selectively focus on the most relevant joints under different interaction conditions, enabling interpretable and context-aware reasoning of human-object dynamics.
We visualize the joint-level attention maps for typical cases in Fig. 14 and Fig. 15 For the first example in Fig. [4, the person bends down to pick up a box. During this full-contact phase, the object is entirely under human control. The attention map highlights not only the hands but also the feet, elbows, and chest suggesting whole-body coordination that is both intuitive and desirable. This supports the idea of a "global response" where object motion is governed by distributed joint influence.
For another example Fig. [5| where the person walking while carrying the box, the attention is more concentrated on the hands and feet. This makes intuitive sense: the hands are directly manipulating the object (contributing to both translation and rotation), while the feet govern the global transition through locomotion. Across multiple sequences, we also observe consistent attention
22

---

on the hips and lower limbs, suggesting their role as stable references for interpreting whole- body movement.
Aetention-weighted joints
SMPLXX Attencion (frame
3.75
85475
3.
3.50
05453
1.0
1.2
00425
0.
1.00
22403
0.040
60375
0.50
88333
0.25
0,000
40325
-14
Figure 14: Pick up box.
Figure 15: Walk forward while holding the box.
F
Details of Classifier-based Guidance
While the use guidance modules (e.g., contact-based constraints) can lead to targeted improvements in motion quality, all comparisons with state-of-the-art baselines are conducted under identical guid- ance settings. This ensures a fair evaluation, as our proposed method HOI-Dyn focuses on internally controlling and optimizing the interaction dynamics during the generation process. Therefore, the guidance configuration is kept fully consistent across all methods and does not introduce any bias in favor of HOI-Dyn.
F. Feet-Floor Contact Guidance
To encourage physical plausibility during locomotion and standing still, we introduce contact guidance loss that penalizes unnatural foot height above the ground, consistent with the CHOIS setting. Specifically, we extract the 3D positions of the left and right toe joints, and compute the support foot height as:
Rapppon min (/ueft-tee hright-tee)
(34)
where h E BxTxl represents the height (Z-axis) across batch B and time T.
We enforce that the support foot remains near the ground a (set at 2 cm) via MSE loss:
Lreet feet--frorr
Σ
h
(b,t) Raspport - a) a)²
(35)
BT
b=1
This loss improves contact realism by promoting plausible foot-ground interaction.
F.2 Hand-Object Contact Guidance
To enhance the realism and stability of hand-object interactions, we use the multi-term auxiliary loss that encourages physically plausible contact and temporal coherence.
Contact Loss: For each timestep, we extract the 3D positions of the left and right palm joints and compute their closest distances the object mesh as follows:
B
Logntact
max(
(0.0)
(6,4)
M(0 - 0) (b..A)
(36)
BT
Conttat
b=1 1=1 (E(LR)
23

---

(b.4) is the palm position, 8 0.02 is the contact where ((b..) is the object mesh at frame 4, Ph threshold, and Pcontact is the predicted binary contact indicator.
Temporal Consistency Loss: For contact frames, we enforce that the palm positions (in the object coordinate frame) remain temporally smooth and aligned by
1 Σ 7
Lonnsistency = 2
(cos(Ot,) My + cos() M
(37)
where 0i is the angle between the relative palm directions at frames and j. and M are symmetric masks marking the frames where contact is active.
Floor Penetration Penalty: We suppress object vertices penetrating below the ground plane (z < 0) through
Lioor
max( -2n, 0).
(38)
N
n=1
The final guidance loss is given by
intteaacionn
Leuntuct
Lcomsisteccy
Lavors with A 100.
(39)
This auxiliary loss encourages accurate and stable contact modeling.
24

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Claims we made accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results It is fine to include aspirational goals as motivation long it is clear that these goals are not attained by the paper.
be expected to generalize other settings.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
The interaction dynamics model may not capture object rotations with high precision, mainly due to limited data fidelity missing fine-grained physical cues. How- ever, our residual-based interaction dynamics loss is specifically designed to mitigate such inaccuracies by focusing differential consistency rather than absolute correctness.
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
25

---

Answer: [Yes]
Justification: We provide a comprehensive justification for the robustness of our residual- based interaction dynamics loss to imperfect model learning, grounded in clear assumptions, theoretical analysis, and empirical evidence.
Guidelines:
The answer NA means that the paper does include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proof' provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
Experimental setups described in detail.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
26

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Our project website is available at httpss//wulin97. httpa//iiw...iiu..o.oooooooooooe and the code is available at httts:///gtthubb.
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run reproduce the results. See the NeurIPS code and data submission guidelines (https /CodeSubmeei /nsps.cccppulic//ggu /odie///ggiddeeeeeouuuuumbibbi ssionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We have described all the details about of experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We have conducted statistical tests in our experiments, error bars are plotted in figures.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
27

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: All computational resources used in our experiments are reported in detail in the experimental setup. Notably, our method is highly efficient: all experiments were conducted on a single NVIDIA RTX A4500 GPU with 20GB memory. We also provide an analysis of model parameters and inference cost, demonstrating that our interaction dynamics model is lightweight.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics htttpp///eeerrsss.cc///bbc//eiit
Answer: [NA]
Justification: Not applicable.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- cration due to laws or regulations in their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [No]
Justification:
Our algorithm has potential negative social impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
28

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [No]
Justification: Code will be released after acceptation, would be open access, no safeguards are required.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging. and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We have cited the existing assets we used in our paper.
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
29

---

If assets are released, the license, copyright information, and terms of use in the thcode has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssit
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [No]
Justification:
We did not introduce any new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
We do not have any experiments research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification:
We do have any experiments or research with human subjects
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
30

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [No]
Justification: No large language models (LLMs) were involved in the design, development, evaluation of the core methods presented in this paper. LLMs were only used as writing assistant to improve the clarity and fluency the text. As such, their usage does not impact the scientific rigor, methodology, or originality of the work.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, ntn-standard components.
Please refer to our LLM policy hcc:ga//inauriissccc///ooreeeeee for what should or should not be described.
31


Reviews:
[{"review_id": "jvQ3VhUP7p", "reviewer": "Reviewer_J5sP", "summary": "This paper introduces HOI-Dyn, a framework for generating more realistic 3D human-object interactions. It models interactions as a \"driver-responder\" system, where human actions drive object reactions. The core uses a lightweight Transformer model to predict object responses and incorporates the loss of a residual dynamic for consistency. This dynamics model is only used during training for efficient inference. Experiments show that HOI-Dyn improves HOI generation quality and provides a new evaluation metric.", "questions": "1. Can this modeling approach be extended to settings with multiple objects or multiple people?\n2. Should the absolute position and pose of the object be considered separately? It seems their reactive behaviors would follow different patterns. If they're mixed, could this lead to overfitting?\n3. What's the fundamental advantage of this method compared to prior approaches that model human, object, and interaction separately? Intuitively, this modeling approach seems more suited for a single human and a single object. In scenarios with more people and objects, this method might limit scalability and significantly increase problem complexity.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The idea of modeling HOI generation as a reactive system is interesting and offers a novel perspective on the problem.\n2. The assumption of an active-passive system is very reasonable because objects shouldn't move without contact.\n\nWeaknesses:\n1. Can this modeling approach be extended to settings with multiple objects or multiple people?\n2. Should the absolute position and pose of the object be considered separately? It seems their reactive behaviors would follow different patterns. If they're mixed, could this lead to overfitting?\n3. What's the fundamental advantage of this method compared to prior approaches that model human, object, and interaction separately? Intuitively, this modeling approach seems more suited for a single human and a single object. In scenarios with more people and objects, this method might limit scalability and significantly increase problem complexity.", "was_revised": true, "final_justification": "The authors have provided a compelling rebuttal that successfully addresses my primary concerns regarding scalability and their unified modeling approach.\n\nThe most significant point is the clear demonstration that their model's core framework is not inherently limited to single-human, single-object scenarios. Their proposed extension, which uses a permutation-invariant aggregation strategy, offers a logical and theoretically sound path to scale the model for multi-human interactions. This thoughtful generalization shows the approach's potential beyond the current, focused implementation.\n\nFurthermore, the authors' empirical study on coupled versus decoupled modeling is a crucial addition. The results are decisive: the coupled model consistently outperforms the decoupled version. This data-driven finding is a strong justification for their design choice, as it proves that modeling an object's translation and rotation jointly provides a beneficial inductive bias. It correctly reflects how these motions are interconnected in real-world human-object interactions. The authors have convincingly shown that their model's design is both well-justified and leads to superior performance.", "rebuttal": "# Response to Reviewer J5sP\n\n## **Question 1**: \"Can this modeling approach be extended to settings with multiple objects or multiple people?\"\n>\n>Thank you for the question. Yes, our modeling approach *can be extended to scenarios involving multiple humans and objects*. For detailed discussions and the formulation of such extensions, please refer to our response to Question 3 below.\n\n## **Question 2**: \"Should the absolute position and pose of the object be considered separately? It seems their reactive behaviors would follow different patterns. If they're mixed, could this lead to overfitting?\"\n\n>Thank you for the insightful suggestion. Our interaction dynamics model focuses on the object’s motion state changes, predicting relative translation and rotation rather than absolute position and pose. This relative formulation better captures reactive behaviors. To investigate joint versus separate modeling of translation and rotation, we compare two design choices as discussed in Section 4.3: (1) **Coupled** modeling, where translation and rotation are jointly predicted by a shared module; and (2) **Decoupled**, where separate modules independently predict translation and rotation. To ensure a fair comparison, we fix the maximum horizon to $K=2$ and control the model size (i.e., size of the transformer module) in both design choices.\n>\n>| Transformer Config | Model Type | Params (M) | VAL Loss | TEST Loss |\n>| :---- | :---- | :---- | :---- | :---- |\n>| Depth4\\_Dim64\\_Head8 | Coupled | 0.483 | 0.1318 ± 0.0004 | **0.3130 ± 0.0004** |\n>| Depth8\\_Dim64\\_Head8 | Coupled | 0.550 | *0.1300 ± 0.0003* | 0.3177 ± 0.0001 |\n>| (Depth2\\_Dim64\\_Head8) $\\times$ **2 branches** | Decoupled | 0.496 | 0.1455 ± 0.0013 | 0.3396 ± 0.0015 |\n>| (Depth4\\_Dim64\\_Head8) $\\times$ **2 branches** | Decoupled | 0.564 | 0.1387 ± 0.0010 | 0.3370 ± 0.0006 |\n>| (Depth4\\_Dim128\\_Head8) $\\times$ **2 branches** | Decoupled | 1.047 | **0.1264 ± 0.0005** | 0.3382 ± 0.0010 |\n>\n>**Key observation**:\n>\n>* Under the current model scale and training setup, *coupled models consistently demonstrate better generalization performance compared to decoupled ones*.\n>\n>Though decoupled models with larger capacity can achieve lower validation loss, their test loss remains higher, indicating a larger generalization gap. This suggests that the coupled design introduces a helpful inductive bias by jointly reasoning about translation and rotation, which often co-occur in physically realistic human-object interactions.\n>\n>In summary, the coupled approach is more parameter-efficient and it better captures the interaction dynamics under the current settings, while decoupled modeling’s benefits may appear in more complex scenarios that need separate treatment of translation and rotation.\n\n## **Question 3**: \"What's the fundamental advantage of this method compared to prior approaches that model human, object, and interaction separately? Intuitively, this modeling approach seems more suited for a single human and a single object. In scenarios with more people and objects, this method might limit scalability and significantly increase problem complexity.\"\n>\n>### (1) Fundamental Advantage of the Proposed Method\n>\n>Our method’s key advantage lies in integrating human and object motions within a unified generative framework, as described in Section 2.3. This framework leverages the strong capability of diffusion models to produce globally coherent sequences, which is crucial for capturing the causal flow in human-object interactions. By explicitly decoupling the driver (human) and responder (object) roles through interaction dynamics, our approach ensures that object reactions are causally consistent with human motion—**addressing common issues where interactions lack clear cause-effect relationships**.\n>\n>In contrast, approaches that model human and object separately often fail to naturally capture their interactions. Even some unified models suffer from artifacts—such as objects moving before human initiation—because **generative models often prioritize overall distributional coherence over detailed local interactions**. Our method not only harnesses the strengths of unified generative modeling but also explicitly enforces fine-grained interaction through structured dynamics.  \n>\n>### (2) Scalability to More Objects and Humans\n>\n>Thank you for the valuable comment. Our current work focuses on single-human, single-object interactions, aiming to isolate and examine core interaction dynamics under a controlled setting. Since our interaction dynamics model is **object-centric**, in multi-human and multi-object scenarios, the modeling focuses on each object individually. The generalization to these scenarios can be naturally decomposed into two typical cases:\n>\n>#### Case 1: Object Being Manipulated by One Human\n>\n>This is fully covered by our current formulation. As a reminder, our single-human interaction dynamics (Eq.2) are expressed as :\n>\n>$$ \\Delta o^{(t)} \\leftarrow \\mathcal{D}(s^{(t)}, o^{(t)}, \\Delta h^{(t)}; \\theta\\_{\\mathcal{D}}) $$\n>\n>where $o^{(t)}$ is the current object state, $\\Delta h^{(t)}$ is the human motion change, $s^{(t)}$ encodes the interaction context such as contact type and object geometry.\n>\n>#### Case 2: Object Being Manipulated by Multiple Humans\n>\n>To handle scenarios where multiple humans simultaneously interact with the same object (e.g., jointly carrying a table), we propose the following generalization of Eq.2:\n>\n>$$ \\Delta o^{(t)} \\leftarrow \\mathcal{D}(  o^{(t)},   {(s_i^{(t)}, \\Delta h_i^{(t)})}_{i=1}^N  ; \\theta\\_{\\mathcal{D}})$$\n>\n>Each pair $(s_i^{(t)}, \\Delta h_i^{(t)})$ captures the interaction context and motion of the $i$-th human. This formulation enables the model to reason over heterogeneous human-object interactions — such as different contact locations, force directions, or limb configurations — in a unified manner.\n>\n>To integrate the set ${(s_i^{(t)}, \\Delta h_i^{(t)})}_{i=1}^N$, we can adopt a **permutation-invariant aggregation** strategy, such as attention pooling or a transformer-based encoder, yielding a compact interaction embedding. This embedding, combined with the object state $o^{(t)}$, serves as input to the dynamics model to predict object motion under multi-human manipulation.\n>\n>We believe this generalization offers a principled foundation for scaling up interaction dynamics modeling from single-human cases to more realistic, multi-human scenarios. However, due to the limited availability of large and diverse multi-human interaction datasets, we have not yet performed extensive experiments in this setting. We will update the manuscript to include these discussions and plan to further explore experimental validation in future work."}, {"review_id": "B5dEL4CCyd", "reviewer": "Reviewer_biUj", "summary": "This work introduces a novel driver–responder perspective for human-object interaction (HOI) generation, where human actions drive object responses. Central to this approach is the concept of interaction dynamics, which governs how objects naturally react to human motion—implicitly handling contact and ensuring physically plausible interactions.", "questions": "1. One point of curiosity is how the model would perform if the object waypoint constraints were removed. Since the current framework relies on predefined waypoints to guide HOI generation, it would be insightful to evaluate whether the interaction modeling still holds up without such guidance. This could better demonstrate the model’s true capacity for reasoning about physical causality and interaction dynamics.", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### **1. Strength:**\n- The entire article is basically complete and the experiments are adequate.\n- The overall qualitative and quantitative results are satisfactory, showing clear improvements over CHOIS across several metrics and visual examples.\n- Interaction Dynamics modeling perform well on the object pose correction.\n\n### **2. Weakness:**\n- The related work section is somewhat rough and lacks precision in categorizing and positioning recent HOI generation methods. For instance, HOI-Diff and CG-HOI are concurrent works, so the statement that CG-HOI is further enhanced is inaccurate and potentially misleading. Additionally, in Section 2.3, I would not categorize CHOIS under joint human-object prediction. Similar to OMOMO, CHOIS relies heavily on predefined object trajectories, effectively simplifying the task. A more appropriate categorization would be to group methods like HOI-Diff, CG-HOI, InterDreamer [1], THOR, HIMO [2], and ChainHOI [3] together, as they focus on jointly modeling human-object interactions without access to future object motion. OMOMO, CHOIS and [4] should be categorized into Object trajectory-guided HOI generation.\n\n\n[1] InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interaction. Sirui Xu et al.\n\n[2] HIMO: A New Benchmark for Full-Body Human Interacting with Multiple Objects. Liang Xu et al.\n\n[3] ChainHOI: Joint-based Kinematic Chain Modeling for Human-Object Interaction Generation. Ling-An Zeng\n\n[4] Human-Object Interaction from Human-Level Instructions. Zeng Wu et al.", "was_revised": false, "final_justification": "", "rebuttal": "# Response to Reviewer biUj\n\n## **Weakness 1**: \"The related work section is somewhat rough and lacks precision in categorizing and positioning recent HOI generation methods\"\n>\n>Thank you for the valuable feedback. We agree that HOI-Diff and CG-HOI [1] are concurrent works and will revise the phrasing accordingly. We also appreciate your suggested taxonomy separating joint modeling methods (e.g., HOI-Diff [2], CG-HOI [1]) from object trajectory-guided ones (e.g., CHOIS [3], OMOMO [4]), and will update Section 2.3 to reflect this clearer categorization.\n>\n## **Question 1**: \"One point of curiosity is how the model would perform if the object waypoint constraints were removed. Since the current framework relies on predefined waypoints to guide HOI generation, it would be insightful to evaluate whether the interaction modeling still holds up without such guidance. This could better demonstrate the model’s true capacity for reasoning about physical causality and interaction dynamics.\"\n>\n>We appreciate your insightful concern.Importantly, our proposed dynamics model **does not rely on using these navigation points** as input. To isolate and assess the actual contribution of the interaction dynamics model, we design a **No WayPoint (NO-WP)** setting, in which we **remove waypoints** from both the Baseline (CHOIS[3]) and the proposed HOI-Dyn models.\n>\n>Under this setting, the comparison metric *GT difference* is no longer meaningful. Therefore, we focus on those metrics that are more relevant to physical plausibility and interactive realism, including the metrics of **foot sliding, penetration, contact F1, contact accuracy, FID**, and **Diversity**. The classifier-based guidance is inapplicable during testing; thus, the reported results reflect the generator’s intrinsic performance. The results are as follows:\n>\n>| Metric | Foot Sliding ↓ | Penetration ↓ | Contact F1 ↑ | Contact Acc ↑ | FID ↓ | Diversity ↑ |\n>| :---- | :---- | :---- | :---- | :---- | :---- | :---- |\n>| Baseline NO-WP | 0.401 | 0.581 | 0.573 | 0.648 | 5.36 | 7.90 |\n>| HOI-Dyn NO-WP | **0.376** | 0.582 | **0.592** | **0.670** | **4.81** | **8.09** |\n>\n>Compared to the baseline, our HOI-Dyn (NO-WP) significantly improves foot sliding, contact F1, contact accuracy, FID, and diversity, while maintaining comparable penetration levels. These results highlight the effectiveness of the interaction dynamics.\n\n---\n\n>**References:**  \n>\\[1\\] Cg-hoi: Contact-guided 3d human-object interaction generation, CVPR 2024\\.  \n>\\[2\\] Hoi-diff: Text-driven synthesis of 3d human-object interactions using diffusion models, CVPRW 2025\\.  \n>\\[3\\] Controllable human-object interaction synthesis, ECCV 2024\\.  \n>\\[4\\] Object motion guided human motion synthesis, TOG 2023\\."}, {"review_id": "HMkYUaM5TI", "reviewer": "Reviewer_UuR7", "summary": "The paper proposes a transformer-based dynamics model that predicts object movements induced from human movements during interaction. The dynamics model is used to calculate dynamics loss when training a conditional diffusion-based generative model which outputs human object interaction from object geometry/text/waypoint conditional input. The paper demonstrates the dynamics-based approach improves human-object interaction quality (consistency, contact, natural and plausible human/object movements, etc).", "questions": "This paper introduces interaction dynamics through a clever and novel approach using dynamics models and residual-based dynamics loss. The method achieves improved HOI motion synthesis results compared to SOTA baselines, which is demonstrated through comprehensive experiments. Despite the presence of motion artifacts (Weakness 1), the work represents a valuable advancement.\nHowever, it would be better if: (1) more clarification that the assumptions made in using the dynamics model for calculating dynamics loss (Weakness 2), and (2) action-specific baseline comparisons to more clearly demonstrate the advantages of the dynamics-based approach (Weakness 3).", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n1. Defining interaction dynamics and consistency as object movement induced by human movement is innovative and intuitive. The integration of dynamics loss into CHOIS's HOI learning diffusion model is clever, introducing no additional inference overhead while enabling conditional generative models to learn interaction dynamics.\n\n2. Defining dynamics as residuals is effective and the paper demonstrates that such formulation mitigates possible errors from the dynamics model, making the approach plausible and robust.\n\n3. The method shows improved results compared to baselines, with extensive experiments conducted across various settings for the dynamics model.\n\nWeaknesses\n\n1. The result videos exhibit common artifacts where objects drift when not in contact with humans (at demo video 00:02, 00:12, 00:16). A contact-aware term that prevents object moving when human-object interaction has not yet occurred would improve realism.\n\n2. While L181 states that the dynamics model is locally smooth and temporally homogeneous, the local smoothness assumption requires more explanation. The claim that \"local smoothness reflects the continuity of physical motion\" needs more detail on how the dynamics model ensures continuity.\n\n3. It would be better if performance improvement is analyzed per action type (e.g., \"put,\" \"kick\") in Table 1. In the video, the CHOIS baseline appears effective for hand-based interaction (carry, push, etc) but degrades significantly for foot-based interactions compared to the paper's method, and it would be clear for readers if this difference is demonstrated numerically. \n\n(minor weakness for better clarity)\n\n4. Please include input text in videos so readers can evaluate whether the output motion is semantically aligned with the given text.", "was_revised": true, "final_justification": "The rebuttal addresssed most of my concerns, and therefore I would keep my original rating", "rebuttal": "# Response to Reviewer UuR7\n>\n## **Question 1**: \"The result videos exhibit common artifacts where objects drift when not in contact with humans (at demo video 00:02, 00:12, 00:16). A contact-aware term that prevents object moving when human-object interaction has not yet occurred would improve realism.\" (Weakness 1)\n>\n>Thank you for your valuable observation. Compared to the baselines, our method has already substantially reduced object drift through the interaction dynamics, which is reflected in improved Contact quality and lower motion errors (Table 1). Only minor drift remains in a few frames, likely due to the soft nature of our constraint. We agree that incorporating stronger contact-aware mechanisms could further enhance realism, and we plan to investigate this in future work.\n>\n## **Question 2**: \"While L181 states that the dynamics model is locally smooth and temporally homogeneous, the local smoothness assumption requires more explanation. The claim that \"local smoothness reflects the continuity of physical motion\" needs more detail on how the dynamics model ensures continuity.\" (Weakness 2)\n>\n>Thank you for raising this point. We clarify how local smoothness is ensured from three perspectives: **theoretical motivation, architectural design, and empirical validation**.\n>\n>**(1)Theoretical Motivation**: The assumption of local smoothness is rooted in the physics of interaction: object dynamics in human-object interactions follow Newtonian mechanics, where object motions evolve continuously in time in response to smooth forces and contact. Since human joint motion $h^{(t)}$ and object state $o^{(t)}$ are both temporally continuous signals in the real world, their influence on $\\Delta o^{(t)}$ should also vary continuously. This forms the basis of the following formulation: \n> $$ \\Delta o^{(t)} \\approx \\mathcal{D}(s^{(t)}, o^{(t)}, \\Delta h^{(t)}; \\theta\\_{\\mathcal{D}}), $$\n>where the learnable function $\\mathcal{D}$ models the object's next-step change as a function of the current state and human-induced motion. Since the inputs vary smoothly across time, the model is expected to learn a smooth mapping unless otherwise constrained.\n>\n>**(2)Architectural Design**: We do not introduce any discontinuous operators (e.g., quantization, hard thresholding). The network is composed of standard differentiable components (MLPs, attention, etc.), which are smooth by construction. Additionally, the input space of $\\mathcal{D}$ is locally dense (i.e., trained on continuous trajectories), promoting the learning of smooth transitions. Temporal homogeneity is further ensured as the model is **time-step agnostic**—i.e., it is applied uniformly across all frames without access to absolute time.\n>\n>**(3)Empirical Validation**: We validate local continuity by adding Gaussian perturbations to the input human joint states and tracking how the predicted object dynamics change. As shown below, the changes in predicted translation and rotation grow smoothly and proportionally with the perturbation scale ($\\epsilon$),  consistent with the behavior of a Lipschitz-continuous system.\n>\n>| Perturbation $\\epsilon$ | Combined (12D) Diff Norm | Trans (3D) Diff Norm | Rot Diff (9D) Norm |\n>| :---- | :---- | :---- | :---- |\n>| $1 \\times 10^{-5}$ | 2.6e-5 | 5.7e-6 | 2.5e-5 |\n>| $1 \\times 10^{-4}$ | 1.2e-4 | 2.4e-5 | 1.1e-4 |\n>| $1 \\times 10^{-3}$ | 1.1e-3 | 1.97e-4 | 1.07e-3 |\n>| $1 \\times 10^{-2}$ | 1.1e-2 | 1.99e-3 | 1.09e-2 |\n>| $1 \\times 10^{-1}$ | 1.0e-1 | 1.96e-2 | 1.00e-1 |\n>\n>To contextualize the perturbation magnitudes, the average norms of the input joint delta motions are approximately:\n>\n>| Component | Average Norm | Perturbation Level Relative to Average Norm (%) |\n>| :---- | :---- | :---- |\n>| Translation (3D) | $4.42 \\times 10^{-4}$ | 2.3%, 23%, 230%, 2300%, 23000% (for $\\epsilon = 1e{-5}$ to $1e{-1}$) |\n>| Rotation (9D) | $2.83 \\times 10^{-3}$ | 0.35%, 3.5%, 35%, 348%, 3480% |\n>| Combined (12D) | $2.87 \\times 10^{-3}$ | 0.35%, 3.5%, 35%, 348%, 3480% |\n>\n>This shows the perturbations range from very small to several times the natural input scale, providing a meaningful test of local smoothness.\n>\n>**Summary**: The local smoothness assumption is physically grounded, structurally preserved, and empirically validated in our setup. We are happy to clarify this in the revised version by expanding the discussion in L181 and including the continuity test results.\n>\n## **Question 3**: \"It would be better if performance improvement is analyzed per action type (e.g., \"put,\" \"kick\") in Table 1\\. In the video, the CHOIS baseline appears effective for hand-based interaction (carry, push, etc) but degrades significantly for foot-based interactions compared to the paper's method, and it would be clear for readers if this difference is demonstrated numerically.\" (Weakness 3)\n>\n>We sincerely appreciate the reviewer’s insightful suggestion regarding analyzing performance improvement by action types.\n>\n>**Action category assignment:** Since each textual description often contains multiple action verbs, we adopt a **multi-label classification** scheme. If a text contains any foot-related action (e.g., “kick”), it is assigned to the Foot-Interaction category; if it contains any hand-related action (e.g., “grab”, “pull”), it is assigned to the Hand-Manipulation category.\n>\n>**Quantitative results:** We conducted detailed analysis of key metrics on these two categories (Hand-Manipulation and Foot-Interaction), summarized in the table below:\n>\n>| Metric | CHOIS-Hand | Ours-Hand | CHOIS-Foot | Ours-Foot |\n>| :---- | :---- | :---- | :---- | :---- |\n>| Foot Sliding ↓ | 4.16 | **3.69 (↓11.4%)** | **3.48** | 3.50 (↑0.6%) |\n>| Contact F1 ↑ | 0.67 | **0.72 (↑7.6%)** | 0.33 | **0.37 (↑12.1%)** |\n>| Penetration ↓ | 0.57 | **0.56 (↓1.8%)** | 1.10 | **1.03 (↓6.4%)** |\n>| MPJPE ↓ | 16.02 | **15.61 (↓2.6%)** | 13.32 | **12.94 (↓2.8%)** |\n>| $T_{\\text{obj}}$ ↓ | 14.36 | **12.46 (↓13.2%)** | 13.32 | **9.80 (↓26.4%)** |\n>| $O_{\\text{obj}}$ ↓ | 0.98 | **0.90 (↓8.2%)** | 0.74 | **0.72 (↓2.7%)** |\n>\n>These results show that our method outperforms CHOIS baseline on both hand- and foot-based interactions, with particularly notable improvements in foot-interaction cases where the baseline struggles more. We will incorporate this fine-grained analysis into the revised manuscript to enhance clarity and help readers better appreciate our method’s advantages."}, {"review_id": "7Xdv8SpANf", "reviewer": "Reviewer_XvKb", "summary": "This paper presents a novel framework that models 3D human-object interactions as a driver-responder system using a lightweight transformer-based dynamics model to predict object reactions to human motions, combined with a residual-based dynamics loss for consistency.", "questions": "1. Unclear writing, especially regarding the workflow in Figure 1(b), which is inadequately described in the paper. Can you explain more about \"causality alignment\"?\n\n2. What is error feedback in L125?", "limitations": "Weakness #2 should be disscussed.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "#### Strength\n1. One notable advantage is the innovative concept of modeling humans as \"drivers\" and objects as \"responders,\" which proves superior to simultaneous generation of human and object motions. This driver-responder framework better captures the causal dependency in human-object interactions, enabling more logically consistent and physically plausible motion generation by explicitly defining how human actions drive object reactions.\n\n2. Quantitative comparisons demonstrate that the proposed method outperforms previous baselines, with significant improvements observed in key metrics with statistical significance, which validates the effectiveness and superiority of the approach.\n\n#### Weakness\n\n1. The absence of an ablation study on the impact of using the interaction dynamics loss during training. \n\n2. Given that physics-based approaches are widely adopted for capturing physical plausibility in human-object interactions, the paper should give comparison or discussion on using physics simulators to model object reactions to human actions.\n\n3. In the demonstration video, noticeable foot sliding artifacts are observed. Additionally, there are instances where the object exhibits premature movement before human contact—for example, the cases starting at 3s and 11s.", "was_revised": true, "final_justification": "After reading authors' rebuttal, I think most of my concerns have been solved.  The key concern of me is why we need a learned object reactor. Based on the advantages listed by the authors, I am inclined to accept this paper.", "rebuttal": "# Response to Reviewer XvKb\n\n## **Weakness 1**: \"The absence of an ablation study on the impact of using the interaction dynamics loss during training.\"\n>\n>Thank you for the valuable suggestion. We had conducted ablation studies to evaluate the effectiveness of our proposed interaction dynamics loss during training. As shown in Table 4 of the submission, our method outperforms the one that does not incorporate an interaction dynamics loss. \n>\n>To further isolate the impact of the **residual formulation** in our dynamics loss (Eq. (10)), we compared it against a **non-residual variant**, which directly penalizes the difference between the predicted object motion $\\Delta \\hat{o}_t$ and the dynamics model output $\\Delta \\hat{o}_t^*$ without compensating for model bias:\n>\n> $$\\mathcal{L}_{\\text{dyn-non-residual}} = \\mathbb{E}_t [ | \\Phi (\\Delta \\hat{o}_t^*, \\Delta \\hat{o}_t ) | ]$$\n>\n>We report the results without applying classifier-based guidance to evaluate the pure generation capability, as follows:\n>\n>| Method | $T_{xy} \\downarrow$ | Foot Sliding $\\downarrow$ | $C_{F1} \\uparrow$ | $P_{\\text{hand}} \\downarrow$ | MPJPE $\\downarrow$ | $T_{\\text{root}} \\downarrow$ | $T_{\\text{obj}} \\downarrow$ | $O_{\\text{obj}} \\downarrow$ |\n>| :---- | :---- | :---- | :---- | :---- | :---- | :---- | :---- | :---- |\n>| Without Interaction Dynamics | 3.05 | 3.63 | 0.54 | 0.61 | 16.05 | 25.34 | 12.36 | 0.99 |\n>| HOI-Dyn (Non-Residual Dynamics) | 3.03 | 0.39 | 0.59 | 0.71 | 15.63 | 25.10 | 11.86 | 0.89 |\n>| **HOI-Dyn (Residual Dynamics)** | 3.03 | 0.37 | 0.60 | 0.61 | 15.56 | 24.61 | 11.67 | 0.91 |\n>\n>In summary, incorporating the interaction dynamics loss clearly improves performance over not using it. Among the various formulations, the residual version is designed to cancel noise and imperfections in the pretrained dynamics model, making it more effective (see Appendix C). The non-residual loss directly penalizes discrepancies, causing errors in the dynamics model to propagate into diffusion generation and potentially degrade performance.\n>\n>Our experiments confirm that **the residual formulation yields more stable and better results**, especially in physical realism and contact accuracy. We hope this clarifies the benefits and necessity of our interaction dynamics loss design.\n>\n## **Weakness 2**: \"Given that physics-based approaches are widely adopted for capturing physical plausibility in human-object interactions, the paper should give comparison or discussion on using physics simulators to model object reactions to human actions.\"\n>\n>Thank you for the insightful suggestion. Different from the purpose of this paper, most existing physics-based methods—such as Intermimic [1], PhysHOI [2], and SkillMimic [3]—typically **rely on black-box physics simulators and reinforcement learning to train one policy per skill**, focusing on human control rather than object dynamics. As a result, they **do not explicitly model the object’s reactive behavior in response to human motion**, and are rarely applicable to HOI generation within a fully differentiable, end-to-end trainable pipeline.\n>\n>Although our approach follows a kinematics-based paradigm for HOI generation, we have made considerable efforts to incorporate physical principles.  \n>\n>Our method **introduces a learned dynamics module that explicitly predicts the object’s motion** conditioned on human pose, contact, and object state. This can be viewed as a conditional world model, allowing our generator to capture temporally coherent and physically consistent reaction patterns, while retaining **full differentiability for efficient gradient-based optimization**.  Moreover, our method follows the approach of CHOIS by explicitly **introducing contact states** to enhance human-object coupling in the diffusion process, and further extends it by **modeling the object’s motion response to human actions**. This enables improved physical reasoning and more accurate generation.\n>\n>We acknowledge the promising direction of integrating learning-based approaches with physics simulators in future work. In particular, incorporating physical properties such as mass, friction, and compliance—as explored in datasets like FORCE [5]—could further enhance realism and fidelity. We will clarify these distinctions and include a dedicated discussion comparing our approach with physics-based methods in the revised version.\n>\n>**References:**\n>\n>[1] InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactions. CVPR 2025.  \n>[2] PhysHOI: Physics-Based Imitation of Dynamic Human-Object Interaction. arXiv 2023.  \n>[3] SkillMimic: Learning Basketball Interaction Skills from Demonstrations. arXiv 2024.  \n>[4] CHOIS: Controllable human-object interaction synthesis. ECCV 2024\\.  \n>[5] FORCE: Dataset and Method for Intuitive Physics Guided Human-object Interaction. 3DV 2025.\n>\n## **Weakness 3**: \"In the demonstration video, noticeable foot sliding artifacts are observed. Additionally, there are instances where the object exhibits premature movement before human contact—for example, the cases starting at 3s and 11s.\"\n>\n>Thank you for your valuable observation. Compared to the baselines, our method has already substantially reduced object drift through the interaction dynamics loss, as reflected in improved contact quality and lower motion errors (Table 1). While minor drift remains in a few frames—likely due to the soft nature of our constraint—overall, our method achieves higher Contact F1 (0.71 vs. 0.66), lower translation (12.47 vs. 14.29) and rotation errors (0.90 vs. 0.99), and improved human motion quality (Foot Sliding 0.37 vs. 0.41, Human FID 0.48 vs. 0.87), suggesting more consistent interactions and more natural motions.\n>\n## **Question 1**: \"Unclear writing, especially regarding the workflow in Figure 1(b), which is inadequately described in the paper. Can you explain more about 'causality alignment'?\"\n>\n>Thank you for the question. **Causality alignment** refers to ensuring that the **object motion is a plausible and physically consistent response to the human motion and interaction context**.\n>\n>While diffusion models can generate realistic sequences, object motion may still be inconsistent with how objects should physically react. To address this, we introduce an **interaction dynamics model** $\\mathcal{D}$, which learns how objects typically respond to human motion and context.\n>\n>During training, we use $\\mathcal{D}$ to simulate object motion from both the generated and ground-truth HOI sequences. We then compare the simulated outcomes to measure whether the generated motion leads to realistic object responses. This forms our **Interaction Dynamics Loss (Eq. (10))**, which enforces **causal consistency** between human actions and object reactions.\n>\n>In short, Figure 1(b) shows this process: **using $\\mathcal{D}$ as a learned judge to align the causal effect of human-object interactions**. We will revise the figure caption and related text in the manuscript for clarity.\n>\n## **Question 2**: \"What is error feedback in L125?\"\n>\n>Thank you for pointing this out. We appreciate the opportunity to clarify the role of the error feedback $e(y_h^{(t)}, y_o^{(t)})$ in our framework.\n>\n>In our formulation, we decouple the human and object dynamics through a **driver-responder** structure: the human acts under internally guided dynamics (as is natural in human motion), while the object motion is externally driven and must respond to physical interaction cues. This asymmetry is reflected in the control signal $u^{(t)}$, which governs the object's behavior in response to the human.  \n>\n>The **error feedback** $e(y_h^{(t)}, y_o^{(t)})$ is a conceptual signal that reflects the mismatch between the human’s intent and the object’s actual behavior. In the context of training, this feedback takes the form of the **Interaction Dynamics Loss (Eq. (10))**, i.e., the difference between the generated object motion and the ground-truth motion via the dynamics model $D$.  \n>\n>Although $u^{(t)}$ is not explicitly parameterized as a separate control module, it is **implicitly optimized through the diffusion process**. During training, each denoising step updates the object’s predicted state to better match the ground truth. This is achieved by minimizing the diffusion model’s reconstruction loss, which, in effect, serves as a surrogate for the accumulated error feedback across time steps. Thus, we interpret the internal optimization of the diffusion model as a form of **control signal learning**—the model is learning to steer object motion in a way that aligns with the human motion and interaction context.  \n>\n>At inference time, although no ground-truth object motion is available, the model has learned to generate synchronized and physically plausible object behaviors by internalizing this driver-responder coordination during training. In this way, the **error feedback drives the learning of the interaction dynamics**, and the **diffusion model acts as an implicit controller**."}]

Author rebuttal:

Meta-review: This paper received overall positive reviews after the rebuttal (3 borderline accepts and 1 accept).

The main reasons to accept the paper are summarized as follows.
- Reviewers consistently identified the core "driver-responder" formulation as a primary strength. They described the idea of modeling HOI generation as a reactive system as "interesting" (J5sP), "innovative" (UuR7, XvKb), and offering a "novel perspective" (J5sP). This framework was seen as better capturing the causal dependency in interactions (XvKb).

- The technical approach of integrating the dynamics model as a training-time loss was recognized by Reviewer UuR7, as it introduces no additional inference overhead. The use of a residual formulation for the dynamics was also noted as "effective" for mitigating potential errors from the dynamics model, making the approach plausible and robust (UuR7).

- The paper was recognized for its satisfactory quantitative and qualitative results. Reviewers noted that the experiments were "adequate" and showed "clear improvements" over the CHOIS baseline across several metrics (biUj), and that the method "shows improved results compared to baselines, with extensive experiments" (UuR7).

At the same time, there are negative concerns, including:
- The paper was criticized for a lack of precision in the related work section. One reviewer (biUj) pointed out that the categorization of prior methods was inaccurate and potentially misleading, and that some concurrent works were not positioned correctly.

- Reviewers noted the presence of visual artifacts in the generated results. One reviewer (UuR7) observed that objects sometimes drift when not in contact with the human, and another (XvKb) pointed to noticeable foot sliding and instances of objects moving prematurely before contact.

- The paper was seen as lacking a sufficient ablation study on the impact of the interaction dynamics loss during training (XvKb). Additionally, a comparison or discussion regarding the use of physics simulators as an alternative for modeling object reactions was noted as a missing component (XvKb).

The AC overall recognizes the significance of this paper on learning interaction dynamics for human-object interaction generation using diffusion models and also that the existence of visual artifacts is inevitable due the difficulty of the task. The AC recommends accepting this paper. The authors are highly recommended to fix the first and third concerns listed above and also incorporate new results in the rebuttal into the revision.
Decision: Accept (poster)