Unleashing Guidance Without Classifiers for Human-Object Interaction Animation
Authors: Ziyin Wang
Sirui Xu
chuan guo
Bing Zhou
Jiangshan Gong
Jian Wang
Yu-Xiong Wang
Liangyan Gui
Venue: ICLR 2026 Poster (2026)
Keywords: human-object interaction
human motion generation
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Generating realistic human-object interaction (HOI) animations remains challenging because it requires jointly modeling dynamic human actions and diverse object geometries. Prior diffusion-based approaches often rely on handcrafted contact priors or human-imposed kinematic constraints to improve contact quality. We propose a data-driven alternative in which guidance emerges from the denoising pace itself, reducing dependence on manually designed priors. Building on diffusion forcing, we factor the representation into modality-specific components and assign individualized noise levels with asynchronous denoising schedules. In this paradigm, cleaner components guide noisier ones through cross-attention, yielding guidance without auxiliary classifiers. We find that this data-driven guidance is inherently contact-aware, and can be further enhanced when training is augmented with a broad spectrum of synthetic object geometries, encouraging invariance of contact semantics to geometric diversity. Extensive experiments show that pace-induced guidance more effectively mirrors the benefits of contact priors than conventional classifier-free guidance, while achieving higher contact fidelity, more realistic HOI generation, and stronger generalization to unseen objects and tasks.

Published as a conference paper at ICLR 2026
UNLEASHING GUIDANCE WITHOUT CLASSIFIERS FOR HUMAN-OBJECT INTERACTION ANIMATION
Ziyin Wang' Sirui Xu'
Chuan Guo
Bing Zhou?
Jiangshan Gong'
Jian Wang Yu-Xiong Wang! Liang-Yan Gui
University of Illinois Urbana- Champaign Snap Inc.
http:// /tt///ziy// nwangl. gitthb.......................... 1L/LTGHT
Lift whitechair, upside down and place on top of the table.
person puts backpack their right shoulder, and walks counterclockwise.
and put down the monitor.
arXiv:2603.25734v [cs.CV] 26 Mar 2026
person cooks with pan, switches hands, adds seasonings.
A person, grabs and inspects torus, then replaces
Lift the smalltable above your head, put
their starting point. Figure I: Given a textual prompt, our LIGHT generates realistic, vivid human-object interaction (HOI) motions via novel classifier-free guidanee scheme.
ABSTRACT
Generating realistic human-object interaction (HOI) animations remains challeng- ing because requires jointly modeling dynamic human actions and diverse object geometries. Prior diffusion-based approaches often rely hand-crafted contact priors or human-imposed kinematic constraints to improve contact quality. We propose LIGHT, data-driven alternative in which guidance emerges from the de- noising pace itself, reducing dependence on manually designed priors. Building on diffusion forcing, factor the representation into modality-specific components and assign individualized noise levels with asynchronous denoising schedules. In this paradigm, cleaner components guide noisier ones through cross-attention, yield- ing guidance without auxiliary classifiers. We find that this data-driver guidance is inherently contact-aware, and can enhanced when training is augmented with a broad spectrum of synthetic object geometries, encouraging invariance of contact semantics to geometric diversity. Extensive experiments show that pace-induced guidance more effectively mirrors the benefits of contact priors than conventional classifier-free guidance, while achieving higher contact fidelity, more realistic HOI generation, and stronger generalization to unseen objects and tasks.
INTRODUCTION
Human-object interaction (HOI) animation, which involves generating dynamic motion sequences of person interacting with objects, has become important problem in computer vision and graphics. Realistic HOI animation underpins applications ranging from virtual reality, gaming, and robotics simulation, where agents must manipulate in generative model have opened new possibilities for synthesizing such animations directly from high-level specifications, egg., text descriptions. In particular, diffusion models (Peng al., 2023;
objects in human-like way. Recent advances

---

Published as a conference paper at ICLR 2026
Diller & Dai, 2024; Li et al., 2023a) have emerged powerful generative framework capable of modeling nigh-dimensional distribution of human motions and contacts, by iteratively denoising noise into plausible HOI sequences.
Despite this promise, a persistent challenge is ensuring high-quality realistic interactions between humans and objects. A plain diffusion model, without physics simulation, often produces artifacts: hands miss their targets, objects drift or penetrate the body, or contact is unstable over time. Prior work has sought to mitigate this through guidance. One line of methods employs external classifiers trained for human-speciifed tasks, e.g., contact or affordance regression (Peng et al., 2023; Diller & Dai, 2024) to steer the diffusion process, but such classifier guidance is cumbersome design, and may overfit to specific priors. Another line introduces hand-crafted rules with kinematic and dynamic constraints, such as forcing hands to align with objects via inverse kinematics (Xu et al., 2024), or introducing physics simulation (Xu et al., 2025b; 2026), but these approaches sacrifice generality and are computationally inefficient. Together, these efforts highlight: existing HOI animation still relies heavily on external priors not directly from the data.
Extending classifier-free guidance (CFG) (Ho & Salimans, 2022) to HOI animation is a natural way to reduce the reliance on external priors, but CFG, especially CFG based on text dropout for text- conditioned generation, mainly improves global distributional alignment and offers limited control over the fine-gre lineggrined. persistent contact central to HOL. We therefore introduce LIGHT, i.e., Learning /mplicit Guidance for Human-object in in''eraction, a complementary data-driven framework where guidance arises from the relative denoising pace of separate components. As illustrated in Figure 1, LIGHT produces diverse and realistic human-object interaction animations with coherent contact dynamics. Concretely, we factor the representation into modality-specific components (e.g. human and object) and assign each its own noise level with asynchronous schedules. And we formulate two paths: (I) a staged schedule, where one modality (e.g., human) is kept cleaner while the other (e.g., object) follows its prescribed noise levels. approximating conditional trajectory; and (II) uniform schedule that applies weak conditioning by assigning all modalities with prescribed noise levels, where the "unconditional" case is realized as noisier conditioning, constituting soft form of CFG. The contrast between these two paths produces a guidance effect analogous to CFG. As the lag between schedules approaches zero, LIGHT collapses joint denoising with guidance; as the lag grows large, it approximates conditioning dropout in CFG. Our experiment shows that hard dropout on text improves global distributional alignment, whereas the soft guidance from LIGHT adjusts more on low-level contact details, indicating that LIGHT learns, purely from data, to reduce contact errors without hand-crafted priors. Note that assigning different noise levels diffusion models naturally corresponds to the diffusion forcing mechanism (Chen et al., 2024), and LIGHT provides a principled extension of diffusion forcing into a guidance framework.
LIGHT relies solely on data priors: however, this does not constrain the model's flexibility to incorpo- rate additional world knowledge like physics provided by humans, as such human-induced priors can also be reflected through data manipulation. We find that LIGHT achieves further improvement when training data are augmented with prior via contact-aware shape-spectrum augmentation. Specifically, objects are augmented with geometrically diverse alternatives from large repositories (Chang et al., 2015; Deitke et al., 2023), and motions are retargeted to preserve contact semantics and relative pose. These synthetic pairs teach the model that contact should remain invariant to irrelevant shape changes. This results in a stronger prior that the asynchronous schedules can exploit, improving generalizability to, e.g., unseen objects from training.
In summary, our contribution lies in three folds:
We introduce LIGHT, a novel guidance mechanism in which asynchronous denoising induces guidance without reliance on external classifiers. In contrast to CFG's reliance hard dropout, our formulation provides a softer and more flexible alternative, naturally extending diffusion forcing into the guidance framework and potentially inspiring future applications in other domains.
We propose contact-aware shape-spectrum augmentation, a strategy that preserves contact semantics while varying object geometry, thereby enabling the model to acquire more robust generative capabilities and improve generalization.
2

---

Published as a conference paper at ICLR 2026
We conduct extensive experimental evaluations demonstrating that the proposed approach consistently outperforms existing baselines and facilitates the generation of vivid and realistic interactions.
2 RELATED WORK
Denoising Diffusion Models for Human Animation. Denoising diffusion models (Sohl-Dickstein et al., 2015; Song et al., 2020; Ho et al., 2020) synthesize data by gradually denoising samples drawn from a noise distribution, effectively reversing a stochastic diffusion process. Recently, these models have achieved notable success in human motion generation, producing realistic and diverse anima- tions (Barquero et al., 2023; Tevet et al., 2023; Zhang et al., 2022a; Raab al., 2023; Zhang et al., 2023b; Shafir et al., 2023; Zhang et al., 2023e). A prominent example, MDM (Tevet et al., 2023), leverages transformer architectures to predict clean human motion trajectories from noisy inputs dur- ing the denoising stages. To generate conditional motion sequences, such diffusion methods typically incorporate external conditions into the diffusion process, including textual descriptions (Petrovich al., 2023; Guo et al., 2022b; Petrovich et al., 2022; Zhang et al., 2022a; 2023d; Tevet al., 2022a; Barquero et al., 2024). Further advancements have extended diffusion-based approaches to more conditioning scenarios, including guiding animations along specific trajectories (Karunratanakul et al., 2023; Rempe et al., 2023; Xie et al., 2023).
Denoising Diffusion Models for Interaction Animation. Recent advancements in interaction anima- tion have shown significant progress in diverse scenarios, including human-human interactions (Liang al., 2023; CMU; Mehta et al., 2018; 8; Xu et al., 2023a;c) and human interactions within static scenes (Hassan et al., 2021; Cao et al., 2020; Hassan et al., 2019). Early human buman-object interaction (HOI) animation methods typically leveraged kinematic models combined with conventional genera- tive frameworks. These earlier approaches primarily addressed simplified scenarios, either involving static or small objects (Xic et al., 2022; Zhang et al., 2020; Wang et al., 2022; Petrov et al., 2023; Taheri et al., 2022; Wu et al., 2022; Kulkarni et al., 2023; Zhang al., 2022b), or focusing exclusively on hand-object interactions (Li et al., 2023c; Ye et al., 2023; Zheng et al., 2023; Zhou et al., 2022a; Zhang et al., 2024a; 2023a). Consequently, they struggled to effectively capture complex, dynamic, whole-body interactions. Recently, denoising diffusion models have emerged as a powerful paradigm capable of modeling sophisticated interactions involving dynamically moving objects (Peng et al., 2023; Diller & Dai, 2024; Li et al., 2023a; Wu et al., 2024a;b; Song et al., 2024; Xu et al., 2024; Zhang et al., 2024b). Concurrently, new datasets have expanded the range of possible interactions from low-dynamic manipulations (Taheri et al., 2020) to highly dynamic scenarios engaging multiple body parts simultaneously (Bhatnagar et al., 2022; Jiang et al., 2023; Huang al., 2022; Zhang et al., 2023c; Fan et al., 2023; Li et al., 2023b; Zhao et al., 2024; Kim et al., 2024b; Jiang et al., 2024; Yang al., 2024). We use the most comprehensive InterAct dataset (Xu et al., 2025a) for evaluation.
Guidance with Denoising Diffusion Models. Diffusion models can be steered by additional guidance signals to better satisfy conditioning constraints. image and text generation, for instance, classifier guidance (Dhariwal & Nichol, 2021) uses an external classifier's gradients to direct the denoising process toward desired outcomes, whereas classifier- 2022) foregoes separate classifier by leveraging the model's own conditional and unconditional predictions for guidance. Inspired by such strategies, recent works in 3D human-object interaction (HOI) generation have explored explicit contact-aware realism. Without explicit constraints, diffussion-bsed HOI models often produce unrealistic artifacts, e.g.. floating objects missing contact points. To mitigate this, InterDiff (Xu et al., 2023b), for example, employs kinematics-inforned predictor that iteratively refines the diffusion output with corrections, improving human-object prediction accuracy. Similarly, HOI-Diff (Peng al., 2023) integrates an auxiliary affordance module to steer the model toward consistent contact and affordance cues throughout generation. These approaches demonstrably boost realism but introduce additional complexity by relying on extra networks or hand-crafted constraints, which can hinder generalization. CG-HOI (Diller & Dai, 2024) cast interaction synthesis as a multi-task learning problem: auxiliary contact prediction task is learned jointly to guide the motion generation, thereby avoiding training separate guidance models. However, it still imposes predefined relationships between predicted contact and object movements, using weighted combinations that embed human- designed assumptions. In contrast to these externally guided methods, we propose a guidance strategy without any assumptions on homan-object contact, where the proposed approach gradually applies
guidance (Ho & Salimans,
muxiliary-guidee approaches ensure
3

---

Published as a conference paper at ICLR 2026
Noise levels
a h
B
==
MLP
Lift the chair and put down.
DistilBERT
Transformer Decoder
Frame-wise
ROPE
Guidance
th frame
2 2 A
MLP
Staged Schedule
Uniform Schedule
MLP
MLP
Modal- wisc PE
Figure 2: Overview of LIGHT. Left: Training. We form different modalities, e.g., body, hand, and object, each diffused with own noise level. After adding modal-wir wisee and frame-wise wise rotary positional encodings, the tokens are processed by shared Transformer decoder and an MLP head to predict clean motion. Right: Inference. We compare a uniform schedule that denoises all modalities synchronously with staged schedule that keeps one modality cleaner from the uniform run.
guidance according to temporal schedule for different components during denoising, promoting new principled manner and and potentially inspiring future applications in other domains.
METHODOLOGY
Overview. We formalize the HOI animation task the synthesis of 3D motion sequence in which human interacts with an object over time horizon of T frames. The input is text description point cloud P. and a body shape parameter 3 from SMPL-H (Loper et al., 2015; Romero et al., 2017), and the output of LIGHT is a sequence comprising both human motion and object motion trajectories. The human state is represented by joint positions j" E R x52xx3 where 30 of total 52 joints are for both hands, and hand scalar rotation angles jrth 330 Object trajectories are translations of E IR3'x and rotations 6D representation (Zhou et al., 2019) o" E RTx6 Thus, each frame æ; is fully characterized by the tuple x, = . of, o.). We omit the time step in the following for simplicity.
d, canonical geometry of the object represented
Preliminaries. Diffusion forcing (Chen et al., 2024) is recent advancement in diffusion modeling that generalizes the conventional diffusion process by allowing independent noise schedules for each token within sequence. Unlike standard diffusion models, which uniformly apply single noise schedule across all tokens at each diffusion step, diffusion forcing treats each token independently, enabling flexible and staged denoising processes tailored to sequential generation tasks. Formally, consider a sequence of tokens (0), representing the unperturbed data. Diffusion forcing introduces token-specific noise levels with each element drawn independently from {0, 1,2...., K), where K is the total number of denoising steps. Each token of (0) is then corrupted individually, x x(A) (Vā()); (0)) + (v1 ã @A)), c), where e ~ (O,0 I) denotes Gaussian noise, and ā(A) defines the cumulative noise schedule for handling tokens corrupted by varying degrees of noise. use dot product (.) because the noise level for different tokens can be varied.
We slightly adjust the original diffusion forcing framework directly predict the clean data ((0) = Go [x A), X, d) from its noised counterpart æ (A), typical solution for the motion generation task (Tevet et al., 2023), rather than predicting the noise. The training objective is formulated as minimizing the reconstruction error,
LDF Ez(o) ([)(0) Gz(
(I)
4

---

Published as a conference paper at ICLR 2026
where (0) denotes the ground truth data. (o(x A, d) denotes the model prediction of clean data from the generative model G. parameterized by 8, conditioned on the partially noised sequence x r()), the noise level À for each token, and the textual description d. For notational simplicity, omit other inputs including human shape parameter 33 and object geometry P here; we describe how these additional conditioning variables incorporated into our generative model below.
Token Separation. Following Cha et al. (2024), we explicitly decompose the representation x into distinct modalities for the human body, hands, and objects, denoted æ², æ², and æ respectively, which formulates these three components as separate token groups, yielding total of 3 x tokens and noise levels X {X*, XA } E Tx As common practice for whole-body motion generation (Lu et al., 2023), this separation is beneficial for differentiating body and hand representations due to their distinct characteristics: the body encompasses 22 joints and typically exhibits larger-scale spatial movements, while the hands contain 30 joints with fine-gran Ene-grained and intricate finger motions. Empirically, we observe that separating the hand component improves motion generation quality in tasks with frequent hand interactions such
the GRAB dataset (Taheri et al., 2020) (Figure 4),
Overall Inference Procedure. As shown in Figure 2, at test time, we perform two coupled denoising passes that share the same denoiser Go. as outlined in Algorithm 1. First, uniform schedule denoises all modalities in syne under a same schedule, enhanced with text classifier-fre guidance. Second, a staged schedule applies modality-speciff schedules. The staged schedule incorporates the cleaner generation from the prior uniform schedule and introduces pace-induced guidance, combined with text CFG, incremental guidance. The final sample is obtained from the staged schedule. Design choices and hyperparameters are provided in Sec. 4.
Inference with Uniform Schedule. The reverse process of diffusion forcing gradually denoises noisy latent sequence zy (A) into a clean sequence æv (0), defined as:
Ty Go (A), ),, d) + wi (G (2) X, d)
(A), (A),
æy 1) 1A-
1), ău) +
where our generative model predicts the denoised sequence Yv given the partially noised input sequence Zy (A), with classifier-free guidance (CFG) with text d dropout. wi is a hyperparameter to control the text CFG. With ãy further noised back to zy(4 yields clearer HOI sequences until the inference advances to step 0. In this scenario, all frames and modalities are denoised simultaneously as plain diffusion models, with set to be the same and start at K. The resulting trajectory æu (-) are incorporated into the staged schedule to formulate LIGHT.
1). this iterative denoising step
Inference with Staged Schedule. Our staged inference strategy assigns asynchronized denoising schedules, enabling distinct denoising paces across modalities. Specifically, for specified modalities mi, m2 satisfying mi 7m2 = 2, m Um2 {b, h, o}. i.e., together m. and m2 cover all modalities without overlap (e.g. Mn {h}, m2 {b, o)). This partitioning is flexible. In Table B, we verify that all possible combinations yield improvements to varying degrees. Then, we set
a's (x:7 mh 8); xz" (Am²)); X' ((AT) 8); >m²)),
(4)
where the vectors T and n are all-k vectors at denoising step k, and (-; denotes the con- catenation operation. The offset vector ô determines how much earlier mi is denoised compared to my thereby creating pacing discrepancy across modalities. This discrepancy allows mi to reach a cleaner state from the previous generation x and when it is combined with the current output of xg to formulate z'g. it can lead to what we refer to as pace-induced guidance. Formally, the guided update is given by
ãs = Go(os(A), A, d) wwilGolas (A), X, Go( w X Go( X,
(5)
(6)
xs (A ( (1)-))((a(AA-1))))))))
( 1111//1/- -()-)),
where Wi,wly the scalar weights controlling the strength of the conditional influence. The resulting guided prediction for the m2 component is then re-noised according to the schedule and propagated to the next denoising step, while my continues to follow the trajectory of zy (k), k (0, 1, 2, .... K).
5

---

Published as a conference paper at ICLR 2026
Algorithm 1 Inference with pace-induced guidance
Require: Total number of denoising steps K; lagged modality index m2; cleaner modality index mi; æt K) and (Ks()) K) initialized with same Gaussian noise; lag offset vector d for k k= K, K ,..... do
1:
2:
Assign as an all-k vector Obtain denoised zy from æy (A) and À with text CFG
3:
Equation b Equation
4:
Add noise back to zy (A 1) and record
5:
end for
6:
for k == K, K 1, -l.,, do
7:
Assign À
all-k vector
8:
if A 8 2 O for all elements then
9:
Obtain the imput of conditional branch x'' (A) by merging zy (A Obtain denoised ø from æs(A) (A) and À with text CFG
Equation Equation Equation 6, incremental guidance
10:
11:
Update Vy with (A) by LIGHT
12:
else
13:
Obtain denoised äg from (A( and À with text CFG end if
14:
15: Add noise back to xs - 1)
D. Equation 7
16:
end for
17:
return (0)
Architecture. The left figure in Figure 2 illustrates the full architecture Ga- (i) Motion tokens. At each frame we form three motion tokens, body zf. hand æ and object oÿ, and attach a modal-wise positional encoding that distinguishes these modalities. A small linear projection is applied to each modality so that the three streams share a common hidden dimensionality before fusion. (ii) Noise-level and lemporal encodings. Each motion token receives an additive noise-level embedding together that encodes the diffusion step with a frame-wise timestep embedding. (iii) Object-geomele token. Object shape is encoded once per sequence. From the point cloud P. we concatenate un-normalised Basis Point Set (BPS) (Prokudin et al., 2019) descriptor with normalized BPS (Zhang al., 2024c) whose maximal point-to-centroid distance is normalized to 0.95; the original object scale (largest radius) is appended as extra scalar. This geometry vector is processed by an MLP and then concatenated to text tokens. (iv) Text token. The input prompt d encoded by frozen DistilBert (Sanh et al., 2019) text encoder, producing sequence of language tokens. These tokens are injected into each of the transformer decoder's s layers through their cross-attention blocks. (v) Transformer decoder and prediction head. The concatenated sequence, including body, hand and object tokens with all positional, temporal embeddings, is passed through a Transformer decoder, tokens injected by cross- coss-attention. A lightweight MLP head maps the
with text and denoised motion latents to final output, which are trained to match the clean ground-truth targets.
Training Schedule. Following Kim al. (2024a); Xiu et al. (2025); Zhang et al. (2024d); Chen et al. (2024), we assign independent noise levels across modalities, formally defined as A X X ~ U10, 1, 2...., K} independently. Training with independent noise levels encourages the model to capture a broad range of asynchronized conditional distributions, enabling diverse conditionalities at inference. Unlike diffusion forcing (Chen et al., 2024), we do not vary noise levels across frames; instead, all frames within the same modality share the same noise level.
Training Loss. The model trained with composite loss consisting of diffusion supervised term LDE and regularization term Treg. The total loss is defined as C. LDF + freg. The regularization term Lye promotes plausible human-object interactions and comprises three components: (i) bone-length loss that penalizes deviations in limb lengths from ground truth, (ii) a contact loss that aligns designated human joints with expected contact regions on the object, and (iii) a velocity loss that matches predicted object and human motion velocity with ground truth. Full details of these components are provided in Sec. A. of the Appendix.
Contact-Awart Shape-Spectrun Augmentation. We employ optimization basse augmentation strategy to enhance our dataset by transferring original HOI sequences onto novel objects from the same category. Specifically, we first train correspondence network following Xie al. (2024) that maps points from the source object's surface to corresponding points on new object surface selected from ShapeNet (Chang al., 2015) and Objaverse (Deitke et al., 2023) within the same object category. With the learned correspondence, we replace the original object with the novel

---

Published as a conference paper at ICLR 2026
tmmunff Joniel lad
TextsHot
InterDiff
HOI- DIf
CHOIS
ueaall
Ours
Figure 3: Qualitative comparison with baselines. Our method yields realistic human-objer interactions, fewer contact/penetration artifacts, more accurate finger positioning, and better text-motion alignment.
Table I: Quantitative comparisons on the InterAct dataset (Xu al., 2025a) between our method and baseline approaches. We report R- R-Precision with hatch sizes 256.
Melool
Divesity
Tip
Top
Tap
Cym
Cve
00000-
7.781 staza
1000-1907
100g
1K0 (Pong aia.. 3023)
0.00
CHUSS cLa at1las
B133
0.515t
013gee0
Teo25k0 (Cha
0.426 362
065373005
0 0.705nex
0.33152
7631 00115-40
0.005mmm:
0.000000
0.11
0dggence
0.53358.0
LIGaTT (Cher) win gridano
0 0,59gex08
6 6.7159989
7
0
0131
0 01250041a
0.730-1044
0.567800ce
0.50915
LIGHT (Dano) wi grrdeace
0:62)m9:14
6.75pmna14
0.18877
721396
0133
0.132
0:33)
84igenice
0,627
object, optimizing the placement so that the original human-object contact points are preserved the new object's corresponding points remain consistently matched to the same human contacts. The optimization objectives are detailed in Sec. A.2 of the Appendix.
4 EXPERIMENTS
Dataset. We conduct experiments mainly on the InterAct dataset (Xu et al., 2025a), and ablate on its major subsets, BEHAVE (Bhatnagar et al., 2022) and OMOMO (Li et al., 2023a). InterAct includes ime--graneed textual annotations accompanying these sequences, and adopt its official training-testing split for all evaluations. As the data originally annotate human motion using SMPL- H (Romero al., 2017) and SMPL-X (Pavlakos et al., 2019), we standardize all representations to SMPL-H and utilize SMPL-H joints consistently across our experiments, using official SMPL conversion. We manually filter out implausible motion sequences from the original MoCap data; for instance, we exclude cases from OMOMO in which hand orientations were incorrectly inverted, and cases from IMHD where the human distorted given their shape and pose. We apply object augmentation to enrich the dataset from 217 objects to 1121 objects, with examples in Figure A.
Metrics. Following the standard practice (Guo et al., 2022a), we measure realism and diversity of the generated HOI sequence, alignment with textual descriptions, and physical plausibility of interactions. To evaluate realism and diversity, we utilize the Fréchet Inception Distance (FID), which quantifies feature distribution similarity between generated sequences and ground- samples, and a Diversity metric, measuring variability across generated HOIs. To assess textual alignment, adopt R-Precision, and Multimodal Distance (MM Dist), quantifying the feature-level distance between generated HOIs and corresponding text embeddings. We introduce three metrics tailored explicitly for assessing the plausibility and quality of generated HOI sequences. The Foot Skating Ratio (FSR) measures the proportion of frames exhibiting unrealistic foot sliding. The Penetration Ratio (Pene) calculates the average fraction of object vertices intersecting the human mesh across the sequence. The Contact Ratio (Contact) assesses the frequency with which the human and object maintain consistent contact throughout the motion. More details can be found in Sec. B.2 of the Appendix. Following Li et al. (2023b), we also report contact precision (Cpree), recall (Crec), and F1 score (Cr1) ) to measure frame-wise contact accuracy, though these may not fully capture plausibility due to the diversity of text-to-HOI generation (see Sec. B.2 of the Appendix).
Existing methods (Peng et al., 2023; Diller & Dai, 2024; Wu et al., 2024a; Song et al., 2024) typically rely on feature extractors trained using relatively small-scale HOI datasets or their evaluator only measure human motion quality, which can negatively impact the robustness and reliability of evaluations. To address this limitation, we retrain feature extraction models using the larger-scale InterAct dataset, leveraging regressed joint representations in conjunction with object BPS (Prokudin al., 2019) features. Additional details can be found in Sec. B.I of the Appendix.
7

---

Published as a conference paper at ICLR 2026
Table 2: Ablation study of token- sepen-sepaattion strategies the InterAct dataset (Xu al., 2025a). We report
A-Prrcision batch size 256.
lazd-booly equasron
oporatio
Try
Try
Tap
Liveratt"
Peasl
Contac
Cyre
lakeatiin
2000
20114
2.712 ed
0.0ggh
0,53250041
50:00
0.627
DUEP
2,80
7.
8:798
0.578-9141
ILOR
0L80
0.572
The person who stands the olarm clock walks the hand and the and then back and stretch
The individusl stands sext a banonn, approsches it, graspe it with the right hand and passes forward before placing it down, thas returas and strenhhes the anus. Figure 4: Qualitative comparison between our method using body and hand merged into single token (left) versus separating body and hand into distinct tokens (right). Unrealistic grasping artifacts produced by the single-token approach are highlighted in red dashed boxes. Our separate-tol token strategy yields better results.
Implementation Details. In the main paper, we fix the modiality pairs by setting mi {b, h), m2 (o}. Additional experimensts on all of the component combinations are presented in the Appendix C. Unless otherwise specified, we set the guidance weights as w1 0.5, w/ = 3.0 with noise-level offset 5 250 while the total denoising steps K 500. During training, object geometry is represented using BPS (Prokudin al., 2019) comprising 1024 points. Our models are trained on NVIDIA A A100 GPUs, leveraging
training with FP16 precision and flash attention. Training converges within approximately 24 hours using a single GPU. For the transformer backbone, adopt 8-layer transformer decoder structure with latent dimension of 512 and feed-forward dimension of 1024. Additional implementation details are in Sec. A.
Baselines. We compare against four recent HOI generation baselines. HOI-Diff (Peng al., 2023), a diffusion-
framework for text-driven HOI, employs a transformer backbone (Tevet al., 2022b) with classifier guidance via an affordance predictor (Dhariwal & Nichol, 2021). CHOIS (Li et al., 2023a), originally requiring inputs beyond text, is adapted here for text-only prompts. InterDiff (Xu al., 2023b) modified by replacing its historical motion encoder with text encoder. Tex12HOI (Cha et al., 2024) is incorporated for full-body HOI generation, following their protocol train static contact-map estimator. To assess our guidance strategy. we further evaluate two variants: (i) our method without guidance and (ii) our full model with LIGHT.
Quantitative Evaluation. As shown Table 1, LIGHT surpasses prior diffusion- text-to-HO) methods on most key metrics: it attains higher R R-precision for tighter text-animation alignment and lower FID and MM Dist scores for better generation quality. Tables D and demonstrate that our method surpass other baselines, when trained on small-scale datasets, mirroring baseline's 's setting. Sec. C demonstrate that our framework is able to generalize on two new tasks without retraining, with guidance showing improvement over counterpart without guidance, even these unseen tasks.
Push the woodchair, release the hands, then drag the woodchair, and set it back down. Figure 5: Qualitative comparison. Left: our LIGHT without guidance. Right: our full method with guidance, which markedly enhances generation quality.
8

---

Published as a conference paper at ICLR 2026
Table 3: Ablation study. We compare models trained with and without data augmentation the InterAct dataset (Xu al., 2025a). Experiments on unseen objects include in-category and cross-category objects never observed during training. We report R- PrPrcission with batch size 256.
138
Dip
Top
Tap
Cree
Cy
Ie-oeegery
B0
0.279 e 012
0.006
on
6.133 1005
9.201
6.596 1.606
0 0. 91
0.912
Onseraltgiyy
0,02--
5.115100
Sangmm
0.06011
0.38g7
0.50p
Conw-ategury
Qualitative Evaluation. In the baseline comparison (Figure 3), our method attains higher contact accuracy, whereas the baseline exhibits noticeable penetration and floating artifacts, and their finger articulation is often incorrect. We also conduct a user study to complement the qualitative validation, which is discussed in Sec. D, demonstrating that ours produces animation with higher quality.
Impact of Pace-Induced Guidance. We evaluate the impact of our proposed guidance. Figure shows examples comparing HOI sequences generated with and without pace-induced guidance. With guidance enabled, generated animations clearly exhibit more precise contact dynamics and fewer unrealistic motion artifacts. This highlights that our approach effectively enhances the plausibility and overall visual quality of generated results. Table 1 corroborates these observations: combining guidance with our pipeline achieves the strongest performance.
Impact of Separating Tokens. In Figure 4, we show that separate hand token modeling yields noticeably more precise grasping motions and object placements. By comparison, the unified-token baseline often produces unnatural grasps and misaligned objects, underscoring the benefits of our separation strategy. Quantitative results in Table 2 reinforce this trend: token separation for three modalities with the our framework achieves superior performance across all metrics. We also conduct ablation on subset evaluation (Tables G and F), to mirror the setting of existing baselines (Li et al., 2023a; Xu et al., 2023b; Peng et al., 2023) trained on smaller dataset.
Impact of Augmented Data. Table A and Figure illustrate that our augmentation framework generates realistic human-object interaction animations well suited for training. We evaluate mod- els trained with augmented data under three distinct train-test splits: (i) Partitioned by sequence, consistent with the main experimental setup. Here, the test set contains novel objects all objects also appear in training. (ii) Partitioned by object category, where the test set consists of in-category objects unseen during training. (iii) Partitioned by object category, where the test set consists of cross-category objects entirely unseen during training. In all cases, the model is trained on combination of the original training data and its in-category augmented data, then evaluated on the corresponding test set using our pretrained evaluator without fine-tuning. Table 3 shows that augmentation consistently improves performance across metrics on the unseen-object evaluation. This demonstrates that object-level augmentation substantially enhances generalizability, benefiting both in-category and cootsoyy unseen objects.
Impact of Guidance Intensity and Denoising lagging. Here, we investigate how the key hy- perparameters, specifically we and 8, affect the effectiveness of our pace-induced guidance. As shown in Figure 6, varying the guidance weight w2 and the denoising offset å reveals a clear optimum. Weak guidance (small wy) ) degrades FID; W2 = 4 injects just enough prior for the best quality; larger we over-constrains, reducing diversity and motion fluidity. Likewise, small offsets (8 < 100) keep the staged and uniform branches too similar for effective correction, 5 = 200 strikes the best bal- ance, and larger offsets might cause excessive divergence and abrupt fusion. Analysis of Guidance Direction. Motivated by the need to understand why guidance works, we follow (Ra- jabi et al., 2025) and analyze the di- rections induced by LIGHT and tra- ditional CFG. Specifically, we mea- sure how each guidance direction cor- relates with two reference directions:
PED
0.1855
7.500
102
6.187
125
6:185
750
0.30
81287
2.5
Figure 6: Ablation study on the schedule-based guid- ance weight we and denoising preceding offiset 8. Left: 5 fixed at 200 while varying wz. Right: wy fixed at 4 while varying 5.
Table 4: Comparison the mean gradient similarity with direction, direction towards GT distribu-
tion. å = 250 is used and mean gradient similarity is calculated on all the guiding steps. we define (a. b)
Tba
Dosaeet
"Doo Dos
"Bumn Ber"
"Bom. Vhyeo?
IntesAct (Ku 3025a)
0.401
0401 +0.002
0217
0.277
OMOMO (1: 3073g)
0 195
0.395 + 0,009
0,299
0,219 81332

---

Published as a conference paper at ICLR 2026
(i) the displacement toward the ground-truth (GT) 9ut = =s()). where +(0) is defined in Sec. as ground truth conterpart, and (ii) the penetration-reduein gradient Lgeen. We provides com- parisons between two produced guidance directions, where (i) the guidance direction of our LIGHT is defined as JLIGIT = (o X' d) - Go(as(A) ()), A),A, X, d), as defined in Equation 6. (ii) the text-efg direction is gCFG = Go (zy (A), X, d) Go( (A). X, 0)) as defined in Equation 2. We report the mean cosine similarity between two guidance directions and two reference directions. Empirically, our pace-induced guidance consistently steers motion toward the desired distribution, achieving GT-aligned similarity on par with text-conditioned CFG. To probe effects on low-level contact, we also measure the cosine similarity between two directions with and the renetration-reducing gradient Leea As shown in Table 4, our LIGHT shows more correlation with this gradient compared to pure text CFG. We hypothesize that the hard-dropout form of CFG biases samples toward the marginal data distribution enhancing global plausibility but diminishing sensitivity to contact- contact-specific cues. In LIGHT, the "unconditional" path retains weak conditioning via noisier, lagged components (e.g., slight human-object surface misalignment). Then, the contrast between clean and noise branch will focus on surface-alignment signals and explicitly promotes depenetration and alignment.
CONCLUSION
We introduced LIGHT, a framework for human-object interaction animation that jointly models dynamic human motion and diverse object geometries without relying on hand-crafted priors or kinematic constraints. By decoupling modality-specins schedules, the model enables cleaner elements to guide noisier ones, yielding inherently contact-aware generation. Generalization is further enhanced through synthetic object augmentation, which serves as a data prior to promote invariance of contact semantics across geometric variations. Collectively, these contributions establish a new form of guidance for HOI animation, providing both scalable path toward more generalizable modeling and a natural extension of guidance within the diffusion forcing paradigm.
components with individualized noise
ACKNOW.EDGMENTS
This work was supported in part by Snap Inc., NSF under Grants 2106825 and 2519216, the DARPA Young Faculty Award, the ONR Grant N00014-26-1-2099 and the NIFA Award 2020-67021-327999 This work used computational resources, including the NCSA Delta and DeltaAI and the PTI Jetstream2 supercomputers through allocations CIS230012, CIS230013, CIS2403 and CIS240428 from the Advanced Cyberinfrastructure Coordination Ecosystem: Services & Support (ACCESS) program, as well as the TACC Frontera supercomputer, Amazon Web Services (AWS), and OpenAl API through the National Artificial Intelligence Research Resource (NAIRR) Pilot.
ETHICS STATEMENT
The realism and flexibility of annimations produced by our approach could facilitate misuse, such as generating convincing yet fictitious scenarios involving human interactions with objects, which might lead to misinformation. To proactively mitigate these risks, we explicitly adopt abstract human-bode representations, specifically, the SMPL model, which inherently lacks identifiable facial or biometric features. Thus, our approach substantially decreases the likelihood of synthesized content being exploited for identity misrepresentation other privacy-invasive applications, thereby ensuring responsible and ethical usage.
REPRODUCIBILITY STATEMENT
We provide a detailed description of the model architecture in Sec. 3. Details of adapted baselines introduced in Sec. 4. For augmentation, we include all the details in Sec. A.2. Comprehensive implementation details, including hyperparameters, evaluation metrics, and the training procedure of the evaluator, reported in Sec. 3 and further expanded in Sec. A, facilitate reproducibility.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
CMU graphics lab motion capture database. http://nocap... cmu wmw.edd/...........m............ 3
German Barquero, Sergio Escalera, and Cristina Palmero. BeLFusion: Latent diffusion for behavior- driven human motion prediction. In ICCV, 2023.
German Barquero, Sergio Escalera, and Cristina Palmero. Seamless human motion composition with blended positional encodings. CVPR, 2024.
Bharat Lal Bhatnagar, Xianghui Xie, Ilya Petrov, Cristian Sminchisescu, Christian Theobalt, and Gerard Pons-Moll. BEHAVE: Dataset and method for tracking human object interactions. In CVPR, 2022. 3, 7, 19, 20, 21
Zhe Cao, Hang Gao, Karttikeya Mangalam, Qi-Zhi Cai, Minh Vo, and Jitendra Malik. Long-term human motion prediction with scene context. In ECCV, 2020.
Junuk Cha, Jihyeon Kim, Jae Shin Yoon, and Seungryul Back. Tex12hoi: Text-guided 3d motion
generation for hand-object interaction. In CVPR, 2024. 5, 7, 21
Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, al. Shapenet: An information-rich 3d model repository. arXiv preprint 15Xiv:151..03012, 2015. 2, 6, 17, 18
Boyuan Chen, Diego Martí Monsó, Yilun Du, Max Simehowitz, Russ Tedrake, and Vincent Sitzmann. Diffusson forcing: Next-token prediction meets full-sequence diffusion. In NeurIPS, 2024. 4,
Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: universe of annotated 3d objects. In CVPR, 2023. 2, 6, 17, 18
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans image synthesis. In NeurIPS, 2021. 3, 8
Christian Diller and Angela Dai. CG-HOI: Contact-guided 3d human-object interaction generation. In CVPR, 2024. 2, 7,7
Zicong Fan, Omid Taheri, Dimitrios Tzionas, Muhammed Kocabas, Manuel Kaufmann, Michael J. Black, and Otmar Hilliges. ARCTIC: A dataset for dexterous bimanual hand-object manipulation. In CVPR, 2023. 3
Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang. Wei Ji, Xingyu Li, and Li Cheng. Generating diverse and natural 3d human motions from text. In CVPR, 2022a. 7, 16, 18
Chuan Guo, Xinxin Zuo, Sen Wang, and Li Cheng. Tm2t: Stochastic and tokenized modeling for the reciprocal generation of 3d human motions and texts. In ECCV, 2022b.
Mohamed Hassan, Vasileios Choutas, Dimitrios Tzionas, and Michael J Black. Resolving 3d human pose ambiguities with 3d scene constraints. In ICCV, 2019. 3
Mohamed Hassan, Duygu Ceylan, Ruben Villegas, Jun Saito, Jimei Yang, Yi Zhou, and Michael Black. Stochastic scene-aware motion prediction. In ICCV, 2021. 3
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12595 2022. 2, 3
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurIPS. 2020. 3
Yinghao Huang, Omid Taheri, Michael J. Black, and Dimitrios Tzionas. InterCap: Joint markerless 3D tracking of humans and objects interaction. In GCPR, 2022. 3
Nan Jiang, Tengyu Liu, Zhexuan Cao, Jieming Cui, Yixin Chen, He Wang. Yixin Zhu, and Siyuan Huang. CHAIRS: Towards full-body articulated human-object interaction. In ICCV, 2023.

---

Published as a conference paper at ICLR 2026
Nan Jiang, Zhiyuan Zhang, Hongjie Li, Xiaoxuan Ma, Zan Wang, Yixin Chen, Tengyu Liu, Yixin Zhu, and Siyuan Huang. Scaling up dynamic human-scens interaction modeling. In CVPR, 2024.
Korrawe Karunratanakul, Konpat Preechakul, Supasorn Suwajanakorn, and Siyu Tang. GMD: Controllable human motion synthesis via guided diffusion models. In ICCV, 2023, 3, 19
Gwanghyun Kim, Alonso Martinez, Yu-Chuan Su, Brendan Jou, José Lezama, Agrim Gupta, Lijun Yu, Lu Jiang, Aren Jansen, Jacob Walker, et al. A versatile diffusion transformer with mixture of noise levels for audiovisual generation. Advances in Neural Information Processing Systems, 37: 11837-11865, 2024a.
Jeonghwan Kim, Jisoo Kim, Jeonghyeon Na, and Hanbyul Joo. ParaHome: Parameterizing everyday home activities towards 3d generative modeling of human-object interactions. arXiv preprint arXiv:2401. 10232, 2024b.
Nilesh Kulkami, Davis Rempe, Kyle Genova, Abhijit Kundu, Justin Johnson, David Fouhey, and Leonidas Guibas. NIFTY: Neural object interaction fields for guided human motion synthesis. arXiv preprint arXiv:2307.0751, 2023. 3
Jiaman Li, Alexander Clegg, Roozbeh Mottaghi, Jiajun Wu, Xavier Puig, and C Karen Liu. Control- lable human-object interaction synthesis. arXiv preprint urXiv:2312.039/3, 2023a. 3, 7, 8, 9, 18, 20,25 21
Jiaman Li, Jiajun Wu, and C Karen Liu. Object motion guided human motion synthesis. ACM Transactions on Graphics (TOG), 42(6):1-11, 2023b, 3, 7, 17, 18
Quanzhou Li, Jingbo Wang, Chen Change Loy, and Bo Dai. Task-oriented human-object interactions generation with implicit neural representations. arXiv preprint arXiv:2303. 13/23, 2023c. 3
Han Liang, Wenqian Zhang, Wenxuan Li, Jingyi Yu, and Lan Xu. InterGen: Diffusion-based multi- human motion generation under complex interactions. arXiv preprint arXiv:2304.05664, 2023.
Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J Black. SMPL: A skinned multi-person person linear model. ACM transactions on graphics, 2015. 4
Shunlin Lu, Ling-Hao Chen, Ailing Zeng. Jing Lin, Ruimao Zhang, Lei Zhang, and Heung- Yeung Shum. Humantomato: Text-aligned whole-body motion generation. arxiv:2310.12977 2023. 5, 18
Dushyant Mchta, Oleksandr Sotnychenko, Franziska Mueller, Weipeng Xu, Srinath Sridhar, Gerard Pons-Moll, and Christian Theobalt. Single-shot multi-person 3D pose estimation from monocular RGB. In 3DV, 2018. 3
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv: 1807.03788 2018. 18
Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed A. A. Osman, Dimitrios Tzionas, and Michael J. Black. Expressive body capture: 3D hands, face, and body from a single image. In CVPR, 2019. 7
Xiaogang Peng, Yiming Xie, Zizhao Wu, Varun Jampani, Deqing Sun, and Huaizu Jiang. HOI-Diff: Text-driven synthesis of 3d human-object interactions using diffusion models. arXiv preprint arXiv:23/2.06553, 2023. 1, 2, 3, 7, 8, 2,
Ilya A Petrov, Riccardo Marin, Julian Chibane, and Gerard Pons-Moll. Object pop-up: Can we infer 3d objects and their poses from human interactions alone? In CVPR, 2023. 3
Mathis Petrovich, Michael J. Black, and Giil Varol. TEMOS: Generating diverse human motions
from textual descriptions. In ECCV, 2022.
Mathis Petrovich, Michael J Black, and Gül Varol. TMR: Text-to-motion retrieval using contrastive
3d human motion synthesis. In ICCV, 2023. 3, 18
12

---

Published as a conference paper at ICLR 2026
Sergey Prokudin, Christoph Lassner, and Javier Romero. Efficient learning point clouds with basis point sets. In ICCV, 2019. 6, 7,8
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. PointNet: Deep learning point sets for 3d classification and segmentation. CVPR, 2017.
Sigal Raab, Inbal Leibovitch, Guy Tevet, Moab Arar, Amit H Bermano, and Daniel Cohen-Or. Single motion diffusion. arXiv preprint arXiv:2302.05905, 2023.
Javad Rajabi, Soroush Mehraban, Seyedmorteza Sadat, and Babak Taati. Token perturbation guidance for diffusion models. arXiv preprint arXiv:2506. 10036, 2025.
Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint ar0iv:1988. 10084, 2019. 18
Davis Rempe, Zhengyi Luo, Xue Bin Peng, Ye Yuan, Kris Kitani, Karsten Kreis, Sanja Fidler, and Or Litany. Trace and pace: Controllable pedestrian animation via guided trajectory diffusion. In CVPR, 2023.
Javier Romero, Dimitrios Tzionas, and Michael J. Black. Embodied hands: Modeling and capturing hands and bodies together. ACM Transactions on Graphics, 36(6), 2017. 4,
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv: 2019. 6
Yonatan Shafir, Guy Tevet, Roy Kapon, and Amit H. Bermano. Human motion diffusion as generative prior. arXiv preprint arXiv:2303.01418, 2023.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In ICML, 2015.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502. 2020. 3
Wenfeng Song, Xinyu Zhang, Shuai Li, Yang Gao, Aimin Hao, Xia Hou, Chenglizhao Chen, Ning Li, and Hong Qin. Hoianimator: Generating text-prompt human-object animations using novel perceptive diffusion models. In CVPR, 2024. 3,7
Omid Taheri, Nima Ghorbani, Michael J Black, and Dimitrios Tzionas. GRAB: A dataset of whole-body human grasping of objects. In ECCV, 2020. 3, 5, 20
Omid Taheri, Vasilcios Choutas, Michael Black. and Dimitrios Tzionas. GOAL: Generating 4d whole-body motion for hand-object grasping. CVPR, 2022.
Guy Tevet, Brian Gordon, Amir Hertz, Amit H Bermano, and Daniel Cohen-Or. Motionclip: Exposing human motion generation to clip space. In ECCV, 2022a.
Guy Tevet, Sigal Raab, Brian Gordon, Yonatan Shafir, Daniel Cohen Oohe and Amit H Bermano. Human motion diffusion model. arXiv preprint arXiv:2209. 14916, 2022b.
Guy Tevet, Sigal Raab, Brian Gordon, Yoni Shafir, Daniel Cohen-or, and Amit Haim Bermano. Human motion diffusion model. In ICLR, 2023. 3,
Xi Wang, Gen Li, Yen-Ling Kuo, Muhammed Kocabas, Emre Aksan, and Otmar Hilliges. Recon- structing action-conditioned human-object interactions using commonsense knowledge priors. In 3DV, 2022. 3
Qianyang Wu, Ye Shi, Xiaoshui Huang, Jingyi Yu, Lan Xu, and Jingya Wang. THOR: Text human-object interaction diffusion via relation intervention. arXiv preprint arXiv:2403. /208, 2024a. 3.7
Yan Wu, Jiahao Wang, Yan Zhang, Siwei Zhang, Otmar Hilliges, Fisher Yu, and Siyu Tang. SAGA: Stochastic whole-body grasping with contact. In ECCV, 2022. 3
13

---

Published as a conference paper at ICLR 2026
Zhen Wu, Jiaman Li, and C Karen Liu. Human-objec interaction from human-level instructions. arXiv preprint arXiv:2406. 17840, 2024b. 3
Xianghui Xie, Bharat Lal Bhatnagar, and Gerard Pons-Moll Chore: Contact, human and object
reconstruction from a single rgb image. In ECCV, 2022. 3
Xianghui Xie, Jan Eric Lenssen, and Gerard Pons-Moll. InterTrack: Tracking human object interac- tion without object templates. arXiv preprint arXiv:2408. 13953, 2024. 6, 17
Yiming Xie, Varun Jampani, Lei Zhong, Deqing Sun, and Huaizu Jiang. OmniControl: Control any joint at any time for human motion generation. arXiv preprint arXiv:2310.08558 2023.
Jingqiao Xiu, Fangzhou Hong. Yicong Li, Mengze Li, Wentao Wang, Sirui Han, Liang Pan, and Ziwei Liu. Egotwin: Dreaming body and view in first person. arXiv preprint arXiv:2508. 13013, 2025. 6
Liang Xu, Xintao Lv, Yichao Yan, Xin Jin, Shuwen Wu, Congsheng Xu, Yifan Liu, Yizhou Zhou, Fengyun Rao. Xingdong Sheng, et al. Inter-x: Towards versatile human- -umann interaction analysis. arXiv preprint arXiv:23122 16051, 2023a. 3
Sirui Xu, Zhengyuan Li, Yu-Xiong Wang, and Liang- Yan Gui. InterDifft Generating 3d human-object interactions with physics-informed diffusion. In ICCV, 2023b. 3, 7, 8, 9, 21
Sirui Xu, Yu-Xiong Wang, and Liangyan Gui. Stochastic multi-person 3d motion forecasting. In ICLR, 2023c. 3
Sirui Xu, Ziyin Wang, Yu-Xiong Wang, and Liang- Yan Gui. Interdreamer: Zero-shot text to 3d
dynamic human-object interaction. arXiv preprint arXiv:2403. 2024. 2, 3
Sirui Xu, Dongting Li, Yucheng Zhang, Xiyan Xu, Qi Long, Ziyin Wang. Yunzhi Lu,, Shuchang Dong,
Hezi Jiang, Akshat Gupta, Yu-Xiong Wang, and Liang- Yan Gui. InterAct: Advancing large-scale versatile 3d hujen-object interaction generation. In CVPR, 2025a. 3, 7, 8, 9, 17, 20, 21
Sirui Xu, Hung Yu Ling, Yu-Xiong Wang, and Liang- Yan Gui. InterMinic: Towards universal whole-body control for maysscs-based human-object interactions. In CVPR, 2025b.
Sirui Xu, Samuel Schulter, Morteza Ziyadi, Xialin He, Xiaohan Fei, Yu-Xiong Wang, and Liang- Liang-Yan Gui. InterPrior: Scaling generative control for physics-based human-object interactions. In CVPR, 2026. 2
Jie Yang, Xuesong Niu, Nan Jiang, Ruimao Zhang, and Siyuan Huang. F-HOI: Toward fine-grained semantic-aligned 3d human-object interactions. In ECCV, 2024.
Yufei Ye, Xueting Li, Abhinav Gupta, Shalini De Mello, Stan Birchfield, Jiaming Song, Shubham Tulsiani, and Sifei Liu. Affordance diffusion: Synthesizing hand-object interactions. In CVPR, 2023. 3
Hui Zhang, Sammy Christen, Zicong Fan, Luocheng Zheng, Jemin Hwangbo, Jie Song, and Otmar Hilliges. ArtiGrasp: Physically plausible synthesis of bi-manual dexterous grasping and articulation. arXiv preprint urXiv:2309.0389) 2023a.
Jason Y Zhang, Sam Pepose, Hanbyul Joo, Deva Ramanan, Jitendra Malik, and Angjoo Kanazawa. Perceiving human-object spatial arrangements from a single image the wild. In ECCV, 2020.
Jiajun Zhang, Yuxiang Zhang, Liang An, Mengcheng Li, Hongwen Zhang, Zonghai Hu, and Yebin Liu. Manidext: Hand-object manipulation synthesis via continuous correspondence embeddings and saiduaalgguided diffusion. arXiv preprint arXiv:2409.09300, 2024a.
Jianrong Zhang, Yangsong Zhang, Xiaodong Cun, Shaoli Huang, Yong Zhang, Hongwei Zhao, Hongtao Lu, and Xi Shen. T2M-GPT: Generating human motion from textual descriptions with discrete representations. In CVPR, 2023b.
14

---

Published as a conference paper at ICLR 2026
Juze Zhang, Haimin Luo, Hongdi Yang, Xinru Xu, Qianyang Wu, Ye Shi, Jingyi Yu, Lan Xu, and Jingya Wang. NeuralDome: neural modeling pipeline on multi-vies view human-object interactions. In CVPR, 2023c. 3
Juze Zhang, Jingyan Zhang, Zining Song, Zhanhe Shi, Chengfeng Zhao, Ye Shi, Jingyi Yu, Lan Xu, and Jingya Wang. Hoi-m" 3: Capture multiple humans and objects interaction within contextual environment. In CVPR, 2024b.
Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, and Ziwei Liu. MifionDiffuse: Text-driven human motion generation with diffusion model. arXiv preprint arXiv:2208. 15001, 2022a.
Wanyue Zhang, Rishabh Dabral, Vladislav Golyanik, Vasileios Choutas, Eduardo Alvarado, Thabo Beeler, Mare Habermann, and Christian Theobalt. Bimart: A unified approach for the synthesis of 3d bimanual interaction with articulated objects. arXiv preprint 242::::::2:0::55 2, 2024c. 6
Xiaohan Zhang, Bharat Lal Bhatnagar, Sebastian Starke, Vladimir Guzov, and Gerard Pons-Moll. COUCH: Towards controllable human-chair interactions. In ECCV, 2022b.
Yaqi Zhang, Di Huang, Bin Liu, Shixiang Tang. Yan Lu, Chen, Lei Bai, Qi Chu, Nenghai Yu, and Wanli Ouyang. Motiongpt: Finetuned Ilms are general-purpose motion generators. arXiv preprint arXiv:2306. 10900, 2023d.
Zihan Zhang, Richard Liu, Kfir Aberman, and Rana Hanocka. TEDi: Temporally-entangles diffusion for long-term motion synthesis. arXiv preprint arXiv:2307 /5042, 2023e. 3
Zihan Zhang, Richard Liu, Rana Hanocka, and Kfir Aberman. Tedi: sypporally-entanglee diffusion for long-term motion synthesis. In ACM SIGGRAPH 2024 Conference Papers, pp. 1-11, 2024d. 6
Chengfeng Zhao, Juze Zhang, Jiashen Du, Ziwei Shan, Junye Wang. Jingyi Yu, Jingya Wang, and Lan Xu. I'M HOI: Inertia-aware monocular capture of 3d human-object interactions. CVPR, 2024. 3
Juntian Zheng, Qingyuan Zheng. Lixing Fang, Yun Liu, and Li Yi. CAMS: Canonicalized manipu- lation spaces for category-level functional hand-object manipulation synthesis. In CVPR, 2023.
Keyang Zhou, Bharat Lal Bhatnagar, Jan Eric Lenssen, and Gerard Pons-Moll. Toch: Spatio-temporal object-to-hand correspondence for motion refinement. In ECCV. Springer, 2022a. 3
Keyang Zhou, Bharat Lal Bhatnagar, Bernt Schiele, and Gerard Pons-Moll. Adjoint rigid transform network: Task-conditioned alignment of 3d shapes. In 2022 international conference on 3D vision (3DV), pp. 1- IEEE, 2022b. 17
Yi Zhou, Connelly Barnes, Jingwan Lu, Jimei Yang, and Hao Li. On the continuity of rotation representations in neural networks. In CVPR, 2019. 4
15

---

Published as a conference paper at ICLR 2026
APPENDIX
In the Appendix, we provide additional methodological details and experimental results: (i) an expanded explanation of our loss formulation and object-augmmntation procedure in Sec. A; (iii) definitions of the proposed metries in Sec. B.2; and (iv) extra experimental results, omitted from the main paper for space, in Sec. D.
LLM Usage, We employ large language models (LLMs), such as ChatGPT, to assist in polishing our paper. Specifically, L.MSs are used to correct grammatical errors, refine word choice, and improve overall fluency. We do not use LLM in formulating methodology running experiments
A ADDITIONAL DETAILS OF METHODOLOGY
A.1 TRAINING Loss
In training, regularize our LIGHT with three loss terms, foot-skating loss (Ln). velocity loss (L.), and contact loss (Leom). The total regularization objective is
Lyeg = A Lis + A Ly + Apont Lecnnt
(8)
where we set AR = 1, = 0.02, Acom = 0.1.
Foot-skating loss. To constrain foot sliding during ground contact, we penalize the deviation
between predicted and ground-truth foot velocities:
T
4
LB
ΣΣ ==
[P. - -P1,)) (36.5 3.-., -:
(9)
where of E (0,1) is the ground-truth foot-contact label for joint time and ii, (resp. }f,) denotes the predicted (resp. ground-truth) 3-D position of that joint. We follow (Guo et al., 2022a) for the foot-joint definitions and contact annotation.
Velocity loss. To encourage smooth temporal evolution of both human and object trajectories, we match first-order differences between predictions and ground truth:
(it - -P-1,)) (ii 3 3
L.
Apv
j=1
Auv
(o) - of-1) - -00 -
(=1
Σ (of o;-1) -66; o;- 11
Aarv
(10)
where jf. is the position of human joint j. while of and of are the object translation and rotation at time t. We set Apv = 1, Amv = 1, Agev 1.
Contact loss. To promote accurate human object interactions, we minimize joint-to-surface dis- tances whenever contact is expected:
I Ligunt
ΣΣ (d V. 2))²
(11)
==
where -) returns the minimum Euclidean distance between joint ir and the set of ground truth object vertices V,, and of is the ground -tuth binary contact label for joint j at time t, where we extract this information from ground truth data whether any joint mesh distance is less than 0.03
16

---

Published as a conference paper at ICLR 2026
Table A: Quantitative evaluation of augmented data quality relative ground truth annotations.
Darasct
Data source
Ponet
Flonting
CH
CH!.
CR
InterAct (Xu et al,, 2025a)
Ground Truth
0.008
0.005
0,947
0.911
Augmentation
0.010
0.011
0.980
0.980
0.980
GRAB (Taheri 2020)
Ground Truth
0.003
0.000
1,000
1.000
1,000
Augmentation
0.004
0.000
0.994
0.998
0.996
A.2 ADDITIONAL DETAILS ON OBJECT AUGMENTATION
We employ an optimization based strategy to enhance our dataset by transferring each original human- object interaction (HOI) sequence onto novel object instance of the same category. Specifically, given an original HOI sequence with human motion and an object trajectory, we first train dense correspondence network following Xie et al. (2024); Zhou et al. (2022b). Specifically, first, we extract the AABB (Axis-Aligned Bounding Box) of the object, and normalize to make the diagram of AABB to be 1, and the center of AABB to be at the origin. Then we train the corresponding network following Zhou et al. (2022b). The model architecture consists of a PointNet (Qi al., 2017) and multilayer perceptron (MLP), which maps an unordered point cloud to the orientation. We follow Zhou et al. (2022b) to supervise the model with a chamfer distance loss. This network learns to map points the source object's surface to corresponding points new object instance selected from a 3D shape repository the same category, e.g., ShapeNet (Chang et al., 2015) or Objaverse (Deitke et al., 2023). Using the learned correspondence, we replace the original object in the HOI sequence with the novel object, obtaining an initial transformed trajectory for the new object.
The goal is to position and move the new object such that the human's contact interactions remain consistent with the original sequence. In other words, for each contact constraint observed in the original sequence, i.e., a specific human joint maintaining contact frame with the original object, the corresponding surface point on the new object, identified via the correspondence network, should likewise remain in contact with that same human joint. We achieve this by optimizing the new object's pose parameters and the original human parameters over time to preserve the spatial alignment of these contact points while maintaining the physical plausibility of the interaction. To refine the new object's is trajectory, we formulate an objective function that is a weighted sum of multiple alignment terms:
L C= Aeoa Aveemmal Inpommal Acalli Luolli + Aimi Linit Auce
(12)
where Acon. Anormal Asalli Ainit and Ause weighting coefficients. Each loss term L. is designed to enforce a particular aspect of alignment between the human and the new object:
Lemn: penalizes any deviation in the distance between the human's 's contact points and the new object's ' corresponding surface points.
? encourages the surface normals of the new object at contact regions to align with those of the source object, ensuring that the orientation of the object relative to the human's 's contacting limb (e.g., a hand grasp) remains consistent with the original interaction.
Lealt: penalizes interpenetration or unintended collisions between the human body and the new object, enforcing that the object remains outside the human geometry except at the intended contact regions.
Linit: regularizes the solution towards the original motion.
Tace: promotes smooth motion by penalizing abrupt changes in velocity and acceleration.
By minimizing L, we refine the human and novel object's 's trajectory so that the human's 's interaction with the object remains natural and consistent with the original sequence. Thus, the new object is placed and moved in such a way that its correspondence-definee contact points remain matched to the human's 's grasp or touch points throughout the sequence.
B ADDITIONAL DETAILS OF EXPERIMENTAL SETUP
B.1 EVALUATOR REPRESENTATION AND EVALUATOR TRAINING
Our evaluator takes input the global human joint locations, object rotation, object translation, and object dynamic BPS. The dynamic BPS is computed using a static basis point set, following Li et al. (2023b); Xu et al. (2025a). Consequently, the evaluator assesses the quality of the entire HOI sequence
17

---

Published as a conference paper at ICLR 2026
Figure A: We visualize our augmentation results, where original MoCap sequenees (left) are enriched by introducing novel objects (right) sourced from ShapeNet (Chang al., 2015) and Objaverse (Deitke et al., 2023). Through optimization, human and object motions are aligned corresponding contact points.
rather than evaluating human object motion in isolation. Following Lu et al. (2023); Petrovich et al. (2023), we jointly train the text and HOI motion encoders of our evaluator. Departing from traditional training on classification tasks (Guo al., 2022a), utilize sequence-level contrastive learning objective based on the InfoNCE loss (Oord et al., 2018), following recent implementations (Lu et al., 2023; Petrovich et al., 2023). Our text encoding module incorporates Sentence-BERT (Reimers Gurevych, 2019).
B.2 EVALUATION METRICS
We outline our metric definitions below. Standard text-to-motion metrics are not restated; we simply extend them with the object and contact representations introduced in the main paper. For their original formulations, see Guo et al. (2022a).
Contact Percentage. This metric measures how consistently the human's 's body joints stay in contact with the object, indicating the quality of physical interaction. Extending Li et al. (2023b) from hand contact to whole-body contact, we define
Contact
(dlPlg) Vi) <7),
(13)
TJ
1=1
where jf, E R the 3-D position of the j-th joint at time L, Vo is the set object vertices at time f, and d(ji" V.o) is the minimum distance from the joint to the object surface. We use the contact threshold ? 0.05 following Li et al. (2023a). The more alignment of values of Equation equation 13 with the ground truth indicates better quality.
Frame-Wiss Contact Matching. Following et al. (2023b), we report frame-wiss contact match- ing, where Cr1, Oprecs and Cree denote the F1 score, precision, and recall of detected contacts between hands and objects. We use a contact threshold of y 0.05, consistent with Li al. (2023a). However, this metric is not entirely suitable for the Hext-to-HOI task, since frame-level alignment is not strictly enforced. In generative tasks conditioned only on text, it is standard practice to evaluate outputs with distributional metrics (e.g., FID in text-to-motion text-to-in -xttte--mmage, rather than element- or Trame-wise measures such as MPJPE or MSE. Nevertheless, we include frame- wise contact scores they provide partial indication of contact quality.
18

---

Published as a conference paper at ICLR 2026
Table B: Quantitative evaluation of unconditional generation the BEHAVE dataset (Bhatnagar al., 2022). R-Precision we adopt batch size 64. We don't I retrain our model unconditional generation.
Interaction
711
17
FID
Diversity
Pene
Contact
Cprse
Cree
CFi
0,98920:231
6.678=0:165
0.165
0. 0.122200001
0.16220-001
0.640
0.61020.013
0.58520.00
o, h
0.95846.222
6.715=9:141
0.172
0. 0.122
0.164=0.03
0.643
0.61240.024
0.585=0000
b,
0,957#0.258
6.672=0.224
0.165=0.211
0.122=00.00
0.167=0.001
0.640=9.002
0.617#0.013
0.588=0.003
b.o
0,96340.240
6.694=0.180
0.173#40.221
0.122=00.06
0.169=0.002
0.646-0.013
0.622#6.020
0.593=0.002
h 0.976
6.691 =0.201
0.
0.122#0.co8
0.070
0.640
0.624#0.028
0.593=0.005
b.
0.932
6.757
0,175
0.124
0.176
0.630
0.622#0.006
0.586
b
0,968
6. 6.708
184
0.172
0.121
0.166 0000
0.641 200.007
0.616
0.586-00005
(a) Baseline vs Ours
(b) Guidance Effect
80%
37.33%
% of Participants
Preference
26.67%
17.33%
10.67%
20%
8.00%
CHOIS HOI-DIFF Text2HO1 InterDiff
Ours
W/o Guidance
w/ Guidance
Figure User study results. 15 participants evaluates our method comparing baselines, and our guidance mechanism. Percentages indicate participant preference.
Penetration.
Penetration quantifies geometric violations between the human mesh and the object. After reconstructing the full human mesh via forward kinematics. we compute
T Σ Penetration = T
min( sds (vo.). 0)/-
(14)
where sdf"' (vo,) is the signed distance from object vertex V., to the human mesh at time t. Negative SDF values indicate penctration depth; clamping positive values to zero ensures that only inter- penetrations are accumulated. Lower scores in Eq. equation correspond to fewer or shallower penetrations.
Foot-Skating Ratio (FSR). FSR gauges how well the character's feet remain stationary during ground-contact phases, a key indicator of motion realism. Following Karunratanakul et al. (2023), we compute
E of (I Jp s.,,5) fps <t).
FSR
(15)
T
where of E (0, is the ground-contand indicator for a foot joint at time t, J is that foot's position, and fps converts frame-to-frame displacement to velocity. We adopt = 0.025 and ground-contact height threshold of 0.05 as Karunratanakul et (2023). FSR ranges from 1; higher values signify less foot sliding and more realistic foot-grod foot-ground interaction.
C ADDITIONAL ANALYSIS OF GUIDANCE
Quantitative Evaluation of Our Guidance Across Tasks. In addition to the Text-to-HOI task presented in the main paper, we evaluate our guidance under different task settings. Specifically, we consider (i) unconditional HOI generation, where the model takes input an object mesh and
19

---

Published as a conference paper at ICLR 2026
Table C: Ablation of modality combinations mi, My InterAct (Xu et al., 2025a), BEHAVE (Bhatnagar al., 2022), and OMOMO (Li et al., 2023a) datasets. We report R-Precision with batch size 256.
Daast
Pil-
Kid bic!
Diserity
PSR
Creter
Vaeustionn
Tep
Cz
naggtnass
0.2541
6
0:14
02
0.591
BERAVE
6.6
0
0
6d61
A277
8.148
625
0
0,97-400
67700065
A
0 011310
02338000
053
0:300
0.43
0.366-14
0100000
020
maraagar and, 2077) 60
bo
0:222
042215
0.12914.2
0.140
0721
0.59-3403
0415
620
0:334
6412
0.5ay
0:209
0413
0,00
0.39944.0
8.13
072
0,555
17
0
07
0.501300
0.A
0423-1
004
0.5
8445 ox
0.12
654
0
6751
511110
0:
0445-
0335
010
039g-300
0.551
A.A
0.30213-0
055674
6
0125
62
0.33 0,59y
S.A.
0.300
670
0121
04
058p
0.383-1-0
053g14
7.672
0.07
013311
0.351
5/66
0123
0.59
7402
0:300
0
6200
3764
5.A
0.302-4-1
8540
0.013:4:14
0.555-300
0M0000
0.382
650
0.013
0.10
0
0
0533
018810
510010
0,333
0.5234400
0,20
8:14
Ere
1315
7.49
6.882
0
b.o
B66
5151
0,575
6.6
0330
050
0.12
3100
B100
0.214
0.344
0.182
0.530
human shape to synthesize complete HOI sequence without extra conditioning, and (ii) controllable HOI generation, where the model is conditioned on a richer set of prpuuts-nclluinn the full object motion sequence, object mesh, human shape, and textual description to generate the corresponding HOI sequence. For both settings, we reuse the model from the main paper without retraining. This is made possible by our independent noise scheduling, which allows the model to noised out unused conditioning signals or overlap one denoising modality with input condition. Table B demonstrates that our guidance remains effective across multiple tasks.
Analysis of Different Component Combinations. In this section, we experiment with multiple component combinations. Table C demonstrates that our guidance effective on flexible component combinations.
D OTHER ADDITIONAL EXPERIMENTAL RESULTS
User Study. We conducted a user study with 15 participants. Among them, 86.7% reported that our augmented data exhibits fewer artifacts than the ground truth (GT), while 100% agreed that the interaction consistency between GT and augmented sequences is preserved. Additional statistics are provided in Figure B. which further demonstrate that (i) our method qualitatively outperforms the baseline, and (ii) the proposed guidance mechanism leads to improved performance.
Quantitative Evaluation of the Quality of Augmented Data. Table A demonstrates that our augmented data can match the quality of the ground-truth set, showing minimal contact errors and demonstrating its suitability as synthetic training data. And our augmentation of small objects on GRAB (Taheri et al., 2020) could also greatly align the contact of the ground truth dataset
Quantitative Evaluation on additional datasets. The Table D, E and ablations in Table F, G, demonstrates that our guidance method is effective across multiple datasets, rather than only large-scale dataset. And also, token separation and independent noise schedule are also effective.
Disscussion of Runtime efficiency. Our guidance requires three model forward passes per step, and a complete inference procedure is needed before applying guidance. Specifically, for a batch of 64 samples, each sample consisting of 300 frames, our guided inference completes in approximately 72 seconds on one A100 gpu. In contrast, HOI-Diff, InterDiff both takes about 15 seconds without guidance.
20

---

Published as a conference paper at ICLR 2026
Table D: Quantitative comparisons on the OMOMO dataset (Li et al., 2023a) between method and baseline approaches. We report R- PPeeccision with batch size 256.
Mettod
Peeciiics
FTD1
MM Dis!
Divertt
FIR
Pone
Cretact
Tay
Top?
Tay
Cyre
Ciee
Cr
Gorand Truck
0.5002001
0791
0000 W0
1.80050000
7.00g5
001254 X0
50070
1.00c: 100
1000500000
ROI DMT Pang olll (2005)
022
0403
0.6777811
2.50g
7.3
0003
D13:
0 0188
0.75
6.00
BoodDu No (2023))
0.312-4.00 a.0
4718
1.587
7.587
0. md
0.000-00m
0 0:300-000
0.900190
0.50015:0
Teaztaot Cla (2024)
0.236 as
CASS 1988
1,8224000
7.399.5.00
B122
0.00 0.005-mam
LIGari (Durs) wil acidance
0.279-990
1.82240000
7,39
0.04 m00
0.09--0a
0344
037511.90
LIGHT (Dan)) puddnace
0.30Q
0,339966
8:000/00m
Table E: Quantitative comparisons on the BEHAVE dataset Bhatnagar et al. (2022) between our method and baseline approaches. We report R- R-Precision with batch size 256.
Merbod
R-Pradacon
NM Duul
Ponzi
Contact
Top
Top
Top
Cye
Gooad Thin
6 101g
D 0.
ompuntac ac
POL DMT Porg at (2033)
A828
0.73
0.50
ws
beadMf Xa (28036)
0.25
044405
0,59190007
046307
0,25g99
0.4437
0.755
0.67p74
Tou3B01 Cha (2024)
0.34248
0
1.29
6.5upme 5.81500o0
033339020
0.17
0.1289001
0.203
0,50m
LIGHT (Ders) gaklone
0.55p
0.50m
6,0291
0.165
0.2299
016761
(Oun) glelasse
033780001
012
3,37790006
Table F: Ablation study of token-separation strategies on the BEHAVE dataset Xu et al. (2025a). We report R-Precision with batch size 256.
hand-booly
equriton
heraanobjoor oppration
Top
Proddoer
Dip 2
Top
MM Dare
Diverity
Peapl
Caatatt
Insoutionn
Cys
A.91 1f
D1 00
B:31 Moo
0.731
2,00 0.5
1738
6:22 0L137
20
0.0025r
0.000501
Table G: Ablation study of token separation strategies the OMOMO dataset (Li ct al., 2023a). We report Preccision with batch size 256.
burd--oaly equertionn
oojoc apperation
Procddee
MM Daek
Divenity
Peapl
Caatat
lnmention
Top
Top 3
Top
50 Du
0553 R1
5,99 Shit
7.44 a13
0,19g
0,80 mo
0,81
32
0.55j
7,366
0.00p54
0,223
busap
0,225
0.106
Limitations. Despite the encouraging results, our approach currently faces several limitations. First, our framework is designed specifically for single-object scenarios, and extending it to multi-object interactions remains an open challenge. Furthermore, the current model does not explieitly incorporate static environmental contexts or detailed scene geometry, potentially limiting the contextual realism of generated motions. Additionally, while our method improves interaction coherence and physical plausibility, it may still struggle with highly dynamic or physically complex interactions requiring precise and rapid manipulations. Finally, due the computational complexity introduced by staged conditioning and diffusion forcing, real-time or interactive applications might be challenging.
21


Reviews:
[{"review_id": "GnWupjNbU0", "reviewer": "Reviewer_2cfF", "summary": "The paper presents LIGHT, a novel framework for generating 3D human-object interaction (HOI) motions from textual descriptions. At the core of the network, the authors propose a pace-induced guidance that avoid the usage of handcrafted contact priors for good contact quality and realism. Instead, LIGHT is formulated with two paths: a uniform pass denopises all modalities (body, hands, object), while a staged pass that uses the outputs from the first pass to guide the noise components. Additionally, a contact-aware shape-spectrum augmentation strategy is proposed to improve the generalization on unseen objects. Extensive experiments have been conducted to present its superior performance over existing techniques.", "questions": "Besides the weaknesses listed above, the reviewer may have some additional questions:\n\n+ Could the authors illustrate more about the failure mode, such as types of interactions and objects? \n\n+ How's the physical quality, such as penetration issues, of the generated HOI sequences? It might also be better to evaluate for the penetration issues?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "+ The proposed pace-induced guidance and contact-aware shape-spectrum augmentation are novel. Specifically, (1) The pace-induced guidance is proven to provide a data-driven altenative, which is even more effective than the priors used in previous methods. (2) The augmentation builds a informative invariance directly into the training data, leading to improved generatlization.\n\n+ The method demonstrates clear quantitative and qualitative improvements over strong baselines (HOI-Diff, CHOIS, InterDiff) across multiple datasets (InterAct, BEHAVE, OMOMO) and metrics.", "weaknesses": "- The method is currently designed and evaluated for interacting with only one single object. Interactions with multiple and complex objects would be more beneficial. \n\n- Comparisons with zero-shot HOI generation methods, such as InterDreamer and ZeroHSI, may also be useful.\n\n- Minor issues: The inference process requires around 72 seconds, which is higher than HOI-DIff and InterDiff (non-guided baselines).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eeaMxDvvXq", "reviewer": "Reviewer_sZdf", "summary": "The paper proposes a novel classifier-free guidance framework for diffusion-based HOI animation. Unlike prior methods relying on external contact classifiers or kinematic constraints, LIGHT achieves guidance through asynchronous denoising schedules. The cleaner (less noisy) modalities guide noisier ones, producing contact-aware behavior. Additionally, the paper introduces contact-aware shape-spectrum augmentation using ShapeNet and Objaverse objects to improve geometric generalization. Extensive experiments on the InterAct, BEHAVE, and OMOMO datasets demonstrate that LIGHT outperforms baselines such as HOI-Diff, InterDiff, CHOIS, and Text2HOI in FID, contact quality, and text-motion alignment, while maintaining realism and generalization across unseen object categories", "questions": "- Just curious about why does the prediction-based method Interdiff with modification (text-conditioning) achieve such competitive results?\n- Some large-scale subsets of the InterAct dataset, such as BEHAVE and OMOMO, lack explicit hand motion (showing the mean hand pose). In this case, is the proposed hand–object separation strategy still meaningful or effective? Additionally, does the data augmentation stage include any hand-related synthesis to compensate for the missing hand motion?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The idea of pace-induced guidance (asynchronous denoising between modalities) is innovative, extending diffusion forcing into a practical HOI setting\n- The quantitative results are comprehensive, with thorough comparisons against existing baselines and well-conducted ablation studies.", "weaknesses": "- From the final visual results, the proposed strategy indeed demonstrates the ability to effectively leverage human priors to generate plausible interaction poses. However, the method still struggles with fine-grained contact modeling, and noticeable artifacts remain at the contact level.\n - The evaluation metrics seem somewhat questionable — the R-Precision scores are all within a similar range, and several other metrics also show minor differences across methods. It is unclear whether these metrics are sensitive enough to effectively distinguish the quality of human–object interactions.\n- Since the video format is the most effective medium for evaluating the quality of human–object interactions, here the visual results are quite limited, with only two baseline comparisons, one augmented-training example, and one ablation study shown. This makes it hard to visually assess the claimed improvements or the model’s generalization capability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RFAlLGXexf", "reviewer": "Reviewer_woqV", "summary": "1. This paper presents LIGHT, an asynchronous-denoising–based guidance mechanism that achieves soft, flexible conditioning without relying on external classifiers.\n\n2. In addition, the paper introduces contact-aware shape-spectrum augmentation, which maintains contact semantics while altering object geometry, enhancing robustness and generalization.\n\n3. Extensive experiments further verify the approach, showing consistent improvements over existing baselines and enabling the generation of vivid, realistic interactions.", "questions": "1. Will the authors release the code and the augmented data used for the Contact-Aware Shape-Spectrum Augmentation?\n\n2. In Section 4, how do you compute x′_S from x^{m1}_U and x^{m2}_S? Is it a simple concatenation or another operation?\n\n3. In the supplementary video at timestamp 1:06, the motions for InterDiff and Text2HOI appear very similar to each other but different from the proposed method. Were different text prompts used for these baseline results?\nPlease check it.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. It proposes a pace-induced guidance mechanism to generate more realistic and plausible human-object interactions.\n\n2. Through extensive experiments, it analyzes the effects of pace-induced guidance, token separation, augmented data, guidance intensity and denoising lag, and guidance direction.\n\n3. The authors also provide fair comparisons by re-implementing and modifying prior baselines (e.g., InterDiff, Text2HOI).", "weaknesses": "1. There is no reference to Figure 5 in the section Impact of Guidance Intensity and Denoising Lagging. Please explicitly link the analysis to the figure.\n\n2. In the Impact of Guidance Intensity and Denoising Lagging experiment, the paper states that the best value of δ is 300, but Figure 5 seems to suggest that 200 performs best. Could the authors clarify which value is correct?\n\n3. I am not fully clear about the settings in Table 2. For the case where hand-body separation is ✓ and human-object separation is -, is the object token concatenated with the hand stream or with the body stream? In other words, which grouping is correct: {b, ho} or {bo, h}?\n\n4. Additionally, if there are no separator tokens, meaning all tokens are combined, then it seems that the staged schedule cannot be applied. Could the authors clarify how the model behaves in this case?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7swvTvRI0G", "reviewer": "Reviewer_wUBQ", "summary": "This paper proposes a new method for text-driven HOI generation. The main technical contributions are two fold. First, it porposes a new guidance mechanism where asynchronous denoising induces guidance without using an external classifier/objective function. Second, a contact-aware shape-spectrum data augmentation strategy is proposed that preserves contact semantics while varying object geometry. Experimental results on InterAct, BEHAVE, and OMOMO are reported.", "questions": "1. $\\textbf{x}_S$ is not defined in the paper. Could you please explain it?\n\n2. In Eq. (4), for $\\textbf{x}_S'$, it it a concatenation of $\\textbf{x}_U^{m_1}$ and $\\textbf{x}_S^{M_2}$? By the way, the symbol of $\\textbf{x}$ in $\\textbf{x}_S'$ is different from that in $\\textbf{x}_U^{m_1}$ and $\\textbf{x}_S^{M_2}$ in the paper.\n\n3. Is ithe pace-induced guidance used in the inference only?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The proposed data augmentation technique is promising. Generalization of the interaction semantics to similar objects is a fundamental challenge, which is not only useful for the HOI generation task studied in this paper, but may also be useful say other domains, say robotics.\n\n2. State-of-the-art results are reported in this paper.", "weaknesses": "1. While the autors have explained **how** the proposed asynchronous denoising of different parts works for HOI generation, they didn't explain **why** it works. In a high-level sense, it works in a way analogous to CFG, but it lacks detailed analysis and investigation. Especially considering the proposed approach is inspired by (or connected to) the duffions forcing mechanism, which is a generic framework (not tailored for HOI generation), showing **insights** why it is useful for HOI generation is critical as future work may better understand when it works and when it may not.\n\n2. The contact-aware shape-spectrum augmentation is a big contribution of this paper. But it lacks sufficient details of how it works (e.g., statistics of number of objects before and after augmentation) and no visual examples are shown in the main paper. Only limited illustrations are provided in the appendix and supplementary video. It makes readers hard to gauge the effectiveness of this part and reproduce it.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)