Disentangled Hierarchical VAE for 3D Human-Human Interaction Generation
Authors: Zichen Geng
Zeeshan Hayder
Bo Miao
Jian Liu
Wei Liu
Ajmal Saeed Mian
Venue: ICLR 2026 Poster (2026)
Keywords: Human Motion
Human-Human Interaction
3D CV
Motion Generation
Primary area: generative models

Abstract: Generating realistic 3D Human-Human Interaction (HHI) requires coherent modeling of the physical plausibility of the agents and their interaction semantics. Existing methods compress all motion information into a single latent representation, limiting their ability to capture fine-grained actions and inter-agent interactions. This often leads to semantic misalignment and physically implausible artifacts, such as penetration or missed contact. We propose Disentangled Hierarchical Variational Autoencoder (DHVAE) based latent diffusion for structured and controllable HHI generation. DHVAE explicitly disentangles the global interaction context and individual motion patterns into a decoupled latent structure by employing a CoTransformer module. To mitigate implausible and physically inconsistent contacts in HHI, we incorporate contrastive learning constraints with our DHVAE to promote a more discriminative and physically plausible latent interaction space. For high-fidelity interaction synthesis, DHVAE employs a DDIM-based diffusion denoising process in the hierarchical latent space, enhanced by a skip-connected AdaLN-Transformer denoiser. Extensive evaluations show that DHVAE achieves superior motion fidelity, text alignment, and physical plausibility with greater computational efficiency.

Published as conference paper ICLR 2026
DISENTANGLED HIERARCHICAL VAE FOR 3D HUMAN- HUMAN INTERACTION GENERATION
Zichen Geng' Zeeshan Hayder² Bo Miao
Jian Liu"
Wei Liu'
Ajmal Mian !
1 'Department of CSSE, The University of Western Australia, Crawley WA 6009, Australia
DDat661, CSIRO, Acton ACT 2601, Australia
3 Australlann Institute for Machine Learning, The University of Adelaide, Adelaide SA 5000, Australia RNERCRRVC, Human University, Changsha 410012, China
zer . genge research uwa edu. au, zeeshan. hwwder wtw.rrddaaall..rro....... bo.mi aoBadelaide edu. au, jianl iu@hnu, edu.cm .n (wei. ajmal @uwa, edu.au
ABSTRACT
Generating realistic 3D Human-Human Interaction (HHI) requires coherent modeling of the physical plausibility of the agents and their interaction semantics. Existing methods compress all motion information into single latent representation, limiting their ability to capture fine-graines actions and inter-agent interactions. This often leads to semantic mis- alignment and physically implausible artifacts, such as penetration or missed contact. We propose Disentangled Hierarchical Variational Autoencoder (DHVAE) based latent diffu- sion for structured and controllable HHI generation. DHVAE explicitly disentangles the global interaction context and individual motion patterns into decoupled latent structure by employing a CoTransformer module. To mitigate implausible and physically inconsis- tent contacts in HHI, incorporate contrastive learning constraints with our DHVAE to promote a more discriminative and physically plausible latent interaction space. For high- fidelity interaction synthesis, DHVAE employs cess in the hierarchical latent space, enhanced by a skip-connected AdaL.N-Trann N-Trrrsformer denoiser. Extensive evaluations show that DHVAE achieves superior motion fidelity, text alignment, and physical plausibility with greater computational efficiency. The official implementation is publicly available at: https//ginbbbemommnmnnneccccn/de
aDDIM-hased diffusion denoising pro-
INTRODUCTION
Humans naturally coordinate with each other through timed and spatially aligned actions, such as shaking hands, dancing, playing sports, or passing objects. Representing and generating such human- inter- actions (HHls) 3D is a core challenge in embodied AI, with broad impact on virtual character animation, human-robot collaboration, and embodied communication. Given a simple natural language prompt like "Person hands an object to Person B," generating motion sequences for both agents that are semantically aligned, temporally coherent, and physically plausible remains an open challenge.
Early progress in motion generation largely focused on single-agent synthesis, where approaches like T2M- Ghang et al. (2023), MotionGPT Jiang al. (2024a), and MDM Tevet et al. (2023) successfully model long-range temporal dynamics conditioned on text. However, extending these techniques to multi-agent interactions is non-trivial. HHI generation presents unique challenges: it requires modeling synchronized dynamics between multiple agents, capturing both mutual awareness and individual autonomy, and handling diverse interaction semantics ranging from high-level coordination to fine- fine-grained local motions. Recent

---

Published as conference paper ICLR 2026
concatenate
X6
x,
X6
X,,
interaction
Xg
Encoder
Encoder
DHVAE
masked auto-
disentangled
diffusion
codebook
latent diffusion
regression
InterLDM
(b) InterMask
(c) DHVAE (Ours)
Figure 1: (a) InterLDM ei al. 2025). (b) InterMask Javed et al. (202)) encode all motion information into
a single latent. (c) Our encodes individual motions and interactions into separate disentangled latents.
advances in generative modeling, particularly latent diffusion models (LDMs) Rombach al. (2021). have shown impressive performance in synthesizing high-dimensional data across domains, including human motion. By operating in compressed latent space, these models enable efficient learning and scalable generation. MI.D Chen et al. (2023) is the first to adopt this paradigm for single-agent motion generation, achieving good results through temporal-awarc latent diffusion.
Extending LDMs to human-human interaction, however, remains underexplored. A few recent attempts, such InterLDM Li al. (2025), have begun applying this idea to HHI. As illustrated in Fig. T (a), these approaches encode both agents into a flat, unified latent representation and apply diffusion in this joint space. While such design enables synchronized modeling. it entangles agent identity with interaction context, leading to limited expressivity and degraded realism, particularly in cases requiring fine-grained coordination or distinct agent behaviors.
Moreover, as illustrated in Fig. [ (b). models such InterMask Javed et al. (2025) encode motion within a unified latent space shared across agents. While this approach has achieved state-of-the-art (SOTA) per- formance, it lacks explicit modeling of global interactions between agents, often resulting in physically implausible outcomes, e.g., hand penetration or missed contact in tasks like "two people shake hands." " These failure cases underscore two fundamental limitations in current methods: the absence of structured interaction representations and limited control over contact realism.
We propose a paradigm for text-conditioned human- human interaction generation: Disentangled Hier- archical Variational Autoencoder (DHVAE) paired with structured latent diffusion. Our key innovation explicitly disentangle the HHI representation into three levels: (1) Za. modeling Person A's individual motion; (2) Za. modeling Person B's individual motion; and (3) 2o, a shared latent capturing the global inter- action context. To fully leverage this hierarchy, we introduce Coanssfformer module that jointly encodes mutual awareness and preserves individual agent identity. Beyond architectural design, we further enhance the learning of the interaction representation z, using contrastive learning strategy. We introduce simple yet effective approach to construct positive and negative HHI pairs, imposing prior-base based supervision on z, encourage to encode meaningful and physically plausible interactions. This design directly addresses the lack of prior-based modeling for physically realistic contact interactions, key limitation in prior works. The structured latents (z., Za, ] are passed through an AdaLN Peebles & Xie (2023) Transformer-based denoiser trained using Denoising Diffusion Implicit Model (DDIM) Song et al. (2221) process in the latent space. To address the scale imbalance and structural heterogeneity between Zo, Za. and Z₁, we introduce segment positional encoding (SPE) to reflect each token's 's role in the interaction, and token scaling to cali- brate feature magnitudes across latent groups. We also adopt skip connections for the AdaLN Transformer stabilize training while allowing flexible conditioning.
Overall, we propose a disentangled and controllable framework for generating HHI motion from natural language with high alignment to text and strong physical plausibility. Evaluations the popular InterHu-
2

---

Published as a conference paper ICLR 2026
man Liang et al. (2024) and InterX Xu et al. (2024a) benchmarks demonstrate that our model significantly outperforms SOTA counterparts across all major metrics, including FID, R- R-Precision, and Multimodal Dis- tance. Ablation studies confirm the value of our architectural decisions. Our main contributions include: We propose a disentangled hierarchical VAE which separates the latent representation of human- human interactions into three components: the individual motion components, and the global inter- action component, enabling controllable and personalized generation.
We propose contrastive learning strategy over the global interaction latent x, to enable prior- based modeling of interaction semantics and improved physical plausibility, especially for contact- sensitive regions.
Our model is the lightest and fastest, and sets new SOTA performance on the InterHuman and InterX benchmarks on multiple metrics, demonstrating superior text-motion alignment and fidelity.
2 RELATED WORK
Human Motion Generation. Recent advances in human motion generation have been fueled by large-scale motion capture datasets and the emergence of generative models. Early methods relied on aligning latent spaces between text and motion using objectives like Kullback-Leibler divergence Guo et al. (2020) 2022bb. or directly learning motion embeddings from paired descriptions Ahujaa & Morency (2019). ). With the advent of Transformer models, autoregressive approaches such as T2M-GPTZZang et al. (2023) and MotionGPT Diang et al.] (2024)) represent motion as discrete tokens and generate them sequentially. Despite producing coherent results, these methods often struggle with long-term dependencies and lack bidirectional modeling- To address these limitations, non-autoreeressivee models have gained popularity, including masked motion transformers such as MoMask Guo et al. (2024) and MMM Pinyoanuntapong et al. (2024), which leverage bidirectional context prediction via masked token reconstruction. In parallel, diffusion-based frameworks like MDM Tevet et al. (2023), MotionDiffuse Zhang et al. (2024). and FLAME Kim et al. (2023) have
achieved promising results in generating temporally consistent and realistic motion. However, these methods are designed for single-person motion and struggle to model complex multi-agent interactions.
Human-Human Interaction Generation. Human-human interaction modeling extends the challenges of motion generation by requiring coherent inter-personal dynamics. Prior works can broadly categorized into reaction-based and joint generation paradigms. Reaction models such as Chopin et al. (2024); Ghosh et al. (2024); Xu et al. (2024b); Ji et al. (2025) generate one agent's 's motion conditioned on another's 's actions, generalization across diverse interaction types. Joint generation approaches like CDmMDM [hafir et al. (2024) and RIG Tanaka & Fujiwara (2023) adopt diffusion- based architectures where two agents are jointly denoised with shared or cross- cooss-oondiiioeed modules. InterGen Liang et al. (2024) extends this by using cooperative denoisers with mutual conditioning. Later methods |hhan et al. Fan et al. 2024)) Li al. (2025) further scale HHT by introducing global relational attention or structured priors. Ponce et al. (2024) realized the post effort to control interaction. and proposed in2IN by generating individual motions prior separately and then refining interaction with guided diffusion.
but often lack symmetry
20024;
:
Wang et al. (2025) noticed the problem of role-aware interaction and put up Role-Aware Scan and Localized Pattern Amplification to enforce the accuracy of interaction. Despite their success, these approaches still face limitations in fine-grad tne-grained spatiotemporal modeling and high-quality coordination across different bodies, especially when interactions are diverse or weakly defined. Hence, Javed et al. (2025) proposed InterMask. a BERT-like Devlin et al. (2019) model which encodes the spatial and temporal representation in a discrete token space and generates the masked token gradually from scratch.
Latent Diffusion for Structured Generation. Latent diffusion models (LDMs) Rombach et al. (2021 have emerged as a scalable solution for high-dimensional generative tasks by performing the diffusion process in a compressed latent space, significantly reducing computational cost while maintaining expressivity. In single-person motion synthesis, latent diffusion has improved sampling efficiency and modeling capacity, as demonstrated by works like MLD, which uses transformer-baa based VAE to compress long motion sequences
3

---

Published as conference paper ICLR 2026
compressed unified latent token. For HHI, Li al. (2025) follows the architecture of MLD and com- presses the two-person motions in a single latent representation. However, these methods typically operate flat latent space and do capture the hierarchical
multi-scale nature HHI motion patterns. To address this, our proposed method introduces a hierarchical latent diffusion architecture that models motion generation and interaction at multiple levels of abstraction. By integrating both local joint-level dynamics and global interaction-leve context in unified framework, we achieve improved realism, controllability. and diversity in HHI motion synthesis.
3
METHOD
disentangled representation
Denoiser
DHVAE Encoder
"hwo people shake hands each other
DHVAE Decoder
time embedding
Fa
interaction
Cempte (For Hy
111
24
Decader
tnlon sealing
Adbl Trandoomer
Iemaaton steps
Sepront
Sib connection
weltigg sperater
Figure 2: Architecture of our DHVAE to encode the structured latent representation Zo. Zu, Zb. The global latent token 2, will learn an interaction plausible space via contrastive learning. The encoded structured representation will be passed into skip-connected AdaLN Transformer to learn the denoise process.
3.1 DISENTANGLED HIERARCHICAL LATENT SPACE ENCODING
As shown in Fig. we propose Disentangled Hierarchical Variational Autoencoder (DHVAE) that dis- entangles motion In a global-individual manner via three latent variables: Za and Zか for Persons A (æ,) and B (za),, ensuring personalized motion details, and joint latent z, capturing their global semantics and interaction.
Encoding, Individual motion features are extracted using Transformer encoders with learnable tokens thy 26 (see 6.4), producing and and individual temporal embeddings. A CoTransformer fuses these individual embeddings to model interactions. Each branch uses the other's output as key and value, with skip connections reducing query distortion. Outputs are concatenated with global token u, and passed through an MLP to form Zo. modeled as Gaussian latent with learnable mean and variance.
Decoding, The global latent z, is first decoded via Transformer to obtain implicit interaction, then fed to two parallel Transformer decoders for Persons A and Each decoder attends to 2, through cross-attention, generating temporally synchronized and semantically coherent motion sequences.
Objective Function. Let x = [xa, x²] the motion of the two persons. conventional (flat) VAE like Li
2025) would model it as:
p(x) Eg(()/)) [log p(x|2)] DxL. lg(z)x)]lp(2)),
(1)
4

---

Published as a conference paper ICLR 2026
While Equation 1 models the joint distribution of x, and Xs, it employs a single latent variable z and therefore does not explicitly separate agent-specific and shared semantic features or global interactions. In contrast, our DHVAE introduces structured latent variables Zu Zb, and Zo capture both private behaviors and shared dependencies:
log p(x, x6) 2 LELBO Eng(a. (x) log p(x Zo: Ra) + log p(x6 Ro. zb)
(2)
- -KL ((zu|xx)) (lp(za) DKL [(zz//x))lppzz DKL ( Za: zo) lpp(o))
where p(x, (n, 2a)) / p(xs Ro; 26)) denote individual likelihoods reconstructed from global and individual latents, g(ze Xx) (zz xx) are individual posterior distributions, g(zo/za, Zú) is the CoTransformer-encodee global interaction latent, and p(zo), p(z.). p(z)) are Gaussian priors. This formulation captures causal or complementary inter-agent dynamics, enabling structured interaction modeling. one-to-many conditional generation, and fine tineggriind semantic control. In contrast, works like Li et al. (2025) model only p(x, Xs, X²/z), z), which overly compresses information and leads to large covariance between agents. While our hierarchical decoding manner reduces the covariance between components, increasing the capability of the denoising process (Please see Sec for theoretical and visualization discussion).
Interaction Contrastive Learning. Prior methods, such as InterGen and InterL.DM, impose penalty on the pairwise distance between agents to encourage interaction-aware behavior. However, this approach assumes fixed spatial proximity between agents, overlooking the significant variability between contact and non-contact interactions. As result, such methods tend to overfit to specific motion patterns and fail to generalize across diverse interaction types. This often leads to implausible behaviors such penetration unnatural detachment.
To address this limitation, we propose contrastive learning objective over the global interaction latent vari- able 2o. which encodes the shared context between agents. By constructing positive and negative motion pairs based on semantic and physical plausibility, our method encourages 2, to capture meaningful interac- tion structures. This not only enhances generalization across different interaction scenarios but also improves the physical realism and coherence the generated motions.
The core idea is summarized in Algorithm For each motion pair (Xa, Xt). we first check for physical contact by computing the overlap between the voxelized human mesh pairs. If contact exists, we create a positive sample x by applying small ground -paane translation within =o otherwise, we allow slightly larger range - A negative sample x, is generated by larger shift from a two-tailed truncated Gaussian, ensuring spatial inconsistency. Passing these perturbed pairs into DHVAE yields interaction latents z and 2. A triplet margin loss then enforces z,, to be closer to z than z by margin m. encouraging sensitivity spatial plausibility. To preserve joint information, we add a joint-position penalty. The overall objective is:
LDHVAE CELBO + Ajoint + Cripplet
where Ljoim is the LI loss on joint positions. For InterHuman, we use global positions, while for InterX, we derive joints via the SMPLX forward kinematics.
3.2 DIFFUSION OF HIERARCHICAL LATENT
DHVAE provides disentangled latent representation (2., Ru, 21} for global interaction and individual mo- tions. To generate realistic and diverse sequences, perform latent diffusion Rombach et al. (202T) using DDIM Song et al. (2021), which enables efficient non-Markovian sampling without loss of quality. The forward process gradually adds Gaussian noise,
q(z, 2, 1) -N(z,; V1 1,,,1,,811),
(4)
5

---

Published as a conference paper ICLR 2026
with reparameterrizatoonn
Vá,20 v1- âre, E~ e~,10,
(5)
where Ci = IT:=1 3.). A Transformer-based denoiser eo sg(ze, 1, reconstructs all latent components conditioned on text c. and the DHVAE decoder synchronizes them back into motion sequences. To stabilize training. we apply token scaling to balance contributions Zo. Za and Zp. Moreover, since these compo- nents have different value ranges, we normalize 2, and 24 by scale factor 81 that z (Zos Za/ 81, 2b/ 81) lies in comparable range.
Besides, to learn the hierarchical order, employ SiLU-based MLP combined with embedding layer instead of traditional sinusoidal embedding. InterGen Liang et al. (2024) first introduced AdaLN for HHI and later works followed. The AdaLN used in InterGen and in2IN has two-parameter setting with only a scale A and a shift whereas we follow InterMask, which has an AdaLN-zero style three-parameter setting. While stacking multiple Transformer layers enables the model to capture multi-level information, deeper stacks alone often lead to vanishing gradients, especially in the denoising process. To improve the capacity and stability the denoising process, we adopt U-Net-like architecture RRnneberger et al.) (2015) within the AdaLN Transformer-bane
denoiser. This design introduces skip connections between the opposite layers in the denoiser, enabling the reuse of low-level latent features in shallow layers.
Classifier-F Guidance. To enhance sample diversity and controllability. we incorporate classifier-free guidance (CFG) Ho Ho Salimans (202T) during denoising. The model jointly trained with and without tex- tual conditions. During inference, the denoised output is guided by interpolating between the unconditional and conditional predictions with a guidance strength wt
Eguided = (1 + w) ca ca(z,, c) w ,o(zz,
4 EXPERIMENTS
Datasets. We perform comparative evaluations on two popular benchmarks for text-conditioned human interaction generation: InterIIuman Liang al. (2024) and InterX Xu et al. (2024a). InterIluman adopts the AMASS Mahmood et al.] (2019) skeleton format with 22 joints, including the root joint. Each joint but for the root is described by the tuple {Pg. yg: rea). where Py R is the global position, V, R is the global velocity, and rea E R represents the local 6D rotation. This results in motion representation tensor m, E R- X262, InterX follows the SMPL-X Pavlakos et al. (2019) format comprising rotations of 55 joints, including the main body, hands, and face joints, along with the root orientation. Each joint and root orientation is represented by red, and the root translation by Py, with additional root velocity v added to the
Algorithm 1 Contrastive Learning for Interaction Latent Z
Require: Input motion pair (Xa, X²)
1: Compute glohal latent: 2, DHVAF(Xx Xa;
2: if' iisscontact (X,, Xb) then
3: x Translate (Xx, A MunncNom(0,,00000 0z)) for 5em, small jitter)
4: else
5: x Translate (xb, A (o, o..)) fou 30cm, looser non-contact jitter)
6: end
7: X3 + Translate A TwoTailed(+1155g =30)) (A 45 45-90em, implausible shift]
z DHVAE((x, x ),
z XHVAE(X,,x;)
9: Compute contrastive loss:
Laipler max(0, d(z,, xx - d(zo, ((20,2)) m)
6

---

Published as a conference paper ICLR 2026
representation. This results in m, IR Nxx666 We retain the native skeleton representations of each dataset to demonstrate the compatibility of our framework with diverse pose representations and joint structures.
Evaluation Metrics. To comprehensively evaluate the performance of our model, we adopt suite of feature-space metrics by Liang et al. (2024). To assess the realism and fidelity of generated interactions, we compute the Fréchet Inception Distance (FID) and between the feature distributions of generated and ground- truth motions and their Diversity. To evaluate the semantic alignment between text prompts and generated motions, we use R-Precision and Multimodal Distance (MMDist), which measure the consistency hetween text input and generated motions. To assess generative quality beyond accuracy, report Multimodality (MModality), quantifying the model's ability to produce multiple motions for the same textual description.
Baselines. We compare against all SOTA HHI generation methods, including InterGen Liang et al. 20224) MoMat-MoGen Cai et al. (2024), InterMask Javed et al. (2025). and in2IN Ruiz-Ponce et al.) (2024) on the InterHuman dataset. For InterX, we follow the evaluation setup of Javed et al. (2025) and report results for T2M (Zang et al. 2023), MDM Tevet et al. 2023)) ComMDM Shafir et al. (2024), InterGen, and Inter Mask. HumannML3D-ssy
Since in2IN is trained based InterX dataset. In addition, we implement an MLD-based variant as a latent diffusion baseline for both datasets. We also include TIMotion Wang al. (2025) for comparison. However, since only the InterHuman variant is officially released, we run their provided code and pretrained weights to reproduce results, and for InterX, we directly use their claimed results.
motion representations, it cannot be applied to the SMPLX-
Implementation Details. We use a latent space of 1 x 256 for Zo, Ze. Zb for the InterHuman and 1 x 336 for the InterX due to different representation dimensions (InterHuman 262 and InterX is 336). For the DHVAE encoder, we employed two sets of Transformer Encoders of 4 layers, and a CoTTransformer of 3 layers with skip connection, which have a hidden dimension of 1024, head number of 4, and a dropout ratio 0.1 for InterHuman; and a hidden dimension of I 1344 for the InterX Dataset. For the DHVAE decoder, all three decoders share the same parameter settings as the corresponding encoders. For the denoiser, we use 13 layers of AdaLN Transformer, and other settings follow the Transformer above. In the reverse diffusion process, we set the 3y = 0.00085 and 3y = 0.012, the diffusion time steps to be = 1000, and the inference time step to be 50. We set the unconditional ratio to be 0.1 and use the CFG strength of 3.5/3.0 3.0 for the Inter/uuman/Iner> datasets. We trained the DHVAE for 2000/1200 epochs for InterHumanm/nterx and 1600/1000 epochs for the denoiser. All experiments were conducted single NVIDIA H100 GPU.
4.1 QUANTITATIVE RESULTS
We conduct comprehensive evaluations on both the benchmarks, where our model achieves SOTA perfor- mance across all major metrics. As shown in Table our method consistently outperforms previous ap- proaches by significant margin with the lowest FID, MMDist, and the highest R- R-precisions. These results demonstrate our model's efficacy in producing realistic and coherent HHI sequences.
As two-step generation pipeline, encoder reconstruction sets an upper bound on the model's fidelity. To evaluate the reconstruction quality of our DHVAE, we compare against the VAEs of two other two-step methods: InterMask 22--Q--VAE and our implementation of MLD, as previously shown in Fig. [ As shown in Table D DHVAE achieves the best reconstruction performance across reconstruction FID (rFID), MPJPE (in meters), and LI loss on Z-normalized features, which suggests that our disentangled prior enables compact and structured representation.
Besides, evaluate computational efficiency in terms of model size and average inference time per sentence (AITS, measured in seconds and excluding the pretrained CLIP text encoder). As shown in Table 3 our model outperforms the SOTA methods TIMotion and InterMask, achieving both the smallest parameter footprint and the fastest inference speed. This highlights the lightweight yet efficient design of our approach. To the best of our knowledge, DHVAE is the first work since InterMask to achieve consistent improvements
7

---

Published as a conference paper ICLR 2026
InterHuman
Model
R-Prec@11
R- R-Prec@#1
R-Preco#3
FID L
MM Dist 4
Diversity
MModality
Ground Truth
0.452#.00s
0.610 808
0.7014400s
0 007
3.735*.000 5008
7.948+t000
InterGen Liang et sl. 2024
0.371 0.3712:000 2:010
0.515
0.624
5.918
5.108
7.3877
2.141 003
MLD* Chen et al 2024
0.392 ous
0.533 006
0.6124.004
6. 11884.002
3.817 003
7.7854.00
1.236
MoMar- MnGen Cin ml
2024
0.44g o044
0.591 003
0.666
5.674 120
3.790 0011
8.02)
1 1.295 0.2
in2IN Ruiz-Ponce et al
2024
0.455 #004
0.611 00000
0.687
5.177 120
3.790
7.940
1.061 008
InterMask Javed
Javee et al.
2025
0.449
0.599 006
0.681##004 4.00
5.153
3.790
h.9444.000
1. 1.737#0000 020
TIMotion Wang ct
0.485 -1885-0007
0.635 4 006
0.712 4-000
5.600 108
7.96
0,952 023
Ours
0.496 #:04
0.647 = 000
0.720 4.000
5.015
3.772 1772"
7.952
0.804 000
InterX
Model
R-Prec@11 1
R- R-Prec@21
R-PrecG#
FID L
MM Dist 4
Diversity
MModality
Ground Truth
0.429-t-00
0.626
0.736
0.002
8.53
9.734
T2M Zhang al. 2023
0.184##011
0.298 010
0.396
5.481
9.57G
2.771
2.761 on
MDM Tevet ct al 2023
0. 0.203
0.329 007
0.426
23.701 #-057
9.5488
5.856
3.490 061
ComMDM Shahr al.
2024
0.090
0.165 001
0.236
29.206 t.07
6.870
4.73.gt.007
0.771 063
InterGen Liang et al 2024
0.207:2.02
0.335 002
0.429
5.207
8.504
7.788tt-008
3.686 2.860
MLD*
202,3
0.386 nos
0.577 006
0.678 oos
1.295 0018
4.056 033
9.008
2.375 017
InterMask
Javed Javed
2025
0.403
0.585 .066
0.705 0.705
0.399:t:013
3.705
9.0a6
2.261
TIMotion Wang al.
2025
0.412#.004
0.601 #
t.704
0.385##.02
3.706 h.706
9.191 1.092
2.437
Ours
0,442 2.00
0.638
00G
0.745 5.04
0.339
3.604
9.378 h.00
2.505
Table I: Comparisons on InterHuman and InterX datasets. The best results are in bold, and the second-best are underlined. Methods with * are implemented by us. All results are run 20 times. For a fair comparison, we set the latent size of MLD to be the same ours, i.c. 3 x 256.
across all core metrics on both InterHuman and InterX, thereby establishing state-of-the-art baseline for text-conditioned HHI generation.
Table 2: Reconstruction results for prior-based and SOTA models, best results in bold.
Table 3: Computational cost of mod- els including latency and size
Task
Model
rFID
MPJPE +
Loss 4
Model
AITS 4
Size
MLD- ALD-VVE
1.011
0.089
0.256
InterMask TIMotion
1.021
74M
Recon
2D- VQ- VAF
0.970
0.129
0.276
1.472
77M
DHVAF (ours)
0.503
0.055
0.218
DHVAE (ours)
0.454
56M
Finally, we evaluate the physical plausibility of generated motions by computing both penetration statistics and contact quality. Following prior work Jiang et al.] (2024b), we voxelize the SMPL-X Pavlakos et al. (2019) mesh at a resolution of 2 cm and measure the degree of interpenetration based on voxel overlap. Specifically, we report the Penetration Volume (PV), defined as the average number of overlapping voxels across all generated sequences. To capture different aspects of penetration, we further introduce two com- plementary metrics: the Penetration Frequency Ratio (PFR), which measures the proportion of sequences that contain any penetration event, and the Penetration Duration Ratio (PDR), which quantifies the temporal fraction of penetration within each sequence. In addition to penetration measures, we compute the contact ratio for sequences with annotated contact. To ensure robustness, apply voxel dilation to both interacting meshes and treat cases where the maximum overlapping voxel count exceeds the volume of 6 cm cube (i.e., 216 ml) as severe penetrations, which are subsequently excluded from being considered valid contact. This prevents false positives caused by deep or unrealistic intersections and ensures that the contact score reflects physically meaningful interactions.
8

---

Published as conference paper ICLR 2026
The results in Table suggest that our method achieves the lowest penetration score and the highest contact ratio among MLD, InterMask, and TIMotion.
Model
PV
PFR
PDR 4
Contact f
MLD
0,503
0.108
0.175
0.427
InterMask
0.873
0.149
0.243
0.349
TIMotion
0.485
0.122
0.104
0.466
Ours w/o triplet
0.446
0.107
0.102
0.445
Ours
0.390
0.064
0.087
0.581
Table 4: Penetration metrics and contact ratio for state-of-the models.
4.2 QUALITATIVE RESULTS
We compare our method with the SOTA baseline InterMask. As no official InterMask implementation exists for InterX and space is limited, we show visual results only InterHuman (Fig. 33. The results with TIMo- tion are presented in Fig. 7 InterMask encodes individuals independently in discrete token space without a global interaction latent, often yielding implausible motions at contact points, body interpenetrations (e-g., "uug the seated person"), or failures to follow complex prompts (e.g., "one standing up. the other reach- ing out hand"). contrast, our method produces more physically plausible and semantically accurate interactions, such as successfully aligning hands for "greet each other by shaking hands."
4.3 ABLATION STUDY
DHVAE We compare our proposed DHVAE with baseline VAE based the MLD framework Chen al. 2023), which uses a unified latent representation equipped with skip-Transse Transfrrasformer. For a fair comparison, we align the major architectural configurations, including latent dimensionality (3 x 256 for MLD-VAN -VD))
InterMask
Ours
two performers greet each other by shaking hands.
one person bends down and gives hug to the seated person.
the first person typing keyboard while the second approaches them and whispers something into their
no whisper
the first stands up, and the second reaches out to grasp their hand.
no graup
Figure: 3: Qualitative Comparison with InterMask Javed et al. (0025) InterHuman Dataset, indicating superior text alignment, fidelity, and physical plausibility. The body meshes are arranged sequentially from left to right, with colors progressing from light to dark.
9

---

Published as conference paper ICLR 2026
DHVAE
Denoiser
Metrics
Triplet Loss %
CoTRM MLD-VAE
SPE
TokenScale
Skip
rFID +
g +
RP@
0.499
5.063
0.492
/
0.669
5.886
0,468
V
0.667
6.005
0.463
V
0.632
5.593
0,476
/
V
1.024
5.694
0.457
/
V
1.089
5.603
0,462
V
7.452
0.423
V
6,531
0,413
5,343
0.468
0.503
5.015
0.496
Table Ablation study of DHVAE and Denoiser components. indicates the component used. For fair comparison with MLD, we set the latent size of MLD-VAE to be 3 x 256
and the denoiser used to model the reverse diffusion process. From Tables and 5 we can observe that by replacing the VAE with the MLD-VAE, there is significant degradation, suggesting the disentangled latent space is better than a uniform flattened one. We further ablate the effect of interaction contrastive learning by removing the triplet loss. This leads to slight drop reconstruction accuracy but improves genera- tion quality. Although the numerical metrics show no obvious advantage, qualitative results in 6.5 and physical evaluations Table reveal that the triplet loss improves the physical plausibility. Additionally, we evaluate the impact of two key components in DHVAE: (1) the CoTrrassfrrmer, and (2) the global latent variable 2,. Specifically, we consider two variants: replacing the CoTransformer with 3-layer MLP with residual connections, and removing the global latent generating motions solely from 2, and (main- taining CoTransformer). As shown in line 4, both modifications result in substantial performance drops reconstruction and generation, confirming the importance of hierarchical latent modeling and effective context aggregation via the CoTransformer. Denoiser. We ablate three architectural design choices for the denoiser: (1) SPE, (2) token scaling, and (3) skip connections. To assess the role of SPE, replace it with standard sinusoidal positional embeddings. As shown in Table removing either SPE or token scaling leads significant performance degradation, highlighting the importance of encoding structural segmentation and maintaining scale consistency across latent components. While omitting skip connections has a smaller im- pact, it still provides measurable benefits, particularly improving convergence during the reverse diffusion process.
CONCLUSION
We proposed Disentangled Hierarchical VAE-based latent diffusion framework for text-conditioned HHI generation. By disentangling global interaction and individual motions and employing a CoTransformer module, our approach addresses the limitations of existing flat latent representations. The denoiser inte- grated with skip-connected AdaLN-Transformer segment positional embedding, and token scaling enables high-quality and controllable motion generation. Experiments on two challenging benchmarks demonstrate that our method achieves new state-of-the- aaetartti in realism and semantic alignment. Our approach offers bust and extensible framework for HHI motion generation, with promising implications for animation, and human-robot collaboration. Future work could explore incorporating social cues, expanding to more than two agents, and extending to 3D avatars or embodied simulation environments.
10

---

Published as a conference paper ICLR 2026
ACKNOWLEDGEMENTS
This research was supported by the Australian Research Council (ARC) Discovery Project DP240101926. We gratefully acknowledge this support.
ETHICS STATEMENT
Our work follows the ICLR Code of Ethics, which emphasizes contributing to society, minimizing harm, respecting privacy, and upholding high standards of scientific integrity. The proposed method does not involve direct human subjects, sensitive personal data, identifiable information. All experiments are conducted on publicly available benchmark datasets under their respective licenses. We ensure transparency and reproducibility by providing detailed methodology and acknowledging all contributions.
We believe our research benefits the community by advancing motion representation learning in a socially responsible way. It avoids discrimination, respects faimess and inclusivity, and does not pose foreseeable risks to health, safety, or the natural environment. Should any unintended negative consequences arise, we are committed to mitigating them in accordance with ethical best practices.
REPRODUCIBILITY STATEMENT
We have taken several steps to ensure the reproducibility of our work. All model architectures, training procedures, and hyperparameter settings are described in detail in Sections 1..1ooons6....110 and Section 4, with addi- tional implementation details and ablation studies provided in the Appendix. The datasets used are publicly available, and all preprocessing steps are carefully documented facilitate reproducibility, we will release the complete source code, pretrained models, and data processing scripts upon acceptance. This will allow other researchers to fully replicate and build upon our results.
the supplementary materials. To further
REFERENCES
Chaitanya Ahuja and Louis-Philippe Morency. Language2pose: Natural language grounded pose forecast- ing. In 2019 International Conference on 3D Vision (3DV). IEEE, 2019.
Zhongang Cai, Jianping Jiang, Zhongfei Qing, Xinying Guo, Mingyuan Zhang, Zhengyu Lin, Haiyi Mei, Chen Wei, Ruisi Wang, Wanqi Yin, Liang Pan, Xiangyu Fan, Han Du, Peng Gao, Zhitao Yang, Yang Gao, Jiaqi Li. Tianxiang Ren, Yukun Wei, Xiaogang Wang, Chen Change Loy, Lei Yang, and Ziwei Liu. Digital life project: Autonomous 3d characters with social intelligence. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 582-592, June 2024.
Xin Chen, Wen Jiang, Biao Liu, Zilong Huang, Bin Fu, Tao Chen, and Gang Yu. Executing your commands via motion diffusion in latent space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.
Baptiste Chopin, Hao Tang, and Mohamed Daoudi. Bipartite graph diffusion model for human interaction generation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 5333-5342, 2024.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidi- rectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers), Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10. 00.18853/v1/N199 1423. URL ottp://ppe////aaaaaaoooooo.ooor.

---

Published as a conference paper ICLR 2026
Ke Fan, Junshu Tang, Weijian Cao, Ran Yi, Moran Li, Jingyu Gong, Jiangning Zhang, Yabiao Wang, Chengjie Wang, and Lizhuang Ma. Freemotion: A unified framework for number-free text-to-motion synthesis. European Conference on Computer Vision, pp. 109. Springer, 2024.
Anindita Ghosh, Rishabh Dabral, Vladislav Golyanik, Christian Theobalt, and Philipp Slusallek. Remos: 3d cendion-ondiiioned reaction synthesis for two-person interactions. In European Conference on Computer Vision, pp. 418 437. Springer, 2024.
Chuan Guo, Xinxin Zuo, Sen Wang, Shihao Zou, Qingyao Sun, Annan Deng, Minglun Gong, and Li Cheng- Action2motion: Conditioned generation of 3d human motions. In Proceedings of the 28th ACM Interna- tional Conference on Multimedia, 2020.
Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng. Generating diverse and natural 3d human motions from text. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022a.
Chuan Guo, Xinxin Zuo, Sen Wang, and Li Cheng. Tm2t: Stochastic and tokenized modeling for the reciprocal generation of 3d human motions and texts. In ECCV, 2022b.
Chuan Guo, Yuxuan Mu, Muhammad Gohar Javed, Sen Wang, and Cheng. Momask: Generative masked modeling of 3d human motions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 1900-1910, 2024.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021.
Muhammad Gohar Javed, Chuan Guo, Li Cheng, and Xingyu Li. Intermask: 3d human interaction gen- eration via collaborative masked modeling. In The Thirteenth International Conference on Learning Representations, 2025.
Kaiyang Ji, Ye Shi, Zichen Jin, Kangyi Chen, Lan Xu, Yuexin Ma, Jingyi Yu, and Jingya Wang. Towards immersive human-x interaction: A real-time framework for physically plausible motion synthesis. arXiv preprint 15X::2508.021000 2025.
Yanli Ji, Feixiang Xu, Yang Yang, Fumin Shen, Heng Tao Shen, and Wei-Shi Zheng. A large-scale rgb- d database for arbitrary-vice human action recognition. New York, NY, USA, 2018. Association for doi: 10.14553240508.32406775
Computing Machinery. ISBN 9781450356657.
/ttts:/::///do
/1411144//34448888 3240675
Biao Jiang, Xin Chen, Wen Liu, Jingyi Yu, Gang Yu, and Tao Chen. Motiongpt: Human motion as foreign language. Advances in Neural Information Processing Systems, 36, 2024a.
Nan Jiang, Zhiyuan Zhang, Hongjie Li, Xiaoxuan Ma. Zan Wang. Yixin Chen. Tengyu Liu, Yixin Zhu, and Siyuan Huang. Scaling up dynamic human-scent interaction modeling. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 1737-1747, 2024b.
Jihoon Kim, Jiseob Kim, and Sungjoon Choi. Flame: Free-form language-bast based motion synthesis & editing. Proceedings of the AAAI Conference Artificial Intelligence, 37, 06 2023. doi: 10.1609/aaai.v3777 v3717. 25996.
Boyuan Li, Xihua Wang, Ruihua Song, and Wenbing Huang. Two-in-one: Unified multi-person interactive motion generation by latent diffusion transformer. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. IEEE, 2025.
12

---

Published as a conference paper ICLR 2026
Jiaman Li, Alexander Clegg, Roozbeh Mottaghi, Jiajun Wu, Xavier Puig, and Karen Liu. Controllable human-object interaction synthesis. arXiv preprint arXiv:2312.03913, 2023.
Han Liang, Wenqian Zhang. Wenxuan Li, Jingyi Yu, and Lan Xu. Intergen: Diffusion-based multi-human
motion generation under complex interactions. International Journal of Computer Vision, pp. 1-21, 2024.
Naureen Mahmood, Nima Ghorbani, Nikolaus F, Troje, Gerard Pons-Moll, and Michael J. Black. AMASS: Archive of motion capture surface shapes. In International Conference on Computer Vision, pp. 5442 5451, October 2019.
Bo Miao, Mohammed Bennamoun, Yongsheng Gao, and Ajmal Mian. penetrrum-guidee multi gualii-raanulrrity referring video object segmentation. In Proceedings of the IEEE/CVF International Conference on Com- puter Vision, pp. 920-930, 2023.
Bo Miao, Mingtao Feng, Zijie Wu, Mohammed Bennamoun, Yongsheng Gao, and Ajmal Mian. Referring human pose and mask estimation in the wild. Advances in Neural Information Processing Systems, 37: 44791-44811, 2024.
Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed A. A. Osman, Dimitrios Tzionas, and Michael J. Black. Expressive body capture: 3D hands, face, and body from a single image. In Proceedings IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 10975-10985, 2019.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference Computer Vision (ICCV), pp. 1172-4182. IEEE Computer Soci- ety, October 2023.
Ekkasit Pinyoanuntapong, Pu Wang, Minwoo Lee, and Chen Chen. Mmm: Generative masked motion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models, 2021.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. Medical Image Computing and Cosputer-Assisted Intervention (MICCAI), 9351, 2015.
Pablo Ruiz-Ponce, German Barquero, Cristina Palmero, Sergio Escalera, and José Gareía- Rodríguez. in2in: Leveraging individual information to generate human interactions. In Proceedings of the IEEE/CVF Conference Computer Vision and Patlern Recognition (CVPR) Workshops, pp. 1941-1951, June 2024.
Yoni Shafir, Guy Tevet, Roy Kapon, and Amit Haim Bermano. Human motion diffusion as generative prior. In The Twelfth International Conference on Learning Representations, 2024.
Mengyi Shan, Lu Dong, Yutao Han, Yuan Yao, Tao Liu, Ifcoma Nwogu, Guo-Jun Qi. and Mitch Hill. Towards open domain text-driven synthesis of multi-person motions. ECCV (65), 2024.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference Learning Representations, 2021.
Mikihiro Tanaka and Kent Fujiwara. Role-aware interaction generation from textual description. In ICCV, 2023.
Guy Tevet, Sigal Raab, Brian Gordon, Yoni Shafir, Daniel Cohen-or, and Amit Haim Bermano. Human motion diffusion model. In The Eleventh International Conference Learning Representations, 2023.
13

---

Published as conference paper ICLR 2026
Yabiao Wang, Shuo Wang, Jiangning Zhang, Ke Fan, Jiafu Wu, Zhucun Xue, and Yong Liu. Timotion: Temporal and interactive framework for efficient human-hume human motion generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 7169-7178, 2025.
Zhenzhi Wang. Jingbo Wang, Yixuan Li, Dahua Lin, and Bo Dai. Intercontrol: Zero-shot human interaction generation by controlling every joint. In NeurIPS, 2024.
Zhen Wu, Jiaman Li, Pei Xu, and C. Karen Liu, Human-object interaction from human-level instructions. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2025.
Liang Xu, Xintao Lv, Yichao Yan, Xin Jin, Shuwen Wu, Congsheng Xu, Yifan Liu, Yizhou Zhou, Fengyun Rao, Xingdong Sheng. et al. Inter-x: Towards versatile numannhuuman interaction analysis. In CVPR, pp. 22260-22271, 2024a.
Liang Xu, Yizhou Zhou, Yichao Yan, Xin Jin, Wenhan Zhu, Fengyun Rao, Xiaokang Yang, and Wenjun Zeng. Regennet: Towards human action- reatttoneeacion synthesis. CVPR, pp. 1759-1769, 2024b.
Jianrong Zhang, Yangsong Zhang, Xiaodong Cun, Shaoli Huang, Yong Zhang, Hongwei Zhao, Hongtao Lu, and Xi Shen. T2m-gpt: Generating human motion from textual descriptions with discrete representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, and Ziwei Liu. Motiondiffuse: Text-driven human motion generation with diffusion model. IEFE Transactions on Pattern Analysis and Machine Intelligence, 2024.
14

---

Published as a conference paper ICLR 2026
6 APPENDIX
6.1 USER STUDY
We conducted a user study at moderate scale, distributing 30 surveys and receiving 18 valid responses. In the study, we compared our results with InterMask and TIMotion across multiple motion categories using the InterHuman test set. The reviewers were asked to select the motion sequences with the highest quality- wwith all model names anonymized during evaluation. As illustrated in Fig. method consistently achieved the highest preference ratings compared to the other two approaches.
boxing
shake hands
whisper
greeting
1:22..96
br 22.296
5.0%
77.6%
hug
dance
fencing
RPS
13.3%
18
8 22.20
72.2%
66.%%
Ours
InterMask
TIMotion
Figure 4: User Study of DHVAE compared to InterMask and TIMotion
6.2 CLASSIFIER FREE GUIDANCE
We investigate the impact of classifier-free guidance (CFG) generation quality by varying the guidance scale from to 10 in increments of 0.5. For each setting, we repeat the sampling process 20 times to account for randomness and report the average results. We compare
method against the InterMask baseline and highlight the regions where our approach surpasses the baseline using gray shading. The evaluation results for the InterHuman dataset presented in Fig. 5 and the corresponding results for the InterX dataset shown in Fig. 6
This analysis provides insight into how different guidance strengths influence various metrics, including FID, R-Precision, and MM-Distance, thereby offering practical recommendations for optimal CFG values.
6.3 QUANTITATIVE COMPARISON WITH INTERLMD
We consider InterLDM et (025) be the first application latent diffusion models the HHI task. However, we observed significant inconsistencies in its reported result--articcuart) HMM-Distane--wher when compared with other state-of-the-art methods. Due to these inconsistencies and the inability to reproduce their results, we have chosen not to include InterLDM in the main comparison table (Table to avoid introducing potentially misleading conclusions. Theoretically, the metrics R-precision and Multimodal Distance share the embedding feature, hence they are correlated, i.e., the higher the R- R-precision is, the lower the Multimodal Distance is. Therefore, we infer that their MMDistance can not match ours based on their R-precisions.
abnormally low
15

---

Published as conference paper ICLR 2026
Metrics CFG Scale (Multiple Models)
R
2.50
Figure 5: Metrics along the change along dlassffer-frre-guiddae seale on InterHuman dataset
Metrics vs CFG Scale (Multiple Models)
VM desance
149
Figure 6: Metrics along the change along cressfiir-freeguuiddae scale on InterX dataset
16

---

Published as a conference paper ICLR 2026
In our attempt fairly benchmark their method, we noted that the authors did not release official code, and our direct inquiries for clarification received no clear response suggestions. Additionally, InterLDM did not conduct evaluations on the InterX Xu et al. (2024a) benchmark, further limiting the completeness of the comparison. Nevertheless, for transparency and completeness, we include side-by-side comparison with their reported results in the appendix. We emphasize that, except for the results on the questionnaire MM-Distance, our method outperforms InterLDM across all other reported metrics. This reinforces the effectiveness and robustness of our approach while addressing the ambiguity in their reported numbers.
Dataset
Model
@-Prec 1
R-Prec@2 f
R-Prec@33 f
FID
MM Dist
Diversity +
Ground Truth
0. 0.452400008
0.610
0.701 #0.008
0.278
3. 7.555 F10.00
7.948
InterGen Liung ul.
2024
4:371
0.5154.012
0.6244-000
5,918 070
5. 1.18::0014
7.3874.020
MLDD Chen et al.
2023
0.392
4.5334.005
0.612
6.15844.0z
3.8174.003
7.785
InterI.DM
t.427
0.55g4.050
0.638
5.61944.01
1.862
7.888
InterHuman
MoMat-MoGinn MoCien Cai et al. 2124
0. 0.44g$5000
0.591
U.6666t0004
5.674
3. 4.79904.001
8.021
InterMask Javed al. 2025
0. 0.
0.599
0.681#0004
5,153
3.790
7.944
in2IN RRiz-Ponce et 2024
t.455
0.611 11:00s
0.687
5.177
3.790
7.940+.047
Ours
0. 0.496
0.647
0.720 0.720
5.015#:086
3.772
7.952
Table 6: Comparison results on InterHuman including InterLDM. The best results are marked with bold font, and the second-ber best results are marked with underline. The group with * is implemented by us. To avoid coincidence, each metric is repeated 20 times, and standard deviation is provided.
6.4 QUALITATIVE COMPARISON WITH TIMOTION
As shown in Fig. 7 we compared the generated sequences with TIMotion with the same prompts, finding
that TIMotion has serious artifacts, shaking, and what's more, position shifting between agents (marked
with blue dash in the figure), which happened to most of the current works Liang et al.] (2024); Tanaka & Fujiwara (2023). Besides, implausible contacts also happen, as marked in red circle. Please see the supplementary.mp4 file to see the actual animations.
6.5 VISUALIZATION OF ABLATION STUDY ON CONTRASTIVE LEARNING
In the main text, have shown that contrastive learning improves the quantitative results. In this section, we present visual cases that, without the Contrastive Learning, part of the generated HHI sequences may not be semantic or physically plausible. As shown in the Fig. 9 for given prompts like "two people shake their hands", without the triplet loss, the hands do not come into contact closely,
Based on the previous analysis, we visualize two PCA projections of the interaction latent z, in the context of the hand-shaking task. Figure & illustrates the PCA embedding of the ground-truth motion, where we show the anchor, positive, and negative samples, encoded by DHVAE with and without contrastive learning. Figure Ep presents the encoded z, from the generated motion sequences given the prompt "hand-shaking" again comparing models with and without contrastive learning. Our DHVAE model with contrastive learning clearly separates positive and negative samples in the latent space. This separation is consistent across both reconstruction from ground-trum -ruth motions and generation from language prompts. The structured latent space induced by contrastive supervision demonstrates not only improved representation learning but also contributes to more plausible and controllable generation quality. For detailed animations, please refer to the supplementary. file.
17

---

Published as conference paper ICLR 2026
TiMotion
Ours
two performers greet each other by shaking hands.
one person bends down and gives hug to the seated person.
the first is typing keyboard while the second approaches them and whispers something into their
the first one stands up, and the second one reaches out grasp their hand.
Figure 7: Comparison with TIMotion on InterHuman
Projection
Representations
Projection Generated Representations.
owi Triptes Loas]
Plrisble Fnoxted Triplek taer
0.20
Photibve Tripic Leag
hepovottl Ereaded was
Nogulave Tripett Loval
Plarabbte Frosted T.Trppee
Component's
Inpos Lanal [Sund]
hupaaall Enceded Tappit Loto
Phottee Trglet [Uues]
0.10
Negative teppet Lourn)
9.26
Principal
-0.00
-0.05
-0.10
-0.15
Principel Component
Principal Componest
(a) PCA on ground ground-truth truth motion
(b) PCA on generated motion thand-shaking
Figure 8: PCA projections of the interaction latent z, in the hand-shaking scenario. (a) shows encodings of ground-truth motion triplets; (b) visualizes zo of generated motions from prompts. Both compare DHVAE with and without contrastive learning.
6.6 INSIGHT INTO HIERARCHICAL LATENTS SUPERIORITY
We provide both theoretical and empirical evidence that our hierarchical latent design {Zo: Ba Zb} is more effective in the diffusion process than the two-branch baseline z = { Zb}.
Lemma. For given diffusion process with denoiser 6 scheduler, and data distribution p(x), higher patch- /channel-wise variance of x leads to more difficult denoising.
18

---

Published as conference paper ICLR 2026
w/o. Contrastive Learning
ours
two performers greet each other by shaking hands.
the first person typing keyhoard while the second approaches them and whispers something into their ear.
Figure 9: Visualization ablation study for Contrastive learning
Proof. The training objective is the expected per-step KL. divergence between the true reverse process and the model:
(()) Ep Ep(ll()
DKL (q(x0-1 ze, xo) |l po( xe. Σ
(7)
Each step has closed-form KL:
1 20 || zo) - (rr, h, { - log 0/
DKL
(8)
where schedule-only constants appear in the second term. Hence, channel statistics affect the loss only via the MSE term.
DDPMs, (x,, A4 B, For prediction error 8, p2 Mía.
Ep( 12 = Tr(A, A Emjh) + |AA + B, po (2, t, h)1l2
(9)
where Σnjh Var(zo h). If channel variance of increases, then Eneh grows in the PSD sense, making the first term nondecreasing. Thus the expected KI., and consequently L(0), grows with oî.
Moreover, since gradient variance scales with 122 larger of induces noisier gradients, requiring smaller learning rates and slowing convergence. Therefore, higher channel variance increases both the KL and the optimization difficulty.
Plat Componants
Components
L-SN Piot of Flattened Latent Componart
100
1-SRE Comparent
20
DSNE Componert
20
e.0e Component
20
440
-20
50
40
-20
40
LSNE Companenc
USNE Conppnn
ESNE Component
T-SNE of DHVAE
(b) T-SNE of Two-Branch
(c) T-SNE of Flattened Latent (DEL = 4.7)
DK = 2.7)
(DL = 3.6)
Figure 10: T-SNE projections of latents the InterHuman test set.
19

---

Published as conference paper ICLR 2026
Fact. As shown in Fig. 10 our DHVAE produces more uniform covariance across latent channels, while the two-branch and flattened VAEs exhibit noticeable imbalance. Specifically, we compare (a) our hierarchical DHVAE, (b) a two-branch VAE encoding only {2a. ), and (c) flattened VAE following MLD[Chen al. (2023). The improved uniformity stems from our hierarchical decoding scheme, which cascades the shared 20 and global motion sequence (Fig.) 2 into the decoders for 2n and Eb.
Although the flattened VAE yields lower cross-covariance than the two-branch, its KI. divergence and re- construction quality are substantially worse, leading to poorer generative performance. In contrast, our disentangled hierarchical structure balances variance across components while preserving reconstruction fi- delity, resulting in stronger diffusion-bant generation. As fact, with the same penalty λKl = 0.001, our DHVAE can encode the latent with the smallest KI. Divergence, resulting in more normalized distributions.
Based on the lemma above, we further explained why Token Scaling is crucial for denoising, since it can further decrease the covariance magnitude.
6.7 ABLATION STUDIES ON CONTACT THRESHOLD
We conduct ablation studies on the contact threshold o, and the non-contact threshold du- First, we fix Ta 0.30 and vary de- When oe less than 00 the results remain stable for both contact and penetration. However, as increases, the margin between negative and positive samples becomes smaller, making the latent space more tolerant larger perturbations. This leads poorer physical plausibility and higher FID scores. Next, we fix o, = 0.05 and vary oa. When is less than 0.2, the model struggles to distinguish between contact and non-contact behaviors. As o, increases further, the penetration score rises slightly, likely because non-contact motions become more tolerant, which can result in collisions.
Setting
PV 4
PFR 4
PDR 4
Contact f
FID |
O. 0.02, σu = 0.30
0.392
0.073
0.088
0.557
5.050
de 0.03, da = 0.30
0.387
0.069
0.082
0.589
5,093
0.05, du 0.30
0.390
0.064
0.087
0.581
5.015
a. 0.06, d. 0.30
0.368
0.067
0.088
0.592
5.047
O. 0.08, a. = 0.30
0.395
0.070
0.085
0.572
5.022
a. 0.10, dμ = 0.30
0,402
0.068
0.090
0.524
5.035
0.15,
0.30
0.416
0.082
0.092
0.508
5,088
O 0.20, G. 0.30
0.421
0.102
0.099
0.484
5.107
0.05, 0.10
0.417
0.109
0.092
0.584
5.044
G. 0.05, d. 0.20
0.387
0.074
0.090
0.586
5,054
O. 0.05, G.
0.390
0.064
0.087
0.581
5.015
D. 0.05, G. = 0.40
0.395
0.066
0.085
0.577
4.997
J 0.05, J. = 0.50
0.393
0.068
0.088
0.580
5.063
0.05,
0.60
0.406
0.075
0.090
0.575
5.125
Table 7: Performance under different a, and a settings.
6.8 PARAMETERS FOR DHVAE
In this section, evaluate the performance of method under varying architectural settings understand its sensitivity to key parameters. Specifically, we analyze the effect of two components: (1) the number of layers in the individual Transformer encoders, and (2) the dimensionality of the latent space.
20

---

Published as conference paper ICLR 2026
For the first of experiments, fix the latent size to I and vary the number of layers in the individual Transformer encoders. For the second, we fix the number of layers to 4 for the individual encoder and 3 for the interaction encoder, while varying the latent size.
As shown in TableBland Table we report both the reconstruction performance, measured by reconstruction FID (rFID), and generation quality metrics, including FID, R-Precision, Multimodal Distance (MM Dist).
We observe that when fixing the latent size, increasing the number of individual encoder layers initially improves the reconstruction quality (i.e., lower rFID), but performance begins to degrade beyond certain depth, likely due to overfitting or vanishing gradients. Interestingly, the generation performance steadily improves with increasing depth, suggesting that deeper encoders help extract more expressive features for sampling.
The best overall performance is achieved when using layers for the individual encoders and layers for the interaction encoder, as reflected by the lowest FID and highest R-Precision scores. This setting provides strong trade-off between reconstruction fidelity and generation realism, confirming its configuration.
default
VAE settings
number
rFID 4
FID 4
R-Prec 1
R-Prec@21
R-Prec@31
MM Dist
Ground Truth
0.273
0.452
0.610
0.701
3.755
(4, 1)
0.539
5.290
0.492
0.645
0.717
3.775
(4, 3)
0.503
5.015
0.496
0.647
0.720
3.772
Layer Number
(4, 5)
0.522
5.145
0.489
0.643
0.712
3.780
(4, 7)
0.486
5.312
0.483
0.635
0.707
3.782
(6, 3)
0.479
5.349
0.499
0.650
0.721
3.770
(8, 3)
0.507
5.612
0.483
0.637
0.715
3.784
Table Ablation study on different DHVAE layer settings on InterHuman. The tuple in Layer Number represents the number of individual Transformer Encoder layers and the CoTransformer Encoder Layer, respectively.
VAE settings
number
rFID L
FID
R-Prec@11 1
R-Prec@21
R-Prec@3 f
MM Dist $
1=1
0.503
5.015
0.496
0.647
0.720
3.772
1=2
0.459
5.116
0.501
0.653
0.725
3.770
Latent Size
1=3
0.447
5.269
0.496
0.644
0.721
3.771
1=5
0.433
5.571
0.490
0.640
0.715
3.778
1=7 0.462
5.670
0.486
0.638
0.709
3.782
Table 9: Ablation study on different latent sizes for DHVAE on InterHuman.
6.9 PARAMETERS FOR DENOISER
In this section, evaluate the effect of varying the number of layers in the denoiser network. As shown in Table TO we report the performance changes on both the InterHuman and InterX datasets across several key metries.
We observe that increasing the depth of the denoiser generally improves R-Precision, while FID tends to stabilize slightly deteriorate beyond certain point. To balance generation quality and alignment per-
21

---

Published as conference paper ICLR 2026
formance, we empirically choose 13 layers as the optimal configuration for the denoiser our final model across both datasets.
Datasel
Number of Layers
R-Prec@1
R-Prec@21
R-Prec@3
FID
MM Dist +
Diversity
Ground Truth
0.452
0.610
0.701
0.273
3.755
7.948
1=7
0.486
0.632
0.708
5.301
3.783
7.928
1-9
0.491
0.642
0.718
5.107
3.772
7.970
InterHuman
0.498
0.649
0.721
5.122
3.770
7.988
1=13
0.496
0.647
0.720
5.015
3.772
7.952
1=15
0.493
0.643
0.716
4.983
3.775
7.940
1=17
0.483
0.637
0.704
5.419
3.780
7.993
Ground Truth
0.429
0.626
0.736
0.002
3.536
9.734
1=7
0.439
0.635
0.741
0.384
3.637
9.308
I=9
0.43
0.637
0.742
0.366
3.602
9.157
InterX
1=11
0.440
0.637
0.720
0.335
3,614
9.325
1=13
0.442
0.638
0.745
0.339
3.604
9.378
|=15
0.445
0.642
0.748
0.378
3.600
9.265
1=17
0.435
0.630
0.738
0.380
3.772
9.420
Table 10: Performance with different number of Transformer Encoder Layer
6.10
LIMITATIONS AND DISCUSSION
While our proposed method achieves strong performance across multiple metrics and benchmarks, several limitations remain that highlight important directions for future research.
worth noting that artifacts still exist in our method, especially on the more challenging InterX dataset, which uses the SMPLX PaviaPaaalkoo et al. (2019) representation. This representation more easily lead to flot-sliding, penetrations, or miscontact. Nevertheless, we provide the corresponding visual- izations and animations, and we hope work inspire further improvements for SMPLX-basee datasets.
Artifacts.
Task Scope and Generalization. Our model designed for dyadic human-human interactions (i.e., two- person scenarios). As noted in the main text, this formulation inherently restricts its applicability to broader multi-person contexts, such as crowd interactions or team-based tasks. Although this choice reflects the current availability of HHI datasets- datasets with dynamic or arbitrary numbers of agents remains critical bottleneck for extending this line of work. We believe that with richer datasets, the hierarchical design of our method could be adapted to multi-agent scenarios by incorporating scalable inter-agent modeling mechanisms. fact, one option is to utilize pose estimation methods like Miao et al. (2024) for multi-person scenes and even involve with object Miao et al. (2023) from video.
of which contain only paired interactions-the development of
Physical Plausibility: Contact. Although our contrastive learning strategy improves the plausibility of generated interactions by encouraging well-structured interaction latent space, it does not provide an ex- plicit mechanism to correct fine-graines physical artifacts such subtle penetrations missed contacts. These issues are especially prominent in tasks requiring high-precision contact, like handshakes or object passing. Since contrastive learning operates the latent level and focuses relational alignment, it lacks direct influence over the decoded physical outcome, We consider integrating post-hoc refinement strategies Wu et al. (2025): Li al. (2023): Wang et al. (2024) such Classifier Guidance contact-aware diserimi-
22

---

Published as a conference paper ICLR 2026
nators (e.g., diffuuionn based constraints or physics priors) as promising future directions further enhance physical realism in a controllable way.
Evaluation Metrics and Human Judgment. Our evaluation follows standard protocol using quantitative metrics such FID, R-Precision, Diversity, and Multimodal Distance. However, these metrics originally designed for single-agent generation tasks and may fail to capture important aspects HHI, such as con- tact quality, synchronization, and plausibility under human judgment. In particular, current metrics are not sensitive to issues like missed hand contact or interpenetration, which often diminish the perceived realism of the motion. We advocate for the development of HHI-specific evaluation protocols-sus as plausibil- incorporating human-in-the-loop
ity scoring, contact ratio, or perceptual realism aseesmments-potentiee evaluations or learned quality assessors. This will be integral part of our future work.
Model Efficiency and Training Cost. Although our model is lightweight compared to other large-scale Transformer-bant based generative models, training variational autoencoder with structured latent diffusion remains computationally demanding, especially when scaling to high-resolution motion extending to longer sequences. Exploring parameter-efficient variants or leveraging pretraining across different motion domains could alleviate some of these challenges.
In summary, while our method demonstrates promising results and novel architectural contributions, we view these limitations as opportunities for further exploration. By addressing the challenges above, future iterations of this framework can evolve into a more versatile and human-aligned HHI generation paradigm.
6.11 MORE VISUAIIZATION FOR BOTH HUMER & INTER
Here we visualize more results for both InterHuman and InterX in Fig. [] and Fig. 12 demonstrating the comprehensive high-quality of the HHI sequence. For InterX visualization, not only the main body parts, our DHVAE can generate dedicated gestures as well, suggesting the full body HHI generation ability and generalizability.
6.12 METRICS
FID: Frechet Inception Distance is used to evaluate the dissimilarity between two distributions as d>(N(u, N,,, Σ²)²² || - + + - 2VEE),
(10)
where the j and p' are the feature's mean values of generated samples and ground truth, and Σ and E' are the feature's covariance matrices respectively. This metric measures the distance between two normal distributions, that generated samples and the ground truth. The less the FID is, the better the model performs. Since generated motion results usually contain hundreds or even thousands of frames, FID for motion samples should be calculated with extracted features instead of raw generated motion data and ground truth. In practice, pre-trained action recognition models by Guo et al. Guo et al. (2020) and by Ji et al. Ji et al. (201)) are utilized to extract features from input samples.
Diversity measures the variance of the generated motions across all action categories. From set of all generated motions from various action types, two subsets of the same size Sa are randomly sampled, and their respective feature sets are extracted as "f,,..., (sa) them is given by:
Diversity:
and {
J. The diversity between
S1
Div =
1112
(11)
5.
1=1
23

---

Published as conference paper ICLR 2026
both they take out stones, scissors, and cloth while bending over.
two performers are practicing fencing, while the first one tries to strike, the second one retreats and guards the counter-attach
the two lift their legs to kick each other simultaneously, followed by the first one attempting another assault.
they make a respectful greeting by nodding their heads towards each other.
one person moves forward and strikes the other person with a right hook, and the other person retaliates by striking the other person with their right leg.
one person spreads both arms and prepares to do backbend, and the other person approaches and pushes against one person's 's chest.
Figure 11: HHI motion sequences generated on InterHuman dataset
24

---

Published as conference paper ICLR 2026
One walks behind the other, placing his/er hands the other person's shoulders and carrying him/her on his/ter back. The other person places hands first person's hands, they walk forward together.
The first person raises hands blows a kiss, while the second person Two people stand side by side, simuloneously raising their right hands and posing.
refusal.
One person sits while the other person stands behind him/her, bends over, and makes
peace hand pose.
person to help lift arms the person lying floor
assists hin/her standing up.
The first person extends both hands to cover his/her mouth, leans clase the left ear of the second person, and whispers.
Figure 12: HHI motion sequences generated InterX dataset, including gestures
25

---

Published as conference paper ICLR 2026
Usually, good generated samples are supposed to have similar diversity value as ground truth.
Multimodality: Different from diversity, multimodality measures how much the generated motions diver- sify within each action type. Given set of motions with C action types. For e-th action, we randomly sample two subsets with same size St and then extract two subset of feature vectors fe.......ff.... and {f,,,,.. ... ). The multimodality of this motion set is formalized as
C
S1
Multimodality C =
[fc. - f
(12)
This metric describes how close the relationship is between the motion features and the text features. It is computed as the average Euclidean distance between the generated motion features and corresponding text features. For given text 'T with feature fr and corresponding generated motion M with feature fM. The multimodal distance is given by:
Multimodal Distance:
MMD
lf fM.ill?
(13)
where n is the sample number of text T. In practice, the text features are extracted by a pre-trained text encoder by Guo et al. Guo al. (022a). Multimodal distance also applied for motion generation tasks driven by other modalities than text, such music and audio.
Also known as motion-retrieval precision, calculates the text and motion's 's top K matching accuracy among R documents based on the Euclidean distance. Actually, most works follow the method of Guo al. [uo al. (2022a). For each generated motion, its ground-truth text description and randomly selected mismatched descriptions from the test set form a description pool. This is followed by calculating and ranking the Euclidean distances between the motion feature and the text feature of each description in the pool. Then accuracy will be calculated for the top 1, 2, and places.
R-P R-Precision:
6.13 DETAILED PROOF OF OBJECTIVE FUNCTION
While Equation does model the joint probability p(xa, X;) in a standard VAE framework, we note that in practice, conventional implementations often assume conditional independence given the latent variable z, which represents their global semantic and interaction features, i.c.
p(xa, xz/z) = p(x, (z) p(x; zz).
Under this factorization, the model cannot capture the direct dependencies between agents' motions. There- fore, the statement in the paper that "Equation 1 ignores the conditional dependency between the agents" refers to this practical limitation rather than the mathematical form the joint distribution. Our DHVAE explicitly models such dependencies through its CoTransformer structure. Here assume that under given 2o, Ta and n3 are conditionally independent. The full proof with our assumption is: Start from the marginal liglliikl:oood::
log p(x, log / p(xx: Xb, Za Zb, Zoo dza dzy dz
Introduce the variational posterior:
g(za, Zo. Zo xx) = g(zu xx)9(Zz) x6) g(z0 Za. Zb))
By Jensen's inequality:
log (X,, Xb) 2 Bag log Dog Xb, Za. 26. 20) log g(z,, 2b, 2o x))
26

---

Published as conference paper ICLR 2026
We assume conditional independence structure in the generative process, where the individual motions Ta and are conditionally independent given their respective local latents and the shared interaction latent. Under this assumption, the joint distribution factorizes as:
p( Xa; Nb. Bas 2b, 2o) p(zaa p( ) p(2)) pxxa Bho 2a p(x6 Zao 26);
and rearranging terms yields:
log log Xb) 2 EBy [xg
Zo: ka) log p(xb Zo, 20)] (a))lp(za)) ) - DKL (9(2o xx)])p(z))
-DKL (g(za
DKL (g(zo
Zu, Zol)o())))))).....
6.14
DETAILED DIAGRAM FOR THE DHVAE
Here we show a more detailed version our DHVAE with both the CoTransformer and individual en- coders/decoders.
DHVAE
CoTransformer
Ensader
meminn ewiagg fowwcch them"
taken scale
Transformer
Trannoemer
Trandorreer Ensoder
Drandonnee Emsoder
CLIP Text Escetkr
Suggeen posilorel amboekinng
Trannfarmer
Trannlsrmer
+
CoTrandfaraer
opanater
~ A/l. 13
skip connection
The ation stepo
Inarable query
sample A.
Tranferuer
learnable iabnnt variatae
anai embenting
Transfrreer Decoder
Transforme Decoder
concutenaain skip connection
Peren
Person
Decoder
Figure 13: Detailed diagram of DHVAE
6.15
USE OF LLMS
In this work, made limited use of large language models (LLMs), specifically GPT-5 and Gemini, only for paper polishing and grammar refinement. There are no parts of the research design, core ideas, experimental implementation, analysis, conclusions that generated by the LLMs. All technical contributions, model development, experiments, and interpretations were edited, executed, and validated exclusively by our human authors. The use of LLMs was restricted to improving readability and presentation quality, without influencing the scientific content or originality of the work.
27


Reviews:
[{"review_id": "DrpO80HUtj", "reviewer": "Reviewer_MZDm", "summary": "This paper presents a novel DM-based architecture for interaction generation. Unlike previous approaches that implicitly encode global interactions within the VAE latent space, the proposed method employs a hierarchical encoding scheme to decouple global interaction information from individual motion, thereby explicitly modeling interactions between two agents. To further constrain and refine the learned interactions, contrastive learning is integrated into the VAE training process. Experiments conducted on two commonly used datasets demonstrate that the proposed method outperforms existing approaches across all evaluation metrics.", "questions": "1. For non-contact cases, contrastive learning in Algorithm 1 suggests that negative samples are synthesized by artificially increasing the distance between agents. In this scenario, the agents become even more separated. Why should such samples be considered valid negatives for non-contact interactions?\n2. The model seems smaller than sota methods. Does the proposed architectural modification introduce additional time delay during inference?\n3. Would increasing the latent space dimension improve performance, or is the method robust to this hyperparameter?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well written and easy to follow. The motivation underlying the proposed methodology is clearly articulated and logically developed.\n2. The approach is conceptually simple yet effective. Both the disentanglement of representations and the incorporation of contrastive learning to promote physical plausibility are well justified.\n3. The experimental evaluation is thorough, and the ablation studies effectively demonstrate the contribution of each component.", "weaknesses": "1. The design of the contrastive learning component requires further clarification. It is intuitive to generate negative samples for contact cases; however, for non-contact cases, Algorithm 1 suggests that negative samples are synthesized by artificially increasing the distance between agents. In this scenario, the agents become even more separated. Why should such samples be considered valid negatives for non-contact interactions?\n2. It would be informative to report inference latency for interaction generation in comparison with InterMask and TIMotion. Does the proposed architectural modification introduce additional time delay during inference?\n3. The sensitivity of the method to the latent code dimensionality is not discussed. Would increasing the latent space dimension improve performance, or is the method robust to this hyperparameter?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FXFxPD7HjO", "reviewer": "Reviewer_h5hK", "summary": "This paper proposes an improved VAE structure, namely DHVAE, for better Human-Human Interaction (HHI) synthesis. Specifically, the individual motions in HHI are separately encoded into latent space and then fused, resulting in two latent variables representing individual motions and one latent variable $z_o$ representing global information. These latent variables are used to train a denoiser that learns to generate HHI-representative latent variables from text. Additionally, the paper proposes Interaction Contrastive Learning to train $z_o$ with the goal of achieving physically plausible results. The denoiser design also incorporates corresponding techniques to adapt to the proposed VAE structure. Quantitative results demonstrate advantages over many existing methods.", "questions": "1. The paper's introduction to the denoiser is relatively brief, but mentions the use of AdaLN, SPE, skip connections, and other techniques. Could a framework diagram of the denoiser be provided in the appendix to more clearly illustrate these techniques?\n\n2. Section 6.7 presents experiments on the layer number and latent size of DHVAE. However, no analysis of the latent size impact is provided in that section. Could the authors explain why the FID metric worsens as latent size increases?\n\n3. Why is the parameter count for TIMotion in Table 2 inconsistent with the data in Table 5 of TIMotion's original paper?\n\n4. Based on weakness 4, given that existing HHI generation methods (including the proposed approach) still suffer from poor physical plausibility, why not introduce post-processing strategies [1, 2] or physics-based simulation [3], as has been successfully adopted in human-object interaction works?\n\n\n**References:**\n\n[1] Wang Z, Wang J, Li Y, et al. InterControl: Zero-shot Human Interaction Generation by Controlling Every Joint[J]. Advances in Neural Information Processing Systems, 2024, 37: 105397-105424.\n\n[2] Li J, Clegg A, Mottaghi R, et al. Controllable human-object interaction synthesis[C]//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024: 54-72.\n\n[3] Wu Z, Li J, Xu P, et al. Human-object interaction from human-level instructions[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2025: 11176-11186.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The figures and tables are well-presented, and the appendix is relatively comprehensive.\n\n2. The method shows advantages in quantitative metrics on both InterHuman and InterX datasets.", "weaknesses": "**Major**\n\n1. **The contribution of the main component—DHVAE—has not been sufficiently discussed.** Compared to \"a flat, unified latent representation\" VAE, DHVAE uses three latent variables to represent an HHI sequence. Does this mean that the dimensionality of DHVAE's latent variables is three times that of a standard VAE? For example, on the InterHuman dataset, is D(DHVAE) : D(VAE) = 3×256 : 1×256? If so, are the comparisons in Table 2 and Table 3 fair?\n\n2. **The paper's presentation and formulations lack rigor.** For example, at Line 178, the paper states that Equation 1 \"ignores the conditional dependency between the agents.\" However, Equation 1 actually models the joint probability between $x_a$ and $x_b$. \n\n   Furthermore, **Equation regarding the ELBO of $\\log p(x_a, x_b)$ lacks rigorous derivation.** Without a formal proof showing that this objective is indeed a valid lower bound of the log-likelihood, it should be treated as a heuristic training objective rather than a theoretically grounded ELBO. The authors should either provide the complete derivation or clarify the assumptions under which this bound holds.\n\n3. **Insufficient evidence for the contribution of Interaction Contrastive Learning (ICL).** Table 3 shows that ICL does not significantly help with FID and R-Prec metrics; Table 5 demonstrates that ICL can reduce penetration. However, a critical experiment is missing: proving that ICL does not reduce penetration simply by pushing people further apart—it should be demonstrated that the introduction of ICL at least does not decrease the contact ratio, i.e., maintaining the contact ratio while reducing penetration.\n\n4. **Despite achieving certain quantitative results, the generated effects still exhibit many artifacts based on the provided supplementary materials,** such as penetration, foot sliding, poor contact, and other physically implausible behaviors. This undoubtedly undermines the paper's claim that \"DHVAE achieves superior physical plausibility.\"\n\n**Minor**\n\n5. **The proposed Interaction Contrastive Learning does not demonstrate its contribution in the main experimental section.** Its contribution to physical plausibility is only shown in the appendix (Table 5 in appendix). As a contribution point mentioned in the abstract, is this arrangement appropriate?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "w3qmLzQp36", "reviewer": "Reviewer_p1j7", "summary": "This paper proposes encoding the motions of two people into an interaction-aware variable $z_{o}$ in the latent space, and employs a contrastive learning strategy to improve the quality of interactions during reconstruction.", "questions": "1. The Contrastive Learning itself may destroy the original distance information between the two actors; if in a sequence their inter-distance changes drastically (from far to close or vice-versa), will the contrastive loss still work satisfactorily?\n2. The authors provide abundant visual results to demonstrate the effectiveness of Contrastive Learning, yet **an additional user study would make the claim more convincing**.\n3. In view of the concerns about the scalability of Contrastive Learning and its actual contribution to quality, I tentatively assign a rating of 4. **If the authors supply strong evidence that contrastive learning brings substantial quality gains, can be extended to multi-person scenarios, or notably improves cases with large distance variations, I will raise my rating**.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The use of contrastive learning for two-person interaction generation is theoretically sound and has been validated in other motion generation domains (e.g., sign-language synthesis).\n2. The paper’s core contributions, the global interaction latent variable $z_{o}$ and the CoTransformer are both empirically verified through experiments.\n3. The manuscript is clearly written; the authors’ intentions can be readily followed from the text, and the Contrastive Learning algorithm is presented in a transparent and unambiguous manner.", "weaknesses": "1. The core contribution of the paper is rather narrow: encoding two-person motions into a single latent variable $z_{o}$ improves generation quality, but it does not yield creative or novel motion combinations.\n2. The effectiveness of the contrastive-loss term is not reflected in the numerical metrics; the authors attempt to demonstrate its role through PCA visualizations in Appendix 6.5 (Figs. 7 & 8) and qualitative renderings. **Nevertheless, further evidence is required, as the high-quality results may primarily stem from the KL-divergence regularization.**\n3. Although the method enhances human-human interaction generation, **it offers little insight into scaling motion synthesis to more than two agents**. Moreover, the adopted contrastive-learning strategy poses new challenges for defining multi-person interactions: **how should “contact” and “away” be characterized among multiple individuals engaged in sustained interaction?**", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "C5qtfkuJTB", "reviewer": "Reviewer_7SSr", "summary": "The paper presents a novel approach to generate two person interaction motion from text called DHVAE. DHVAE obtain encoding for both motion then fuse these encoding to obtain an interaction encoding. Then a positive/negative pair of encoding is created to help learning the interaction trought a triplet loss. Finally, diffusion is used to denoise the interaction encoding that can then bit split into two motion encoding for both humans. Extensive experiments show that the method beat SOTA on two commonly used datasets quantitatively and qualitatively.", "questions": "See Weaknesses.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- In the context of the datasets used creating positive and negative encoding is a good idea to limit body penetration.\n- method and implementation are clear and well explained \n- extensive experiment and ablation show the superiority of the method\n- Qualitative results are convincing.", "weaknesses": "- It is not clear how \"contact\" is determined. Is it siply trought distance between the different body part ?\n- While the idea of building positive and negative encoding is nice it is limited to direct physical interaction or simple non physical interaction (which comprise nearly all current interaction datasets). But at the same time all lot of human interaction are indirect  (e.g a person point somewhere and the other turn to look) and a lot more subtle. How would this method works in those contexts ? Especially how will the negative encoding work for non contact action when 45–90cm translation might not \"break the interaction ?\n- In table 1 we see that the proposed method has the lowest multimodality on interhuman while the results are not as bad on interx but there is no discussion of this in the paper. How do these results affect the generation qualitatively ?\n- In the supplementary video we can see that penetration still happens in the sample generated by the proposed method. How could this be mitigated using the positive/negative encoding. An ablation on different value for the various  \"σ's\" used  and the on how contact are decides could have been interesting to see how much the penetration issue can been mitigated.\n- I might be wrong but in in the appendix the authors consider InterLDM [1] \"to be the first application of latent diffusion models to the HHI task\" but [2] appear to be older.\n\n[1]Boyuan Li, Xihua Wang, Ruihua Song, and Wenbing Huang. Two-in-one: Unified multi-person interactive motion generation by latent diffusion transformer. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)\n[2] CHOPIN, Baptiste, TANG, Hao, et DAOUDI, Mohamed. Bipartite graph diffusion model for human interaction generation. In : Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2024", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)