Text2Interact: High-Fidelity and Diverse Text-to-Two-Person Interaction Generation
Authors: Qingxuan Wu
Zhiyang Dou
chuan guo
Yiming Huang
Qiao Feng
Bing Zhou
Jian Wang
Lingjie Liu
Venue: ICLR 2026 Poster (2026)
Keywords: Human Motion Generation; Two-person Motion Generation
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Modeling human–human interactions from text remains challenging because it requires not only realistic individual dynamics but also precise, text-consistent spatiotemporal coupling between agents. Currently, progress is hindered by 1) limited two-person training data, inadequate to capture the diverse intricacies of two-person interactions; and 2) insufficiently fine-grained text-to-interaction modeling, where language conditioning collapses rich, structured prompts into a single sentence embedding. To address these limitations, we propose our Text2Interact framework, designed to generate realistic, text-aligned human–human interactions through a scalable high-fidelity interaction data synthesizer and an effective spatiotemporal coordination pipeline. First, we present InterCompose, a scalable synthesis-by-composition pipeline that aligns LLM-generated interaction descriptions with strong single-person motion priors. Given a prompt and a motion for an agent, InterCompose retrieves candidate single-person motions, trains a conditional reaction generator for another agent, and uses a neural motion evaluator to filter weak or misaligned samples—expanding interaction coverage without extra capture. Second, we propose InterActor, a text-to-interaction model with word-level conditioning that preserves token-level cues (initiation, response, contact ordering) and an adaptive interaction loss that emphasizes contextually relevant inter-person joint pairs, improving coupling and physical plausibility for fine-grained interaction modeling. Extensive experiments show consistent gains in motion diversity, fidelity, and generalization, including out-of-distribution scenarios and user studies. Code will be released at github.com/Qingxuan-Wu/Text2Interact.

Published as a conference paper at ICLR 2026
TEXT2INTERACT: HIGH-FIDELITY AND DIVERSE TWO- PERSON INTERACTION GENERATION FROM TEXT
Qingxuan Wu', Zhiyang Dou! Chuan Guo?, Yiming Huang', Qiao Feng',
Bing Zhou', Jian Wang', Lingjie Liu' 'University of Pennsylvania, The University of Hong Kong, $Snap Inc.
ABSTRACT
Modeling human-human interactions from text remains challenging because it requires not only realistic individual dynamics but also precise, text-consistent spatiotemporal coupling between agents. Currently, progress is hindered by 1) limited two-person training data, inadequate to capture the diverse intricacies two-person interactions; and 2) insufficiently fine-grained text-to-interaction mod- eling, where language conditioning collapses rich, structured prompts into a single sentence embedding. To address these limitations, we propose our Text2Interact framework, designed to generate realistic, text-aligned human-humai interactions through scalable high-fidelity interaction data synthesizer and effective spa- tiotemporal coordination pipeline. First, we present InterCompose, scalable synthesis-by-cone bocompositionn pipeline that aligns HLLM-generale interaction descrip- tions with strong single-person motion priors. Given prompt and a motion for an agent, InterCompose retrieves candidate single-person motions, trains a conditional reaction generator for another agent, and uses neural motion evaluator filter weak or misaligned ture. Second, we propose InterActor, iontero-itttrattion model with word-level conditioning that preserves token-level cues (initiation, response, contact ordering) and an adaptive interaction loss that emphasizes contextually relevant inter-person joint pairs, improving coupling and physical plausibility for fine-grained interaction modeling. Extensive experiments show consistent gains in motion diversity, fidelity, and generalization, including out-of--istributt scenarios and user studies. Code will be released at gitub..omm/ingguua Bal.uuaannnnn/ttat//a Wu/Text2nnteract.
interaction coverage without extra cap-
INTRODUCTION
Modeling realistic and controllable two-person interactions from natural language remains a central problem human motion generation with broad implications for animation, extended reality, and embodied AI. Compared with single-person motion, interactions require (i) plausible individual dynamics, (ii) precise spatiotemporal coupling between agents, and (iii) semantic consistency with text prompts that often specify initiation, response, and contact phases. Despite recent progress in single-person synthesis (Tevet et al., [022b) Zhhou et al., 2024; Wan et al.) 2024 Cong et al.) 2024). current approaches struggle extend to diverse two- person scenarios for two principal reasons. Data coverage is insufficient: two-person corpora are markedly smaller than single-person counterparts (e.g., INTERHUMAN (Liang al. 2024): < 8k sequences vs. HUMANML3D (Guo et al. 2022): > 14k), which limits the support of coordination patterns and degrades out-of-distributinn generalization. Language conditioning is coarse: while interaction captions are long and structurally informative (median 21 tokens in INTERHUMAN vs. 7 in HUMANMI.3D) prior work (Liang et al., 2024. Tanaka Fujiwara 2023) Javed et al.) 2024)
2024 Xu al., 2025 Guo
typically compresses the prompt into single sentence embedding holistically, discarding token-level spatial-temporal cues essential for text-to-inte text-to-interaction alignment.
In this paper, we argue that in text-to-interaction generation, coverage can be obtained by scalable motion composition and faitkfulness can be achieved by fine-grained language supervision and coupling
aappling-awareeee aware objectives. To this end, present Text2Interact, a framework that generates realistic, text-aligned human human interactions via a scalable high- fighliicliity data synthesizer and a streamlined spatiotemporal coordination pipeline.

---

Published as a conference paper at ICLR 2026
(8)
(b)
One swings high kick, while the other blocks with their forearm.
Two people quarraling, pushing and shoving each other.
person playfully object high; The other scrambles to catch it, leaping and stumbling but finally grasping it mid-air.
They are dencing Latin, stomping their feet and swaying their hips.
One person extends hand apology, while the other person their
Two people drilling taekwondo teahnique, including front and back kicks.
Figure I: (a) Our generative two-person motion composition framework, InterCompose, synthesizes plausible and diverse interactions from generated textual descriptions and a single-perye person motion condition (yellow). (b) Our interaction generation framework InterActor generates high-quality and plausible interactions faithful to text. A deeper color indicates later time.
First, we introduce InterCompose, our scalable data synthesizer, based on the view that many interac- tions be composed from aangl-person motion primitives when composition is text text-grounded and constrained by inter-agent geometry and timing. Given a prompt x = (I1,-- .ry and an observed (or synthesized) motion for agent tic alignment with and spatiotemporal consistency with A. Our scalable synthesis-by-come byccoposiitioonn framework aligns LLM-ggnnrated interaction descriptions with strong single-person motion priors produce diverse two-person sequences beyond existing distributions. We pair rich interaction texts with concise single-person summaries via LLM (Liu al. 2024), retrieve candidate ängle-person motions from a pretrained generator (Guo et al. 2024), and train conditional reaction model to produce B given A and æ. neural motion evaluator filters weak or misaligned samples, yielding synthetic data that broadens interaction support without additional capture.
learn conditional generator for agent B that enforces seman-
Second, we propose InterActor to achieve fine-grained ing word-level cross-allention rather than sentence-level conditioning (Liang et al. 2024 Javed et al., 2024] Ruizz-Pncee et al. 2024) in InterActor, allowing motion tokens to attend to all textual tokens and thus preserve cues about initiation, response, and contact ordering. To further effectively capture the detailed body interactions and facilitate spatial-tempon interaction loss that weights inter-person joint pairs according to context-dependent proximity and relevance (e.g., hands handshakes, forearms in sparring), in contrast to diffusion-based baselines that treat all pairs uniformly (Liang et 2024 Ruiz-Ponce al. 2024)..
modeling by employ-
consistency, we introduce an adaptive
Extensive experiments demonstrate state-of-the-art performance of our method in the standard benchmark (Liang al.) 2024) of two-person motion generation, showing that InterActor outperforms prior art in terms of motion fidelity, faithfulness to text, and generalizability, Moreover, our ablation studies validate the effectivvnees of each component in the proposed framework, especially in scenarios where real interaction data is sparse.
To faithfully and comprehensively measure the effectiveness of the InterCompose framework, we conduct user preference study that evaluates the improvement on generation quality and text- aligmment from in-the-will text prompts when InterActor is fine-tuned on synthetic data. This measures improvements in generalizability that are undetected by quantitative metrics calculated from the InterGeen-triied (Liang et al. 2024 evaluator embeddings.
In summary, our contribution is threefold:
A scalable synthesis-and and- filtering strategy (InterCompose) that constructs high-quality, diverse two-person interactions from L.M text priors and single-person motion priors.
2

---

Published as a conference paper at ICLR 2026
A word-level attention conditioning module (InterActor) with adaptive interaction loss for semantically faithful and spatiotemporally coherent two-person generation.
state-of-the-and results on standard benchmarks and superior performance in challenging out-of- distribution settings via a broad user study.
2
RELATED WORKS
2.1
TEXT-TO-HUMAN MOTION GENERATION
Text-to-motion generation aims to synthesize human motion sequences from natural language de-
scriptions (Fan et al.
2024 Tanke et al. 2023 Jeong et al. 2024 Jiang et al. 2023 Guo et al.) 2022
Zhang et al. 2023a Wan et al. 2024 Lu et al. 2024| Guo et al. (024). Carly methods such as
Text2Action (Ahn et al. 2018) and Language2Pose (Ahuja & Morency 2019) utilized GANs and architectures to map text to motion, laying foundational work in this area. Sub- sequent approaches leveraged variational autoencoders (VAEs) for probabilistic generation, including Guo et al. (Guo al. 2022) and TEMOS (Petrovich et al. 2022), which improved motion diversity and fluency. More recent advancements have focused on powerful generative models. Following the wide-spread application of diffusion models in various modalities (Ho al. 2020 Chi al.
soqueece-to-sequeee
2023 Pinaya et al.
2022 Lan et al. 2025), diffusion- based motion generation approaches such as MDM (Tevet al.
2022b) and latent diffusion via MLD (Chen al. (023) significantly improved
motion realism and sample efficiency. T2M-GPT (Zhang et 2023a employed autoregressive trans- formers for grained motion synthesis, while MoMask (Guo et al. 2024) introduced generative masked transformers to enhance fidelity under the autoregressive paradigm. ReMoDiffuse (Zhang ct al. 2023b) further enhanced generation quality by retrieving reference motions from a motion database. Parallel to improving generation quality, increasing attention has been given to controllable text-to-motion generation. Techniques have explored conditioning on spatial trajectories (Shafir et al. 2023 Karunratanakul et al., 2023 Wan et al. 2024] Xie et al., (023) and linguistic constraints (Wan et al. Huang et al.] 2024) to provide more precise control over generated outputs. Addition- ally, MotionCLIP (Tevet al. 2022a) aligned motion and language embeddings in a shared space, enabling zero-shot text-to-motion generation. Despite stellar results in single-person motion genera- tion, extending them to two-person interactions introduces additional challenges such as modeling inter-agent coordination and handling semantically richer text descriptions. Our work builds on these foundations by proposing scalable framework that composes diverse and semantically aligned two-person interactions from single-pent -ellee motion priors and language models.
2.2 HUMAN-HUMAN INTERACTION GENERATION
Although some progress has been achieved in multi-human interaction modeling (Fan et al. 2024 Tanke et al.] 2023; Jeong et al. 2024), prior works on human interaction modeling have been mostly focused on the two-person interaction problem. A pioneer work, ComMDM (Shafir et al.) 2023), explores two-person motion generation by using a bridge network to compose the outputs of two single-person motion diffusion models (Tevet et al. 2022b). RIG (Tanaka & Fujiwara 2023) and InterGen (Liang et al.) 2024) first trained dedicated networks to directly model two- person interaction. in2IN (Ruiz-Ponce et al., 2024) explores the simultaneous interaction descriptions to enhance textual alignment and generation quality. MoMat-MoGGas (Cai al.) 2024) proposes enhance generation quality by retrieving from a motion database and generative framework that models interactive behaviors between agents, considering personality. motivations, and interpersonal relationships. InterMask (Javed al. 2024) utilizes the generative masked transformer architecture and spatial-temporal attention to enhance generation quality and text-motion alignment. TIMotion (Wang et al. 2024), contemporaneous work, proposes to model the human interaction sequence in causal sequence, leveraging the temporal and causal properties of human motions. Although these methods have achieved impressive results, there remains significant possibilities of improvement due to their common flaw of limited training corpus and inadequate text modeling granularity. In this paper, we aim to tackle these two key issues with our generative interaction composition framework and fine- grained word-level conditioning module.
of individual and
3

---

Published as a conference paper at ICLR 2026
(a) InterCompose
Semple two person and correspondngg single: person toxt descriptions
2 Generate single-prson person motion from single prompt
Gonerate reection besed on singe person condition text
Theme: Conflier
Two-Person Prompt: the two guys engage phyyiccl altercation, Single Prompt A: The stops forward egaresshvelyy thrusts their arms out, and plants their feet
Text Condition
Tags: Interse, Prysical, Reference examplesi
Single-Person
Reaction Gen Model
Model
LLM
Single Prompt B: ...
Si Single Mction Condition
Synthesized Interaction
(b) InterActor
first onto
xN blocks
kicks violently towards other
Mator-Mooton Interaciinn
onto the ground.
Word Level Conditioning
Cross Attn
CLIP
Conditioning
Self Attn
Cross Attn
Modion Modon-Moton Interation
Predicted Intaraction
Person B Tokens
Word-tevel
porson
ground
Figure 2: Overview of the proposed frameworks. (a) InterCompose" sample interaction and single- person descriptions via an LLM, generate a single-person motion from a motion prior (Guo et al.) 2024), then compose the second agent with reaction model conditioned the two-person prompt and the motion prior. (b) InterActor: an N-block generator with word-level conditioning and motion-motion interaction. Each block cross-attends motion tokens to CLIP word tokens (Radford) et al., 202T), followed by self-attention and inter-agent cross-attention to model individual motion and interactions.
METHOD
Problem Formulation. Given a text prompt Ct, the task of human-human interaction generation from text involves generating two-person interaction sequence X [x1, x2] ] 2=2xx Nx3 that is both semantically and spatially coherent and faithful to the original text prompt, where x; denotes the i-th person's 's motion sequence, T is the sequence length in frames and N is the number of joints. Following standard practice single human and interaction generation (Guo et al.) 2022] Liang et al.) lip- cyl. where motion state of the i-th person at time I, x is defined as a collection of global joint positions j E IR velocities jー E 3R in the world frame, 61) representation local rotations j" E CR&N in the root frame, and binary foot-ground contact Cy E I
2024. Ponce et al. 2024), we use extended representation formulated as: x;
3.1 INTERCOMPOSE: GENERATIVE TWO-PERSON MOTION COMPOSITION
3.1.1 DATA GENERATION FROM SINGLE-PERSON MOTION AND LANGUAGE PRIORS
To address the limited diversity of existing two-person motion datasets, propose modular pipeline, InterCompose, that synthesizes realistic two-person interactions by sampling coherent wo-person and single-person motion descriptions and composing individual motion sequences generated from the descriptions. Specifically, we first use an LLM (Liu et al. 2024) to annotate the text descriptions InterHuman (Liang et al. 2024). classifying them into a discrete space of coarse- grained themes (e.g. greeting, dancing, conflict) and fine-grained tags (e.g. excited, synchronized, disarm) that further describes the interaction. By systematically combining plausible theme-tag combinations, we can generate interaction descriptions that remain stylistically consistent with InterHuman but span a broader range of behaviors by sampling from the LLM in the joint theme-tag space: TLLM (theme, tags). Then, given generated interaction text C, we decompose into two (thm(thmme, role-specific sub-descriptions (c. cf) using an additional LLM prompt. Each of describes the motion of person i independent of the other, while taking the context information into account. Please refer
4

---

Published as a conference paper at ICLR 2026
to Fig. 2 (a) for illustration. We use (c), ci) to generate corresponding single (X1. X2) via a pre-trained single-person text-to- motion generator MoMask (Guo et al. 2024) trained on sangle-person motion datasets (Guo et al. 2022). enabling it to generate motions beyond the single-person motion distribution of InterHuman (Liag et al. 2024).
single-person person motions
To model dependencies between the interactants, train conditional diffusion model Dy that synthesizes the second agent's motion X2 given the first agent's motion X and the shared interaction description Ct. Formally, we model the conditional distribution pa (x2 x2) using a denoising diffusion probabilistic model (DDPM) with an 8-layer Transformer architecture. At training time, Da aims to recover an interaction sequence (x1,Xg) sampled from InterHuman (Liang et al.) (024)
from one ground -ouundtruthhan and one noised interactant (x1. x2). During inference, we sample using MoMask then generate X2 using DD conditioned on x1, producing a complete interaction (X1, X2) that is semantically aligned with c and physically coordinated.
This compositional approach significantly enlarges the diversity of two-person interactions compared to existing datasets, as it decouples single-person motion priors and recombines them under guided conditions. Unlike direct generation approaches, which must learn joint coordination from sparse data, our formulation leverages both rich single-person priors and role-specific semantics to scaffold plausible and varied interactions from structured textual prompts. In addition, the inference- based nature of our data composition process allows it to be extremely scalable and cost-efficient compared to the traditional MoCap-based data collection process.
3.1.2 HIGII-QUALITY AND DIVERSE DATA FILTERING.
To ensure the quality and diversity of the synthesized two-person motions, we propose a two-stage filtering pipeline that considers text-motion alignment and distributional regularization. We first train a contrastive encoder using the InterHuman (Liang et al. 2024) two-person interaction dataset to project both text and motion into shared embedding space. Specifically, we freeze pretrained text encoder (CLIP (Radford et al. 202T)) with a trainable Transformer (Vaswani et al.] 2017) feature extractor head fhead, and learn a motion encoder fo based on the Transformer architecture. The training objective is a symmetric cross-entropy (CE) loss over cosine similarities between normalized embeddings. A held-out subset of the InterHuman dataset is reserved provide reference embedding distribution for diversity filtering.
After training, we apply the encoder to the synthetic dataset Day [(xx(( and compute the cosine similarity between each motion and its paired text. We discard samples with similarity scores below threshold 6 0.58, empirically chosen based on performance on a validation split, This step eliminates low-quality or semantically misaligned samples.
To further enforce motion diversity and promote high-quality samples that are underrepresented in the original two-person dataset, we perform a distributional filtering step using the two-person motion embeddings from the held-out InterHuman subset & = [xo(x)) as reference. For each synthetic motion embedding [x),, we compute its Fuclidean distance to the & nearest neighbors in Eraal and retain only those whose average distance falls within a predefined anmulus: Tmin S d((f(x)). Eraa)) <
Tmax- This preserves synthesized motions that are novel (outside the inner radius Tmin) but not far from the real data distribution (inside the outer radius Tmmx). Please refer to Sec. Appendix for more details and an illustration of the distributional filtering process.
Sec. BB. and Fig. 13 in
This dual- stage filtering framework ensures that the final synthetic dataset exhibits both semantic fidelity and distributional diversity. Detailed analysis of the effects of 6, r mmins and I" Fmax is in Sec. 4.3
3.2
INTERACTOR: FINE-GRAINED INTERACTION MODELING
3.2.
WORD-LEVEL ATTENTION MODELING OF LANGUAGE AND INTERACTION DYNAMICS
Having diversified our training distribution with synthetic data, we now address the issue of insuffi- cient granularity in two-person text semantics modeling. To tackle the issue and improve semantic alignment between natural language and generated motion, we design cross-attention-base word- level text-motion conditioning architecture that injects fine-grained text information throughout the generation process. Unlike prior methods that inject sentence-level embedding into motion tokens via AdaLN (Liang et al. 2024 Javed et al. 2024 Ponce et al., 2024) or sentence-level cross-attention (Tanaka & Fujiwara 2023), our architecture allows each motion token to dynamically
5

---

Published as a conference paper at ICLR 2026
attend to individual word-level tokens, preserving the nuanced motion semantics and apatial-tempora alignment cues in semantic-riee -rch interaction prompts.
Formally, given tokenized interaction description Ct {wi...., WL), extract word-level embeddings T { ...... } using a frozen CLIP text encoder. The architecture is composed of alternating processing modules, each consisting of two types: 1) Word-level Conditioning Module Mu: A Transformer block with cross- attentitnnio between single agent's motion tokens x; and the full text embedding sequence T, enabling each motion token to focus on semantically relevant parts of the prompt. This block preserves temporal resolution and injects lexical cues aligned with event structure. 2) Motion-Motiin Interaction Module Mm A two-stage module where motion tokens X; first perform self-attention over their own sequence (intra-agent context), followed by cross-attention over the other agent's 's motion tokens x, (j * i), which models inter-agent physical and temporal dependencies such as push-put pull or synchronization. Please see Fig. 2 (b) for an illustration.
Each update step consists of a word-level conditioning module followed by motion motion interac- tion module; these two modules together form full block that is applied an alternating fashion: first to one agent, conditioning the next step. Leveraging the symmetry of two-person interactions, the blocks B,, and B,, are shared across agents, ensuring architectural symmetry and parameter efficiency. The alternating structure allows each agent to respond adaptively to both the linguistic description and the dynamic behavior of their partner, while preserving causal and temporal coherence.
the text and the other agent's motion, and then to the other agent in
Overall, the network design enables high-fidelity generation that is both semantically grounded and interaction- aware. allowing nuanced conditioning through the word-level representation and fostering motion patterns that are faithful to the described scenario.
3.2.2 ADAPTIVE INTERACTION SUPERVISION
We use the standard velocity loss Lvel, foot contact loss Lrort bone-length loss CBL, and relative orientation loss CRO. For these objective functions, refer to InterGen (Liang et al. 2024) for details.
In addition to the above objective functions, we designed a new objective LAdalmeerntt to enhance the generation of plausible interaction semantics, crucial element of eex-to-interraction generation. Motivated by the insight that joint pairs that are closer to each other carry more importance in the interaction semanties, we propose a novel adaptive interaction loss that supervises the pairwise distances between human-human joint pairs with an adaptive weighting:
N
Cadalnteract
dig + |Ides diglle
1
(I)
Where dij, dij are the ground-truth and predicted distances between the joints i and j respectively, and == 0.1 is an empirically set constant. By putting more emphasis on spatially proximate inter-agent joint pairs, our adaptive interaction objective function provides strong guidance for the model to adhere to the interaction semantics.
4 EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Dataset. We use the InterHuman (Liang et al. 2024) dataset for training and evaluating our model. InterHuman contains 6,022 two-person interacting motions and 3 textural descriptions per motion in the training split, and 1.177 two-person interacting motions in the test split. Additionally, a synthesized dataset of 25,000 text-motion pairs before filtering and 1,200 text-motion pairs after filtering is used for fine-tuning. All models are first trained on the InterHuman training split. For fine-tuning, the model is fine-tuned on the InterHuman training split augmented by the filtered synthetic dataset. All metrics are calculated using the InterHuman test split.
Metrics. Following standard practice in human human interaction generation Liang et al. 2024
Ruiz-Ponce et al., 2024) Javed et al.) 2024: Cai et al. 2024), we use the R- R-Precision (Top- I. 2, 3), Frechet Inception Distance (FID), Multimodal Distance (MM Dist), Diversity, and Multimodality

---

Published as a conference paper at ICLR 2026
R Precision
Method
FID
Dist.
Diversily
MModalily
Top
Top 2
TTp
Ground Truth
0.452 nos
0.610 o0e
0.701 o0s
0.273 007
3.755 cos
7.94g=-061 061
T2M (Gio et al
0.238
0.325#.0.0
04464=014
13.700-0072 007
5.731 015
7.046##022
1.387
MDM level 2022h
0.153
0. 26000
0
9.167
7.125 ous
7.602*-045
2.350
ComMDM Shallr et dl X023
0.223*.000
0.3331
0.166=000
7.069*.054
6,212
7.204=
1 1.822
Tanaka Füjiwar 2016
0.285
0.100
0.621
6.775
6.876 o012
7.311
2.0965.006
Intertien (1 Lann 2134)
0
0.515
0.624=010
5.918t.07m
5.108
7.387
2.141
MoMat MoUlen Cai a
2024
0.4494-oor
0.591 L.occ
0.686 m04
5.6744.055
3.790
8.021 3.330
1 1.295 Lo03
in2IN [Ruiz-Ponce 4044
0.125
0.5 0.576
0.002
5.535 h.13
3.803
3.803+-002 0002
7.953
7.953
1.215#.028
InterMask (Javed 2134
0.44g
0.50g
0.683 0044
5.154 061
3.790 17700--0o0 0012
7.944
1.737
Ours
0.483
0.638
0.638
0.717
5.191 4.
3.778
7,900
1.051
Table 1: Performance on the InterHuman (Liang et al. 2024) test sets. + indicates 95% confidence interval and + means closer to ground truth the better. Boldface indicates the best result.
(MModality) for evaluation models. Please refer to InterGen (Liang et al. 2024 for the detailed definition of these metrics.
Implementation Details. Our model consists of 12 attention blocks and 12 word-level conditioning blocks, positioned in an interleaved manner. We utilize a frozen
(Radford et al.
2021) model for extracting steps to 1,000 and use cosine noise schedule (Nichol & Dhariwal 2021). The model is trained with NVIDIA A 100 GPUs for 200,000 steps, with a Se-5 learning rate and batch size of 16 with the Adam W (Loshchlovv & Hutter 2017) optimizer, cosine learning rate scheduling, and 1000-step warm-up. During sampling, we use the DDIM (Song al.) 2020) sampling with 50 timesteps. with classifier-free guidance (Ho & Salimans, 2022) weight of 3.5.
the text encoder. We set the number of diffusion (Ho et al.) 2020
4.2 COMPARISON WITH THE STATE-OF-THE-ARTT
Quantitative Comparison. Tab. contains the quantitative comparison between InterActor and state-ofi tate-offthhe-ar methods. Each experiment repeated 20 times, after which the mean and 95% confidence interval of each metric is recorded. InterActor achieves state-of-the-an results all three R-precision metrics, surpassing the previous state-of-the-arr, InterMask (Javed et al., 2024). by significant margin, highlighting the effectiveness of the word-level conditioning design choice in text-motion aligmment. In terms of motion quality, our InterActor also achieves the best MM Distance and the second-best FID, with a small FID margin (0.037) from the state-of-the-arrs InterMask, and surpassing all other prior arts. Notably, our FID is within the 95% confidence interval of InterMask's FID, highlighting an equal level of generation quality from a statistical perspective.
Ours
InterMask
person pulls rope towards oneself while other person is being pulted by towards one person. Eventually, one person emerges the winner tug war.
forwarrd forward leg strikes while moving forward. Once the training session is over, both participants pay respect one another by bowing.
people are training taekwondo by performing
The first holds their face with both hands, white the second kneels down with their right knee and reaches both hands the first
Figure 3: Qualitative comparisons of interaction generation results from InterActor and Inter- Mask (Javed et al., 2024). Our method produces results with better text-motion alignment and more robust to implausible poses. A deeper color indicates a later time.
7

---

Published as a conference paper at ICLR 2026
Qualitative Comparison, We provide qualitative comparison between method and the current state-of-the-art, InterMask (Javed et al. 2024). As shown in Fig. 3 our model exhibits stronger adherence to text, higher robustness, and more plausible interaction semantics.
Specifically, in the first generation result of our method, the agent marked in red successfully pulls the agent marked in green, and the red agent emerges as the winner (frame 5). In contrast, the motion generated by InterMask only exhibits pulling, and does not reflect this final part of the motion. In the second row, the InterMask result exhibits implausible human pose outputs in frames I and and does not reflect the final bowing action, while our model generates plausible results faithful to the complete semantic meanings of the text. In the third row, the kneeling human generated by InterMask again exhibits an implausible human pose in frames 2, 3, 4, and 5. These results highlight our InterActor's 's pose robustness over InterMask, an aspect not adequately measured by the evaluator and the FID metric, while confirming InterActor's lead in text to motion alignment.
User Study Results on InterActor vs InterMask. To compare the generalizability of our InterActor outside the InterHuman (Liang et al. 2024) distribution with the current SOTA, InterMask, we conduct a user study comparing InterActor and InterMask (Javed et al. 2024) using newly generated text prompts from LLM, that follow the format of InterHuman (Liang et al. (024) interaction descriptions but differ in interaction semantics. Then, we generate motions with InterMask (Javed et al., 2024) and InterActor using the synthetic text prompts and request users to compare the text-motion consistency and motion quality between each pair of interactions.
Results are presented in Tab. 4 With 81.1% of users favoring Text2Interact's text-motion consistency and 80% of users favoring Text2Interact's 's motion quality, our method presents an overwhelmingly superior -utto--distributton semantic generalization result compared to InterMask.
4.3 FURTHER EVALUATION
Method
Procision?
FID
MM Dist.
Diversity++
MModality
Top 1
Top 2
Top 3
Before Fine tuning
0.485
0.644 aos
0.721 4 000
h.07
7.904 cus
1.081 2.019
Finctune (0.25
0,485 oue
0.641 001
0.717
5.981 006
3. 7.7788 001
7.916 on1
1.080 0as
Finetune (0.3 < d < 0.6)
0.450
0.035 nod
0.715
5.682
3.779
7.909
1.058#009 080
Finetune (0.35 < d < 0.6)
0,483 ous
L.63
0.717 ons
5.191 4.005
3.778 4.77
7.900 com
1.051 4.0
Table 2: Quantitative Results of InterActor after fine-tuning on synthetic data generated by InterCom- pose. d denotes the Fuclidean distance between synthetic data sample point and its closest held-out data point in the embedding space of the neural evaluator.
Quantitative Evaluation of TuninTunig and Filtering, We present the results of fine-tuning the
model on combined dataset consisting of InterHuman (Liang et al. 2024) training split and filtered synthetic data for 50k steps, with a learning rate of 5e-6. As shown in Tab. [ the model exhibits a similar level of text-to-motion matching (R-Precision) after fine-tuning and significantly improved FID in the best case, highlighting the improvement in generalizability. Notably, the FID exhibits a clear increasing trend when the minimum Euclidean distance d of the filtering process is increased within a reasonable range, confirming the effectiveness of the proposed filtering pipeline in achieving synthetic data quality and diversity at the same time. The results also indicates that the increased dataset diversity by synthetic data improves the model's generalizability.
Qualitative Visualization of Synthetic Data. In Fig. we present exemplar results of our data synthesis pipeline, InterCompose. The agent marked yellow is generated by the single-person motion generator (Guo et al., 2024) while the agent marked blue is generated by the reaction generation model. As shown in the figure, our pipeline synthesizes high-quality and diverse motions from single-persee motion and text descriptions, with close adherence to the text. Moreover, the textual deseriptions resemble real-life numan-human interaction situations with emotional interactions or inter-person collaboration instilled into the text prompts.
User Study Results Fine- Tunin.. We present the user preference study results conducted with 51 participants on 10 samples generated with Fig. 5 shows the users' strong preference for the model after fine-tuning. in terms of both motion
texts using our data generation pipeline.
8

---

Published as a conference paper at ICLR 2026
first person waves their arms wildty, while the other
people sweep pushing the other, who gathers neatly into dustpan.
theirs frustration.
One jumps startling other person rushes catch
One backs away slowly, guiding other tray drinks, their
Figure 4: Qualitative samples of InterCompose. Prompts are synthesized by an LLM (Liu al. 2024). The yellow is synthesized by the single-person motion generator, while the blue is generated by the reaction model with the yellow as the condition. A deeper color indicates a later time.
quality and text-motion matching. This result confirms our model's improved generalizability to out-of-distribution samples after fine-tuning.
User Study Results on InterCompose vs InterActor. Fig. presents a user preference study between motions synthesized with InterCompose and generated by InterActor. Two studies are conducted for InterCompose (a) without and (b) with distributional filtering, where only high-quality and novel motions retained after the filtering process. The results show that: (a) InterActor shows stronger consistency in motion quality compared to InterCompose, rendering the former suitable for general text-to-inte -exttto-ntteraction tasks; (b) with the distributional filtering step, motions from InterCompose have higher quality compared to motions generated by InterActor, confirming the quality of the synthesized and filtered motion dataset used for fine-tuning.
User Study Results Scaling, To further investigate the results on the quantity of scaling data, we generated an additional 25k pairs of synthetic data with InterCompose and performed the filtering process described in Sec. 3.1.2 resulting in additional 1.2k pairs of data, bringing the total amount of filtered synthetic data from 1.2k to 2.5k.
To investigate the effect of scaling synthetic data model generalization, fine-tune InterActor the 2.5k synthetic interaction pairs combined with InterHuman (Liang et al. 2024) data and compare it with the model fine-tuned on 1.2k synthetic pairs combined with InterHuman. We conducted user preference study to evaluate the generated interactions from the two models using LLM- -eg--eertdeed synthetic text prompts. The results Tab. show that the users clearly prefer the interaction generated by the model fine-tuned with 2.5k data in terms of both motion quality and text-motion consistency. This highlights the scalability of our pipeline and its effectiveness in improving generalizability.
Comparison With and Without Finetuning
Comparison between InterCompose and InterActor
InterCompose InterActor
IntorCumpose InterActor
Finetuning
With Finetuning
User Preference EN)
Preferance
56.0%
User Preffrnnnce
60
62.7%
80
40
42.7%
40
38.7%
40
20
20
Quality (b) With Filtering
Toxt-Motion Matching
Motten
Without Filtering
Figure S: User preference study re- sults of InterActor with and without fine-tuning on synthetic data.
Figure 6: Comparison motion generation results using In- terCompose and InterActor, (a) without filtering, and (b) with filtering. The motion quality and text-motion matching of In- terCompose surpass InterActor only after filtering.
Ablation Study. After thoroughly investigating the effectiveness of the data synthesis and fine- tuning pipeline, now analyze the effectiveness of the proposed sub-components: the word-level

---

Published as a conference paper at ICLR 2026
conditioning module (WLC), the adaptive interaction loss (AIL), and the synthetic data fine-tuning process (FT). When removing the word-level conditioning module, we replace it with sentence- level condition injection by AdaLN; for ablation of adaptive interaction loss, replace it with the flat-weight distance map function LDM proposed by InterGen (Liang et al. 2024); for ablation of fine-tuning, the model trained only on InterHuman (Liang al., 2024) without the fine-tuning process. Tab. shows significant improvement of our model after adding each proposed component, terms of text-motion matching (R-Precision), FID, Multimodal Distance, and diversity.
R Precision?
Method
FID
MM Dist.
Diversity
MModality1
Top
TTop2
Top 3
All Proposed Components
0.441 #
0.608= 005
0
6.237
3.781
7.959
1.068
w.o. AIL, FT
0.48441.00
0.632- ons
0. 0.710-1.00
3.779 6.1921..00
7.853
1.081
wo. WL.., FT
0.4844 900
0.629= 005
0.711 000
5.877 061
3.779 001
7,851 #.044
0.996
w.o. FT Ours
0,485 1.4855-mm
0.611
0.721 -o00
5.701
3.777
7,901
1.046 1.066-o0z
0 0.183
0.638
0,717
5.191 9.055
3.778 001
7,900
1.051
Table 3: Ablation Study: Effect removing one or more the proposed components: Adaptive Interaction Loss (AIL), Synthetic Data Fine- Fine-Tuning (FT), Word-Level Conditioning (WLC).
Metric
InterActor
InterMask
Metric
Finetune 1.2k
Finetune 2.5k
Text-Motion Consistency Motion Quality
81.1 %
18.9%
Text-Motion Consistency
30.5%
69.5%
80.0%
20.0%
Motion Quality
35.8%
64.2%
Table 4: User Preference Study Results from 19 independent evaluators evaluating gener- ation results on out- of-iisirbiuuti prompts, comparing InterActor and InterMask.
Table 5: User Preference Study Results from 19 independent evaluators evaluating gener- ation results on out-of-distribtioon prompts, comparing InterActor fine-tuned with 1.2k and 2.5k pairs of synthetic interactions generated by InterCompose.
5 CONCLUSION
In this paper, presented InterCompose, novel and effective framework that composes single- person motions into two-person interaction from LLM-generated text descriptions, and InterActor, a high-quality and fine-grained two-person interaction generation framework equipped with word- level conditioning. The effectiveness of InterCompose has been confirmed by an ablation study, user study, qualitative results, and latent visualizations. Utilizing data generated by InterCompose, InterActor achieves significant FID boost, achieving SoTA-level R-precision and FID, setting new state-of-the--rt for the two-person motion generation task.
I Limitations and Future Work. While InterActor demonstrates strong motion fidelity and faithful- ness, it does not account for physical plausibility during generation, which can result artifacts such as floating motions and ground penetration. Incorporating physics priors offers promising avenue for future work. Additionally, although InterCompose provides an effective synthesis framework. extending it to learn motions directly from video remains an interesting and relatively unexplored direction.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Hyemin Ahn, Timothy Ha, Yunho Choi, Hwiyeon Yoo, and Songhwai Oh. Text2action: Generative adversarial synthesis from language to action. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 5915-5920, IEEE, 2018.
Chaitanya Ahuja and Louis-Philippe Morency. Language2pose: Natural language grounded pose forecasting. In 2019 International conference on 3D vision (3DV), pp. 719-728. IEEE, 2019.
Zhongang Cai, Jianping Jiang, Zhongfei Qing, Xinying Guo, Mingyuan Zhang, Zhengyu Lin, Haiyi Mei, Chen Wei, Ruisi Wang, Wanqi Yin, et al. Digital life project: Autonomous 3d characters with social intelligence. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 582 592, 2024.
Xin Chen, Biao Jiang, Wen Liu, Zilong Huang, Bin Fu, Tao Chen, and Gang Yu. Executing your commands via motion diffusion in latent space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 18000-18010, 2023.
Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. In Robotics: Science and Systems, 2023.
Peishan Cong. Ziyi Wang. Zhiyang Dou, Yiming Ren. Wei Yin, Kai Cheng. Yujing Sun, Xiaoxiao Long, Xinge Zhu, and Yuexin Ma. Laserhuman: Language-guin guided scene-aware human motion generation in free environment. arXiv preprint arXiv:2403. 13307, 2024.
Ke Fan, Junshu Tang, Weijian Cao, Ran Yi, Moran Li, Jingyu Gong, Jiangning Zhang, Yabiao Wang, Chengjie Wang, and Lizhuang Ma. Freemotion: A unified framework for number-free text-to-motion synthesis. European Conference on Computer Vision, pp. 93-109. Springer, 2024.
Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng. Generating diverse and natural 3d human motions from text. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 5152-5161, 2022.
Chuan Guo, Yuxuan Mu, Muhammad Gohar Javed, Sen Wang, and Li Cheng. Momask: Generative masked modeling of 3d human motions. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1900-1910, 2024.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-68511 2020.
Yiming Huang, Weilin Wan, Yue Yang. Chris Callison Callison-Burch, Mark Yatskar, and Lingjie Liu. Como: Controllable motion generation through language guided pose code editing. European Conference on Computer Vision, pp. 180-196. Springer, 2024.
Muhammad Gohar Javed, Chuan Guo, Li Cheng, and Xingyu Li. Intermask: 3d human interaction generation via collaborative masked modelling, arXiv preprint urXiv:2410.100 10010, 2024.
Jaewoo Jeong, Dachee Park, and Kuk-Jin Yoon. Multi-agent long-term 3d human pose forecasting via interaction-aware trajectory conditioning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 1617-1628, 2024.
Biao Jiang, Xin Chen, Wen Liu. Jingyi Yu. Gang Yu, and Tao Chen. Motiongpt: Human motion foreign language. Advances in Neural Information Processing Systems, 36:20067-2007, 2023.
Korrawe Karunratanakul, Konpat Preechakul. Supasorn Suwajanakorn, and Siyu Tang. Guided motion diffusion for controllable human motion synthesis. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 2151-2162, 2023.

---

Published as a conference paper at ICLR 2026
Zitong Lan, Yiduo Hao, and Mingmin Zhao, Guiding audio editing with audio language model. arXiv preprint arXiv:2509.21625, 2025.
Han Liang, Wenqian Zhang, Wenxuan Li, Jingyi Yu, and Lan Xu. Intergen: ceffusion-basee multi- human motion generation under complex interactions. International Journal of Computer Vision, 132(0):3463-3488 2024.
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint 1rXiv:1711.05101, 2017.
Shunlin Lu, Jingbo Wang, Zeyu Lu, Ling-Hao Chen, Wenxun Dai, Junting Dong, Zhiyang Dou, Bo Dai, and Ruimao Zhang. Scamo: Exploring the scaling law in autoregressive motion generation model. CVPR 2025, 2024.
Leland McInnes, John Healy, and James Melville. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint /02.v:/802.03446 2018.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference on machine learning, pp. 8162-8171, PMLR, 2021.
Mathis Petrovich, Michael J Black, and Gül Varol. Temos: Generating diverse human motions from textual descriptions. In European Conference on Computer Vision, pp. 480-497. Springer, 2022.
Walter HL Pinaya, Mark S Graham, Robert Gray, Pedro F da Costa, Petru-Daniel Tudosiu, Paul Wright, Yee Mah, Andrew MacKinnon, James Teo, Rolf Jager, et al. Fast unsupervised brain anomaly detection and segmentation with diffusion models. arXiv preprint arXiv:2206.0346) 2022.
Pablo Ruiz Ponce, German Barquero, Cristina Palmero, Sergio Escalera, and Jose Garcia- Modrii-Ro in2in: Leveraging individual information generate human interactions. arXiv preprint arXiv:2404.09988, 2024.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PmLR, 2021.
Pablo Ruiz-Ponce German Barquero, Cristina Palmero, Sergio Escalera, and José García Rodríguez. in2in: Leveraging individual information to generate human intcractions. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 1941-1951, 2024.
Yonatan Shafir, Guy Tevet, Roy Kapon, and Amit H Bermano. Human motion diffusion as generative prior. arXiv preprint wxXiv:2303.01411 2023.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020.
Mikihiro Tanaka and Kent Fujiwara. Role-aware interaction generation from textual description. In Proceedings of the IEEE/CVE international conference on computer vision, pp. 15999-16009, 2023.
Julian Tanke, Linguang Zhang, Amy Zhao, Chengcheng Tang, Yujun Cai, Lezi Wang, Po-Chen Wu, Juergen Gall, and Cem Keskin. Social diffusion: Long-term multiple human motion anticipation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9601-9611, 2023.
Guy Tevet, Brian Gordon, Amir Hertz, Amit H Bermano, and Daniel Cohen-Os Motionclip: Exposing human motion generation to clip space. In European Conference on Computer Vision, pp. 358-374. Springer, 2022a.
12

---

Published as a conference paper at ICLR 2026
Guy Tevet, Sigal Raab, Brian Gordon, Yonatan Shafir, Daniel Cohen- Or, and Amit H Bermano. Human motion diffusion model. arXiv preprint arXiv:2209.149101 2022b.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
Weilin Wan, Zhiyang Dou, Taku Komura, Wenping Wang, Dinesh Jayaraman, and Lingjie Liu. Ticontrol: Trajectory and language control for human motion synthesis. ECCV 2024. 2024.
Yabiao Wang. Shuo Wang, Jiangning Zhang, Ke Fan, Jiafu Wu, Zhengkai Jiang, and Yong Liu. motion generation. arXiv preprint
Temporal and interactive modeling for efficient arXiv:2408. 2024.
Yiming Xie, Varun Jampani, Lei Zhong, Deqing Sun, and Huaizu Jiang. Omnicontrol: Control any joint any time for human motion generation. arXiv preprint arXiv:2310.08580, 2023.
Shuyang Xu, Zhiyang Dou, Mingyi Shi, Liang Pan, Leo Ho, Jingbo Wang, Yuan Liu, Cheng Lin, Yuexin Ma, Wenping Wang, and Taku Komura. Mospa: Human motion generation driven by spatial audio. NeurlPS 2025 Spotlight, 2025.
Jianrong Zhang, Yangsong Zhang, Xiaodong Cun, Yong Zhang, Hongwei Zhao, Hongtao Lu, Xi Shen, and Ying Shan. Generating human motion from textual descriptions with discrete representations. In Proceedings of the JEEF/CVF conference on computer vision and pattern recognition, pp. 14730-14740) 2023a.
Mingyuan Zhang, Xinying Guo, Liang Pan, Zhongang Cai, Fangzhou Hong. Huirong Li, Lei Yang. and Ziwei Liu. Remodiffuse: netrieval-augmennne motion diffusion model. In Proceedings of the IEFF/CVF International Conference on Computer Vision, pp. 364-373, 2023b.
Wenyang Zhou, Zhiyang Dou#t Zeyu Cao, Zhouyingcheng Liao, Jingbo Wang, Wenjia Wang, Yuan Liu, Taku Komura, Wenping Wang, and Lingjie Liu. Emdm: Efficient motion diffusion model for fast and high-quality motion generation. In ECCV 2024. 2024.
13

---

Published as a conference paper at ICLR 2026
SUPPLEMENTARY MATERIALS
A INTERACTOR IMPLEMENTATION DETAILS
A.1 WORD-LEVEL. TOKENIZATION
We use the CLIP (Radford et al. 2021) ViT-L/14 encoder for encoding the text. The text is tokenized by the CLIP tokenizer into word-level tokens for short words and sub-word-level tokens for long words, with <SOT> and <EOT> tokens inserted at the start and end of the text. The maximum number of text tokens is 75. If the number of tokens after tokenization is longer than 75, the text tokens truncated and additional tokens are discarded.
A.2 WORD-LEVEL CONDITIONING
Out Project
Query
Key
Value
timestep
AdaLN
AdaLNN
timestop
Single
Figure 7: Illustration of the Word-Level Conditioning Block
Fig. 7 iillustrates the details of the proposed Word-Level Conditioning block. One of the interacting agents' motion features (Single Motion Features) is passed through an Adaptive Layer Norm (AdaLN) for the injection of timestep information. The word-level tokens are passed through separate AdaLN of the same structure but with different parameters. Then, the normalized and modulated features are passed through the linear layers to yield the query, key, and value tensors, where the query comes from the single motion features and the key and value come from the word-level tokens. Then, an attention output embedding is obtained using query Q, key K, and value V with the attention mechanism /Vaswani et al. 2017):
QKT JTk
Attention (Q, K) softmax
Finally, the output of the attention layer is passed through a linear layer for the reprojection and mixing of the attention head outputs.
A.3 MOTION-MOTION INTERACTION
A.3.1 SELF-ATTENTION
The Self-Attention module in the Motion-Motion Interaction block is responsible for the processing of one the interacting agents' motion features. Fig. B an illustration of this. The motion features are first modulated by an Adaptive Layer Norm (AdaLN) block, which injects timestep information by scaling the features with mean and variance determined by the timestep. Then, projection layers calculate the query, key, and value tensors separately. which are used to calculate the attention output using the attention mechanism (Vaswani et al. 2017). Finally, the attention output is projected with linear output projection layer to give the final output.
14

---

Published as a conference paper at ICLR 2026
Out Pioject
Amanion
Attomtoo
v
Querry
Kay
Value
Query
Key
tmeetep
AdoLN
AdsLN
tiwertpp
Mation
Modion 6
Single Moxon
Figure 8: Illustration of the Self-Attentio mod- ule in the Motion-Motion Interaction Block.
Figure 9: Illustration of the Cross-Attention module in the Motion-Motion Interaction Block.
A.3.2 ATTESTITTNNN
The Pooss-Attetioon module in the Motion-Motion Interaction block is responsible for modeling the inter-agent interaction. Fig. provides illustration. In the Cross-Attention module, the motion features of agent and agent (vice versa) are used to calculate features using separate AdaLN blocks for the timestep information. Then, the motion features of agent A are passed through the query projection layer to give the query features. The motion features of agent are passed through the key and value projection layer to calculate the key and value features. The query, key, and value features undergo an attention mechanism to obtain the attention output feature, which is subsequently projected by linear output layer form the final output.
B INTERCOMPOSE IMPLEMENTATION DETAILS
TWO PERSON PROMPT SYNTHESIS
We use a prompt template for synthesizing diverse and plausible two-person interaction descriptions from an LLM (Liu et al., 2024) based on coarse- grained themes and fine ane-ggrined tags, along with real examples from the InterHuman (Liang et al. 2024) dataset for styling reference. The complete template provided below Fig.
coarse-grained
B.2 SINGLE PERSON PROMPT SYNTHESIS
After obtaining two-person descriptions, we use a separate prompt template to synthesize pairs of single-person descriptions that are self-contained, coherent, and consistent with the given two-person descriptions. The LLM infers the corresponding single-persod motion according to the provided two- person interaction information, while reasoning the plausible single-person motion the two-person prompt does not provide complete information. The complete prompt template is given in Fig. Examples of two-person prompts and corresponding single- single-person prompts are given in Fig. 12
B.3 SINGLE PERSON MOTION GENERATION
We use MoMask (Guo al. 2024) to generate the single-person motion conditions for the subsequent reaction generation. In addition, we trained a length estimator on the InterHuman (Liang et al. 2024) text-motion pairs to estimate the correct length of the corresponding motion given two- two-person motion prompt, and used the predicted length by the estimator to guide the single-perse person motion generation.
For each two-person prompt, we use the LLM (Lu et al. 2024) to provide two prompts for the two interactants and generate single-person motions, with each prompt.
15

---

JSON array of Published as a conference paper at ICLR 2026
You write compact, vivid descriptions of **two-person interactions**.
Each output sentence MUST:
mention exactly two unnamed people ("one person... the other person.... ...son...).). "),,
focus body / arms / legs (ignore faces / fingers / appearance),
be <=25 words,
clearly match the given *Theme* and *Tags*, be entirely different from the examples.
Theme: {theme}
Tags : {tags}
Reference examples (k):
(example1)
(example2)
(examplek)
Now craft [m) brand-new descriptions. strings.
Return **only**
Figure 10: Prompt template for generating two-person interaction descriptions.
Given the following description of two-person interaction:
(two-person text)
Independently describe the motion of each person involved, using only information implied by the full interaction. Do not mention or refer to the other person in either description. Focus only body, arms, and legs ignore facial expressions, fingers, or appearance.
Use "the person" to refer to each. Assume shared context (e.g., dancing, greeting, arguing), but isolate each description.
Output JSON in this exact format:
(["1": (("person1": "(description1)", "person2": "(descriptionn2")]
Each description must be one sentence, <=15 words, specific, and motion-focused with relevant context.
Figure 11: Prompt template for generating single-person interaction descriptions.
B.4 Two PERSON MOTION COMPOSITION
We trained a reaction generation network that uses given sequence of joints as condition Xcond E R7 x2xx3 to generate the reaction E T2x2266 consisting of the full interaction in the complete InterGen [Liagg et al. 2024h joint representation.
The network is trained on the InterHuman (Liang 2024) training dataset with person's joints not noised and all other terms noised to simulate the reaction generation tasks. At test time, the condition joints provided each time step of denoising and after the final denoising step.
16

---

Published as a conference paper at ICLR 2026
Example 1.
two-person text: One person leans back, arms outstretched, while the other steps forward, pressing their chest lightly against the first's, hands resting on their hips.
single-person text A: The person leans back with arms outstretched.
single-person text B: The person steps forward, chest pressed lightly, hands hips.
Example 2.
two-person text: One person claps twice, and the other responds by jumping in place, their legs kicking out wildly wi with exci
single-person text A: The person raises both arms and brings hands together sharply twice.
single-person text B: The person leaps upward, legs swinging outward vigorously.
Example two-person text: One person lunges wi punch, the other person blocks with crossed arms and counters with a swift kick to the thigh. single-person text A: The person steps forward, extending one sharply in a punching motion.
single-person text B: The person raises both arms cross in front, then swings one leg outward quickly.
Example 4.
two-person text: one person steps forward aggressively, arms raised, while the other person backs away, hands outstretched to resist the advancing confrontation.
single-person text A: The person steps forward aggressively with arms raised. single-person text B: The person backs away with hands outstretched to resist.
Example
two-person text: One person stumbles backward from alcohol, and the other person swiftly wraps an arm around their waist to steady them. single-person text A: The person stumbles backward, legs unsteady from clhol.
single-person text B: The person moves an arm quickly wrap around waist.
Example
two-person text: One person shoves the other's shoulder, causing them to stagger, then crosses their arms in defiance as the other retreats. single-person text A: The person extends their arm sharply, then pulls it back and crosses both arms tightly.
single-person text B: The person stumbles backward from sudden force, then turns away while stepping back.
Figure 12: Examples of generated and two-person interaction descriptions with corresponding single- person descriptions.
B.5 MOTION FILTERING
Before filtering, first use the trained neural motion evaluator project all generated motions to the s12-dimensional motion latent space, to compare with the latents of a 500-sample held-out motion dataset. The motion filtering step consists of k-nearest neighbors filtering with maximum 20 nearest neighbors for each sample in the held-out set. We also calculate the distances between each of the nearest neighbors with the held-out motion sample to make sure the distance is in the
17

---

Published as a conference paper at ICLR 2026
(a)
(b)
Text Embedding Space Visualization
Motion Embedding Space Visualization
UMAP Dimension 2
UMAP Dimension
10-
12-
11-
10-
6-
GT Text
GT Motions Synthetic Motions
Synthetic Text
5
10
11
UMAP Dimension
UMAP Dimension 1
Figure 13: UMAP (Melnnes et al. 2018) visualizations of evaluator and CLIP (Radford al. 2021 embeddings of (a) text and (b) two-person motions from the InterHuman (Liang et al. 2024) held-out subset and filtered synthesized dataset.
predefined annulus dmin < dmaz- Finally, we use the neural motion evaluator to filter out all the remaining motions with text-motion cosine similarity less than 0.58, empirically set threshold.
C MOTION EMBEDDING SPACE VISUALIZATIONS
Fig. 13 demonstrates cuimensionality-rddueee visualization of the text and two-person motion embeddings of the InterHuman (Liang et al. 2024) held-out dataset, extracted from CLIP (Radford et al. 2021) and trained motion evaluator. We utilize UMAP (McInnes et al. 2018) (Uniform Manifold Approximation and Projection), a popular dimension reduction technique that preserves the local and global structure of high- nigh-dimensional data in a low- now-dimensional space. As shown in the figure, both the generated text (Fig. 13 (a)) and motion (Fig. 13 (b)) descriptions from our pipeline have good coverage in most high-density areas of the held-out dataset, while covering many underrepresented areas that lacks held-out data samples, highlighting our pipeline's 's capability in enhancing data diversity.
D USER STUDY DETAILS
We conducted user study to evaluate our InterActor model with and without fine-tuning, Human evaluators are asked to choose between 10 pairs of two-person interaction videos and determine the one out of each pair that is more faithful to the text prompt more natural as interaction. Fig. [4] an illustration of how the user study conducted.
E SOCIETAL IMPACTS
Our work on Text2Interact introduces a scalable framework for high-fidelity and diverse text-to-two- person interaction generation. While the technology has the potential to significantly benefit domains such as animation, virtual reality, assistive robotics, and embodied AI, also raises several ethical and societal considerations.
Positive Impacts. The proposed method can facilitate content creation media, education, and human-computer interaction automating the generation of complex, realistic human interactions.
18

---

Published as a conference paper at ICLR 2026
Human Preference Study: Two Person Interaction
groupe
Each group containe 5 videos teo-person Interactions generated from the some text prompt.
For each pair, please first read the text prompt carefully, then evamine the viceos and answer the following questione.
Test Prempt: one person coucces to pick fallen boos, while the other bends elightly to guide into their waitinghands
00147. betifonc person crouches to pick taller book, when
03147 PotiPonn person crouches to piik fal belee book, wiec- the atlicr be ids slightly to quice into Bl walting
the otfic be lis sligltigg Bylltty to quide into tiik motting
Froeee E. End whiieeeeeeeeeeaaooooooo
Nowe
Video Pair
Prempt:
000011
othe
sneaks and protends snaten it, rausing othe
smevts and prstends unalch nausing one
fl and clutch it tighter.
linch and clutch "ighter.
Figure 14: A screenshot of the user study
19

---

Published as a conference paper at ICLR 2026
It may lower the barrier for creating high-quality motion data, particularly for under-resoureed languages or motion types, and help simulate social interactions for training embodied agents or improving accessibility tools for individuals with disabilities.
Risks and Limitations. As with many generative models, there is a potential risk of misuse, such as generating deceptive or misleading content (e.g., synthetic surveillance or manipulated footage). Although our method focuses solely on body motion and excludes facial expressions or identity features, generated motion could still be used out of context or embedded narratives. Additionally, there is risk of dataset bias being amplified if the single-person priors or L.M--generated text reflect culturally specific or stereotyped behaviors. We recommend future users apply careful evaluation and transparency practices when deploying this technology.
misleading visual
Mitigations. Our dataset curation and filtering process emphasizes diversity and alignment with real-world motion distributions reduce representation biases. Furthermore, model does not generate personally identifiable information or faces, and we encourage its use only in applications that respect human dignity, consent, and privacy.
20


Reviews:
[{"review_id": "zzvJSkR909", "reviewer": "Reviewer_LLQ2", "summary": "This work examines the problem of learning two-person human interaction generative models conditioned on text descriptions. The work has two main parts: creating a dataset and learning a model. The dataset aspect involves creation of a large-scale synthetic dataset of two-person human interaction by combining LLM descriptions to prompt existing one-person motion generation algorithms and an interaction model conditioned on the first motion. The interaction model is trained on motion-capture data from interhuman. The synthetic data is then used to train a two-person interaction model. The two-person interaction model uses word-level text embeddings and interleaved attention between self and partner sequences and text descriptions. Experiments show the model can provide more realistic two-person interactions that previous motion generation models for a broader range of text descriptions.", "questions": "What is the purpose and advantage of the second stage model? Why not just focus on the data generation process as an interaction model in its own right? What are the metrics and user preference for the second stage model vs the data it was trained on?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "* The first half of the approach is an interesting way to create a broader range synthetic human motions than what is available in interaction motion capture data. LLM prompting and SOTA single person human motions can cover a fairly wide range of human actions. By learning a conditional interaction model and synthesizing reactions to the single-person generations, a wider variety of interaction scenarios can be synthesized with reasonable accuracy because the single-person motion provides a strong starting point for the reaction model.\n* The quantitative and qualitative results provide reasonable evidence that the proposed method can lead to more effective modeling of two-person interactions.", "weaknesses": "* The main weakness of this work is that learning the generative model in the second stage of the paper does not seem to provide anything beyond the first stage. In the first stage, the work essentially defines a way to sample from the distribution of two-person interactions by making use of existing models/datasets (and learning a new conditional model from Interhuman). I am not sure what the benefit of distilling this approach into a second model is (other shifting from a conditional to joint model, which for the purposes of generation is not a major advantage in my opinion). Furthermore, shouldn't we expect the second stage model to be further from the distribution of natural motions than the synthetic data it is trained on (due to imperfect learning)? And since the training data is already synthetic, what is the purpose of learning a second synthetic distribution and not just focusing on the data generation process as a model on its own?\n* The work still relies on InterHuman as a way to learn human interactions. While the novelty provided by the single person motion can increase variety of interaction situations, at the end of the day the proposed method does not have a way to model interactions that are not quite close to the interaction distribution from InterHuman.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mVisdy2aRx", "reviewer": "Reviewer_yqjY", "summary": "The paper targets limitations in two-person interaction generation: (1) limited real data and weak spatiotemporal coordination, and (2) insufficient semantic grounding from sentence-level conditioning. The authors propose:\n\n- **InterCompose**, a scalable synthesis pipeline that generates and filters synthetic text-motion pairs via two-stage quality and diversity filtering;\n- **Text2Interact**, which employs word-level cross-attention and an adaptive interaction loss emphasizing semantically important cross-human joint pairs.", "questions": "1. How does the model perform when fine-tuned on unfiltered synthetic data?\n2. Can the authors show a performance curve (e.g., FID vs data size) to demonstrate the scalability claim?\n3. Could you explain why R-Precision slightly drops after fine-tuning?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. **Originality:** The two-stage data synthesis and filtering pipeline is novel and intuitive. The word-level attention and adaptive interaction loss are well-motivated and effective.\n2. **Quality:** Achieves SOTA on InterHuman with solid ablations. The user study adds credibility to perceptual quality.\n3. **Clarity:** Writing and figures are clear, and each module is conceptually coherent.\n4. **Significance:** Addresses two key challenges—data scarcity and semantic granularity—in a well-balanced framework.", "weaknesses": "1. **No validation of unfiltered synthetic data.**\n    The paper claims that filtering improves quality but does not report the result of fine-tuning with unfiltered synthetic data. Without this, the necessity of the two-stage filtering pipeline remains unproven.\n1. **No scaling analysis of synthetic data.**\n    Although the synthesis pipeline is described as “scalable,” the paper does not analyze how model performance changes with varying data volume or filtering ratios.\n1. **δ = 0.58 lacks justification.**\n    The threshold is only said to be “empirically chosen” in Appendix B.5, with no validation or sensitivity analysis.\n4. **Minor observation:** In Table 3, the “w.o. FT” row yields slightly higher R-Precision than “Ours.” Some explanation would clarify this trade-off between text alignment and motion realism.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VdDdDdEDOL", "reviewer": "Reviewer_yhm1", "summary": "In this paper, it introduced a scalable synthesis-and-filtering strategy to generate high-quality interaction from LLM and single-person motion prior. Later, it designed a Text2Interact module to generate two-person interaction motion. The proposed method was validated on InterHuman dataset and presented better performance than many previous works.", "questions": "1. Whether the proposed method design a data synthesis method to create high-quality interaction data.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. It shows better performance than many previous works.\n2. The visual performance is better than the baseline.", "weaknesses": "1. The method is only validated on one dataset, the generalization ability of the proposed method is not validated.\n2. Generating the second human motion given the motion of the first person has been utilized in previous multi-person motion generation. The difference should be further discussed.\n3. In the manuscript, the relationship between InterCompose and Text2Interaction should be detailed.\n4. More recent works should be considered for comparison.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yKPkZ200zz", "reviewer": "Reviewer_kXxF", "summary": "This paper introduces InterCompose and Text2Interact. The former synthesizes the motion of one person through text conditioning and single-person motion priors, generating human-human interaction-text paired data to alleviate data scarcity. The latter employs word-level attention to achieve fine-grained text-motion alignment and emphasizes the reproduction of interaction effects through weighted body parts, enhancing the realism of motion generation.", "questions": "1. There appears to be ambiguity in Fig. 2(a) regarding InterCompose. The diagram suggests that the motion of person A is first generated using A's prompt, and then the motion of person B is generated using the reaction prompt and A's motion. However, in Section 3.1.1, line 210, it is stated that A and B are generated separately. Does the paper intend to explain that during the data preparation stage, MoMask is used to generate the motion pair ($x_{1}$, $x_{2}$) as the ground truth? Then, during the training phase, a diffusion model $D_{θ}$ is trained, with inputs including the text prompt and the motion $x_{1}$ of one person, and the training target is $x_{2}$? It is recommended to distinguish between the data synthesis and training processes in Fig. 2(a) for better clarity.\n\n2. The data filtering process described in Section 3.1.2 also lacks intuitive visual explanations. Adding relevant illustrations in the Appendix would significantly reduce the reading difficulty for the audience.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 2, "contribution": 3, "strengths": "1. The paper pioneers the InterCompose framework, which is capable of synthesizing a large volume of dual-human interaction data. The methodology for filtering and selecting the data is also highly rigorous.\n2. Text2Interact designs a sophisticated cross-attention mechanism that injects fine-grained textual information throughout the entire generation process, ensuring precise prompt control and correct temporal resolution.\n3. The rendering results are highly impressive. Text2Interact indeed generates high-quality interactive motions, and we look forward to the authors open-sourcing their work for the benefit of the community.", "weaknesses": "1. Section 3.1.1 of the paper is not clearly articulated and is kind of difficult to follow. **It would be better to clarify the composition method of $x_{1}$ and $x_{2}$, as well as their role in the subsequent training of $D_{θ}$.**\n2. Section 3.1.2 describes the data filtering process. **It is recommended to include more images to reduce the reading difficulty for the audience.**", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)