MoReact: Generating Reactive Motion from Textual Descriptions
Authors: Xiyan Xu
Sirui Xu
Yu-Xiong Wang
Liangyan Gui
Venue: Accepted by TMLR ()

Abstract: Modeling and generating human reactions poses a significant challenge with broad applications for computer vision and human-computer interaction. Existing methods either treat multiple individuals as a single entity, directly generating interactions, or rely solely on one person's motion to generate the other's reaction, failing to integrate the rich semantic information that underpins human interactions. Yet, these methods often fall short in adaptive responsiveness, \ie, the ability to accurately respond to diverse and dynamic interaction scenarios. Recognizing this gap, our work introduces an approach tailored to address the limitations of existing models by focusing on text-driven human reaction generation. Our model specifically generates realistic motion sequences for individuals that responding to the other's actions based on a descriptive text of the interaction scenario. The goal is to produce motion sequences that not only complement the opponent's movements but also semantically fit the described interactions. To achieve this, we present MoReact, a diffusion-based method designed to disentangle the generation of global trajectories and local motions sequentially. This approach stems from the observation that generating global trajectories first is crucial for guiding local motion, ensuring better alignment with given action and text. Furthermore, we introduce a novel interaction loss to enhance the realism of generated close interactions. Our experiments, utilizing data adapted from a two-person motion dataset, demonstrate the efficacy of our approach for this novel task, which is capable of producing realistic, diverse, and controllable reactions that not only closely match the movements of the counterpart but also adhere to the textual guidance. Please find our webpage at https://xiyan-xu.github.io/MoReactWebPage.

Published in Transactions on Machine Learning Research (09/2025)
MoReact: Generating Reactive Motion from Textual Descriptions
Xiyan Xu University of Mlinois Urbona- Channn-g
riyonauu@illioois.c edu
Sirui Xu
siruimu26Hllinooissccc edu
University of Blinois Urbana- Chaana--aaapiggn
Yu-Xiong Wang
yno@ilknois. edu
University of Blinois Urbona- Chaam-onn
Liang- Yan Gui
lgui@illinois. edu
University of Ilinois Urbana- Caaanaa-haapiggn
Reviewed on OpenReview: o/penreveee net/ forum? id- 4zuT73heqm
Abstract
Modeling and generating human reactions poses a significant challenge with broad applica- tions for computer vision and human-computer interaction. Existing methods either treat multiple individuals as a single entily, directly generating interactions, or rely solely on one person's 's motion to generate the other's reaction, failing to integrate the rich semantic infor- mation that underpins human interactions. Yet, these mothods often fall short in adaptive responsiveness, i.e., the ability to accurately respond to diverse and dynamic interaction scenarios. Recognizing this gap, our work introduces approach tailored to address the limitations of existing models by focusing text-driven human reaction generation. Our model specifically generates rcalistic motion sequences for individuals that responding to the other's actions based on a descriptive text of the interaction scenario. The goal is to produce motion sequences that not only complement the opponent's ' movements but also semantically fit the described interactions. To achieve this, we present MoReact, diffusion- based method designed to disentangle the generation of global trajectories and local motions sequentially. This approach stems from the observation that generating global trajectories first is crucial for guiding local motion, ensuring better alignment with given action and text. Furthermore, we introduce novel interaction loss to enhance the realism of generated close interactions. Our experiments, utilizing data adapted from two-person motion dataset, demonstrate the efficacy of our approach for this novel task, which capable of produc- ing realistic, diverse, and controllable reactions that not only closely match the movements of the counterpart but also adhere the textual guidance. Please find our webpage at httpp:://iiyannzuggithbbbbbbbb wtw.gggtthw..ooooeettttttteeeeee
1 Introduction
Humans able to naturally interact with one another, adopting appropriate actions based their inten- tions and the movements of others. For instance, when someone extends their hand, we understand it an invitation for a handshake, allowing us to react accordingly either by engaging in the handshake or choosing to ignore and walk away. Since interactions are a fundamental aspect of the real world, reproducing this behavior pattern virtual characters holds profound implications for various fields, such as robotics, animation, VR/AB and healthcare. It enhances realism and functionality across these fields, enriching user experiences and interactions with technology.

---

Published in Transactions on Machine Learning Research (09/2025)
The two guys punch each other with their fists
One falls down, and the other rushed them
Two persons looking phone game together
original motion
add noise local motion
add noise to bolyy metion
add noise global traj.
(a)
(c)
Figure 1: (a) Our model, MoReact, learns to generate lifelike reactions, represented by the blue mesh, based the textual description and the actor's motion, represented by the red mesh. (b) As an important motivating analysis for developing our approach, we introduce noise of the same scale to local motion, full-body motion, and global trajectory, respectively. The results indicate that the precision of the global trajectory has a greater impact on the perceptual realism of the reaction. trajectory's significant influence on the motion's semantic information in certain scenarios, such as fall actions.
(c) We demonstrate global
However, achieving realistic generation of human reactions is significant challenge. Current approaches to Hunan-Human Interaction (HHI) generation have not yet fully captured the complexity of human reactions, particularly in generating adaptive reactions to diverse and dynamic social interactions. Current studies, while striving to generate reactive motions for variety of actions (Ghosh et al. 2004 Xu et al. 2024a b Liu
et al. 2024 Tan et al. 2025 Liu al.) 2023 Cen al. 2025 Cong al. 2025,, typically lack integration of the rich semantic information that underlies human interactions, such the intent conveyed text descriptions.
Addressing these limitations, propose novel approach that focuses on enhancing text-driven luman reaction generation. Specifically, we aim to more accurately generate one's (named reactor) reaction based on both the motion of another individual (named actor) and a corresponding textual description, as shown in Fig. Πa). This approach not only seeks to align generated reactions with physical movements but also with the semantic context provided by text, presenting unique challenges maintaining both alignment and authenticity of interactions.
Naïvely adapting existing text-to-motie (Shafir al., 2023) text-to-interaction (Liang et al. 2024 models fails to generate high-quality reactions based text descriptions, often resulting in artifacts like interaction misalignment. The underlying reason is that these methods struggle to adequately model the relationship belween global trajeclory and local motion. To begin with, most existing methods for two-person interaction generation either overly focus on local motion or treat these two equally. However, our analysis indicates that the global trajectory serves the foundation for both local motion and interaction realism. Incorrect global trajectory makes difficult for the local motion align with the action and text description, having more detrimental impact interaction realism than incorrect local motion. As shown in Fig. 1b).
2

---

Published in Transactions on Machine Learning Research (09/2025)
minor deviations in local motion have little impact on interaction quality, while adding the same scale of deviations in global motion leads to the reaction being misaligned, though the single-human motion can still be reasonable if the interaction is not considered. Furthermore, most existing methods neglect the substantial inffuence the global trajectory on the semantics of local motion; instead, they persist in exploring a broad spectrum of potential reactions, rather than locusing on a more confined and plausible subspace delineated by the global trajectory. For example, as shown in Fig. c), the descent of the global trajectory to a lower height suggests that the reactor ought to fall, rather than stand or walk, to achieve a lower position.
Building upon these observations and insights, we propose novel framework, MoReact, to tackle the text-driven human interaction generation task. MoReact incorporates two primary components: (1) Trajec- tory Diffusion Module: This diffusion-based generator predicts the reactor's global trajectory based on the text description of the interaction and the actor's motion. (2) Full-Body Motion Diffusion Module: This diffusion-based generator incorporates text, actor's motion, and reactor's trajectory from the Trajectory Diffusion Module as inputs to synthesize the reactor's full-body reaction. It further enhances the reaction's realism by incorporating the reactor's trajectory information into the denoising process through an inpaint- ing mechanism. By decoupling the reaction generation into two phases, MoReact places strong emphasis on gencrating accurate global trajectories. The infilling of accurate global trajectories generated in the first stage to the full-body motion diffusion model further guides the local motion generation to a specific, refined plansible subspace. Doing so guarantees the semantic integrity of local motion, contributing to the overall realism of the generated reaction.
Within this two-stage modeling framework, we further introduce the interaction loss a specialized loss fune- tion tailored to characterize reactions and interactions. Rather than solely depending the reconstruction loss, which targets global absolute coordinates or local joint features, we highlight the significance of relative motion in interactions. Specifically, we focus on the motion of the reactor's joints in relation to the actor's joint movements, crafting a weighted interaction graph to emphasize the influential relative motion. For instance, in scenarios where two joints are close to each other, e.g., two hands in a handshake interaction, we specifically utilize the interaction loss to promote such contact to be accurately represented.
Based on these features, MoReact generates realistic and high-quality reactions that follow the text guidance faithfully while also syncing scamlessly with the actor's motion. Furthermore, MoReact demonstrates re- markable control capacity - it is capable of synthesizing varied reactions to the actor's s same motion based on different textual descriptions, and conversely, generating diverse reactions to actors' various motions guided by the same text, as shown in Fig. 4
In summary, our contributions are: (a) We develop a novel two-stage diflusion-based modeling framework, MoReact, which incorporates inherent motion pattern in reactions and synthesizes global trajectory and full- body motion in a sequential manner, ensuring the generated reactions are of high quality. (b) We introduce novel interaction loss that enhances the modeling of relative joint motions in interactions, utilizing weighted interaction graph to accurately model close interactions such as handshakes. (c) We conduct extensive experimental evaluation that demonstrates MoRcact's remarkable fiexibility and controllability in the reaction generalion process, enabling the synthesis of diverse reactions based on varying conditions. Comprehensive analyses are provided to validate our design decisions, showcasing the efficacy of MoReact in creating realistic reactions.
2 Related Work
Human Motion Generative Models. Generative models have achieved remarkable advancements in
synthesizing human motion based on various inputs, such action labels (Guo et al.) 2020 Petrovich
et al.
2021
Lee et al.
2023
Athanasiou et al. 2022). audio signals (Li et al. 2022
Tseng et al.
2023 Li
ct al.
2021
Yi et al.
2023
Zhou & Wang 2023), prior motions (Xu et al.
2022
Barquero et al
2023
Chen et al.
2023a :
movementt trajectories [Kaufinann et al.] 2020; Karunratanaa
el al. 2023
Rempe
et al. 2023
Xie et al. 2024), the surrounding environment (Cao et al.) 2020 Hassan et al. 2021 Wang
et al. 2021b
c 2022a by Huang et al.] 20233 Zhao et al. 2022. 2023 Tendulkar et al., [0023] Zhang ct al.
2024a), and including methods that generate motions without any specific conditions (Raab et al.) 2023a
Particularly in text-based human motion generation (Petrovich et al.] 2023 Guo et al. 2022b retich
3

---

Published in Transactions on Machine Learning Research (09/2025)
et al.
2022
Tevet et al.
2023
Chen et al.
2023b Zhang et al.
2022 Jiang et al.,
2023 Zhang et al.
2024b
2023b;
Tevet et al. 2022
Abuja & Morency 2019; Guo et al. 2022a Kim et
al. 2023 Lu et al..
2023
Raab
et al. 2023b
Zhang et al. 2023a Shafir et al. 2023 Dabral et al. 2023:
Zhhang et al. 2023c Wei
ct al.
2023 Zhang et al. 2024c Athanasiou et al.) 2023 Kong ct al. 2023). significant progress has been
made through the integration of diffusion models (Sohl-Dickstein et al. 2015 Song et al. 2020 Ho el al.
2020). A notable advantage of diffusion models is their capacity to itcratively refine the generation process by reintroducing available information, thus tailoring the outcomes to specific conditions. An instance of this is PhysDiff (Yuan et al., 2023), which incorporates a physics- physics-based motion imitation policy into the diffusion process to produce physically realistic motion. GMT) (Karunratanakul et al. 2023) is capable of generating human motions tailored to specific goals, such as following trajectory or achieving certain leyframes. This is also accomplished by strategically incorporating information into the diffusion process. In our work, we adopt a simple but effective inpainting mechanism to incorporate global trajectory information into the full-body motion generation process, cnsuring the full-body motion aligns coherently with the intended trajectory.
Interactive Motion Synthesis. Interactive motion generation takes into account both human movements and the dynamics of interactive entities, including objects and other humans. The goal is to generate motions that are both realistic and appropriately responsive to the interactive context. For example, human-object
interaction generation Xu et al.) 2023b Li et al. 2024
Diller & Dai 2024 Peng et al. 2025
Starke et al.
2019
Wang et al. 2023 Merel et al.
2020:
Hassan et al.
2023 Bae et al.
2023
Corona et al. 2020 considers
the dynamics of both humans and objects. Human- human interaction generation (Wang et al.
2021a Xu
ct al.
2023a c
Adeli et al.
2020
2021:
Guo al.)
2022c
Tanke et al.
2023a
Zhu ct al.
2023
Tanke et al.
2023b
Peng et al.]
2023
Liu et al.
2023
Cai et al.]
2024
Wang al.
2024 Liang et al.
2024
Shafir et al.
2023
Rempe et al.
2023
Xu ct al. 2024a b Liu ct al.
2024) involves motions from two more persons. For
methods aimed at modeling interactions in crowds with numerous participants, there is a notable emphasis
on global trajectories synthesis. For instance, DuMMF (Xu et al.) [023c] separates the modeling of local and global representations in social interactions, placing constraints on global motion. Trace and Pace (Rempe) et al. 2023) present a trajectory-guided diffusion model that allows for the manipulation of trajectories while considering the context of the surrounding environment. However, in two-person interaction generation, the crucial role of global trajectory modeling has been overlooked. In our work, we generate global trajectory and local motion scquentially, allowing the global trajectory to guide the generation of local motion. Despite the advancements in this field, these attempts at modeling multi-person interactions do not adequately capture individual reactions to others. For example, they often struggle to synchronize or align with rapid and varied movements. In contrast, some recent works (Ghosh et al. 2024 Xu et al. 2024b Liu et al.) 2024 Tan et al. 2025 Liu et al. 2023 Cen et al. 2025 Cong et al. 2025) focus on reactive motion generation, emphasizing the spatio-temporal coherence between the actor's 's and reactor's movements. However, these models are limited as they generate reactions either solely based on the actor's motion or by combining the actor's motion with action label, which fail to capture the full diversity of potential reactions. To address this, we enhance the reactive motion generation by incorporating textual guidance.
3 Methodology
Overview. In this section, we begin by formally defining the text-driven reaction generation task. We then
introduce the overall architecture of MoReact in Sec. 3.1 In Sec. 3.2 and Sec. 3.3 we delve into the model's design and the detailed configurations of MoReact's training process. Finally, in Sec. 3.4 we introduce the inpainting mechanism we used during the inference stage.
Task Definition. Given a motion sequence of the actor and a sentence describing the interaction between the actor and the reactor, our goal is to generate the reactor's motion that not only harmoniously coordinates with the actor but also aligns coherently with the textual deseription. In this context, actor refers to the character with known motion, while reactor refers to the character for whom we aim to synthesize motion, a.k.a,, reaction.
The reactor's motion soquence with frames, is represented as x = [ ",...., x JT, where each x E 203 contain the pose information of the reactor at the i-th frame in an adapted HumDnML3D (Guo et al. 2022a) representation. Specifically, each pose at consists of both global trajectory information g¹ E IR. and local
4

---

Published in Transactions on Machine Learning Research (09/2025)
Stage
Trajectory Diffusion Model
Self Attention
Cross Attention
Cross Attention
Cross Attention
FFN
Global Trajectory
Actor's motion
The people pivot their bodies
Stage
90 degrees to the left side
CLIP
TE
Full-Body Motion Diffusion Model
Text Prompt
MLP
MLP
MLP
1
Global Trajectory
Reaction
y
())
(b)
Figure 2: Overview of MoReact. (a) Our approach to lext-driven reaction generalion employs a two-stage framework. First, we employ a trajectory diffusion model to generate the global trajectory of the reactor, based on the actor's full-body motion and the text description. Subsequently, we apply a full-body motion diffusion model to generate the reactor's full-body motion, based on the actor's full-body motion, the text description, well the synthesized reactor's trajectory. (b) Our full-body motion diffusion model is built upon transformer-base trajectory diffusion model mirrors this architecture but omits the cross-attention layer that integrates global trajectory into the generative process.
architecture, where the 'TE' in the figure denotes Transformer Encoder. The
pose information 1' E 259 Global trajectory information g¹ consists of root orientation along the y-axis and 3D translation the global coordinate, while local pose information I includes joint position, joint velocity, joint rotation, and foot contact in the reactor's local coordinate. Similarly, we use y y denote the motion sequence of the actor.
.........""
...
We use w [wi, "2,.... ... Wn] to denole the n-words lextual description of the interaction between the actor and the reactor. Our goal is to model the conditional probability distribution p(x) y. w), from which we can then generate the reactor's motion sequence through sampling.
3.1 Framework Overview
As shown in Fig. D(a), we decouple the process for text-driven reaction generation into two sequential stages, consisting of two diffusion modules, f.e., the Trajectory Diffusion Module and the Full-Body Motion Diffusson Module. In the first stage, we use the Trajectory Diffusion Module to generate the global trajectory of the reactor, informed by the motion of the actor and the text description. Following this, in the second stage, utilize the Full-Body Motion Diffusion Module to generate the full-body motion of the reactor, based the actor's motion, the previously generated reactor's global trajectory, and the text description. During the inference stage, we adopt an inpainting mechanism during the denoising process to ensure the final full-body motion faithfully adheres to the intended trajectory pre-generated by Trajectory Diffusion Module.
3.2 Text-Driven Reaction Generation
Trajectory Generation. Based on our key observation that global motion siguificantly outweighs local motion in generating reactions, our approach prioritizes the reactor's overall trajectory by generating it the beginning, and using it as the condition for the full-body generation. We use Geraj to denote our diffusion model for this generation. More specifically, Guaaj takes time step t, a noised global trajectory gt: the actor's motion sequence y, and the text condition description w input. We observe that instead of

---

Published in Transactions on Machine Learning Research (09/2025)
estimating the clean denoised signal gu, estimating the noise & (t.aj(t.g: y, w) added in the forward diffusion, and recovering Do with & leads to better trajectory generation, which is consistent with the finding in GMD Karunratanakul et al., 2023
Full-Body Motion Generation. In this stage, our goal to generate realistic, coherent, and semantically aligned full-body reactions based on a synthesized global trajectory. Similarly, we use Grull to denote our diffusion model for the reactor's full-body motion generator. We input time step t, a noised full-body motion Bt, the actor's motion y, the text condition description w, well as the synthesized reactor's 's trajectory g into the model f Following MDM (Tevet et al., 2023), the model Grull is designed to directly predict the clean motion En as the output. Note that during the training phase, we utilize the reactor's s ground truth trajectory as the input for the model, while we directly use the synthesized trajectory during the inference phase.
Network Architectures. Fig- bb) illustrates the architecture of Grall- Given noisy motion 2t; our model feeds it inlo transformer-style architecture and obtains the denoised motion Vo. This archilecture consists of self-attention blocks, cross- attestintnnionn blocks, and gred-forware networks. The cross-attention layers integrate the motion feature with the features extracted from w, y, and g.
3.3 Loss Formulations with Training Details
We train the trajectory generation diffusion model, Geraj, and the full-body motion generation model, Grall: independently. And they can be seamlessly combined during inference. The loss formulations and the training strategies are elaborated in the following.
Trajectory Generation. In the trajectory generation stage, we use single reconstruction loss term train our model. Formally, the loss function Liraj is defined as:
Lenj |e =
where ē= = (f, y, w, represents the estimated noise.
Full-Body Motion Generation. At the stage of full-body motion generation, the loss function consists of three terms: the reconstruction loss Ln, the kinematic loss LK, and the interaction loss L. We define the training objective Lia in this stage as:
Lill = ARLR + AKLK + A1LI.
(2)
Here, ÅR, Ax, Ar the weights assigned to Ln, Iu,, Ia, respectively.
(1) Similar to the reconstruction loss used in the trajectory generation stage, we define LR = ||20 2 11 as the reconstruction loss here. However, solely relying on the reconstruction loss does not necessarily lead to realistic reaction.
(ii) To address the problem such as foot sliding or jittering artifacts, following (Tevet et al. 2023 Liang et al. 2024 Ghosh et al.) 2024), we use a kinematic loss term Lk. To specify, LK consists of subterms, which are foot Lyel I L reprosenting foot skating loss, velocity loss, global rotation loss, and global trajectory loss, respectively. With the weights Atoot Avel, Arot, Auraj LK can be formulated as:
TK Afoot T. + X. vel + Aro Ti + Atra) Tima
(3)
For more details on the formulation of these subterms, please refer to Appendix D
(iii) We introduce an interaction loss, L, that emphasizes the spatial relationships within interactions. Inspired by (Zhang et al.] 2023d), this approach models human interactions as interaction graph where each joint of the actor and reactor serves as node, and edges represent joint pairs. To compute L1, we design weighting function that emphasizes pairs of joints that are closer together, deeming pairs that are farther apart as less critical for interaction.
Specifically, the interaction loss L1 consists two terms: position interaction loss LP" and velocity interaction loss I. as shown Eq. where A, and A are corresponding weights:
11 1 + A.
(4)
6

---

Published in Transactions on Machine Learning Research (09/2025)
To compute L;, we first use forward kinematics to calculate the joint coordinates of the generated reaction and the actor's motion in the global coordinate, denoted as Px and P3 E RJ where T is the motion length and J is the number of joints. Next, we calculate the interaction graph M' E |xxJXXX3, where jp = Pyli] Pa [f] representing the difference in coordinates for cach frame between the i-th joint of the reactor and the j-th joint of the actor. We also compute distance graph Dr E RJxJxT where [p] = [4. jlll,, representing the distance between joint pair (i, Similarly, we calculate Mp and Dp of the ground truth reactor and the actor. The position interaction loss Li can be formulated
L
1
E (i..AJES
W,li,,, M j, Mplii,AA11.
S
where = ((I,), I c) is the set of foints-frame pairs that the distance between the i-th joint of the reactor and the j-th joint of the actor is below threshold in the ground truth distance graph. The weighted term Wp RXxXX is defined as:
W, = (oD)))
(6)
where of (i)
Σ., eap( cap(D[a..y.)) is a softmax function along the joint pair axis, and (D)
exp (..).)))))))
D. The intuition behind and weighted terms W, is that distant joint pairs are less important for interaction, while closer joint pairs are more critical. We can compute the velocity interaction loss T. similarly. Please refer to Appendix D for details.
As highlighted (Yuan al., 2023 Liang et al., 2024 Xu 2023b). we find that the denoising motion in the initial denoising phase does not have reasonable kinematic and interaction properties. Based on this observation, it is not reasonable to apply kinematic and interaction losses if the diffusion time step t is large. Thus, we adopt thresholding scheme among loss functions during the training of our full-body motion generator. Specifically, we set a threshold t and only apply the kinematic loss and the interaction loss if t is no larger than t. Therefore, the final form of Luall can be written as:
Lilll = AR + T(t < () + ARTR).
(7)
3.4 Inference
In the inference stage, we incorporate an inpainting mechanism into the denoising process, similar to the motion infilling scheme in Tevet et al. 2023 Shafir et al.] 2023 Raab et al. 2023b). Specifically, at each time step t, after estimating the clean sample 2o, we fuse the To with the previously generated global trajectory from the first stage to form io. This process can be formulated as:
(8) where M represents the mask for the dimensions that describe the global trajectory information in the motion feature and @ the Hadamard product. The modified result, æa is then used to sample #1-1- Through this inpainting mechanism, continuously incorporate the known global trajectory g into the denoising process, ensuring the final full-body motion faithfully adheres to g.
to (1 (1 M) C 2o + M @g.
4 Experiments
In this section, we begin by introducing the experimental setup of our work, which includes evaluation metrics, baseline settings, and implementation details in Sec. 4. Subsequently, we present the quantitative
results of our method Sec. 4.2 followed by the qualitative results in Soc. 4.3. Finally, Sec. 4.4 we
discuss the ablation study conducted on our model.
4.1 Experimental Setup
Dataset. We conduct our evaluation on the InterHuman Liang et al.) 2024) and CHI3D FFeraru et al.
2020) datasets. InterHuman features 6,022 motion sequences across various interaction categories, annotated

---

Published in Transactions on Machine Learning Research (09/2025)
with 16,756 unique descriptions. We use the official training and testing split as specified in InterHuman. To demonstrate the generalizability of MoReact, we also evaluate its performance on the action-driven reaction generation task using the CHI3D dataset, which contains 376 interaction sequences with action labels. We split this datasct into training and testing sets at a 2:1 ratio.
Evaluation Metrics. We adopt the evaluation metrics used prior works (Liang et al. 2024 Xu et al. 2024b) for our quantitative analysis, evaluating the actor and reactor's motion as a whole. Specifically, in the experiments on the InterHuman dataset, R-Precision mcasures the relevance of the generated interaction to the provided text description. The Fréchet Inception Distance (FID) evaluates the realism of the generated reaction relative to the actor's motion. Additionally, we use Multi-Modaliit Distance (MM Dist. to assess the aligmment between the text and the generated interaction in shared latent space. We also measure Diversity to examine the variation in the generated motions. For the experiments on the CHI3D dataset, we further evaluate the generated interactions by measuring their action recognition Accuracy using a pretrained action classifier.
Baseline. Given that text-driven human reaction generation is a novel task, there is no cxisting work or code publicly available that directly serves baseline for comparison. To facilitate comparison, in our evaluations, we modify InterGen (Liang et al. 2024) to incorporate an inpainting mechanism during the inference stage, as described in Sec. 3.4 This modification ensures that InterGen takes the actor's motion as a condition to generate the reactor's motion accordingly. We also adapted MDM (?), a widely used method in text-driven human motion generation, to suit the text-driven reaction generation task. For more details, please refer to the Appendix D
Table 1: Quantitative Comparison on InterHuman and CHI3D. + represents the 95% confidence interval, and + indicates that values that closer to the Real are better. indicates the model is evaluated without motion infilling mechanism.
InterHHuman
CHI3D FID|
Methods
3- P-Precisionf
FIDJ
MM Dist
Diversity +
Accuracyt
Diversity->
Real
20.00
0.206 0009
00.01
7.79g 20.031
0.60440.000 005
0.084 20.0000000 o0s
20.5000000
MDM
0.532 10.025
3.763
10.000 056
3.844 10.000 001
7.751
1.0021
0.49640.011
13.850 10.37
3.997-1.000
MDM-GRU
0.640
19.000
12.758
1..188
3.812 1.0001
7.640
0.0.22
0.345 01.011
39.280
11.397
1.09740.08p
InterGen
0.631 00.005
7.207 114
0.0001
7.692
2.0000.000 038
0.531 0.531+0.001 00.017
46.531
50.699
4.082
InterGen"
00000 0000
7.
3.821 3.821#0.000 20000
0.001
0.661 0.661#0.000 20.005
14.772 148
40.082
MoReact
0.015
2.412#0.050
3.813#0.002 002
7.775#0.0000
0.687
10.801 #0.313
3.582 063
The results demonstrate the effectiveness of kinematic loss Lx, interaction loss L,, thresholding scheme and two-stage framework.
Table 2: Ablation Studies on InterHuman dataset.
Methods
3-Precision"
FID+
MM Dist+
Diversity
Real
0.704
0.206=0.008
3.784
7.799
MoReact (w/o LK; Lr)
0.623 00007
3. 3.164 0.002
3.808 +00001
7.719 7.719 t0.031
MoReact w/o Lr)
0.594 0.594+0.000 00.00
2.456 0.000
3.816 +0.002
7.832 +00037
MoReact(w/o Lx)
0.618
2.673=0.02 020
3.810 to 002
7.784#2.000 23000
7.784
MoReact(w/o threshold scheme)
0.61340.000
3.403
3.816 Lo 002
7.796 7.796-0.033
MoReact(singleestagg))
0.591 #0.007
2.776 0. 082
3.822 +0.0000000 0.0
7.761 #0.051
MoReact
0.615 10.007
2.412 10.050
3,813 Lo.002
7.775 L0.046
Implementation Details. As a pre-processing step, we normalize the actor's motion by relocating it to the origin and rotating it to make the actor facing z axis. Subsequent transformations are applied to the reactor to maintain the spatial relationship between the actor and reactor unchanged. Similar to previous work (Tevet et al.) 2023 Zhang et al.) 2022). we use a pretrained and frozen CLIP (Radford et al. 2021) model to encode text prompts into text features, while the rest of MoReact is trained from scratch. The trajectory generation model trained for 1, 1,200 epochs, and the full boll-body motion generation model is trained for 2,000 epochs. We train both models using learning rate of Ir le 4 and the AdamW optimizer,
8

---

Published in Transactions on Machine Learning Research (09/2025)
with a batch size of 32. Our models are implemented in PyTTorh and trained on two NVIDIA A40 GPUs. Following Teeeet et al. 2023 adopt classifier-free approach (Ho & Salimans 2022 in the generation process. For evaluation, we adopt the MotionClip (Tevet et al. 2022) provided by InterGen (Liang et al, 2024 to evaluate our model's performance on InterHuman dataset. We follow the implementation of ST- GCN Yan et al.) 2018 to train the action recognition model and compute action-elassffiatione ion accuracy on CHT3D (F Fieraru et al. 2020) dataset. Additional implementation details are provided in the Appendix D
4.2
Quantitative Results
We compare MoReact with InterGen (Liang et al. 2024 and MDM (eeeet ct al. 2023 on the test sets of the InterHuman and CHI3D datasets. For a fair comparison, we include the results of both adapted InterGen with actor motion infilling and the original InterGen without motion infilling. The main results are summarized in Table I Our approach, MoReact, outperforms the baselines across various metrics, particularly achicving significant improvements
interaction quality and alignment, as evidenced by a substantial reduction in FID. As described in Sec. 4.1 FID is the most critical metric for reflecting the quality of the generated reactions. The results also indicate that naively adapting existing text-interaction or text-motion models fails to generate satisfactory reactions. This underscores the need for new paradigms in text-driven reaction generation, implying the importance of our proposed method.
4.3 Qualitative Results
The qualitative comparisons between MoReact and baselines are demonstrated in Fig. 3 As shown in the first two rows, the baselines fail to generate realistic reactions based on the textual description and the actor's motion. Specifically, the reactions generated by InterGen and MDM do not faithfully align with the textual description and fail coordinate harmoniously with the actor's motion. For instance, in Fig. (c.), InterGen is unable to synthesize motion that the reactor falls onto the ground the text described. This exemplifies how the previously generated global trajectory act constraint, narrowing the search space for full-body motion generation and resulting in interactions that better align with the text. Moreover, in Fig. 3(a), the reactor and actor overlap in the same space, leading to implausible body penetration artifacts. All visualization results from MDM fail to generate realistic reactions that align well with the actor's 's motion, even if the reaction itself coherent with the text description. These issues highlight the importance of a plausible global trajectory for realistic interaction and demonstrate the effectiveness of MoReact.
Moreover, we demonstrate MoReact's capability for diverse control, illustrated in Fig, 4 MoReact effec- tively synthesizes diverse reactions to the same motion of the actor based on differing textual descriptions. Conversoly, it is also adept at generating varied reactions to different motions of actors when guided by the same lext.
4.4 Ablation Studies
We conduct an ablation study on the InterHuman datasel to evaluate the effectiveness of our loss designs and two-stage framework, shown in Table 2 The results indicate that the kinematic loss Lk, interaction loss L1, and thresholding scheme all contribute to generating more realistic reactions and achioving lower FID scores. To validale the superiority of the two-stage framework, we develop a baseline model that operates in single stage, directly generating full-body motion from the text description and the actor's motion. The quantitative results in Tableeddemonstaat that our two-stage model significantly outperforms the single-stage model across all metrics. Additional details of the ablation studies can be found in the A Appendix
9

---

Published in Transactions on Machine Learning Research (09/2025)
One person moves forward, the other person moves towards left front Suddenly. other falls backwardls to the ground, and
viduals towards each other, At the same time, the lifts points their index finger towards the second then corees complete stop.
rushes help them up, Alter that, one person holds out their right hand left hand.
One person attempts lunge forward and stab the other's body, but the other person dodges with a sword their hand.
The guys rush towerds enother embrace tightly.
grasps
MDM
InterGen
MoReact
(a)
(b)
(c)
(d)
Figure 3: Qualitative comparison. We show that MoReact consistently generates more realistic reactions than MDM InterGen, avoiding issues such as body penetration (b), text-motion mismatch (c), and inter- action misaligument. (d).
The first person speaks animatedly while second with phone, not paying attention.
The first person talks excitement while the second stands and listens carefully.
Two people are having fight.
(a)
(b)
Figure 4:
Qualitative evaluation of the control capacity.
10

---

Published in Transactions on Machine Learning Research (09/2025)
5 Conclusions
In this work, we propose inmovative method, MoReact, to solve the text-driven human reaction generation. Utilizing a two-stage framework that sequentially synthesizes the global trajectory and full-body motion, MoReact effectively creates high-quality, realistic reactions. These reactions not only align accurately with the text but also harmonize with the actor's movements. Morcover, MoRcact also demonstrates remarkable flexibility and controllability in the reaction generation process, enabling the synthesis of diverse reactions based on varying conditions. Experimental results demonstrate our methods' superiority over baselines and validate the efficacy of our model's design.
Acknowledgments
This work was supported in part by NSF Grant 2106825, NIFA Award 2020-67021-32799, the Amazon-llinois Center on Al for Interactive Conversational Experiences, the Toyota Research Institute, the IBM-Illinois Discovery Accelerator Institute, and Snap Inc. This work used computational resources, including the NCSA Delta and DeltaAl and the PTI Jelstream2 supercomputers through allocations CIS230012, CIS230013, and CTS240311 1 from the Advanced Cyberinfrastructure Coordination Foosystem: Services & Support (ACCESS) program, well as the TACC Frontera supercomputer, Amazon Web Services (AWS), and OpenAl API through the National Artificial Intelligence Research Resource (NAIRR) Pilot.
References
Vida Adeli, Ehsan Adeli, Ian Reid, Juan Carlos Niebles, and Hamid Rezatofighi. Socially and contextually aware human motion and pose forecasting. R.L., 2020.
Vida Adeli, Mahsa Ehsanpour, Ian Reid, Juan Carlos Nicbles, Silvio Savarese, Ehsan Adeli, and Hamid Rezatofighi. Tripod: Human trajectory and pose dynamics forecasting in the wild. In ICCV, 2021.
Chaitanya Abuja and Louis-Philipe Morency. Language2pose: Natural language grounded pose forecasting. In 3DV, 2019.
Nikos Athanasiou, Mathis Petrovich, Michael Black, and Gül Varol. Teach: Temporal action composition for 3d humans. In 3DV, 2022.
Nikos Athanasiou, Mathis Petrovich, Michael Black, and Gül Varol. SINC: Spatial composition of 3d human motions for simultaneous action generation. In ICCV, 2023.
Jinseok Bae, Jungdam Won, Donggeun Lim, Cheol-Hui Min, and Young Min Kim. Pmp: Learning to physically interact with environments using part-wisc motion priors. In SIGGRAPH, 2023.
German Barquero, Sergio Escalera, and Cristina Palmcro. BeLFusion: Latent diffusion for behavior-driven
human motion prediction. ICCV, 2023.
Zhongang Cai, Jianping Jiang, Zhongfei Qing, Xinying Guo, Mingyuan Zhang, Zhengyu Lin, Haiyi Mei, Chen Wei, Ruisi Wang, Wanqi Yin, et al. Digital life project: Autonomous 3d characters with social intelligence. In CVPR, 2024.
Zhe Cao, Hang Gao, Karttikeya Mangalam, Qi-Zhi Cai, Minh Vo, and Jitendra Malik. Long-term human motion prediction with scene context. In ECCV, 2020.
Zhi Cen, Huaijin Pi, Sida Peng, Qing Shuai, Yujun Shen, Hujun Bao, Xiaowei Zhou, and Ruizhen Hu. Ready- to-react: Online reaction policy for two-character interaction generation. are preprint Xiv:2502.203300 2025.
Ling- Hao Chen, Jiawei Zhang, Yewen Li, Yiren Pang, Xiaobo Xia, and Tongliang Liu. HumanMAC: Masked motion completion for human motion prediction. In ICCV, 2023a.
11

---

Published in Transactions on Machine Learning Research (09/2025)
Xin Chen, Biao Jiang, Wen Liu, Zilong Huang, Bin Fu, Tao Chen, and Gang Yu. Executing your commands via motion diffusion in latent space. In CVPR, 2023b.
Peishan Cong, Ziyi Wang, Yuexin Ma, and Xiangyu Yue. Semgeomo: Dynamic contextual human motion generation with semantic and geometric guidance. In CVPR, 2025.
Enric Corona, Albert Pumarola, Guillem Alenya, and Francesc Moreno-Noguer. Context-aware human motion prediction. In CVPR, 2020.
Rishabh Dabral, Muhammnd Hamza Mughal, Vladislav Golyanik, and Christian Theobalt. MoFusion: A framework for denoissing-ciffuussoo--beed motion synthesis. In CVPR, pp. 9760 9770, 2023.
Christian Diller and Angela Dai. CG-HOI: Contact-guider 3d human-object interaction generation. In CVPR, 2024.
Mihai Fieraru, Mihai Zanfir, Elisabeta Oneata, Alin-lonut Popa, Vlad Olaru, and Cristian Sminchisescu. Three-dimensional reconstruction of human interactions. In CVPR, pp. 7214-7223, 2020.
Anindita Ghosh, Rishabh Dabral, Vladislav Golyanik, Christian Theobalt, and Philipp Slusallek. Remos: 3d motion-conditioned reaction synthesis for two-person interactions. In ECCV, 2024.
Chuan Guo, Xinxin Zuo, Sen Wang, Shihao Zou, Qingyao Sun, Annan Deng, Minglun Gong, and Li Cheng. Action2mation: Conditioned generation 3d human motions. In ACMMM, 2020.
Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng. Generating diverse and natural 3d human motions from text. In CVPR, 2022a.
Chuan Guo, Xinxin Zuo, Sen Wang, and Li Cheng. Tm2t: Stochastic and tokenized modeling for the reciprocal generation of 3d human motions and texts. In ECCV, 2022b.
Wen Guo, Xiaoyu Bie, Xavier Alameda-Pineda, and Francesc Moreno-Noguer. Multi-person extreme motion prediction. In CVPR, 2022c.
Mohamed Hassan, Partha Ghosh, Joachim Tesch, Dimitrios Tzionas, and Michael Black. Populating 3d scenes by learning human-scene interaction. In CVPR, 2021.
Mohamed Hassan, Yunrong Guo, Tingwu Wang, Michael Black, Sanja Fidler, and Xue Bin Peng. Synthe- sizing physical character-scene interactions. In SIGGRAPH, 2023.
Jonathan Ho and Tim Salimans. Classifier-free diffusson guidance. arXiv preprint arXic:2207.12598, 2022.
Jonathan Ho, Ajay Jain, and Picter Abbeel. Denoising diffusion probabilistic models. In NewrIPS, 2020.
Siyuan Huang, Zan Wang, Puhao Li, Baoxiong Jia, Tengyu Liu, Yixin Zhu, Wei Liang, and Song- Shng-Chun Zhu. Diffusion-based generation, optimization, and planning in 3d scenes. In CVPR, 2023.
Biao Jiang, Xin Chen, Wen Liu, Jingyi Yu, Gang Yu, and Tao Chen. MotionGPT: Human motion as a foreign language. In NewrIPS, 2023.
Korrawe Karmnratanakul, Konpat Preechakul, Supasorn Suwajanakorn, and Siyu Tang. GMD: Controllable human motion synthesis via guided diffusion models. In ICCV, 2023.
Manuel Kaufmann, Emre Aksan, Jie Song, Fabrizio Pece, Remo Ziegler, and Otmar Hilliges. Convolutional autocncoders for human motion infilling. In 9DV, 2020.
Jihoon Kim, Jiseob Kim, and Sungjoon Choi. Flame: Free-form language-basee based motion synthesis & editing, In AAAI, 2023.
Hanyang Kong, Kehong Gong, Dongze Lian, Michael Bi Mi, and Xinchao Wang. Priority-centric human motion generation in discrete latent space. Tn ICCV, 2023.
12

---

Published in Transactions on Machine Learning Research (09/2025)
Taeryung Lee, Gyeongsik Moon, and Kyoung Mu Lec. Multiact: Long-term 3d human motion generation from multiple action labels. In AAAT, 2023.
Buyu Li, Yongchi Zhao, Shi Zhelun, and Lu Sheng. Danceformer: Music conditioned 3d dance generation with parametric motion transformer. Tn AAAT, 2022.
Jiaman Li, Alexander Clegg, Roozbeh Mottaghi, Jiajun Wu, Xavier Puig, and C Karen Liu. Controllable human-object interaction synthesis. In ECCV, pp. 54-72, 2024.
Ruilong Li, Shan Yang, David A. Ross, and Angjoo Kanazawa. Learn to dance with AIST++: Music conditionod 3d dance gencration. arXiv preprint arXi::210.008779 2021.
Han Liang, Wenqian Zhang, Wenxuan Li, Jingyi Yu, and Lan Xu. Intergen project page: Diffusion-based
multi-human motion generation under complex interactions, 2023. https://gi gighh........ wwt.cooo//o/// foom/tr3e/Init Interr/Innnren
Han Liang Wenqian Zhang, Wenxuan Li, Jingyi Yu, and Lan Xu. Intergen: Diffusion-based multi-humas
motion generation under complex interactions. IJCV, pp. 1-21, 2024.
Yunze Liu, Changxi Chen, and Li Yi. Interactive humanoid: Online full-body motion reaction synthesis with social affordance canonicalization and forecasting. arXiv preprint arXiv:2312 088883, 2023.
Yunze Liu, Changxi Chen, Chenjing Ding, and Li Yi. Physreaction: Physically plausible real-time humanoid reaction synthesis via forward dynamics guided 4d imitation. In ACMMM, pp. 3771-3780, 2024.
Shunlin Lu, Hao Chen, Ailing Zeng, Jing Lin, Ruimao Zhang, Lei Zhang, and Heung- Yeung Shum. JlumanTOMATO Text-aligned whole-body motion generation. arXiv preprint arXiv:2310. 12978, 2023.
Josh Merel, Saran Tunyasuvunakool, Arun Ahuja, Yuval Tassa, Leonard Hasenclever, Vu Pham, Tom Erez, Greg Wayne, and Nicolas Heess. Catch & carry: reusable neural controllers for vision-guided whole-body tasks. ACM Transactions on Graphics (TOG), 39(4):39-1, 2020.
Xiaogang Peng, Siyuan Mao, and Zizhao Wu. Trajectory-aware body interaction transformer for multi- multi-person pose forccasting. In CVPR, 2023.
Xiaogang Peng, Yiming Xie, Zizhao Wu, Varun Jampani, Deqing Sun, and Huaizu Jiang. Hoi-diff: Text- driven synthesis of 3d human-object interactions using diffusion models. In CVPR, pp. 2878-2888 2025.
Mathis Petrovich, Michael J Black, and Güi Varol. Action-conditioned 3d human motion synthesis with transformer vac. In ICCV, 2021.
Mathis Petrovich, Michael J. Black, and Gül Varol. TEMOS: Generating diverse human motions from textual descriptions. In ECCV, 2022.
Mathis Petrovich, Michael J Black, and Gül Varol. TMR: Text-lo-motion retrieval using contrastive 3d human motion synthesis. In ICCV, 2023.
Sigal Raab, Inbal Leibovitch, Peizhuo Li, Kfir Aberman, Olga Sorkine-Hornung and Daniel Cohen-Or. MoDi: Unconditional motion synthesis from diverse data. In CVPR, 2023a.
Sigal Raab, Inbal Leibovitch, Guy Tevet, Moab Arar, Amit H Bermano, and Daniel Cohen-Or. Single motion diffusion. arXiv preprint Xiv:22300 05900, 2023b.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML, 2021.
Davis Rempe, Zhengyi Luo, Xue Bin Peng, Ye Yuan, Kris Kitani, Karsten Kreis, Sanja Fidler, and Or Litany. Trace and pace: Controllable pedestrian animation via guided trajectory diffusion. In CVPR, 2023.
Yonatan Shafir, Guy Tevet, Roy Kapon, and Amit H. Bermano. Human motion diffusion as generative prior. Xiv preprint or arXiv:2303.01418 014 2023.
13

---

Published in Transactions on Machine Learning Research (09/2025)
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In ICML, 2015.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. ar preprint arXiv:2010.02500 2020.
Sebastian Starke, He Zhang, Taku Komura, and Jun Saito. Neural state machine for character-scene inter- actions. ACM Trans. Graph., 38(6):209-1, 2019.
Wenhui Tan, Boyuan Li, Chuhao Jin, Wenbing Huang, Xiting Wang, and Ruihua Song. Think then react: Towards unconstrained action-to-reaction motion generation. In ICLR, 2025.
Julian Tanke, Oh-Hum Kwon, Felix B Mueller, Andreas Doering, and Juergen Gall. Humans in kitchens: A datasct for multi-person human motion forccasting with scene context. In NewrIPS, 2023a.
Julian Tanke, Linguang Zhang, Amy Zhao, Chengeheng Tang, Yujun Cai, Lezi Wang, Po-Chen Wu, Juergen Gall, and Cem Keskin. Social diffusion: Long-term multiple human motion anticipation. In ICCV, 2023b.
Purva Tendulkar, Didac Surís, and Carl Vondrick. FLEX: Full-body grasping without full-body grasps. In CVPR, 2023.
Guy Tevet, Brian Gordon, Amir Hertz, Amit H. Bermano, and Daniel Cohen-Or. Motionclip: Exposing human motion generation to clip space. In ECCV, 2022.
Guy Tevel, Sigal Raab, Brian Gordon, Yoni Shafir, Daniel Cohen-or, and Amit Haim Bermano. Human motion diffusion model. In ICLR, 2023.
Jonathan Tseng, Rodrigo Castellon, and Karen Liu. Edge: Editable dance generation from music. In CVPR, 2023.
Jiashun Wang, Huazhe Xu, Medhini Narasimhan, and Xiaolong Wang. Multi-person 3d motion prediction with multi-range transformers. In NewrIPS, 2021a.
Jiashun Wang, Huazhe Xu, Jingwei Xu, Sifei Liu, and Xiaolong Wang. Synthesizing long-term 3d human motion and interaction in 3d scenes. In CVPR, 2021b.
Jingbo Wang, Sijie Yan, Bo Dai, and Dahua Lin. Scene-aware generative network for human motion synthesis. In CVPR, 2021c
Jingbo Wang, Yu Rong, Jingyuan Liu, Sijie Yan, Dahua Lin, and Bo Dai. Towards diverse and natural scene-aware 3d human motion synthesis. In CVPR, 2022a.
Yinhuai Wang, Jing Lin, Ailing Zeng, Zhengyi Luo, Jian Zhang, and Lei Zhang. Physhoi: Physics-basod imitation of dynamic human-object interaction. arXiv preprinl urXie:2312. 043393,,, 2023.
Zan Wang, Yixin Chen, Tengyu Liu, Yixin Zhu, Wei Liang, and Siyuan Huang. HUMANISE: Language-
conditioned human motion generation in 3d scenes. In NewrIPS, 2022b.
Zhenzhi Wang, Jingbo Wang, Yixuan Li, Dahua Lin, and Bo Dai. Intercontrol: Zero-shot human interaction generation by controlling every joint. Tn NewrIPS, 2024.
Dong Wei, Xiaoning Sun, Huaijiang Sun, Bin Li, Shengxiang Hu, Weiqing Li, and Jianfeng Lu. Under- standing text-driven motion synthesis with keyframe collaboration via diffusion models. arXio preprint arXiv:2305.197 137733 2023.
Yiming Xie, Varun Jampani, Lei Zhong, Deqing Sun, and Huaizu Jiang. Omnicontrol: Control any joint any time for human motion generation. In ICLR, 2024.
Liang Xu, Ziyang Song, Dongliang Wang, Jing Su, Zhicheng Fang, Chenjing Ding, Weihao Gan, Yichao Yan, Xin Jin, Xiaokang Yang, al. ActFormer: A gan-based transformer towards general action-conditioned 3d human motion generation. Tn ICCV, 2023a.
14

---

Published in Transactions on Machine Learning Research (09/2025)
Liang Xu, Xintao Lv, Yichao Yan, Xin Jin, Shuwen Wu, Congsheng Xu, Yifan Liu, Yizhou Zhou, Fengyun Rao, Xingdong Sheng, Yunhui Liu, Wenjun Zeng, and Xiaokang Yang. Inter-x: Towards versatile human- human interaction analysis. In CVPR, 2024a.
Liang Xu, Yizhou Zhou, Yichao Yan, Xin Jin, Wenhan Zhu, Fengyun Rao, Xiaokang Yang, and Wenjun Zeng. Regennet: Towards human action-reaction synthesis. In CVPR, pp. 1759-1769, 2024b.
Sirui Xu, Yu-Xiong Wang, and Liang- Yan Gui. Diverse human motion prediction guided by multi-level spatial-temporal anchors. In ECCV, 2022.
Sirui Xu, Zhengyuan Li, Yu-Xiong Wang, and Liang- Yan Gui. InterDiff: Generating 3d human-object interactions with physics-informed diffusion. In ICCV, 2023b.
Sirui Xu, Yu-Xiong Wang, and Liangyan Gui. Stochastic multi-person 3d motion forecasting. In ICLR, 2023c.
Sijie Yan, Yuanjun Xiong, and Dahua Lin. Spatial temporal graph convolutional networks for skeleton-based action recognition. In AAAI, volume 32, 2018.
Hongwei Yi, Hualin Liang, Yifei Liu, Qiong Cao, Yandong Wen, Timo Bolkart, Dachong Tao, and Michael J Black. Generating holistic 3d human motion from speech. In CVPR, 2023.
Ye Yuan, Jiaming Song, Umar Iqbal, Arash Vahdat, and Jan Kautz. PhysDiff: Physics-guided human motion diffusion model. In ICCV, 2023.
Jianrong Zhang, Yangsong Zhang, Xiaodong Cun, Shaoli Huang, Yong Zhang, Hongwei Zhao, Hongtao Lu, and Xi Shen. T2M-GPT: Generating human motion from textual descriptions with discrete representa- tions. In CVPR, 2023a.
Jianrong Zhang, Yangsong Zhang, Xiaodong Cun, Yong Zhang, Hongwei Zhao, Hongtao Lu, Xi Shen, and Ying Shan. Generating human motion from textual descriptions with discrete representations. In CVPR, 2023b.
Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, and Ziwei Liu. Mo- tionDiffuse: Text-driven human motion generation with diffusion model. arXiv preprint arXiv:2208. 15001, 2022.
Mingyuan Zhang, Xinying Guo, Liang Pan, Zhongang Cai, Fangzhou Hong, Huirong Li, Lei Yang, and Ziwei Liu. ReMoDiffuse Retrieval-augmented motion diffusion model. Tn ICCV, 2023c.
Wanyue Zhang, Rishabh Dabral, Thomas Leimkühler, Vladislav Golyanik, Mare Habermann, and Christian Theobalt. Roam: Robust and object-aware motion generation using neural pose descriptors. 3DV, 2024a.
Yaqi Zhang, Di Huang, Bin Liu, Shixiang Tang, Yan Lu, Lu Chen, Lei Bai, Qi Chu, Nenghai Yu, and Wanli Ouyang. Motiongpt: Finetuned Ilms are general-purpost motion generators. In AAAI, volume 38, pp. 7368-7376, 2024b.
Yunbo Zhang, Deepak Gopinath, Yuting Yo, Jessica Hodgins, Grog Turk, and Jungdam Won. Simulation and retargeting of complex multi-character interactions. arXiv preprint 2rXic:2305.20041, 2023d.
Zihan Zhang, Richard Liu, Kfrr Aberman, and Rana Hanocka. Todi: Temporally-cntangled diffusion for long-term motion synthesis. In SIGGRAPH, Technical Papers, 2024c. doi: 10.1445/3641519.365711
Kaifeng Zhao, Shaofei Wang, Yan Zhang, Thabo Beeler, and Siyu Tang. Compositional human-scene inter- action synthesis with semantic control. In ECCV, 2022.
Kaifeng Zhao, Yan Zhang, Shaofei Wang, Thabo Beeler, and Siyu Tang. Synthesizing diverse luman
motions in 3d indoor scenes. In ICCV, 2023.
Zixiang Zhou and Baoyuan Wang. Ude: A unified driving engine for human motion generation. In CVPR, 2023.
15

---

Published in Transactions on Machine Learning Research (09/2025)
Wentao Zhu, Jason Qin, Yuke Lou, Hang Ye, Xiaoxuan Ma, Hai Ci, and Yizhou Wang. Social motion prediction with cognitive hierarchies. In NeurIPS, 2023.
16

---

In detail, for a reaction Ho, we use a diffusion style forward process to apply a sequence of Gaussian noise additions to zo and obtain the noised full-body reactions I1, 22, Tr, where ær ~ NN,,,) I). Based on and Published in Transactions on Machine Learning Research (09/2025)
A Appendix Overview
In this appendix, we present further analyses, implementation details, and additional experimental results. Specifically: (1) We provide detailed video demonstration in the supplementary materials, with cor- (2) We delve deeper into our key insight - global trajectory serves as the foundation for both local motion and interaction rcalism - and further validate this claim with quantitative experimental evidence in Sec. C (3) Additional information regarding the implementation of
responding explanation provided in Sec. B
MoReact and the baseline models detailed in Sec. D (4) Sec. B illlstratees extra ablation studies show the efficacy of MoReact's ' framework. (5) We discuss MoReact's limitations and social impacts in Sec. F
B Visualization Demo
In the supplementary materials, we include demo video that shows the visualization results associated with figures in the main paper. The video features: (1) visualizations of our main insights; (2) qualitative comparisons with baseline models; (3) showcases of MoReact's controllability on both text and motion; and
(4) qualitative results of MoReact in action-driven rcaction generation task on CHI3D dataset. Please watch the video for further results and details.
C A Further Investigation Our Key Insight: The Central Role of Global Trajectory
As discussed and qualitatively validated in the main paper as well as in the demo video, a crucial insight of our work that the global trujectory serves as the foundation for both local motion and interaction realism. Incorrect global trajectory makes it difficull for the local motion to align with the action and text description, having a more detrimental impact on interaction realism than incorrect local motion. Here to further validate this insight in quantitative manner, we perform experiment. examining the impact of equivalent levels of noise on both global trajectory and local motion and their effects on the overall motion's 's realism.
the noised full-body reactions {xr)7= we can obtain reactions with noised global trajectory reactions with noised local motion {ar 1T with the following equations:
x = (1 M9) @ xo + M ② Zt
(9)
x; (1 M æo + M
(10)
where M5, M' represent the masks for the dimensions that describe the global trajectory information and local motion information in the motion feature respectively, and @ the Hadamard product. For a set of time steps {t), we compute æt, æy and æ for every reaction æ in the test dataset. Subsequently, we evaluate the realism of interaction between the actor's motion and the noised reaction by calculating the Fréchet Inception Distance (FID) across the entire test dataset. The FID is computed by the MotionClip (Tevet ct al. 2022b provided by InterGen (Liang et al.) 2024). The results, depicted in Fig. C.1 and consistent with the demo video, show that adding noise to the global trajectory has a more detrimental effect on the realism of interactions compared with adding noise to the local motion, thus motivating the design of our MoRcact framework.
D Implementation Details
Formulation of Velocity Interaction Loss Lj. Beyond the position interaction loss introduced in the main paper, we also employ a velocity interaction loss to enhance the model's ability to generate realistic close interactions. Similar to the computation of If, for LJ, we first compute V. and V,, E (T- representing the joint velocities of the reactor and the actor. We then caleulate the velocity interaction graph M, XXTx((11)) where M, = Vylū V. We also calculate M, for the ground truth reactor
17

---

Published in Transactions on Machine Learning Research (09/2025)
FID Over Time Steps
Global Traj. Local Motion
Traj.
Local Motion
40
10
200
400
600
800
time steps
Figure C.I: Change of FID for different noising modes and diffusion steps. Adding noise to the global trajectory has more detrimental effect on the realism of interactions compared with adding noise to the local motion.
and the actor. The velocity interaction loss Li can be formulated as:
L
Σ (i..A))
W.lii.AAl
M. (...,
(11)
S'
where S" = (( k) S k < < is a set of index pairs and W, (Dp) + +Dpp) is the weighted term. The definition of Dp, Dp, and a are consistent with those in the main paper.
Formulation of Kinematic Loss LK. As shown in Sec. 3.3 of the main paper and building upon Tevet kinematic loss term, Ik, to prevent artifacts
et al. 2023 Liang et al., 2024 Ghosh et al. 2024), we like foot sliding jittering. Moreover, we aim to utilize the kinematic loss LK to make model focus more on the generation of global trajectory. This focus is crucial because, despite the greater importance of global trajectory compared with local motion, the motion representation allocates only 4 values for the global trajectory versus 259 for the local motion. As shown in GMD (Karunratanakul et al., 2023), such a disparity could lead the model to prioritize local motion generation. Therefore, we want to use the kinematic loss LK eliminate such bias.
Specifically, Lg consists of 4 subterms, which can be formualted as:
Lk Atoot L + Aun vel + Arot Lgg rot + Atraj traj
(12)
Here, foot. LK rot. and L correspond the foot skating loss, velocity loss, global rotation loss, and global position loss, respectively. The coefficients Afoots Avels Arot, and Ausj denote the weights assigned to these four loss terms. To compute these losses, first compute joint positions Pz. P. RJ337 and joint velocities V,, V, E (x3(-1 of the generated reaction and ground truth reaction. We further use Rg, R. E RT to denote the global rotation of the reactor along the y-axis. For clarity, we omit the subscript x in subsequent formulations.
To compute the foot skating loss Lis we first calculate Á E (T which signifies the height each joint across the previous T frames. The formulation of L foot is then articulated follows:
Cl1 [[IV 7v) I(H m))
(13)
L foot
Σ
Vi C(i..
(14)
Σ EieFootJolnts
Clil
ie Join.
18

---

Published in Transactions on Machine Learning Research (09/2025)
Here, To and ? serve as thresholds for calculating C, where C' E {0, 10 indicates the contact be- tween each joint and the ground in each frame. FootJoints C [1,"., represents the subset of indices corresponding to foot joints.
We use similar equations to compute velocity loss Lyel, global rotation loss LR" and global position loss LK traj which are expressed as follows:
LK
J (T - Σ [ 2
(15)
1
L T 1ll R
(16)
tra] 1
K
P|root]] - P Prot]]1
(17)
T
where 'root' denotes the index of the root joint of the reactor.
Experimental Sctup of InterGen. Originally designed for text-driven human interaction generation, InterGen (Liang et al.] 2024 processes a text prompt w lo generate interactions = [ y) between two people with respect to w. However, it is not directly applicable to our task of text-driven human reaction generation. To adapt InterGen for this new task, we integrate an inpainting mechanism into the inference process of InterGen, which is similar to the method described in Sec. 3.4 of the main paper. At each time step of the denoising process of InterGen, after estimating the clean interaction Eg [o]: we embed the known actor's motion into &g to obtain 2g. This operation can be expressed as:
20 = [1, 0 @ 20 + [0, 1| @ 0, yl = = y)-
(18)
Here, O denotes the Hadamard product. The modified result, 2o [2o. gol, is subsequently utilized to calculate μ4 and to sample - from N(,, E.). By employing this inpainting mechanism, continuously integrate the known actor's motion throughout the denoising process, guaranteeing that the resulting interaction x = [æ, y] accurately conforms to y. Consequently, 2o in the ultimate denoising outcome zo- = Lop y/] represents the reaction generated with respect to both the textual prompt w and the actor's motion y.
Through communication with the authors of InterGen (Liang et al. 2024). we discovered that the publicly released checkpoint (Liang et al., 2023) of InterGen was trained using both the training and test sets to produce best demonstrations. Therefore, for fair comparison, train InterGen from scratch using their codebase, strictly following the experimental setup presented in their paper.
Experimental Sctup of MDM. We adapted the official code of MDM to suit the text-driven reaction generalion task. Specifically, by concatenating the action and reaction features before leeding them into the model, we enable the model to be aware of the interaction between two people instead of focusing on just one person. We experimented with two backbones for MDM: transformer encodcr-only backbone and a GRU backbone. For the transformer encoder-only backbone, we utilized N=8 blocks, each with latent dimension of 1,024, and equipped each attention layer with 8 heads. For the GRU backbone, we set N-8 GRU layers with a latent dimension of 1,024. Both models were trained for 2,000 epochs using the AdamW optimizer, consistent with the training settings of MoReact.
Detailed Model Configurations. In the transformer-style architecture of our full-body motion diffusion model, we utilize N-8 blocks, each with a latent dimension of 1,024, and we equip each attention layer with 8 heads, consistent with the setup in InterGen (Liang et al., 2024). Before inputting the noised reaction vector 2, into the transformer layers, we use a linear layer to adjust its dimension to match the transformer's input dimension. Similarly, the output from the transformer layers is processed by another lincar layer to match the motion feature's dimension. For lext processing, we utilize a frozen CLIP-ViT-L-14 model to encode the text prompt into text features for cross-attention. Moreover, following InterGen, we extract the most salient text feature embedding, combine with the diffusion timestep feature, and employ this composite feature within the adaptive layer norms of the transformer blocks. To encode the actor's motion y, transformer encoder layer comprising 2 blocks, latent dimension of 1,024, and 8 heads per attention
19

---

Published in Transactions on Machine Learning Research (09/2025)
Two individuals are seated on chairs, the first individual is looking at the phone, while the other suggests that the former should look ahead.
The other person forcefully raises their right hand and slaps the left cheek of one person, causing them to collapse to the ground. The other person stands by the side afterwards.
MoReact
MoReact (1- Stagge)
MoReact
MoReact (1 SSagge)
Figure E.1: Ablation study on the design choice within MoReact.
layer utilized prior to incorporating y for cross-attention. Except for the absence of a cross-attention layer, the architecture of the trajectory diffusion model mirrors that of the full-body diffusson model.
During training, 2022) that randomly masks 10% of the text conditions, 10% of the actor's 's motion conditions, and 10% of the global trajectory condition independently. During inference, we use the DDIM (Song et al. 2020) sampling
1,000-step diffusion process and adopt classifier-free technique (Ho & Salimans,
strategy with 50 time steps and = 0, and set the classifier-free guidance coelficient. = 3.5. For the hyperparameters used in the training of the revised model, we set (AR, Ak: A1, foot Avel ART Xtraj L. Li) (7.0, 1.0, 1.0, 300.0, 110.0, 1.5, 10, 5.0, 25.0), respectively. In addition, we the threshold for applying the kinematic loss Lg and the interaction loss L 700.
Details for experiments CHI3D dataset. To demonstrate the generalization ability of MoReact, we adapted it to suit the action-driven reaction generation task and evaluated it on the CHI3D (Ficraru et al. 2020) dataset. Specifically, instead of using CLIP to extract features from text as in the text-driven reaction generation task, we employed learnable action embedding to encode the action features. Additionally, compared to the architecture shown in Fig. 2(b) the main paper, we eliminated the cross-attention layer that fuses the textual features into the denoising process. We reduced the latent dimension 512 and the batch size to 16. The model made corresponding adjustments to the bascline MDM model (reducing the latent dimension, adjusting the batch size, and training seltings) to ensure fair comparison. We follow the official implementation of ST-GON (Yan et al. 2018) to build our evaluator, an interaction classifier trained on CHI3D.
trained for 1,000 epochs using the AdamW optimizer. We also
E
Additional Ablation Studies
Two-Stage vs. Single-Stage. Beyond the quantitative analysis of the design choice Sec. 4.4 of the main paper, we present some visual results generated by both the two-stage and single-stage frameworks. As illustrated in Fig. E. and supplementary vidoo, our two-stage framework generates more natural and lext-aligned reactions compared to the single-stage baseline, validating the eflectiveness of our two-stage approach.
Predicted Term of Trajectory Diffusion Model. As mentioned in Sec. 3.2 of the main paper, diffusion models can employ two kinds of strategies during the denoising process to derive #.-1 from the noised data wr: predicting the noise e, or predicting the clean data Øo. Here, we conduct experiments to determine which approach more effective for the trajectory diffusion model. The results, displayed Table E.T] indicate that the variant focusing on noise prediction e outperforms, aligning with the conclusions drawn by GMD (Karunratanakul et al. 2023
20

---

Published in Transactions on Machine Learning Research (09/2025)
Table E.1: Ablation studies on predicted term of trajectory diffusion model.
The trajectory model that
predicts achieves better performance in R-precision, FID and Multi-Modality Distance.
Methods
Traj. Model
3- Precision
FID+
MM Dist
Diversity
Real
0.704 40.005
40.009
3.784
7.799
MoReact
predict æo predict
0.568 00000
2.959
2.959
03.030
3. 3.826 tu.00
+..008000030
MoReact
1.615=0.007
2.41220.058
3.813#0.000
7.775#0.046
Interaction Loss. While some existing work also employed interaction loss to facilitate interaction genera- tion, their implementations differ from ours in some important aspects. For example, ReMoS (Ghosh al., 2024) only considers corresponding joints of the interacting individuals in its interaction loss, thus failing to capture diverse joint interaction patterns present in real-world scenarios. InterGen (Liang et al. 2024). on the other hand, does not incorporate a weighting mechanism, preventing it from effectively penalizing unrealistic close interactions or appropriately de-emphasizing irrelevant distant ones. In contrast, our inter- action loss introduces a novel weighting mechanism that dynamically adjusts the importance of joint pairs based on both ground-truth and generated interactions, thereby enabling more realistic reaction generation. Additionally, we re-implemented the interaction losses employed by InterGen and ReMoS within MoReact and conducted quantitative comparisons with method. As demonstrated in Table E.2, approach consistently achieves superior performance in terms of R-precision, FID, and MM Dist, highlighting the effectiveness of our weighted interaction loss.
Table E.2: Quantitative comparison of different interaction loss designs. Our weighted interaction loss consistently outperforms InterGen (Liang et al. 2024) and ReMoS (Ghosh et al. 2024 losses on R-precision, FID, and MM Dist, demonstrating its superior effectiveness in generating realistic reactions.
Methods
3-Presisionn
FID+
MM Dist
Diversity
Roal
0.704
0.206
70.001
7.799#0.031
InterGen [Liang et
2024
Loss
0.596 +0.009
+0.077
3.826 1.826+0.000 0,002
7.887
ReMoS Ghosh et al.
2024
Loss
0.608 +0.007
2.817700.070
3.819 1..19+00008 000002
7.792 7.792+0.036 +.0035
MoReact
0.615
2.412 2.412#0.000
20.000
3.813
7.775
F Limitations and Social Impacts
Limitations and Future Work. MoReact is designed to generate reactions by considering both textual descriptions and the motion of another individual. Future research will aim to generalize method to broader contexts, for example, generating reactions based on text and the motions of multiple people.
Potential Social Impact. We recognize the potential application of reaction synthesis in military training contexts. With our model, the military might generate virtual soldier who can dodge and counteract response real soldier's movements, thereby simulating authentic battlefield scenarios train soldiers.
21


Reviews:
[{"review_id": "XK9zJTLQQV", "reviewer": "Reviewer_PFaC", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\nOverall, the paper defines the problem in a clear way, provides extensive literature review and explain the proposed method as well as experiments in a clear way. Training details have been explained for reproduction and both qualitative and quantitative results have been shown and discussed.\n\nWeaknesses:\n\nI have listed some of my concerns which are expected to be further clarified by authors:\n\n1. In the second last paragraph of page 4, I recommend authors further clarify how the global coordinate and local coordinate are defined? a good illustration figure would be helpful.\n\n2. Overall, it is not clear to me whether the method is image-based or video-based? Authors can further explain this.\n\n3. Since authors mentioned that \"_Given that text-driven human reaction generation is a novel task, there is no existing work or\ncode publicly available that directly serves as a baseline for comparison._\". My expectations are this work can (1) setup a baseline/benchmark for following tasks; (2) experiments and comparisons are to be extensive on a varieties of datasets and environments, indoor, outdoor, single/multiple person, etc. Given the current experimental results (Table 1, Table 2), this work (evaluation breadth and evaluation depth) is not strong enough.\n\n4. I wonder how quality of text descriptions can affect generation results?\n\n5. I did not see any failure cases.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper propose a new reactive motion generation framework conditioned on text descriptions, which also introduces a new workstream which has been under-explored before. The proposed method consists of two modules, a trajectory diffusion module and a full-body motion diffusion module, and authors have been exploring global motion and local motions during the modeling human motion generation. Authors also conduct experiments to validate the proposed method as well as losses used during the model training.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I have listed some of my main concerns in the weakness section. My main concern lies in evaluations since this paper proposes a new task, reactive motion generation with text descriptions. I would expect more extensive evaluations to support this work and benefit following work.", "broader_impact_concerns": "Authors have discussed the social impacts in the appendix.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "bLxAt7QYbq", "reviewer": "Reviewer_cwLr", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- This paper decomposes reaction generation into global trajectory generation and full-body motion generation, reducing the complexity of the task.\n- The motivation is clear. The writing is easy to follow.\n\nWeaknesses\n- How is the quality of the global trajectory ensured? An inaccurate global trajectory may severely impact the quality of the subsequent motion generation.\n- The interaction loss actually penalizes the relative joint positions between two characters, a concept that has already been introduced in previous works [A, B]. Therefore, the differences between this work and those prior approaches should be clearly discussed.\n\n     [A]  \"Closely interactive human reconstruction with proxemics and physics-guided adaption.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.\n\n     [B] \"Intergen: Diffusion-based multi-human motion generation under complex interactions.\" International Journal of Computer Vision 132.9 (2024): 3463-3483.\n- While the diffusion architectures are effective, they are largely based on prior works and do not introduce significant innovations specific to the challenges of reaction generation.\n- Since InterGen and MDM are not specifically designed for reaction generation tasks, the proposed method should also be compared with baseline approaches that are explicitly tailored for this problem. Regennet can also input action label.\n\n     [A] \"Regennet: Towards human action-reaction synthesis.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.\n\n     [B] \"Remos: 3d motion-conditioned reaction synthesis for two-person interactions.\" European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024.\n- \"black mesh\" in the caption of Fig.1 might be a typo.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a framework for generating reactive motions based on an actor's movements and textual descriptions. The method decouples motion generation into global trajectory planning and local full-body articulation, adopting a two-stage design for reaction synthesis. To enhance the realism of the generated motions, the authors introduce an interaction loss that constrains relative spatial relationships between interacting agents. Experimental results demonstrate the effectiveness of the proposed approach, showing improved motion quality and interaction plausibility.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- The paper should include comparisons with additional baseline methods to better demonstrate the effectiveness of the proposed approach.\n- The paper would benefit from a clearer discussion of how the proposed loss function differs from those used in previous work.", "broader_impact_concerns": "N.A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "CEUJXoBNHm", "reviewer": "Reviewer_r2bf", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- The authors propose an interesting new task: reactive motion generation.\n- The authors proposed some novel training losses - kinematic loss and interaction loss given the task.\n- The paper is well written and easy to understand\n\nWeaknesses\n- I am confused about the ablation study in Table 2, it seems that the method without kinematic loss and with interaction loss show better overall performance in all metrics. Please explain the underlying reason. I am not completely convinced that the kinematic loss is effective at this point.\n- Although the proposed task is novel, I feel it is much simpler than the two-body motion estimation proposed in InterGen, because in this task we additionally have the known motion of one human body with paired text prompt, which greatly simplifies the task. Therefore, even if you provide the actor's motion information in inference, I don't think it is a fair comparison with InterGen, and I am not surprised that the proposed model is better than other methods.\n- The figures are a bit misleading because it is shown with the SMPL model, while the full method does not use SMPL in fact, and the interaction loss does not consider the contact and disjointness between the two SMPL meshes.\n- The proposed two-stage prediction method is not novel idea in motion generation.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper focuses on the task of reactive action generation given text prompts and actor actions. The authors propose a two-stage approach: first generating trajectories, then generating responsive actions based on the trajectories. The authors demonstrate the effectiveness of the approach both quantitatively and qualitatively compared to baseline methods and present an extensive ablation studies on design choices.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see my detailed comments in the [Weaknesses] section above. I think the paper needs significant revision to meet TMLR standards.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)