CooHOI: Learning Cooperative Human-Object Interaction with Manipulated Object Dynamics
Authors: Jiawei Gao
Ziqin Wang
Zeqi Xiao
Jingbo Wang
Tai Wang
Jinkun Cao
Xiaolin Hu
Si Liu
Jifeng Dai
Jiangmiao Pang
Venue: NeurIPS 2024 spotlight (2024)
Keywords: Physics-based Character Animation
Human-Object Interactions
Multi-Agent Cooperation
Primary area: reinforcement_learning
TL;DR: A novel framework for physics-based animation of multi-person carrying tasks.

Abstract: Enabling humanoid robots to clean rooms has long been a pursued dream within humanoid research communities. However, many tasks require multi-humanoid collaboration, such as carrying large and heavy furniture together. Given the scarcity of motion capture data on multi-humanoid collaboration and the efficiency challenges associated with multi-agent learning, these tasks cannot be straightforwardly addressed using training paradigms designed for single-agent scenarios. In this paper, we introduce **Coo**perative **H**uman-**O**bject **I**nteraction (**CooHOI**), a framework designed to tackle the challenge of multi-humanoid object transportation problem through a two-phase learning paradigm: individual skill learning and subsequent policy transfer. First, a single humanoid character learns to interact with objects through imitation learning from human motion priors. Then, the humanoid learns to collaborate with others by considering the shared dynamics of the manipulated object using centralized training and decentralized execution (CTDE) multi-agent RL algorithms. When one agent interacts with the object, resulting in specific object dynamics changes, the other agents learn to respond appropriately, thereby achieving implicit communication and coordination between teammates. Unlike previous approaches that relied on tracking-based methods for multi-humanoid HOI, CooHOI is inherently efficient, does not depend on motion capture data of multi-humanoid interactions, and can be seamlessly extended to include more participants and a wide range of object types.

CooHOI: Learning Cooperative Human-Object Interaction with Manipulated Object Dynamics
Jiawei Gao +1.2, Ziqin Wang +133 Zeqi Xiao ,Jigho Wang', Tai Wang', Jinkun Cao', Xiaolin Hu Si Liu Jifeng Daif2, Jiangmiao Pang Shanghai AI Laboratory, Tsinghua University, Beihang University, Nanyang Technological University, 'Carnegie Mellon University,
Abstract
Enabling humanoid robots to clean rooms has long been pursued dream within humanoid research communities. However, many tasks require hulti-humanoid collaboration, such as carrying large and heavy furniture together. Given the scarcity of motion capture data on multi-humanoid collaboration and the efficiency challenges associated with multi-agent learning, these tasks cannot be straightfor- wardly addressed using training paradigms designed for single-agent scenarios. In this paper, introduce Cooperative Human- -bjaett Interaction (CooHOI). framework designed to tackle the challenge of multi- halli--humaoid object transporta- tion problem through two-phase learning paradigm: individual skill learning and subsequent policy transfer. First, a single humanoid character learns to interact with objects through imitation learning from human motion priors. Then, the humanoid learns to collaborate with others by considering the shared dynamics the manipulated object using centralized training and decentralized execution (CTDE) multi-agent RL algorithms. When one agent interacts with the object, resulting in specific object dynamics changes, the other agents learn respond appropriately, thereby achieving implicit communication and coordination between teammates. Unlike previous approaches that relied on tracking based methods for muuli-humannii HOI, CooHOI is inherently efficient, does not depend on motion capture data of multi-humanoid interactions, and can be seamlessly extended include more participants and wide range of object types.
Introduction
Imagine the scenario where you're moving home and seeking help from humanoid robots. How- ever, certain items, like beds and sofas, are too large and heavy for a single robot to manage effectively. Despite significant advancements in learning and control of physics-based human- object interactions 66,
7 and humanoid robots performing complex opera- 39 the area of multiple humanoids collaboratively transporting objects Abjects-pprriicuarry when single individual may find it challenging handle heavy or long items-remaing relatively inder-investigated. Some previous efforts tried to address these challenges using tracking-basse methods, capturing interactions between two characters during tasks like box carrying, and training policies to mimic these actions. However, obtaining comprehensive motion data on interactions involving multiple participants is costly, and tracking-based methods struggle to adapt to scenarios with varying object sizes and an increased number of agents. Another straightforward idea might be directly train a multi-agent policy from scratch, using approach
43
21
37 20
45
tions (29 19 26
2
10,
9
"Equal contributions. Email: winstongu206 @ gmail.com, wzqin @ @huaa.cdu.ccom Corresponding author. Email: pangjiangmiaod @ gmail.com
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

COOHOL
Multi-Agent Transportation and Cooperation
Carry Objects In A Natural Style
Interaction with Diverse Dally Objects
Figure 1: Our framework empowers physically simulated characters to execute multi-agen human-
object interaction (HOI) tasks with naturalness and precision.
similar that for single agents. However, the expanded action sampling space significantly slows down the overall training process, preventing the policy from converging properly.
Inspired by how humans learn cooperation skills, i.e. beginning with mastering tasks independently and then developing strategies for collaborative efforts coordination tasks, we propose a two- stage framework for training multi-agent cooperation strategies, named Cooperative
Cuiman-Object Interaction, CooHOL In the initial stage, we train a single-agent object carrying policy utilizing the AMP framework (25 7 To ensure agents focus significantly on the dynamics of the objects they tasked with transporting, we include the state and velocity information of the object's 's bounding box in the agent's observation space. The second stage aims transfer these individual carrying skills into collaborative strategies. When two humans collaborate to carry long object, they typically hold at opposite ends. Moreover, when one individual takes action, it affects the dynamies of the object, enabling the other individual to perceive this change and adjust their actions accordingly for effective adjust the observation for both agents focus on the bounding boxes
cooperation. Therefore, located at the ends of the long objects. This refinement aligns with the observation space used during the single-agen agget training phase, effectively leveraging the previously developed single carrying skills. Additionally, the inherent rigid body characteristic of long objects facilitates implicit communication between agents. This setup allows an agent to adeptly adjust their teammate's actions by observing changes in the object's is dynamics, thereby enhancing coordination and cooperation in carrying tasks. The overall framework illustrated Figure 2
To validate the effectiveness of our framework, we conducted experiments where we trained control policies for two humanoid characters carry various long objects. such as boxes and sofas. Our results demonstrate that our framework enables these characters to exhibit natural-looking behaviors while successfully completing cooperative tasks, utilizing only motion capture data from one single agent. We compared our approach against the baseline method of training from scratch and performed detailed ablation studies to evaluate the impact of our design decisions, also testing the limitations of our framework. In summary. our main contributions are as follows: 1)We have developed an efficient and robust framework, CooHOL, for training physically simulated characters cooperative object transporting tasks, demonstrating significant effectiveness. 2) have established that utilizing object dynamics for communication proves to an effective strategy learning cooperative object transporting tasks.

---

Related Work
Physies-based Hluann-Ojjet Interaction Motion Synthesis. Synthesizing natural and physically plausible human-scene interactions, such as humanoid characters sitting on chairs, lying on beds, and carrying boxes, crucial for advancements in character animation and robotics. Physics-bass basse methods leverage physics simulators [88 35] to control characters modeled as interconnected rigid bodies through joint torques and deep reinforcement learning methods. To facilitate the training process, some strategies employ tracking-based techniques [16] 23
38
20 44], which rely on the
availability of high-quality reference motions. This reliance often limits their application in human- scene interactions due to the scarcity of suitable data and affects their versatility across different scenarios. Recently, the Adversarial Motion Priors Priors(AMP) framework [25] introduced the use of discriminator ensure that generated motions align with the distribution of reference motions. This approach has shown success various downstream tasks [1] 24 34], including human-scene interactions 27 37). Despite these advancements, there has been limited focus synthesizing cooperative behaviors among multiple characters interacting with objects gap that our work aims to address.
While significant advancements have been made synthesizing mo- tions for single agents, the realm of multi-character animation remains relatively unexplored. Existing approaches predominantly rely on kinematic-basse methods [6 14 32 30 51 and datasets of multi- character interactions
Culti-Characttr Control.
31 4 These methods, however, require high-quality interaction data and often fall short ensuring physical plausibility or adequately handling multi-character cooperative interactions with objects. Physics- based techniques for character animation have primarily focused on aspects like crowd navigation [8] (27. limiting themselves to behaviors such as pedestrian movement and collision avoidance. Although 44 showcases interactions among multiple characters and object manipulation, such as carrying long boxes, these tracking-basse approaches necessitate motion capture data of human interactions or human-object interactions and struggle to scale an arbitrary number of agents or different types of objects. In contrast, our framework requires only single agent motion capture data for multi-character object transporting tasks and can easily extend to different types of objects and different numbers of agents.
13
3 Methodology
Figure 2 shows the overall framework of our approach. Our method utilizes motion data from individual agents and involves a two-stage learning process to create cooperative control policies. First, we train policy for single-agent carrying tasks, using the dynamics of the manipulated object
Policy
Diseriminator
Skill Transfer
Policy
Policy
Shared
r
a
Mation Data
Large Objects
Single Agent Skill Learning
Objects
Multi Agent Cooperation Learning
Our framework employs two-phase learning paradigm. In the first phase, depicted on the left, we train single-agent carrying skills by imitating from human motion priors. the second phase, we transfer these single-agent skills to a cooperative context. Notably, we use the dynamics of the object as feedback information, as illustrated by the bounding box shown the figures.
Figure 2:

---

as feedback, as described in Section 3.2 Then, we use parallel training to develop cooperative strategies, with changes the object's dynamics acting as form of implicit communication. This further explained in Section 3.3
3.1 Preliminary
Physics-based Character Control. We formulate physics-based character control a goal- conditioned reinforcement learning task. At each time step 4, the agent samples an action from its policy (ax St, gr) based on the current state S, and the task-specific goal feature ge. When this action is applied to the character, the environment transitions to the next state St+1, and the agent receives a task reward , (st, ge: St+1). To train control policies that enable characters to achieve high-level tasks in a natural and life-like manner, we adopt the AMP framework [25]. While this framework aims to optimize the expected cumulative task reward J(TT), introduces a discriminator to encourage the character to produce behaviors similar to those in the dataset by providing a style reward S S141)) The agent's reward rz each time step is defined by w (st. gr, S141) ) + w n (st, S1+1). More details can be found in our appendix and the original AMP paper [25].
(z:
Multi-Agent Reinforcement Learning. We formulate our cooperative task as a Partially Ob- servable Markov Decision Process (POMDP) [T3]. POMDP with n agents defined by (S, ... AA.. OI. O,, R, P, J) where S is the state space, A; is the action space for agent i, O is the local observation for agent i, and is the shared reward function. Each agent uses its own policy To (a: o.) to take action E A, based on its local observation environment transitions according to the function (S St; a1, a.), where S,, St4 are states of time step / and + 1. respectively. The agents then get a reward r, based on the states and 81+1- The goal of multi-agent reinforcement learning algorithms is to jointly optimize the discounted accumulated reward J(0) Eng at [
E O, The
3.2 Single Agent Carrying Skills Training
In developing our approach for single-agent object manipulation, we introduce several advancements based on previous methods. We integrate the dynamics of the manipulated object into the observation space and introduce a reward function framework for object manipulation tasks. These enhancements allow the trained policy easily adapt multi-agent settings.
3.2.1 Enriched Goal Feature with Manipulated Object Dynamics as Feedback.
For successful object carrying tasks, we emphasize the critical role of using the dynamics of the manipulated object feedback. These dynamics captured through the eight vertices of the object o's bounding box 6y its rotation angle b, facing its velocity by and its angular velocity by" as described
in Equation
D. = concatenate(1)" facing Dy b.t).
(1)
By incorporating these dynamics information into the observation, we establish a feedback mechanism that keeps agents continually informed about the outcomes of their actions. This also equips agents with the capability to react appropriately, whether engaged in single-agent tasks or collaborative multi-agent environments. Along with the state of the agent s, and the position of the target dpos formulate the dynamics-aware task observation as:
concatenate(ss, dposs D.).
3.2.2 Enriched Task Design facilitating Efficient Skill Transfer.
To facilitate the transition from single-agent to multi-agent object carrying tasks, decompose the carrying process into three sub-tasks: walking towards the objects, lifting the object from the ground, and carrying the object its intended destination. Consequently, the reward system is structured into three components: Twalk Theld and F Furget-
To encourage the agents choose the face of the object they will face while carrying, we introduce an additional goal feature called stand points stand During training, these stand points are randomly allocated to positions directly in front of the various faces of the object at time step t. This strategy is

---

designed to facilitate multi-agen agent cooperative training, helping the agents learn to avoid walking to the long side of the object where is difficult to grasp and carry. We then define "walk as the distance between the agent and the stand point, as specified in Equation
Twalk exp (1=7
standing
Additionally, in scenarios where multiple agents are transporting an object, the object's 's considerable size often makes it difficult for agents to find an appropriate grip point for lifting. To address this, we introduce a novel concept called held points. Specifically, we select the geometric center hn of each end of the object as the held point and encourage the agents to interact at these points. The reward function Theld, defined in Equation 4 uses 2t hand to represent the mean position of the agent's 's two hands. This design aids the transition from individual to collective task environments by encouraging agents to identify the held points at the geometric center of each end of the long object.
Theld = exp (1|=)
hell
We then define T Iaarget to encourage agents carry the object the destination:
Vlarget =
exp
(1 larger
x: 12).
object
(5)
3.3 Cooperation Strategy Training
Upon mastering single-agen tasks involving object carrying, it is crucial effectively transfer and further enhance these abilities to boost the cooperative learning process. Initially, we facilitate the skill transfer by guiding each agent to lift and transport one end of large object. Subsequently, we refine the control policy through the application of the Multi-Agent Proximal Policy Optimization (MAPPO) algorithm [40]. leveraging the dynamics of manipulated objects for feedback and implicit communication.
3.3.1 Efficient Skill Transfer Using Dynamics Information.
Initially, we replicate the single-agent object-carrying policy for all agents and then fine-tune it within a cooperative framework. However, handling a long object poses challenge due to its size, which complicates the agents' ability to identify suitable lifting points, thereby hindering the efficient application of their object-carrying skills. To address this issue, we encourage agents to observe the cettiiion--eefiicallly.. the state and velocity data of the bounding box at each end of the long object, as outlined Section [32.T] This approach allows agents to simulate carrying smaller box positioned at the long object's ends, using the dynamics of this "smaller box" as feedback, similar to the single-agent skill training process. Additionally, we incorporate previous methodologies, such as stand points and held points described in Section 3.2.2 to enhance the smooth transition of skills.
dynamics
Moreover, this design of dynamics as observation acts as implicit communication channel between agents. When an agent takes an action, the object dynamics will change, and because we use the manipulated object dynamics as feedback in single-agent training, the change in object dynamics will result in corresponding change in strategy for the other agents. This method of implicit communication presents straightforward yet effective approach for enhancing teamwork in multi- agent settings. Furthermore, this framework proves to be adaptable with variations in the number of participating agents.
3.3.2 Cooperation Training using CTDE Scheme.
The non-stationary enviromment of multi-agent reinforcement learning, coupled with sample efficiency issues, presents a significant challenge in training agents to collaborate effectively. During the cooperation training phase, we continue to use a reward function similar to that used in the single- agent training phase and employ the centralized training and decentralized execution (CTDE) [15] scheme for coordination training. During the training phase, we utilize the Multi-Agent Proximal Policy Optimization (MAPPO) algorithm 40 to develop cooperative strategies among agents. This approach involves updating the value function network V, according to Equation (6), using the trajectories D that are accumulated and shared among all agents. More details can be found in our
5

---

Table 1: This table presents our results for single-agent and two-agent carrying of the Box object. "CooHOl" refers the policy trained using the complete CooHOI framework both single-agent and two-agent settings. "CooHOI+ Weighttugg indicates that we applied the same weight augmentation design as InterPhys 77.
Agent Number
Methods
Weight (kg)
Distance(m)
Success Rate(%)
Precision (cm)
InterPhys CCoHO1--eighhAuu(OOn
[5,26]
[1,10]
94.3
8.3
Single Agent
[2,26]
[1,20]
93.98
4.8
(Our))(Ours)
[2,13]
[1,20]
96.48
6.9
From Scratch
[15,40]
[2,20]
0
NAN
Two Agent
CooHOI(Ours)
[15,40]
[2,20]
89.54
3.86
Figure Carrying performance for objects of different categories. From left to right: Table, Armchair, and High Stools. All objects were required to be moved to a location meters away.
appendix and the original MAPPO paper 40
T
Øk+1
arg min |D T
Σ TED t=0
Σ
Vo (ot, St, St, Ra) Ri)2
(6)
In addition, given the homogeneous roles of agents in the collaborative task of carrying long objects, we adopt the strategy of parameter sharing, a method proven to enhance performance across various cooperative tasks [3] 33 40). Specifically, share the parameters of both the policy and value networks among all agents to improve the training of cooperative behaviors.
Experiments
We conducted extensive experiments to test the effectiveness and also the boundary capabilities of our framework. The basic experiment setups are explained in Section 4.1 and in Appendix. We
evaluate our framework on various object-carrying tasks in Section 4.2 To better understand the importance of different design decisions in framework. performed extensive ablation studies in Section 4.3 Since our method primarily focuses on interactions between characters and objects, also provide extensive visual analysis and presentations demonstrate our framework.
4.1 Experiments Setup
Datasets and Initialization. Our primary source of motion data is the AMASS dataset [17]. which provides motions encoded in SMPL (22 parameters. We collected total of four types of basic reference motion data, including motions related to walking, related to picking up, related to carrying, and 5 related to putting down. To enhance the robustness of the carrying process, we randomly initialized the weight and size of the object, as well as the distance from the person to the destination. Specifically, for single individual, the object's weight ranged from 5KG 25KG, its size varied between 0.5 to 1.5 times its original scale, and the distances between the agent and the object, as well as between the object and the destination, ranged from m 20 m. To enhance the robustness of the carrying process, we randomly varied the weight, size, and of the object, as well as distance from the person to endpoint. Furthermore, considering that the process multiple people carrying might involve situations where someone walks backward, we ensured that the single malti-person collaboration by introducing additional motion
agent mastered the basic movements data involving backward and side walking. More training details can be found in the appendix.

---

Table 2: daily life with simple fine-tuning. We tested the performance of our policy model across different objects.
The trained policy exhibits the ability to handle various object categories encountered in
Agent Number
Object Category
Distance(m)
Success Rate(%)
Precision (cm)
Table
[1,20]
97.07
5.23
Single Agent
Armchair
[1,20]
97.26
5.05
HighStool
[1,20]
99.21
4.02
Two Agents
Sofa
[2,20]
84.17
10.12
Get close to the sofa
Pick up the sofa
Put Down sofa to destination
Get close to the box
Pick up the
Caarry" destination
Figure 4: Visualization of cooperative carrying in the multi-agent scenario.
Metrics. Following InterPhys 77 we utilize Success Rate and Precision the primary metrics. Specifically, a task is considered successful if the distance between the object and its destination is less than 0.2m. Precision measures the distance between the object and the target point across all completed tasks. Moreover, all evaluations are based on average results across 4096 environments and random seeds.
Evaluation on Object Carrying Tasks
Comprehensive Evaluation on Box Carrying. We evaluate our policy in both single-agent and two-agent scenarios. Table presents the performance statisties for carrying task. Consistent with the settings described in Interphys (7). randomize the weight and distance outlined in 1ectior4.1] is important to note that InterPhys [7] includes the weight information objects as part of the observation, enabling it to achieve broader weight range. To ensure result comparability, as shown in Line 2, we also incorporated this information into individual training. The results indicate that, with similar success rate, increased the transportation distance and improved transportation accuracy, even when using discrete motion data rather than the costly whole-body motion capture data used in InterPhys [7]. As shown in Line although removing the weight information reduces the weight range the agent can handle, it represents a more realistic setting and provides comparable baseline results for future research. Lines 4-5 show the results in multi-agent scenarios. Without using our CooHOI framework and simply employing parallel training for multi-agent tasks, the training fails. More analysis and visualizations will be provided in the following sections.
Extend to Diverse Daily Life Objects. Moreover, we are not satisfied with merely transporting simple boxes. Therefore, we have sampled nearly 40 common everyday objects from (,, including three types of single-person objects-Tafl Table, Armchair, and HighStool person object, the Sofa, as illustrated in Figure and Line Figure Additional visualizations of these objects can be found in appendix. Additionally, present the results of quantitative
well one type of two-
7

---

Success Rate and Steps of One-Agent-Policy and Two-Agent-Policy
20.0
Twe Agunt-Polcyy Success Rate
Ona-Agante Policy Succeen Rate
320
Two Agent Policy Success Rato
Ons-Agant-Pullcy Succest
Neo-Agent-Policc Success Rate
Seesdgeent-Palig Policy Succees Rade
100
15.0
Succoee Rate (50
Success Rake 150
100
60
12.5
80
12.5
12.5
10.0
10.0
80
10.0
7.5
7.5
7.5
5.0
5.8
5.0
20
One-Agent-PPlicy Palliyy step
One Per--AgenntPPlll Policy step
One-Agent-Palicy step
2.5
Pro-Agent-PPlicy step
Teo-Agent-Policyy step
Tho Poao--ggntPPaccycc.....o stop
10
20
60
0.6
6.8
3.0
1.2
2.5
5.0
16.0 12.5 15.0 17.5 20.0
Figure 5: Detailed ablation experiments on single and two agents cases. "Step" measures the average consumed time in the successful cases. In the 2nd figure, the green circle represents the single-agent scenario without scaling the object's width, while the purple circle represents the multi-agent scenario.
experiments in Table which indicate that our policy can effectively handle most daily life objects with high success rate. Due to the greater variability in the shape and weight of sofas, the accuracy in Line 4 shows slight drop compared to the result in Line 4 of Table [ It is important to note that the sigmificant variations in object shapes make it challenging to learn a unified object representation with limited amount of data, especially since our input is merely simple bounding box. Therefore, for single-person person transport, different types of objects need to be trained separately from scratch, similar to how handle boxes. For multi-person transport, objects like sofas, similar to boxes, require fine-tuning based on the weights obtained from single-person box transport.
4.3 Ablation Studies
To evaluate and understand the importance of different design choices in CooHOI, we conduct detailed analysis of scenarios involving single and multiple agents. This includes boundary analysis, which explores factors that lead decrease in the success rate during object carrying. Furthermore, to fully validate the scalability of our method, we also tested and included the results in a four-person scenario. To ensure the reliability of our results, our experiments continue to utilize the average of random seeds as previously mentioned.
Single-Agent-Based Box Carrying. We have demonstrated our framework's 's effectiveness in Table [ Moreover, we aim to further investigate the maximum potential of our approach. We examined the robustness of the single-agent policy from various perspectives, including object shape, scale, weight, and trajectory length, as shown in Figure 5 First, we observed that a single agent can handle approximately 13 kg weight. Continuously increasing the weight makes difficult for the agent to lift the object. As mentioned in 4.2 since our policy input does not include object density, it is challenging for the agents to generalize across different weights. When the weight reaches 20 kg. the accuracy drops to just 30%. Additionally, due the limited reach of human arms and the lack of dexterous hands in our agent, it is challenging for the agent to lift boxes that are either too large or too small. To validate this hypothesis, we restricted the object's width to Ix while scaling the length and height to 1.5x. As shown by the green circle in the second figure of Figure 5) the success rate could reach 97.8%. Furthermore, our strategy is quite robust to distance variations and is generally unaffected by them.
Salti-Agget-Baasee Box Carrying. We conducted upper-limit testing on the multi-agent policy using a similar evaluation method as the single-agent case. As shown in Figure the conclusions for two agents are similar those for single agent. The curves in the first figure demonstrate that with an increase in the number of agents, we can easily lift larger and heavier objects that a single person cannot handle, highlighting the necessity of having multiple agents. However, due to the increased complexity of coordinating two agents, boundary conditions have more pronounced impact on the policy. For instance, objects that are excessively large or small can cause the policy to fail. As shown by the purple circle in the second figure of Figure we conducted a similar experiment to the agent scenario. By restricting the object's 's width to 1x and scaling the length and height to 1. 1.5x, the success rate increased from 0 to 88.67%.
8

---

Analysis of CooHOI Framework. To thoroughly investigate the contribution of CooHOI, we analyzed the results of each method mentioned in Dection[[ssppratte] as shown by the training curves in Figure 8 The first factor is the influence of the Stand Point, which refers to whether an extra point introduced in front of the object to encourage the agent to walk toward During the experiments, we discovered that without this, the agent sometimes fails to approach the shortest edge of the object, resulting in a lower hold reward. This leads to an incomplete lift and the subsequent inability to carry the object. Dynamic Observation is the second factor, indicating whether we use dynamic information as observation for each agent. Without it, the observation for each agent is limited to the state information of the long object. We found that without the dynamics information, the agent just stands in front of the object, seemingly unsure of what to do. Reverse Walk indicates whether the training process includes motion data for walking backward and a reward function focused on learning this movement. We found that if the policy for a single agent is restricted to only forward movement, training with two agents then leads to a deadlock state. As shown in Figure 8 the agents might be able to contact the box, but they cannot carry it to the destination. Initialization refers to whether the two-agent policy is initialized using the single-agent policy and then be fine-tuned. In our experiment, even with extended training duration, training the two-agent policy from scratch still failed to achieve successful carrying, as shown in Figure X Based on the results above, the absence of any of the aforementioned methods causes our policy to fall into locally suboptimal solution, preventing the completion of the transportation task. Moreover, you can also find more interesting visual examples of the above failure cases while extending agent numbers from one to two in the appendix.
5 Conclusion and Limitation
In this paper, we present Cooperative Human-Object Interaction (CooHOI), a framework designed to address cooperative object transporting tasks through a two-phase learning approach. By initially focusing on individual skill mastery via the Adversarial Motion Priors (AMP) method, followed by a strategic transition to multi-agent collaboration using Multi-Agent Proximal Policy Optimiza- tion (MAPPO), our approach facilitates a sophisticated interplay of shared dynamics and implicit communication among agents, resulting in an efficient and generalized performance.
However, within the scope of this paper, our huamnoid characters lack dexterous hands, which limit their ability to manipulate slippery objects or perform more precise actions. We also utilize object bounding box information as the goal feature for our task, which limits our framework's 's capacity to generalize
diverse range of object shapes. Additionally, our experiments are limited to humanoid characters in simulation, without any real-world deployment. In future research, we aim to incorporate dexterous hands to enable the manipulation of wider variety of objects, as well as integrate active perception and navigational abilities to make our framework more generalizable.
Acknowledgments
This work is supported by Shanghai Artificial Intelligence Laboratory. Ziqin Wang is supported in part by Key R&D Program (2022ZD01 15502). Natural Science Foundation (NO. 62122010, U23B2010), Zhejiang Provincial Natural Science Foundation under Grant No. LDT23F02022F02. Key Research and Development Program of Zhejiang Province under Grant 2022C01082, "Pioneer" and "Leading Goose" R&D Program (No. 2024C01161).
References
[1] Yu-Wei Chao, Jimei Yang, Weifeng Chen, and Jía Deng. Learning to sit: Synthesizing human- chair interactions via hierarchical control. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 5887-5895, 2021.
[2] Xuxin Cheng, Yandong Ji, Junming Chen, Ruihan Yang, Ge Yang. and Xiaolong Wang. Expres- sive whole-body control for humanoid robots. arXiv preprint arXiv:2402. 16796, 2024.
[3] Filippos Christianos, Georgios Papoudakis, Arrasy Rahman, and Stefano V. Albrecht, Scaling multi-agent reinforcement learning with selective parameter sharing. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning,

---

ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pages 1989-1995, PMLR, 2021.
[4] Mihai Fieraru, Mihai Zanfir, Elisabeta Oneata, Alin-lonut Popa. Vlad Olaru, and Cristian Sminchisescu. three-dimensiones reconstruction human interactions. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 7214-7223, 2020.
[5] Anindita Ghosh, Rishabh Dabral, Vladislav Golyanik. Christian Theobalt, and Philipp Slusallek. Remos: Reactive 3d motion synthesis for two-person interactions. arXiv preprint arXiv:231 70577, 2023.
[6] Mohamed Hassan, Duygu Ceylan, Ruben Villegas, Jun Saito, Jimei Yang, Yi Zhou, and Michael Black. Stochastic scene-aware motion prediction. Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 1 11374-1133 11384, 2021.
[7] Mohamed Hassan, Yunrong Guo, Tingwu Wang, Michael Black, Sanja Fidler, and Xue Bin Peng. Synthesizing physical character-scene interactions. arXiv preprint arXiv:2302.008: 2023.
[8] Brandon Haworth, Glen Berseth, Seonghyeon Moon, Petros Faloutsos, and Mubbasir Kapadia. Deep integration of physical humanoid control and crowd navigation. Proceedings of the 13th ACM SIGGRAPH Conference on Motion, Interaction and Games, pages 1-10, 2020.
[9] Tairan He, Zhengyi Luo, Xialin He, Wenli Xiao, Chong Zhang, Weinan Zhang, Kris Kitani, Changliu Liu, and Guanya Shi. Omnih2o: Universal and dexterous numan-to-humann whole-body teleoperation and learning. arXiv preprint arXiv:2406. 08858, 2024.
[10] Tairan He, Zhengyi Luo, Wenli Xiao, Chong Zhang, Kris Kitani, Changliu Liu, and Guanya Shi. Learning suman-to-huuannee real-time whole-body teleoperation. arXiv preprint arXiv:2403.04430 2024.
[11] Jordan Juravsky, Yunrong Guo, Sanja Fidler, and Xue Bin Peng. Padl: Language-directer physics-based character control. In SIGGRAPH Asia 2022 Conference Papers, pages 1-9, 2022.
[12] Diederík P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 14Xi::1412.6980, 2014.
[13] Jogendra Nath Kundu, Himanshu Buckchash, Priyanka Mandikal, Anirudh Jamkhandi, Venkatesh Babu Radhakrishnan, et al. Cross-conditioned recurrent networks for long-term synthesis of inter-person human motion interactions. In Proceedings of the IEEE/CVF winter conference applications of computer vision, pages 2724-2733, 2020.
[14] Han Liang, Wenqian Zhang, Wenxuan Li, Jingyi Yu, and Lan Xu. Intergen: Diffusion-based multi-human motion generation under complex interactions. arXiv preprint arXiv:2304.05684, 2023.
[15] Michael L. Littman. Markov games framework for multi-agen reinforcement learning. In William W. Cohen and Haym Hirsh, editors, Machine Learning. Proceedings of the Eleventh International Conference, Rulgers University, New Brunswick, NJ, USA, July 10-13, 1994, pages 157-163. Morgan Kaufmann, 1994.
[16] Libin Liu, KangKang Yin, and Baining Guo. Improving sampling-based motion control. In Computer Graphics Forum, volume 34, pages 415-423 Wiley Online Library, 2015.
[17] Naureen Mahmood, Nima Ghorbani, Nikolaus F Troje, Gerard Pons-Moll, and Michael J Black. Amass: Archive of motion capture as surface shapes. Proceedings the IEEF/CVF international conference computer vision, pages 5442-5451, 2019.
[18] Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, et al. Isaac gym: High performance gau-based physics simulation for robot learning. arXiv preprint arXiv:2108. 10470, 2021.
10

---

[19] Yutaro Matsuura, Kento Kawaharazuka, Naoki Hiraoka, Kunio Kojima, Kei Okada, and Masayuki Inaba. Development of a whole-body work imitation learning system by biped and bi-armed humanoid. In 2023 IEEE/RS/ International Conference on Intelligent Robots and Systems (IROS), pages 10381. IEEE, 2023.
[20] Josh Merel, Saran Tunyasuvunakool, Arun Ahuja, Yuval Tassa, Leonard Hasenclever, Vu Pham, Tom Erez, Greg Wayne, and Nicolas Heess. Catch & carry: reusable neural controllers for vision-guided whole-body tasks. ACM Transactions on Graphics (TOG), 39(4))3--,, 2020.
[21] Liang Pan, Jingbo Wang, Buzhen Huang, Junyu Zhang, Haofan Wang, Xu Tang, and Yan- gang Wang. Synthesizing physically plausible human motions in 3d scenes. arXiv preprint arXiv:2308. 09036, 2023.
[22] Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed AA Osman, Dimitrios Tzionas, and Michael J Black. Expressive body capture: 3d hands, face, and body from a single image. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10975-10985, 2019.
[23] Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel Van de Panne. Deepmimic: Example- guided deep reinforcement learning of physics- character skills. ACM Transactions On Graphics (TOG), 37(4):1-114 2018.
[24] Xue Bin Peng, Yunrong Guo, Lina Halper, Sergey Levine, and Sanja Fidler. Ase: Large-scale reusable adversarial skill embeddings for physically simulated characters. ACM Transactions On Graphics (TOG), 41(4):1 2022.
[25] Xue Bin Peng, Ze Ma, Pieter Abbeel, Sergey Levine, and Angjoo Kanazawa. Amp: Adversarial motion priors for stylized physics-based character control. ACM Transactions on Graphics (ToG), 40(4):1-20, 2021.
[26] Ilija Radosavovic, Tete Xiao, Bike Zhang, Trevor Darrell, Jitendra Malik, and Koushil Sreenath. Learning humanoid locomotion with transformers. CoRR, 230220333333,,, 2023.
[27] Davis Rempe, Zhengyi Luo, Xue Bin Peng, Ye Yuan, Kris Kitani, Karsten Kreis, Sanja Fidler, and Or Litany. Trace and pace: Controllable pedestrian animation via guided trajectory diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13756-13766. 2023.
[28] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.063: 2017.
[29] Mingyo Seo, Steve Han, Kyutae Sim, Seung Hyeon Bang, Carlos Gonzalez, Luis Sentis, and Yuke Zhu. Deep imitation learning for humanoid loco- mcc-manipulation through human teleopera- tion. In 2023 TEEE-RAS 22nd International Conference on Humanoid Robots (Humanoids), pages 1-8. IEEE, 2023.
[30] Yonatan Shafir, Guy Tevet, Roy Kapon, and Amit Bermano. Human motion diffusion as a generative prior. arXiv preprint arXiv:2303.014: 2023.
[31] Yijun Shen, Longzhi Yang, Edmond SL Ho, and Hubert PH Shum. Interaction- based human activity comparison. IEEE Transactions on Visualization and Computer Graphics, 26(8):2620-
2633, 2019.
[32] Mikihiro Tanaka and Kent Fujiwara. Role-aware interaction generation from textual description. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1 15999- 16009, 2023.
[33] Justin K Terry, Nathaniel Grammel, Sanghyun Son, Benjamin Black, and Aakriti Agrawal. Revisiting parameter sharing in multi-agent deep reinforcement learning. arXiv preprint arXiv:2005. /3625, 2020.
[34] Chen Tessler, Yoni Kasten, Yunrong Guo, Shie Mannor, Gal Chechik, and Xue Bin Peng. Calm: Conditional adversarial latent models for directable virtual characters. In ACM SIGGRAPH 2023 Conference Proceedings, pages 1-9, 2023.

---

[35] Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model- based control. In 2012 JEEE/RS/ international conference on intelligent robots and systems, pages 5026-5033. IEFE, 2012.
[36] Jungdam Won, Kyungho Lee, Carol O'Sullivan, Jessica K Hodgins, and Jehee Lee. Generating and ranking diverse multi-character interactions. ACM Transactions on Graphics (TOG), 33(6):1-12, 2014.
[37] Zeqi Xiao, Tai Wang, Jingbo Wang, Jinkun Cao, Wenwei Zhang, Bo Dai, Dahua Lin, and Jiangmiao Pang. Unified human-scene interaction via prompted chain-of-contacts. arXiv preprint arXiv:2309.07918, 2023.
[38] Zhaoming Xie, Jonathan Tseng, Sebastian Starke, Michiel van de Panne, and Karen Liu. Hierarchical planning and control for box loco- mecommnnppulatio. arXiv preprint arXiv:2306.09532, 2023.
[39] Haoru Xue, Chaoyi Pan, Zeji Yi, Guannan Qu, and Guanya Shi. Full-order sampling- based mpe for torque-level locomotion control via diffusion-style annealing. arXiv preprint wrXiv:2409.16111 2024.
[40] Chao Yu, Akash Velu, Eugene Vinitsky, Jiaxuan Gao, Yu Wang, Alexandre M. Bayen, and Yi Wu. The surprising effectiveness of PPO in cooperative multi-agent games. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems 35: Annual Conference Neural Information Processing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28 December 9, 2022, 2022.
[41] Chong Zhang, Wenli Xiao, Tairan He, and Guanya Shi. Wococo: Learning whole-body humanoid control with sequential contacts. arXiv preprint arXiv:2406.06005, 2024.
[42] Qiang Zhang, Peter Cui, David Yan, Jingkai Sun, Yiqun Duan, Arthur Zhang, and Ren- jing Xu. Whole-body humanoid robot locomotion with human reference. arXiv preprint a4Xiv:2402. /8294, 2024.
[43] Xiaohan Zhang, Bharat Lal Bhatnagar, Sebastian Starke, Vladimir Guzov, and Gerard Pons- Moll. Couch:
Towards controllable human-chair interactions. In European Conference Computer Vision, pages 518-535. Springer, 2022.
[44] Yunbo Zhang, Deepak Gopinath, Yuting Ye, Jessica Hodgins, Greg Turk, and Jungdam Won. Simulation and retargeting of complex multi-character interactions. arXiv preprint 2:55v:2305.20041, 2023.
[45] Kaifeng Zhao, Yan Zhang, Shaofei Wang, Thabo Beeler, and Siyu Tang. Synthesizing diverse human motions 3d indoor scenes. arXiv preprint arXiv:2305. /2411, 2023.
[46] Yi Zhou, Connelly Barnes, Jingwan Lu, Jimei Yang, and Hao Li. On the continuity of rotation representations in neural networks. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 5745-5753, 2019.
12

---

Appendix
In this section, we categorize our discussion into three main parts. Initially, we delve into the sources and processing methods for motion data used in training. Following that, we explore how observations are constructed and how reward functions are established. Finally, we describe the implementation details including physics simulation and hyperparameters in network training.
A Sources and Processing of Motion Data
collected a total of four types of basic reference motion data, including motions related to walking, 5 related to picking up, 4 related to carrying, and 5 related putting down. All these data are SMPL format and recorded at 30 fps over 139 frames. They all originate from the ACCAD subset of the AMASS [17] dataset. Additionally, to ensure the stability of cooperative tasks involving multiple individuals, we included data for sidewalk and reverse carry motions. The sidewalk data comes from the CMU subset within AMASS, while reverse carry data scarce. Therefore, created reverse carry data by reversing the process the carry data. In total, we used 26 motion data as references. Additionally, performed simple visualization of the extended objects in Figure which sampled from dataset [6]
ArmChair
HighStool
Table
Sofa
Figure 6: Some visualization of daily-life objects.
B Task Formulation
We formulate our approach goal-conditioned reinforcement learning. At each discrete step the policy π (a: s;, g.) generates an action a,, based on the current state and goal-specife feature ge. Following this action, the environment transitions into subsequent state, and the agent receives reward r;. An episode concludes either after reaching a predetermined length or if conditions for early termination (ET) are met. Further details are provided below.
B.1 Task Observation
The observational for the task is divided into two primary elements: the state feature s, which encapsulates the character's bodily configuration, and the goal feature g, which pertains tasks involving object manipulation.
The state feature is constituted by 22--dimensional vector, encompassing:
Height of the root: dimension.
13

---

Rotation of the root: 6 dimensions.
Linear and angular velocity of the root: dimensions.
Position of local joints: 42 dimensions.
Rotations of local joints: 84 dimensions.
Linear and angular velocity of local joints: 84 dimensions.
While the root height is measured in the global reference frame, all other components are defined in the frame local to the character. Rotations follow 6-dimensional representation for continuity [46].
The simulated character aligns with [25 24 7 21 featuring 12 internally movable joints and total of 28 degrees of freedom.
The goal feature g comprises 75-dimensional vector, including:
Position of the object: 3 dimensions.
Rotation of object: dimensions.
Dynamics of the object, which cover the bounding box position, along with linear and angular velocities: 33 dimensions.
Target location: 3 dimensions.
Target orientation: dimensions.
Dimensions the target's bounding box: 24 dimensions.
These are measured in the frame local to the character.
B.2 Reward Functions
The agent's reward r, at each time step / is defined by
F w G (St, gr, 81+1) S (8t,
Follow the formulation the AMP framework [25 the style reward S calculated according to
the discriminator:
3 (St, 81+1) = log (1 D (sc,
(8)
And the discriminator is trained by the following objective: arg min BBM((,)44)) [log (D (s, 8:41))]
Eg [log (1 (s,1))
(9)
+ 8P EJM
V D() (
The task reward function G is generally segmented into three components, as in Equation (10): 1) r.G "walk- which encourages the agent to approach the object intended for manipulation. 2) G held- which
encourages the agent align the center of its hands with the center of the box. 3) riurga: which encourages the agent to transport the object to the specified destination.
0.2 r walk + 0.4
helld + 0.4 * r.G
Ltarggt
(10)
The walk reward w is formulated as Equation where x standing point near the object,e* denotes the target velocity, and d= denotes the desired direction from root to the object.
standing denotes the position of the
mat 12² +
0.4
-0.5 F,
sandinng
G
0.4exp
-2. ||u
droot
12)+
(11)
0.2 ||d"
d
|| x
> 0.2m
1.0,
otherwise
14

---

The held reward helld is formulated in Equation 12 where z=!ind denotes the center of the agent's two hands and h, the position of the object helding point.
held exp (-5.0l hell2)
(12)
The target reward Iarget consist of two parts, Tearry and Tface. described in Equation 13
G target
0.5 Vearry + 0.5 Tface-
(13)
The face reward Place guides the agent to walk either forwards or backward. As shown in Equation (14) this achieved by comparing the agent's velocity direction with its orientation relative to the endpoint's 's location, thereby cultivating the agent's proficiency in bidirectional locomotion.
It
face
Ut
facc
faee It
(de
2000) N 0
Place
face
face
It (x. de)
(14)
The earry reward Teamy: designed guarantee that the object delivered to the precise location at specific angle. As outlined in Eq. [5] we constrain the agent's movement direction, alongside the proximity to the end destination and the intended angle. Within this context, x signifies 3D coordinates of the destination, while Pž represents the 2D destination coordinates. Similarly, Pt indicates the 3D position of the agent's root. Furthermore, rot designates the object's desired orientation.
0.25 rt dir 0.25 rs for 0.5 rt
| xt pox
0.1m
Tearry
(15)
* rt rt
+ 0.25 dir + 0.25,
otherwise,
where
re exp
-0.5 ||p p!.011))
neur exp (-10.0 | .c. 2et 102
object (2 rot,
rot" dir i
B.3
Reset and early termination condition
An episode ends either after reaching a predetermined duration upon the activation of early termination (ET) conditions. During our experiments, we observed that lower object heights could lead to kicking actions, where the agent tend kick the object destination, significantly slowing down the training process. To address this, we assess the object's velocity and height to determine the presence of kicking phenomena. If the height of the object is lower than 0.3m and its velocity in x-y plane is greater than I m/s, the kicking early termination (KET) condition is triggered. Experimental results show that this strategy significantly stabilize the training process.
C Implementation Details
C.1 Training Details.
Adopting the methodology of AMP (25), we develop low-level controller encompassing both policy and discriminator networks. The policy network is bifurcated into critic and an actor-network. each initiating with CNN layer and proceeding two MLP layers configured with [1024, 1024, 512] units. The discriminator network is similarly structured, featuring two MLP layers with [1024, 1024, 512] units. We select PPO (28 as the primary reinforcement learning algorithm, coupled with the Adam optimizer [12 at learning rate of 2e-5. The only difference between the multi-agent setting and the single-agent setting during training is whether a pre-trained weight loaded. Our experiments are conducted on the IsaacGym simulator [18] using single Nvidia GTX 3090Ti GPU. We run 4096 parallel environments across 15,000 epochs, which takes approximately 15 hours to complete.
15

---

C.2 Hyperparameters
Following previous work 25
7 21 we use the Isaac Gym simulator 18
The simulation runs at
60Hz and the control policy runs 30Hz.
Besides, the hyperparameters we used the training process is detailed below:
Table 3: Hyperparameters for CooHOL.
Parameter
Value
Number of Environments
4096
WG Task-Rewart Weight
0.5
ws Style-Reward Weight
0.5
PPO Minibatch Size
16384
AMP Minibatch Size
4096
Horizon Length
32
Learning Rate
2e 5
PPO Clip Threshold
0.2
y Discount
0.99
GAE (A)
0.95
T Episode Length
600
D More Ablation Studies and Visulizations.
Failure case visualization. Here, we conducted visual analysis of the fail cases. First, for the case lacking a stand point, we can clearly see that the agent moves towards the nearest face, even though it image, in the absence of dynamic input, we observe that the agent stands still, unable even to squat. In the third image, which depicts the scenario without reverse walking, the agent is able lift the box, but because it cannot learn the backward gait, the two agents end up pushing the box against each other, causing deadlock.
the shortest edge, which leads the agent's inability carry the object. In the second
wrong face
stand still
deadlock
No stand point
No Dynamic Observation
No Reverse Walk
Figure Some visualization failure cases. "Stand point" means leading point behind the object to encourage the agent to walk to the object. "Dynamic Observation" means that each agent has its unique input. "Reverse Walk" indicates whether single agent possesses the skill walk backward. Without any of the methods we propose, the policy cannot be successfully trained.
Ablation on performance of CooHOI under noisy scenarios. In CooHOI, all state information
provided using ground truth data from simulators. However, in real-world settings, input data often noisy and prone to enror.. To evaluate the robustness of our framework in such conditions, we introduce random noise into the observation space of our policy and assess its performance under observation noise, as shown in Table 4
16

---

Table 4: Results of our policy under noisy conditions: We tested both single-agent and two-agent box-carrying scenarios. The noise level is defined by the standard deviation of the Gaussian noise used. SR stands for success rate. The definitions of success rate and precision are consistent with those in Section 4.1 of our paper.
Agent # Weight(kg)
Noise
SR (%)
Precision (cm)
10
0
96.85
5.76
10
1
95.80
6.82
10
2
78.56
9.28
1
10
3
60.03
10.75
10
48.48
8.62
2
20
0
90.33
8,80
2
20
1
90.23
8,96
20
87.98
8,92
20
3
84.86
2
20
4
79.93
9.28
Training reward curves. To enhance visualization for ablation studies, we plot the training curves for both the carry reward and held reward in the two-agent training setup. The results, shown Fig illustrate the effectiveness of our CooHOI framework.
Held Reward Curve
Carry Reward Curve
0.50
0.25
0.15
Reward
Reward
0.15
w/o Dynamic Osservation w/o Stand Point
0.30
Dynamic Observation
wio Stand Point
0.10
Intielliation(Scale
w/o Reverse Walk
mio Rever se Walk
0.05
0.00
0.00
2000
4000
8000
2000 4000
6000
800n
Epoch
for carry reward and held reward in the two-agent setting, using four random seeds, consistent with the definitions provided Section To ensure different models were trained for the same duration, extended the training steps for the 'From Seratch' model by factor of 4, as indicated by 'Scale 4' in the graph. The curves were plotted by sampling every four frames.
Figure 8: Training
17

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We clearly state our assumptions and contributions in the abstract and intro- duction. These claims match the experimental results shown in our paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We discussed the limitations of our methods at the last section of our main paper. We also test the influence of different factors our work in ablation studies.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
18

---

Answer: [NA]
Justification: We propose a novel framework for cooperative object transporting tasks. Our work dows not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We fully described our method's ' details our paper and also provided the code.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
19

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: The motion capture data used in our work is a public dataset, and our code is provided in the supplementary materials.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/
ooeioooooiiccy for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//nips.ccpublle pwwppppguuuiieeeeep httpc///uuidees/deeuuuubiiis
(CodeSubiisss wwtaaabmmsssooni ssiiiissicccyy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We fully described the training and test details of our experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification:
We reported error bars and random seeds our experiments in the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for
example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
20

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma enror bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We described type of computing resources, time of exccution of our experi-
ments.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
We conform with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: We mainly focus on addressing cooperative object transporting tasks of humanoid characters, there may not be any societal impact of our work.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
21

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [No]
Justification: Our work does not have a high risk for misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We properly credited and followed the licence of the code and data we used.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in package should be provided. For popular datasets, paperswithcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
22

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We not releasing new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/moodl as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
23


Reviews:
[{"review_id": "UsVZCJ1tTJ", "reviewer": "Reviewer_ehnN", "summary": "This paper proposes a framework for multi-agent cooperative manipulation in the context of humanoids carrying and transporting large furniture. This task is decomposed into several steps. First, they train a single humanoid to learn how to hold and carry relatively small objects. The agent is trained with ground-truth object information and AMP for natural humanoid behavior. Next, they train a multi-agent policy initialized from the single-agent policy for transporting larger objects. The authors present several ablations and report high performance compared to baseline methods in simulation.", "questions": "In Table 1, what does the “w/o CooHOI” mean? Does that mean training the entire policy from scratch? Is that policy trained with AMP and the same rewards? And what is the definition of CooHOI? What is included in CooHOI?\n\nRegarding the robustness of the system: How would the policy perform in noisy situations? For example, when the object bounding boxes are not accurate (in real scenarios, object perception will have errors).\n\nIn Figure 6, can you elaborate more on why policies, except for CooHOI, fail to carry objects even when they can still hold objects? Specifically, what are the failure cases? Do these policies fail to maintain a stable holding motion, or do they fail to accurately move the objects to the target location?\n\nIn addition, r_carry is only defined in the supplementary material. I suggest including it in Section 3.2.2.\n\nIt would be better to plot the standard deviations in Figure 6, given that you evaluate four seeds.\n\nWhat is the action space of the policy? Is it the target joint position for each joint?", "limitations": "The authors provide a comprehensive limitation section of their method.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "This paper studies a practical task and solves it with a well-designed engineering system. It is also well-written, making it easy to read.\n\nThe paper presents comprehensive ablation experiments on both the single-agent and multi-agent policy parts, revealing important information for the framework.\n\nThe framework can learn natural humanoid behavior.", "weaknesses": "I’m curious about the robustness of the proposed system. If I understand correctly, the bounding box information is given as ground-truth parameters. However, in reality, these quantities are far from perfect.\n\nThe task is restricted to moving and transporting large furniture-type objects.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable time and insightful comments. We hope the following clarifications address your concerns.\n\n> W1: I’m curious about the robustness of the proposed system. If I understand correctly, the bounding box information is given as ground-truth parameters. However, in reality, these quantities are far from perfect.\n\nThank you for pointing this out. Yes, in our work, the bounding box information is provided as ground-truth parameters. While this may seem “unrealistic” in the real world, it is a **common setting** adopted by the physics-based character animation community [2][3][4][5][6][7][8][9][10][11][12][13], including the baseline method [1] we compared in our study. Additionally, the **main focus of our work is on training cooperative human-object interaction tasks**, such as carrying everyday objects. Therefore, we did not introduce special designs for noisy scenarios that may occur in real-world object detection. We believe this allows us to concentrate on the primary contribution of our work.\n\n> W2: The task is restricted to moving and transporting large furniture-type objects.\n\nOur method can handle 40 different shapes of common daily-life furniture-like objects. However, as we mentioned in the limitations section, incorporating dexterous hands would make our framework more generalizable to smaller and unusually shaped objects found in daily life. This is a direction of our future work.\n\n> Q1: In Table 1, what does the “w/o CooHOI” mean? Does that mean training the entire policy from scratch? Is that policy trained with AMP and the same rewards? And what is the definition of CooHOI? What is included in CooHOI?\n\nWe apologize for the confusion regarding the term “w/o CooHOI” in Table 1. As stated in lines 233-235, “Without using our CooHOI framework and simply employing parallel training for multi-agent tasks, the training fails.” This refers to training the entire multi-agent policy with AMP and the same rewards from scratch.\n\nThe definition of CooHOI **encompasses the entire framework design**. Specifically, it involves the insight that object dynamics information is crucial for both single-agent skill learning and coordination learning. The “bounding box” design serves as an interface for efficiently transferring single-agent skills to cooperative learning. \n\nIn contrast, “w/o CooHOI” means not using object dynamics information when training multi-agent cooperation skills. This has two conditions:\n1. Directly train the whole policy from scratch, with agents observing the box as is, while the AMP reward and the reward function remain the same. This corresponds to the “w/o Initialization” curve in Figure 6.\n2. First training the single-agent skills, then fine-tuning using MAPPO, but in the multi-agent training stage, not utilizing object dynamics (the state of the bounding box at each side) and directly including the object state as the goal feature. This scenario is illustrated in Figure 6 and Appendix D, Figure 2, “No Dynamics Observation.” This results in agents standing near the object and seemingly forgetting how to interact with it.\n\nThank you for pointing out this unclear explanation. We will clarify it in the paper’s next version.\n\n> Q2: How would the policy perform in noisy situations?.\n\nAlthough we didn't account for noise situations or introduce special designs for robustness training under noise conditions, we can still test our policy's performance. Specifically, we added Gaussian noise to the object dynamics information and varied the noise level by adjusting the standard deviation of the noise. A noise level of 1 indicates that the standard deviation of the noise is 1 cm. **Please refer to Table 1 in the PDF included in the global rebuttal for our results**.\n\n> Q3: In Figure 6, can you elaborate more on why policies, except for CooHOI, fail to carry objects even when they can still hold objects?\n\nWe apologize for the confusion. A detailed analysis of Figure 6 can be found in Section 4.3, “Analysis For CooHOI Framework,” lines 283-302. Additional visualizations are available in Appendix D. To reiterate:\n- Without the “stand point” design, the agent sometimes fails to approach the shortest edge of the object and cannot hold it successfully.\n- Without the “dynamic observation” design, the agents would just stand in front of the object, seemingly unsure of what to do.\n- Without the “reverse walk” design, the agents may hold the objects but cannot reach the target position successfully, resulting in a deadlock.\n- Without “Initialization,” meaning if we train from scratch, the two-agent policy fails to achieve successful carrying. They will just stand near the objects, but don't know how to carry them to the destination.\n\n> Q4: In addition, r_carry is only defined in the supplementary material. I suggest including it in Section 3.2.2. It would be better to plot the standard deviations in Figure 6, given that you evaluate four seeds.\n\nThank you for your detailed advice. We placed the formulation in the appendix due to the NeurIPS page limit, but we acknowledge that this caused some confusion. In the next version of our paper, we will include $r_{carry}$ in Section 3.2.2, following the definition of $r_{target}$. Additionally, we appreciate your suggestion to incorporate the standard deviation plot in Figure 6, and we will include it in the next version.\n\n> Q5: What is the action space of the policy? Is it the target joint position for each joint?\n\nYes, the action space consists of the target joint position for each joint. This is then converted into force using PD control: `(target_pos - dof_pos) * stiffness - dof_vel * damping`. A detailed explanation of this can be found in our response to Reviewer TU4S Q1. Please refer to that if you have further questions. Note that this approach is a **common paradigm** adopted by the physics-based character animation community [1][2][3][4][5][6][7][8][9][10][11][12][13].", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EiDXe0EIfY", "reviewer": "Reviewer_Zw8u", "summary": "In this paper, the authors introduce a novel framework, Cooperative Human-Object Interaction (CooHOI), aimed at tackling the problem of multi-agent object transportation. The framework consists of two phases: initially, a single agent learns to perform tasks, followed by multiple agents learning to collaborate through shared dynamics of manipulated objects. The utilization of shared object dynamics has proven effective in learning cooperative tasks. The authors conduct comprehensive experiments to validate the efficacy of their approach and explore its capability boundaries.", "questions": "* While acknowledging the idea of utilizing the object’s dynamics as implicit communication between agents, such communication is not synchronous. Specifically, agents adjust their actions only after one agent changes the object dynamics, resulting in a delay between the change in object dynamics and the response actions taken. I wonder if this delay could lead to jitter of the manipulated object. If not, could the reviewers discuss why such jitter would not occur?\n* The held point is described as the geometric center of each end of the object. However, in Figure 1, when cooperatively carrying a large box, the characters grasp the edges of the box instead of the center of its face. I am unsure if I misunderstood the definition of the held point, or if the current definition is inadequate, leading agents to choose other positions they find more suitable for completing the task.\n\n* In the second row of Figure 4, I am curious about why the four agents fail to put down the box. The authors attribute this failure to the absence of dexterous hands and limited friction, however, similar limitations exist in two-agent manipulation tasks. Besides, the multi-agent put-down operation closely resembles the multi-agent pick-up operation (but in reverse), while the agents can successfully pick up the box, they struggle to put it down. Could the authors provide more discussion or visualizations for better understanding?", "limitations": "The authors have provided a detailed discussion of limitation and failure cases.", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "* The idea of sharing manipulated object dynamics among multiple agents as implicit communication to facilitate multi-agent collaboration, is intuitively appealing and has proven effective. This approach aligns with object-centric concept, designed to leverage object dynamics to guide agent actions and promote collaboration. It provides valuable insights for future research.\n* The authors conduct extensive experiments to validate the effectiveness of their approach, including many ablation studies to assess the impact of different design choices, as well as exploration into the framework's capability boundaries.\n* The paper is well-organized and easy to understand.\n* The inclusion of diverse visualizations enhances qualitative comprehension of the method's performance.", "weaknesses": "* It appears that the shapes and categories of the manipulated objects are not very diverse, potentially constraining the method's ability to generalize to novel objects. I recommend that the authors summarize the counts of shapes and categories used in the experiments.  Additionally, if the shapes of the manipulated objects are limited and the grasp actions are abstracted in the simulation, the authors are encouraged to compare CooHOI with some heuristic methods. An intuitive heuristic baseline could involve having two agents grasp each end of a long object, and then move at the same speed to the target position.\n* While the authors provide detailed explanations in the appendix, it would enhance comprehension if they provide simple explaination when introducing a new concept in the main paper. Here are some examples:\n  * In Section 3.1, the authors introduce the task-specific goal feature $g_t$ without explanation.\n  * While it is understandable that the proposed style reward can help the method produce behaviors similar to those in the dataset, it would be better to provide a simple explanation of how to evaluate such similarity.\n  * It would be beneficial to provide a brief introduction to the baseline InterPhys in one or a few sentences.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments and feedback.\n\n> W1.1 : It appears that the shapes and categories of manipulated objects are not very diverse, potentially constraining the method's ability to generalize to novel objects.\n\nWe sampled nearly 40 common everyday objects for training, which fall into six categories: box, table, armchair, and high stool for single agents, and long box and sofa for two agents. We will summarize the counts of shapes and categories of the objects we used in Section 4.1, “Datasets and Initialization,” in the next version of our paper.\n\nIn addition, the main focus of our work is on training multi-agent cooperative object transportation, so we did not introduce special designs for different shapes and categories. This allows us to concentrate on the main contribution of our work. Through fine-tuning, our framework can successfully enable single and multi-agent transportation of **common mid-size daily-life objects** (Table 2). As a comparison, [3] used 40 object shapes, [17] used 17 object shapes. [1] used 350 object shapes, although most of these shapes were for humanoids to sit or lie on. For carrying tasks, [1] primarily used the simple box shape. Therefore, we believe that the 40 shapes and 6 categories of objects we used are **sufficient to validate our framework**.\n\n> W1.2: The authors are encouraged to compare CooHOI with some heuristic methods. An intuitive heuristic baseline could involve having two agents grasp each end of a long object, and then move at the same speed to the target position.\n\nWe believe this intuitive heuristic baseline is quite similar to the “w/o Dynamic Observation” scenario described in Section 4.3 and shown in Figure 6. The key difference between this baseline and our CooHOI method is that the baseline does not use dynamics information from each end of the object to facilitate skill transfer. Training multi-agent cooperation using only reward training is challenging because coordination between agents cannot be easily hard-coded.\n\n> W2: While the authors provide detailed explanations in the appendix, it would enhance comprehension if they provide simple explaination when introducing a new concept in the main paper.\n\nWe will incorporate your suggestions in the next version of our paper to make it easier for readers to follow. Thanks for your suggestion!\n\n> Q1: I wonder if this delay of communication could lead to jitter of the manipulated object. If not, could the reviewers discuss why such jitter would not occur?\n\nIn the evaluation of the trained policies, we did not observe significant jittering of the manipulated object. You can **see the visualization of our results in the “DemoVideo_CooHOI.mp4” included in our supplementary materials**.\n\nWe believe this is due to the following reasons:\n1. We model the objects as rigid bodies, so changes in object dynamics occur “instantly,” without causing large delays.\n2. Our framework is similar to the “object-centric” concept, where object jittering would result in humanoid motion jittering. Since we use AMP to provide a “style reward,” this motion jittering is not encouraged, thus forcing the humanoid to learn effective cooperation and avoid object jittering.\n2. We train the multi-agent policy in a cooperative setting, so implicit communication helps achieve coordination learning by maximizing the reward and avoiding object jittering since jittering does not maximize the reward.\n\nWe believe that explicit or implicit communication in multi-agent systems should carefully address latency issues, especially in **real-world object-carrying tasks**. Thank you again for your valuable comment.\n\n> Q2: The held point is described as the geometric center of each end of the object. However, in Figure 1, when cooperatively carrying a large box, the characters grasp the edges of the box instead of the center of its face.\n\nThe held point is defined as the **3D geometric center** of each end of the objects. We used the held point and the corresponding $r_{held}$ to encourage the agents to reach out their hands near the object and learn to manipulate it, where $r_{held}$  encourage the humanoid to put the center of its 2 hands closer to the held point.\n\nThe held point design in our multi-agent HOI setting serves as \"abstract indicators\" for primarily distinguishing the guidance for each agent during the interaction with the same object, rather than specifying the exact position for them to place their hands. The precise position varies for each object shape, and we encourage the agents to learn this during training. We believe this is why our method can generalize to different object shapes without requiring their mesh information as input.\n\n> Q3:I am curious about why the four agents fail to put down the box.\n\nWe attribute this failure to the absence of dexterous hands and limited friction. During the process of carrying the box to its destination, we found that the large box sometimes slips from the humanoid agents’ hands. This is because the hand model of the humanoid agents is **spherical**, making it very difficult to hold the corners of the box, causing it to be **“squeezed out” of their hands**. \n\nThe difference between the four-agent scenario and the two-agent tasks is that, in the four-agent scenario, the humanoids can **only grasp the corners of the box**. Additionally, when the four humanoids are picking up the object, the two humanoids positioned diagonally can push forward against each other to prevent the box from slipping out of their hands. However, this is very difficult during transportation and often leads to the box constantly falling. As a result, they **never have the opportunity to learn the “putdown” action**, causing the entire process to fail.\n\nHowever, we found a way to bypass this limitation without dexterous hands: we had the four agents carry an “H-shaped” large box and successfully complete the task. **Please refer to the global rebuttal for more details**.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "I4u7xfOIOv", "reviewer": "Reviewer_kufk", "summary": "This work address the problem of multi-character collaboration for object transporting tasks. Different from previous works approaching the multi-character HOI task with tracking-based method, this work learns a physics-based multi-agent policy with reinforcement learning. Instead of directly training a multi-agent policy from scratch, the authors suggest using a two-phase learning approach: CooHOI uses a two-phase learning approach: individual skill acquisition through Adversarial Motion Priors (AMP) and then collaborative learning with Multi-Agent Proximal Policy Optimization (MAPPO). Agents learn to coordinate implicitly by responding to changes in object dynamics caused by others.", "questions": "See more in the weakness section :-)", "limitations": "Yes, the authors addressed the limitation.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The manuscript is well-written, the methodology is clearly presented and easy to follow.\n    * This paper is addressing an under-studied and important application - the physics-based multi-agent cooperation HOI, where most previous works explores in the single-agent setup.\n    * The proposed two-state learning pipeline is reasonable and few important designs including the stand point, held point shows to be effective in addressing this multi-agent cooperation tasks.\n    * The ablation study and analysis provide comprehensive analysis and capacity boundary test for the proposed pipeline and these provide useful insights for the community.", "weaknesses": "* This paper build a framework for multi-agent cooperation policy learning with AMP for single-agent policy pertaining and MAPPO for cooperation skill learning. The technical contribution to the community is minor.\n* The method include few heuristic designs to simplify the problem, for example, the manually defined held point, and object bounding box observation, making it not easy to generalize and scale to more diverse objects.\n* The current framework can be hardly generalise to objects with different size and different shape, and the human motion styles are quite limited.\n        * Regarding generalization to objects weights and scales, are the results presented in Figure 5  the transferring results to different weights and scales or the per-object training results.\n        * Regarding the motion styles, in the training dataset, limited motion sequences are involved with only walking, lifting up and putting down, would the proposed method build upon AMP be able to cover more diverse skills, and this could be helpful to generate more natural and diverse HOI motions.\n* It is presented that when increasing the number of characters, the policy has problem in putting the object down due to the absence of dexterous hands and limited friction. Could the author elaborate more on the major bottlenecks encountered when dealing with a larger number of characters? Assuming that the friction parameter can be adjusted in the simulator and should not be a significant issue for motion generation tasks purely within the simulation, what specific contributions could dexterous hands make to this scenario? Additionally, while the introduction of dexterous hands would undoubtedly enhance the agent's capabilities, what other factors should be considered?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful comments. Many of the weaknesses you pointed out are exactly the areas we plan to address in future work, as we aim to make our method more generalizable.\n\n> Q1: This paper builds a framework for multi-agent cooperation policy learning with AMP for single-agent policy pertaining and MAPPO for cooperation skill learning. The technical contribution to the community is minor.\n\nWe acknowledge that the AMP and MAPPO algorithms we used are not new to the community. However, we believe our contribution lies in **presenting a proof-of-concept framework for this important and relatively under-studied subfield**: physics-based multi-agent cooperation in HOI. Our **core insight** is that manipulated object dynamics can serve as feedback in single-agent skill training, an implicit communication channel in multi-agent coordination, and an interface to facilitate efficient skill transfer from single to multi-agent scenarios. We believe this will inspire many in the kinematics-based animation, physics-based animation, and robotics communities.\n\nAdditionally, some previous tracking-based efforts [18] on physics-based multi-agent cooperation in HOI have addressed this issue by utilizing **high-quality multi-agent mocap data**. Their methods cannot train successfully without multi-agent mocap data, as simply using AMP (or other types of style rewards) and MAPPO would be **too challenging**. This is demonstrated by the \"w/o CooHOI\" result in Table 1 and the \"w/o Initialization\" in Figure 6. In comparison, our framework only requires single-agent motion capture data and can extend to different types of objects and varying numbers of agents by utilizing the manipulated object dynamics.\n\n> Q2: The method includes a few heuristic designs to simplify the problem, for example, the manually defined held point, and object bounding box observation, making it not easy to generalize and scale to more diverse objects.\n\nOur work primarily focuses on multi-person object transportation tasks, validating that using manipulated object dynamics as an implicit communication channel can efficiently facilitate skill transfer and coordination learning. The designs in our methods are mainly intended for efficient skill transfer from single-agent skills to multi-agent coordination learning. We **did not introduce special designs for objects with different shapes, but it can still generalize to 40 different shapes of objects**. \n\nWe believe that handling different object meshes is primarily a perception problem, whereas our focus is on the control aspect of cooperative HOI tasks. Incorporating object mesh-related information or other perception-related designs would certainly enhance the generalizability of our method. This will be a future direction of our study.\n\n> Q3: The current framework can be hardly generalize to objects with different size and different shape, and the human motion styles are quite limited. \n\n- Regarding object weights and scales, the results in Figure 5 are the **transfer results to different weights and scales**. This is a **boundary analysis** of our framework: we first train agents to carry objects with different weights and scales, and then test when they fail with out-of-distribution scales and weights. The results indicate that the policy performs well with out-of-distribution scales and weights but eventually fails when these parameters deviate too far from the training conditions.\n- Regarding object shapes, our framework can generalize to 40 different common mid-size daily-life objects. We have thoroughly discussed this in our answer to your question Q2.\n- Regarding generalization to motion styles, since our work mainly focuses on multi-agent cooperative object transportation, we only chose walking, lifting up and putting down objects, and reverse walking to cover the motion styles.\n\nYour feedback on the generalizability of our methods for both object shapes and motion styles is invaluable to us. We plan to enhance our framework to address **more diverse cooperative HOI tasks in future work**. This is also discussed in our limitations section. After incorporating dexterous hands, we will work on enabling the humanoid characters to cooperatively handle objects of different sizes using object mesh-related information in our training pipeline and aim to achieve various motion styles by utilizing motion priors or motion latents.\n\n> Q4: It is presented that when increasing the number of characters, the policy has problem in putting the object down due to the absence of dexterous hands and limited friction. Could the author elaborate more on the major bottlenecks encountered when dealing with a larger number of characters? \n\n- We attribute this failure to the absence of dexterous hands and limited friction because, during the process of carrying the box to the destination, the large box sometimes slips from the humanoid agent’s hands because the hand model is spherical, making it very difficult to hold the corners of the box, which tends to be “squeezed out” of their hands. However, we found a way to bypass this limitation without dexterous hands: we had the four agents carry an “H-shaped” large box and successfully complete the task. **Please refer to the global rebuttal for more details.**\n\n- Regarding the major bottlenecks encountered with a larger number of characters (more than 4), we believe the main challenge lies in ensuring that these agents can effectively communicate with each other using object dynamics. The complexity of this problem increases exponentially as the number of agents grows.\n\n- We believe introducing dexterous hands would allow the agents to handle smaller daily-life objects or objects with unusual shapes, such as two agents cooperatively carrying a TV together.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "P2bkKx5xHm", "reviewer": "Reviewer_TU4S", "summary": "This paper works on learning multi-human cooperative object manipulation, specifically the collaborative carrying of objects.\nThis paper proposes a two-stage method to learn collaborative object carrying. In the first stage, the agent learns single-person object carrying from motion capture data and heuristic-based task rewards following the Adversarial Motion Priors (AMP) framework. In the second stage, the agent policy is fine-tuned using multi-agent reinforcement learning to learn multi-agent collaborative object carrying.\nExperiments show that the trained policy can successfully control the agents to complete collaborative objects carrying tasks given objects of varying shapes and weights.", "questions": "1. How the object weights can affect the humanoid in the simulation? Is there an upper bound for the joint force of the simulated humanoid? How does the humanoid apply forces to the object in the simulation?\n\n2. Are the standing point and held point also part of the goal inputs? If not, during the training, the same motion sequence can have different rewards given differently sampled standing points? How are the standing points sampled given an arbitrary object mesh?", "limitations": "The author discussed the limitation of lacking dexterous hands, which limits the agent interaction capacity and is left as future work. \nPlease also refer to the weakness discussion above.", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. The proposed cooperative interaction learning framework can learn multi-human collaborative behaviors requiring only single-human motion capture data. In addition, the learned policy has better generalization compared to tracking-based methods.\n\n2. The implicit communication via object dynamics unifies the observation in the single-human and multi-human stages, reducing the gap between the two stages and facilitating the learning of collaborative object carrying.\n\n3. This paper presented extensive ablation study and boundary analysis to show how and to what extent the proposed method works.", "weaknesses": "1. This paper simulates an oversimplified humanoid without hand modeling considering the fact that it works on object manipulation problems, for which the hands play essential roles. The authors also recognize this issue leads to capacity limitations.\n\n2. The agent observation features may not be sufficient for more complex manipulations. In the presented paper, the agent observation mainly consists of the self-motion and cropped object bounding box features. However, more complex collaborative tasks like carrying articulated objects and assembling a sofa require a global overview of the object instead of only a local observation. In addition, the agent also needs to observe the collaborators to avoid interference such as grasping at the same point.\n\n3. The proposed method needs to train a separate policy for a category of objects with similar shapes (L246-247), which indicates a generalization problem.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments and feedback. We hope the following clarifications address your concerns.\n\n> W1: This paper simulates an oversimplified humanoid without hand modeling considering the fact that it works on object manipulation problems, for which the hands play essential roles. The authors also recognize this issue leads to capacity limitations.\n\nThis is indeed a limitation of our work. Our framework currently handles 40 different common mid-size daily life objects, but incorporating dexterous hands would definitely enable humanoids to interact with small objects or objects with unusual shapes. This is a future direction for our research.\n\n> W2: The agent observation features may not be sufficient for more complex manipulations. In the presented paper, the agent observation mainly consists of the self-motion and cropped object bounding box features. However, more complex collaborative tasks like carrying articulated objects and assembling a sofa require a global overview of the object instead of only a local observation. In addition, the agent also needs to observe the collaborators to avoid interference such as grasping at the same point.\n\nIn our work, we minimize interference between different agents by utilizing designs such as standing points and held points. These designs have proven effective for multi-agent object transportation tasks.\n\nHowever, for more complex tasks like carrying articulated objects or assembling a sofa, additional observation input is indeed necessary. These tasks may require explicit observation and communication between agents, as well as specialized designs.\n\n> W3: The proposed method needs to train a separate policy for a category of objects with similar shapes (L246-247), which indicates a generalization problem.\n\nYes, we train a separate policy for each category of objects with similar shapes. Our main focus is on training multi-agent cooperative object transportation, so we did not introduce special designs for different shapes and categories. This approach allows us to concentrate on the primary contribution of our work. \n\nAlso, we think that handling different object meshes is primarily a perception problem, whereas our focus is on the control aspect of cooperative HOI tasks. Incorporating object mesh-related information or other perception-related designs would certainly enhance the generalizability of our method. This will be a future direction of our study.\n\n> Q1: How the object weights can affect the humanoid in the simulation? Is there an upper bound for the joint force of the simulated humanoid? How does the humanoid apply forces to the object in the simulation?\n\nIn our experiments, we use the same humanoid model as previous work[1][3][5][6][7][11]. To reiterate, the action $a \\in \\mathbb{R}^{28}$ specifies the target positions for PD controllers at each joint, and the forces are calculated as `(target_pos - dof_pos) * stiffness - dof_vel * damping`. \n\n- We are not entirely sure what you mean by “How do object weights affect the humanoid in the simulation?”. Perhaps you are asking about the forces at each joint when carrying objects of different weights? We have visualized this in Figure3 and Figure4 in the PDF in the global rebuttal.\n- Yes, there is an upper bound for the joint force of the simulated humanoid. Specifically, since we are using positional control, every degree of freedom (dof) has limits, namely, dof_limits_lower and dof_limits_upper, and the resulting force is bounded. \n- When the humanoid manipulates objects, there will be friction forces between the hands of humanoid and the objects, which are calculated by the PhysX phyics engine of Isaac Gym. \n\n> Q2: Are the standing point and held point also part of the goal inputs? If not, during the training, the same motion sequence can have different rewards given differently sampled standing points? How are the standing points sampled given an arbitrary object mesh?\n\nWe apologize for any confusion. In our work, the standing points are part of the goal input, while the held points are not. During single-agent training, the standing points are **randomly sampled** to positions directly in front of various faces of the object. During multi-agent training, the standing points are sampled in front of each end of the long object. The held points are not part of the goal inputs because we set them at the geometric center of the object, allowing the agent to “infer” the position of the held point based on the object’s position and its bounding box information. We will briefly introduce the constitution of goal feature in Section 3.2.1 in the next version of our paper.\n\nIf the standing points and the object’s position and rotation are given, then the same motion sequence will yield the same rewards.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank the reviewers for their detailed, valuable, and insightful feedback. We are pleased that the reviewers recognize our dedication to **addressing the under-studied and important application** of physics-based multi-agent cooperation HOI (Reviewer kufk). We appreciate the acknowledgment that our ideas on **utilizing manipulated object dynamics can provide valuable insights** for future research (Reviewer Zw8u). Our paper’s **comprehensive ablation experiments and boundary analysis** were noted positively (Reviewers ehnN, Zw8u, kufk, Tu4S), and the reviewers commended **our well-written paper** (Reviewers ehnN, Zw8u, kufk) and **well-designed framework** (Reviewers ehnN, kufk). Furthermore, our visualizations have been highlighted for demonstrating **natural HOI motion** (Reviewers ehnN, Tu4S).

We are pleased to report that we have conducted the analysis suggested by the reviewers on why the four agents failed to carry objects in our previous experiments. We have also designed a new scenario to bypass the limitation of the absence of dexterous hands, demonstrating the effectiveness and generalizability of our methods.

### Successful 4-Agent Coordination in Carrying an H-Shaped Box

In the limitations section, we discussed that our framework fails when four agents collaborate to carry big box. We attribute this failure to **the absence of dexterous hands and limited friction**. During the process of carrying a box to its destination, we observed that the large box sometimes slips from the humanoid agents’ hands. This is because the hand model of the humanoid agent is spherical, making it difficult to hold the corners of the box, which leads to the box being **“squeezed out” of their hands**. 

However, we found a way to bypass this limitation without dexterous hands: we had the four agents carry an **“H-shaped” large box** and **successfully complete the task**. We report the success rate of our policy when carrying objects of different weights. When carrying 60kg H-shape box, our policy achieves 90.63% success rate and 23.56 cm precision. Considering the H-shaped object is relatively large (3m x 3m x 0.5m), we define success as having the center of the object within 0.5m of the target position. Please refer to **Fig1 and Fig2 in the attached PDF** for more visualizations. The success rate and precision of the 4-agent carrying policy are detailed below:


| Agent Number | Object Category | Weight of the Object (kg) | Success Rate (%) | Precision (cm) |
| ------------ | --------------- | ------------------------- | ---------------- | -------------- |
| 4            | H-shape Box     | 60                        | 90.63            | 23.56          |
| 4            | H-shape Box     | 70                        | 88.28            | 24.97          |
| 4            | H-shape Box     | 80                        | 76.95            | 24.72          |
| 4            | H-shape Box     | 90                        | 69.53            | 25.01             |


### References for All Reviewers

**Dear reviewers, due to the rebuttals' character limit, we've placed the references for all rebuttals below. Thank you for your time and consideration**.

[1] Hassan, Mohamed, et al. "Synthesizing physical character-scene interactions." *ACM SIGGRAPH 2023 Conference Proceedings*. 2023.

[2] Tessler, Chen, et al. "Calm: Conditional adversarial latent models for directable virtual characters." _ACM SIGGRAPH 2023 Conference Proceedings_. 2023.

[3] Xiao, Zeqi, et al. "Unified human-scene interaction via prompted chain-of-contacts." arXiv preprint arXiv:2309.07918 (2023).

[4] Peng, Xue Bin, et al. "Deepmimic: Example-guided deep reinforcement learning of physics-based character skills." ACM Transactions On Graphics (TOG) 37.4 (2018): 1-14.

[5] Peng, Xue Bin, et al. "Amp: Adversarial motion priors for stylized physics-based character control." ACM Transactions on Graphics (ToG) 40.4 (2021): 1-20.

[6] Juravsky, Jordan, et al. "Padl: Language-directed physics-based character control." _SIGGRAPH Asia 2022 Conference Papers_. 2022.

[7] Peng, Xue Bin, et al. "Ase: Large-scale reusable adversarial skill embeddings for physically simulated characters." ACM Transactions On Graphics (TOG) 41.4 (2022): 1-17

[8] Juravsky, Jordan, et al. "SuperPADL: Scaling Language-Directed Physics-Based Control with Progressive Supervised Distillation." _ACM SIGGRAPH 2024 Conference Papers_. 2024.

[9] Rempe, Davis, et al. "Trace and pace: Controllable pedestrian animation via guided trajectory diffusion." _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 2023.

[10] Luo, Zhengyi, et al. "Perpetual humanoid control for real-time simulated avatars." _Proceedings of the IEEE/CVF International Conference on Computer Vision_. 2023.

[11] Wang, Yinhuai, et al. "Physhoi: Physics-based imitation of dynamic human-object interaction." arXiv preprint arXiv:2312.04393 (2023).

[12] Luo, Zhengyi, et al. "Grasping Diverse Objects with Simulated Humanoids." arXiv preprint arXiv:2407.11385 (2024).

[13] Luo, Zhengyi, et al. "SMPLOlympics: Sports Environments for Physically Simulated Humanoids." arXiv preprint arXiv:2407.00187 (2024).

[14] He, Tairan, et al. "Learning human-to-humanoid real-time whole-body teleoperation." arXiv preprint arXiv:2403.04436 (2024).

[15] He, Tairan, et al. "OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning." arXiv preprint arXiv:2406.08858 (2024).

[16] Rudin, Nikita, et al. "Learning to walk in minutes using massively parallel deep reinforcement learning." _Conference on Robot Learning_. PMLR, 2022.

[17] Pan, Liang, et al. "Synthesizing physically plausible human motions in 3d scenes." _2024 International Conference on 3D Vision (3DV)_. IEEE, 2024.

[18] Zhang, Yunbo, et al. "Simulation and retargeting of complex multi-character interactions." _ACM SIGGRAPH 2023 Conference Proceedings_. 2023.

Meta-review: This paper introduces CooHOI, a framework for learning cooperative human-object interactions in physics-based character animation. The method demonstrates the ability to learn multi-agent object transportation tasks using only single-agent motion capture data, with good generalization to objects of varying shapes and weights. All reviewers recommended acceptance, praising the paper's well-designed approach, comprehensive experiments, and valuable contribution to an under-studied area. The reviewers consistently highlighted the effectiveness of using manipulated object dynamics for implicit communication between agents and the framework's ability to transfer single-agent skills to multi-agent scenarios.

Initial concerns focused on the limited generalization to diverse object shapes, the use of simplified humanoid models without dexterous hands, and questions about the robustness of the system. The authors provided detailed responses, clarifying their approach to object diversity, explaining the rationale behind their design choices, and offering additional experiments demonstrating the framework's capabilities. These responses addressed many reviewer concerns, though some reservations remain about generalization and the limitations imposed by simplified hand models.

Despite these limitations, the consensus is that CooHOI represents a significant contribution to the field of physics-based character animation and multi-agent cooperation. The paper's thorough evaluations, clear presentation, and novel insights outweigh its current limitations. For the final version, I encourage the authors to incorporate their rebuttal explanations, particularly regarding object diversity and the rationale behind their design choices. Additionally, discussing potential future work on incorporating dexterous hands and improving generalization to more diverse objects would strengthen the paper further.
Decision: Accept (spotlight)