CrossLoco: Human Motion Driven Control of Legged Robots via Guided Unsupervised Reinforcement Learning
Authors: Tianyu Li
Hyunyoung Jung
Matthew Gombolay
Yong Cho
Sehoon Ha
Venue: ICLR 2024 poster (2024)
Keywords: Human Motion Driven Control
Legged Locomotion
Unsupervised Reinforcement Learning
Primary area: applications to robotics, autonomy, planning
TL;DR: a guided unsupervised reinforcement learning framework that simultaneously learns robot skills and their correspondence to human motions.

Abstract: Human motion driven control (HMDC) is an effective approach for generating natural and compelling robot motions while preserving high-level semantics. However, establishing the correspondence between humans and robots with different body structures is not straightforward due to the mismatches in kinematics and dynamics properties, which causes intrinsic ambiguity to the problem. Many previous algorithms approach this motion retargeting problem with unsupervised learning, which requires the prerequisite skill sets. However, it will be extremely costly to learn all the skills without understanding the given human motions, particularly for high-dimensional robots. In this work, we introduce CrossLoco, a guided unsupervised reinforcement learning framework that simultaneously learns robot skills and their correspondence to human motions. Our key innovation is to introduce a cycle-consistency-based reward term designed to maximize the mutual information between human motions and robot states. We demonstrate that the proposed framework can generate compelling robot motions by translating diverse human motions, such as running, hopping, and dancing. We quantitatively compare our CrossLoco against the manually engineered and unsupervised baseline algorithms along with the ablated versions of our framework and demonstrate that our method translates human motions with better accuracy, diversity, and user preference. We also showcase its utility in other applications, such as synthesizing robot movements from language input and enabling interactive robot control.

Published as a conference paper at ICLR 2024
CROSSLOCO: HUMAN MOTION DRIVEN CONTROL OF LEGGED ROBOTS VIA GUIDED UNSUPERVISED REIN- FORCEMENT LEARNING
Tianyu Li, Hyunyoung Jung, Matthew Gombolay, Yong Kwon Cho, Schoon Ha
Georgia Institute of Technology
Atlanta, GA 30332. USA
(t1i471, jung331 sehoonha @egatech. edu matthew, gombol ayê ec.cc... gatech. edu
yong. . chocce. www.chhwwwccccccceeeeeeeeeeee gatech.ecuh..
ABSTRACT
Human motion driven control (HMDC) is an effective approach for generating nat- ural and compelling robot motions while preserving high-level semantics. How- ever, establishing the correspondence between humans and robots with different body structures is not straightforward due to the mismatches kinematics and dynamics properties, which causes intrinsic ambiguity to the problem. Many previous algorithms approach this motion retargeting problem with unsupervised learning, which requires the prerequisite skill sets. However, it will be extremely costly to learn all the skills without understanding the given human motions, par- ticularly for high-dimensional robots. In this work, introduce CrossLoco, guided unsupervised reinforcement learning framework that simultaneously learns robot skills and their correspondence human motions. Our key innovation is introduce a cycle-consistency. -onsestenee-bbaeed reward term designed to maximize the mu- tual information between human motions and robot states. We demonstrate that the proposed framework can generate compelling robot motions by translating di- verse human motions, such as running, hopping, and dancing. We quantitatively compare our CrossLoco against the manually engineered and unsupervised base- line algorithms along with the ablated versions of our framework and demonstrate that our method translates human motions with better accuracy, diversity, and user preference. We also showcase its utility in other applications, such as synthesizing robot movements from language input and enabling interactive robot control.
INTRODUCTION
The concept of telcoperating robots through human movements, known Human Motion Driven Control (HMDC), has been illustrated in various forms of media, including animations, movies, and science fiction, such Madö King Granzört (Tuchi 1989), Pacific Rim (del Toro, 2013), and Ready Player One (Spielberg. 2018). In these media, HMDC technology allows operators to intuitively control robots using their body movements. Compared to fully autonomous control, this teleop- cration offers the essential dexterity and decision-making capabilities required for tasks demanding precise motor skills and situational awareness. Consequently, this property makes HMDC promising for various applications, including entertainment, medical surgery, and space exploration.
The key challenge of HMDC how establish the correspondence between robot states and hu- man motions, which can also be referred to as motion retargeting. For certain types of robots, such as humanoids or manipulators, this correspondence might be simple enough be approached by assuming the mapping of end-effectors in Cartesian space and solving the formulated inverse kinematics problem (Gleicher) 1998; Tak & Ko 2005). However, when we consider robots with significantly different morphological structures, such as quadrupeds, hexapods, or quadrupeds with mounted arms, the correspondence becomes nontrivial due to the intrinsic ambiguity of the prob- lem. Therefore, researchers often have approached this motion retargeting problem by applying supervised learning techniques to the paired datasets (Sermanet et al., 2018 Delhaisse et al. 2017 Rhodin et al. 2014). Nonetheless, creating paired datasets can be challenging and labor-intensive

---

Published as a conference paper at ICLR 2024
Running
Sharp Turning
Dancing
Figure 1: We introduce CrossLoco, a guided unsupervised reinforcement learning framework for
translating human motion robot control.
task that requires significant engineering expertise. To address this issue, some researchers have proposed using unsupervised learning techniques to learn the correlation from unpaired human and robot motion datasets (Li et al., 2023b Choi et al. 2020 Smith et al., 2019). In this case, the robot dataset serves as prior knowledge indicating the motion pattern of the robot. However, obtaining motion datasets can expensive because we do not know the required skills for the given human motion. In addition, control itself is challenging due the complexity of the quadrupedal robot and its underactuated dynamics. This leads to our research question: can we learn morphooyppyylogy HMDC without prior knowledge of the robot?
The research question presents three primary challenges. Firstly, the significant difference in kine- matics and dynamics between the human and the target robot makes it difficult to establish cor- respondence. Secondly, we cannot build a predefined motion database for the rohot due to the complexity of the problem. Finally, the problem itself ambiguous. For instance, there exist many different quadrupedal gaits that can capture the essence of human walking. To address these challenges, drew inspiration from the recent unsupervised skill discovery techniques, such as Ey- senbach et al. (2018) and Peng et al. (2022), and aim to simultancously learn robot skills and robot- human motion correspondence by maximizing the mutual information between human and robot motions.
In this work, we introduce CrossLoco, guided unsupervised reinforcement learning framework that enables simultaneous learning of human-robot motion correspondence and robot motion con- trol (Figure [ Our key approach is to introduce a sycle-consistene--bee ward term that maximizes the mutual information between human motions and the synthesized robot movements. We implement this cycle consistency term by training both robot-to-human and saman-to-robol reconstruction networks. Our formulation also includes regularization terms and a root-tracking reward to guide correspondence learning. Simultaneously. we train a robot control policy that takes human motions and sensory information as input and generates robot actions for interacting with the environment.
correspondence re-
We demonstrate that CrossLoco walking, running, and dancing. Even for locomotion, the robot exhibits two distinct strategies, trot- ting and galloping, inspired by human walking motions with different styles. We quantitatively compare our method against the baseline, DeepMimic (Peng et al., 2018a), along with the ablated versions of our CrossLoco framework and show that our method can achieve better quantitative results in terms of accuracy, diversity, and user preference. We further showcase the potential appli- cations of our framework: language2text motion synthesis and interactive motion control.
translate large set of human motions for robots, including
2 RELATED WORKS
Learning Locomotion Skills. There are various methods for robots to learn locomotion skills. One approach involves maximizing reward function designed by experts using reinforcement learning. as demonstrated in several studies such as Tan et al. (2018) 8): Haarnoja et al. (2018); Xie et al. (2018)
Li et al.) (201)); Rudin et al. (2022). Another method is motion imitation, where the control policy trained with imitation reward, shown in Peng al. (2015 b): Li et al. (2023a); Bergamin ct al. (2019); Won Lee (2019): Ling et al. (2020); Peng et al. (2020). This reward is calculated based on the distance between the robot's current pose and reference pose from the demonstration
2

---

Published as a conference paper at ICLR 2024
trajectory. The closer the distance, the larger the reward. Generative adversarial imitation learning (GAIL) (Ho & Ermon 2016) is another approach that trains the policy to deceive a discriminator that distinguishes real and fake demonstration data. Finally, without the need for engineering reward functions or demonstration data, some studies such as Eysenbach et al. (0188; Sharma et al. (2019) focus on unsupervised skill discovery from interaction data through intormation-theoreti methods.
Motion Retargeting. Transferring motions between different morphologies has been an important topic in both roboties and computer graphics communities to produce natural motions for various robots and characters. Researchers have investigated various approaches, such as designing man- ual correspondences (Gleicher 1998 Tak & Ko 20055 Grandia al., (023). learning from paired datasets (Sermanet et al., 2018 Delhaisse et al. [2017] Jang et al., 2018], or developing modu- lar/hierarchical policies (Won & Lee| 2019, Hejna et al., 2020. Sharma et al. 2019). More recent
works (Zhang et al. [020] Aberman et al. [020 Viilleas et al. 2018 Li et al., 2023b; (Smith et al.)
2019 Kim et al. [220] [Shankar et al. 2022) aim to learn the state and action correspondence from unpaired datasets via unsupervised learning. However, these methods often require a pre-collected
dataset of both domains, which not available for robots in our problem.
Sycle-Consistency. Our work is inspired by previous research on cycle-consistency (Zhou et al., 2016 Zhu et al.) 2017 Liu et al. 2017, Rao et al.] 2020, Bousmalis et al., 2018). For instance, CycleGAN (Zhu et al. 2017) combines cycle-consistency loss with Generative Adversarial Net- translation. By adding domain knowl- be extended to video retargeting (Bansal et al., 2018) and domain adapta- tion (Hoffman et al., 20118). In robotics, a similar approach has been investigated for sim-to-real transfer (Stein & Roy 2018 James al., 2019). Besides alignment in image space, a few re- searchers (Zhang et al., 2020 Shankar et al.) 2022) adopt cycle-consistency to align agents in dif- ferent dynamics and structures, while the others (Aberman al., 2020. Villegas et al. 2018) apply cycle-consistency for motion retargeting between similar buman-likee robots or characters. Inspired by these works, we aim to co-train control policy for the diverse motor skills of a quadrupedal robot while establishing cycle consistency between the robot and human motions.
works (Goodfellow et al. 2014)) for unpaired
edge, CycleGAN
3 PRELIMINARIES
Cidii-ondi Reinforvement Learning. We formulate our framework as a skill-conditioned reinforcement learning problem, where agent interacts with an environment to maximize an ob- jective function by following a policy л. At the beginning of each learning episode, a condition term is sampled from the dataset system st, then takes an action sampled from the policy a, ~ Taylss: z) to interacts with the envi- ronment. After executing the actions, the environment takes the agent to a new state sampled from the dynamics transition probability S1+1 ~ p(5 (s at). A scalar reward can be measured using a reward function rt r(s;, a,. St+1, z). The agent's 's objective is learn policy that maximizes its expected cumulative reward J (TT)
p(z). At each time step, the agent observes the state of the
J(x) Ex-p(x) (-p(p) (()))))))))))))
Σ rr))
Here, T is a state and action trajectory with the length T, where its distribution can be computed as p(T[IT,)) z) p(so) IT p(s St, TT)T(a (s z) is the likelihood of the trajectory under policy n. The initial state So is sampled from the distribution p(So) and 7 E [0. 1) is a discount factor.
Skill Discovery By Maximizing Mutual Information. Eysenbach et al. (018)) and Peng et al. (2022) formulate the skill discovery problem as an unsupervised reinforcement learning problem, where the objective is to maximize the mutual information between the robot state and latent vector sampled from distribution z ~ p(z): [(S Z) = H(S)- (S, Z). This equation can be interpreted as the policy æ is to learn to produce diverse behaviors while each latent vector z should correspond to distinct robot states.
However, this equation is intractable in most scenarios where the state marginal distribution is un- known, and two tricks are commonly implemented to tackle this. The first trick is to take advantage of the symmetry of mutual information:
I(S: Z) I(Z: = HZ) (Z(S).
(2)
3

---

Published as a conference paper at ICLR 2024
Input Human Motion
Reconstructed
|Ip²
P111
Policy
R2H-Mapper
H2R-Mapper
x
P.
P
P
1| - 1,11
$/
Simulated Robot Motion
Reconstructed Robot Motion
Figure 2: Method Overview. A robot control policy utilizes human motions and sensory information generate robot actions for interacting with the environment. The Robot2Human -apaper is then used to reconstruct the input human motion pose from the robot's pose. Lastly, the Human2Robot Mapper used to reconstruct the robot pose from the reconstructed human pose. Both mappers trained via supervised learning using the difference between the real and reconstructed poses. These differences are also utilized for constructing a correspondence reward function, which used to update the policy (see Equation 8
This trick removes the need for measuring the marginal entropy of robot state HS) by instead measuring the entropy of the latent vector H (Z) which remains constant in fixed skill prior p(z). The second trick is
variational lower bound proposed by |Esenbbchh et al. (2018) and
Gregor et al. (016) to approximate the mutual information as follows:
/ (Z: = (Z) - (Z 2 max (z) + Ex--ple)===7ddii))))o
(3)
where q(z[s) is a variational approximation of the conditional distribution p(z[s) and the lower bound is tight if = p. This skill discovery objective encourages policy to produce distinct behaviors for different skill vectors z by designing a reward based on the measurement of (z|ss)
CROSSLocO
CrossLoco is guided unsupervised reinforcement learning framework designed learn robot lo- comotion control policy driven by human motion. The framework establishes a correspondence between human and robot motions, enabling the robot acquire locomotion skills from human mo- tions. The method overview is shown in Fig 2 In this section, we first introduce how we formulate the problem. Then, present our recle-oosisstency-haee human-robot correspondence. Lastly, we provide additional implementation details.
method for learning locomotion and
4.1 PROBLEM FORMULATION
Our goal is to train a robot control policy, denoted as π,, that can produce various robot motions based on different human motion inputs. Therefore, we can view our problem as Markov De- cision Process conditioned on the given human motion. Let us define P and P as the human and robot kinematic poses. Then the human motion is defined
sequence of pose vectors: m" = [p. .pi.pt.-. ... p]-]]- The robot state s represents both the kinematic and dynamic status of the robot, hence to motor commands, such as target joint angles. At each time step, the policy takes the robot state vector Se and the augmented human motion feature xf =<p.') input to generate action a, ~ π (a, ss x)) Then our goal to maximize the given reward function r:
view s the superset of the pose P/. The robot action corresponds
[(x) E xpm(")) " xxx))
rr(s at)]
(4)
t=0
where the robot trajectory is defined sequence of the robot states T" [s] si. -
This formulation leads to the question of designing an effective reward function r that builds the relationship between the human and robot poses, P and pç_ In addition, the reward function should
4

---

Published as a conference paper at ICLR 2024
Algorithm 1 CrossLoco pseudocodc
Require: Human Dataset M..
1: Initialize: Policy æ, Value function V, R211-Mapper ahah .122R-apper qhar, Data Buffer D.
2: repeat
3:
for trajectory i= m. do
4: + {(2= si (e, Fx) collect trajectory by rolling out æ.
5:
update я and using PPO with data from D. update qath and g* using data from D by minimizing 2 (Equation)
6:
7:
Reinitialize Data Buffer D.
8:
until Done
include some regularization terms, such minimizing the energy, avoiding self-collisions, or pre- serving the predefined semantic features. We will discuss the design of our reward function in the following section.
4.2 MEASURING CORRESPONDENCE VIA CYCLE-CONSISTENCY
To develop a reward function that represents the correspondence between human and robot motions, we borrow the information-theoeees the correspondence reward term such that it maximizes the mutual information between human and robot pose, given by (p] IT). From Equation 3 this formulation can be approximated by:
approach mentioned in the previous section. We formulate
T(P P (T) 2 (p.)) + I "mm"""(("))) ppp ppplllp ar) [log(1) (ogg(a"" 2h (p.) [P))].
(5)
where we assume that human motion prior is from the fixed dataset and refer gº2h Robot-to- Human Mapper (R2H-Mapper). Because the first term H(p²) is constant, we can find the optimal policy by maximizing the second term, logig-2h 2h (p |P A higher value represents that R2H- Mapper is more certain about the human pose given the robot pose, hence indicating that the human pose is distinetive given the robot pose.
We model the R2H-Mapper as a Gaussian distribution with fixed covariance 80th (p. (p:) = N (p?). .) where μ 2d (p?) is the mean of the distribution while or is the constant covariance matrix. The R2H-Mapper can be trained by minimizing loss function Lr2h,
arg min 2h = E (ptp:>(pppp/"<p(pp(e""e 2h
[Ilp 22th 11
(6)
where d' (p) is the likelihood of observing robot pose p by executing policy x given the human pose p/. Similarly, can design correspondence reward to minimize the given term Ilp -1- 12
However, R2H-Mapper does not prevent multiple robot poses p" from being mapped to the same human pose pd because it only considers one-directional mapping, which may cause degenerated motions. To address this issue, gh22 (p (p?lp!"), which is used for mapping the human pose back to the robot pose. We use a cycle- consistency formulation of Zhu et (2017), where we first map the robot pose the human pose, followed by mapping the generated human pose back to the robot pose. This results in an objective loss function rrr for H2R-Mapper and R2H-Mapper as:
add a Human-to-Rothe
Mapper (H2R-Mapper), denoted as
arg min 2r
2h -Ep
[|p h2r (1 11)))
2k tarr
Finally, from our mutual information maximization and cycle-consistency loss minimization, formulate the correspondence reward as follows:
exp( |pp 11 2h (p:1e
|p.
1 h2 (p.))
(8)
4.3 IMPLEMENTATION DETAILS
During the training process, the policy æ, as well as the H2R-Mapper and R2H-Mapper, are updated iteratively. The policy is trained using Proximal Policy Optimization (PPO) Schulman et al. (001))
5

---

Published as a conference paper at ICLR 2024
Human arm motron le tranglumed robot front lep
Source Human Motion
Unable to resonstruct are motiog durtig running
Simulated Robot Motion
Reconstructed Human Motion
Accurate reconstruct
Erron-0.07
EmrorD0.00
Errors0.01
Errdm-b.64
Emor-O.de
Ende-0.32
Error00330
Reconstructed Robot Mation
Error-0001 Error-0.02
Eiror-D.00 D0
Error-0,01
Erron-0.19
Emror00077
Erron-0,01
Figure 3: Visualization reconstructed human and robot pose with reconstruction error. We show that the mappers can accurately reconstruct human and robot poses when the robot actively follows human movement. When the human is relatively stationary, human arm movements can be mapped onto the robot's front legs. In scenarios like human running, the framework focuses primarily on leg motion.
Meanwhile, the H2R-Mapper and R2H-Mapper are trained using supervised learning. The learning framework is summarized in Algorithm
Model Representation. Human pose (p* E R23) and robot pose (p" E R!7) consist of local information, including root height, root orientation, and joint pose. The robot state (s R²7) contains all the information in p* as well as root and joint velocity, and previous action. The human feature vector (x R888) includes human pose, root velocity, and joint velocity information the future 1, 2, 10, and 30 frames.
Complete Reward Function. In addition the correspondence reward mentioned earlier, our re- ward function includes several terms to regulate the training and preserve high-level semantics. A root tracking reward, denoted as rfoot exp(-11ss level movements by minimizing the deviation between the normalized base trajectories of the hu- man (8poot) and the robot (sroot). In this context, both trajectories include the root position and height, which are normalized by their respective leg lengths. Without this term, the resulting correspondence can be arbitrary: c.g., human forward walking motion can be mapped into a robot's lateral movements. To prevent unrealistic movements, torque penalty. yoor -I[a, and joint limits penalty, p/um = -I are borrowed from Rudin et al. (2022). Here, p is the pose limit of the robot. The overall reward is calculated the weighted sum of all these terms: T2 y. opd + r + tor + iimm To optimize the weights, increas- ing wird can improve the correspondence between robot and human motion, but it may nega- tively affect root tracking performance or increase energy consumption. Increasing woot puts more emphasis on root tracking. Small values for wtor and fim can result in unnatural mo- tions, while excessively large values can lead to overly conservative motions. In our setting, we use Cwd roat tor lim = [1.0, 1.0, 0.0001, 5.0J.
s.oot()) I]). is designed to preserve high-
Network Structure. The policy, critic, H2R-Mapper, and R2H-Mapper are modeled by fully- connected network consisting of three hidden layers with 512 nodes each. ELU is used as the activation function for the policy and critic, while ReLU is used for the mapper networks.
EXPERIMENTS
We conduct series of experiments to investigate three key aspects of the proposed work: firstly, the feasibility of acquiring a human-motion-ddrveen robot controller, which referred to as the 'human2robot' controller; secondly, the comparative performance against alternative baseline ap- proaches; and lastly, the influence of the correspondence reward on the training process.
We evaluate the effectiveness of our approach by transferring a set of human motions to Aliengo quadrupedal robot (unitree 2023) with 12 joints, which has a significantly different morphology compared to humankind. We take human motion from LaFANI dataset (Harvey et al.) 2020). Our human dataset consists of 50 human motion trajectories with eight seconds of the average clip length. The dataset contains various types of human movements, including walking, running, hopping, and

---

Published as a conference paper at ICLR 2024
Forward Jumping
In-place Dancing
Forward Walidnn
CrossLoco Result
CrossLoco Result
CrossLoco Result Crawling Motion.
Foot still in contact, no jump up.
Over Conservative Stil.
<<<
DeepMimic
Tasked-RL Result
R2H
Figure 4: Comparison of generated motions. Our method, CrossLoco, synthesizes more compelling motions compared the baselines by preserving both high-level semantics and fine motion details.
dancing. All experiment environments are conducted using Isaac Gym (Makoviychuk et al. 2021). nigh-performance GPU-based physics simulator. During training, 1024 environments are simulated in parallel on a single NVIDIA GeForce RTX 3080 Ti GPU for period of about 3 days.
5.1 MAIN RESULTS AND ANALYSIS
We present the results of our human motion driven control experiments in Figure Our method successfully learns human2robot controller that can transfer various human motions robot with a different morphology. We observe agile motions from the robot, such as running and sharp turning when the human performs fast locomotion. The results also demonstrate that the robot can creatively follow human dancing motions, which is hard to manually design. This showcases the capability our method to establish automatic correspondence between humans and robots while learning diverse robot skills. All the motions can be best seen in the supplementary video.
In Figure: 3 we demonstrate the effectiveness of our proposed correspondence reward in conjunc- tion with the R2H and H2R mappers, which enable the robot to mimic human movements accurately. It's important to note that such synchronization unattainable without the correspondence reward. Without it, the robot would remain stationary merely track the human's root position. Our ex- periments have revealed noteworthy findings. For instance, even when humans walk at a similar speed but with varying styles-sure as different step frequencies- the robot adapts its movement to match the human's ' frequency. Another intriguing observation is the relationship between the corre- spondence and root tracking. When human is stationary moves minimally, the robot responds effectively to the human's 's arm movements. However, during rapid movement, the robot prioritizes leg motion. We hypothesize that during high-speed locomotion, the root tracking reward becomes more dominant, leading to balancing act in the CrossLoco framework between root tracking and achieving high-quality correspondence.
In some scenarios, the robot is not able to perfectly mimic the given human motion. For instance, when human moves backward and makes a sharp 180-degree turn, the robot cannot follow the desired orientation. Additionally, CrossLoco may struggle to transfer large side stepping. There are two potential reasons for these imperfections. First, the neural network may incapable of captur- ing all motions. Second, the robot's 's morphology may prohibit it from performing certain motions that are easy for humans, such as swift turning. During training, we observed that the diversity of human motion is critical to the training outcome. This is because the learned mappers can overfit to specific scenarios. For instance, if the dataset contains only hand-waving motions, the robot might slightly vibrate the root to maximize the correspondence reward hence lead to undesirable motion transfer results.
5.2 BASELINE COMPARISON
We further quantitatively compare our method against the following baseline methods:
Engineered Motion Retarget + DeepMimic (DeepMimic)) This baseline contains two stages. Firstly, an expert manually designs a motion retargeting function to translate human motions to robot motion referenced trajectories. Then, the robot is trained to track these

---

Published as a conference paper at ICLR 2024
ACR
DIV T
RTR 1
CR 7
PR
CrossLoco (Ours)
0.785
2.853
0.743
65.5%
43%
DeepMimic
0.558
2.231
0.579
67.5%
16%
Task-Only
0.556
2.494
0.740
30.0%
14%
R2H-Only 0.683
2.779
0.729
42.5%
27%
Table 1:
Quantitative Results. Our CrossLoco outperforms all the criteria except for being the second-best at correctness with small margin.
reference trajectories using DeepMimic (Peng et al. 2018ap. It is important to note that the result of this baseline heavily relies on the quality of the retargeted motions, which requires a significant amount of effort from the expert. In our case, we have designed the retargeted motions by matching the human foot and robot foot with fixed tripod gait.
This baseline is designed to investigate the impact of the proposed correspon- dence reward training outcomes. As such, we compare this approach to CrossLoco, where the weight of correspondence set to zero (w 0). Therefore, this policy is trained solely on a root tracking reward and other regularization rewards.
Task-Only:
R2H-Only: This baseline is designed by removing the robot pose cycle-consistency part from CrossLoco and only keeping the human pose's consistency. As there is no robot pose consistency, this baseline, the correspondence reward defined as ri opd.r2h cxp(-Ilp! -p" 2h (p;2)pp)))))))) (2))
Since we assume we have no robot motion dataset, hence, we don't include any GAN-based baseline methods, such as Adversarial Correspondence Embedding (Li al. 2023b).
Our objective to quantitatively assess the effectiveness of establishing correspondence between human and robot motion, well the diversity of the robot motion of these methods. In order to achieve this, we utilize the following metrics:
Averaged Correspondence Reward (ACR): This term measures the correspondence be- tween human and robot motions. A higher ACR indicates better correspondence. Even though correspondence reward is used in each baseline training procedure, co-train R2H-Mapper and H2R- Mapper to measure the correspondence reward.
Diversity (DIV): This term has been used for measuring motion diversity in many SOTA works (Shafir et al. 2023 Guo et al. 2023). A higher DIV indicates robots can acquire of all generated motions from different source human motions, two subsets of the same size Sa are randomly picked. The diversity of this set of motion is (s"")) (s()) and (ss")) are features extracted from robot state. Here, we pick robot root velocity and joint pose as the feature.
more skills. From
defined as: DIV 1 Su - [V{(s
52
Averaged Root Tracking Reward (RTR): This term measures the learned policy can track the desired root trajectory. The root tracking reward is defined in Section 4.3
In addition these metrics, we conducted a user study with subjects to evaluate the performance from subjective perspective.
Correct Rate (CR): We first investigate whether users can identify a correct match be- tween the given human and synthesized robot motions. A user is tasked to find matched pair from one human animation and four retargeted robot motions. One robot motion is retargeted from the given human motion, while the other three generated from different inputs. We examined the combination of four human motions and four methods, and then measured the percentage of correct matches. Ideally, good transfer should accurately capture the style of the human motion, resulting in easy matching for the user.
Preference (PR): In the second part, we provided users with robot motions generated with different approaches. We asked the users to select the motions that they believed repre- sented a good transfer. We then measured the ratio at which each method was chosen.
Our results summarized in Table I The quantitative analysis indicates that CrossLoco outper- forms the baseline methods all metrics suggesting that it can effectively learn a controller that can translate different human motions diverse robot motions while tracking the desired root trajectory.
8

---

Published as a conference paper at ICLR 2024
MDM
LMM
Cross
Loco
"Squats down then jumpa."
Language2Robot
Interactive Robot Control
Figure 5: Illustration of two applications: Language2Robot and Interactive Robot Control.
Although DeepMimic is designed as a one-to-one mapping between human and robot poses, achieved lower correspondence reward than CrossLoco (0.558 0.785). This could be attributed to the fact that an engincering mapping function may not be physically feasible for all input human motions, and the learning process may sometimes sacrifice the desired posing tracking for higher desired root tracking reward. Moreover, since the engineered robot desired motion can sometimes be physically infeasible, the root tracking reward of DeepMimic is also lower than that of CrossLoco (0.579 vs 0.743). Based on the results of the user study, it was found that DeepMimic's i motion (with a 16% PR score) was not preferred by users. However, it achieved the highest CR score (67.5 (67.5%) This can be because users can match human and robot motions based on the most obvious frames, even with poor overall motion quality.
As for Task-only, since it is trained only with root tracking and regularization rewards, it produces conservative motions by ignoring human leg motions some cases, such as different in-place danc- ing motions. All human in-place dancing motions are mapped to the robot standing with slight root movements by Task-only baseline. However, the correspondence reward in CrossLoco triggers robots learn diverse skills that correspond different human motions, evidenced by its superior performance in terms of correspondence reward, diversity term, and user study results compared to Task-only. We also obverse that for CrossLoco achieves slightly higher root tracking reward, indi- cating CrossLoco's s great capability.
The results show that CrossLoco achieves higher correspondence reward and more diverse motion compared to R2H-Only. Additionally, users found the results of CrossLoco to be more distinguish- able. This could be attributed the effective regularization provided by the cycling of human back to robot, resulting in more distinguishable outcomes.
5.3 APPLICATIONS
Numerous studies have delved into human motion synthesis using a variety of input sources, such as text (Bahl et al. 2022), music Tseng et al. (2023), and user inputs (Holden et al. 2020). Our learned human2robot controller can be seamlessly integrated with these modules, utilizing human motion the interface for applications. In this context, we have implemented two examples: Language2Robot and Interactive Robot Control. An illustration of these examples is presented module (MDM (Bahl) ct al. 2022) ) with CrossLoco, enabling the generation of robot motion based on verbal instructions.
in Figure 5 Language2Robot merges the established text-to-human-mote
Interactive Robot Control combines an existing humanoid character controller (LMM (Holden et al. 2020) with a moossLoco-triine policy to facilitate interactive robot control without the need retrain a large-scale interactive robot control policy from scratch. For more details, please refer to our supplementary videos and the appendix.
6 DISCUSSION
We introduce CrossLoco, an unsupervised reinforcement learning framework designed to en- able robot locomotion control driven by human motion. This framework incorporates cycle- consistency-bases reward function, which facilitates the discovery of robot skills and the establish- ment of correspondence between human and robot motion. Our experimental results demonstrate the effectiveness of our approach in transferring wide range of human motions to control a robot that has a different morphology.
Our next steps involve exploring two directions. Firstly, we aim to extend our framework beyond locomotion control to more complex scenarios, including long-horizon human demonstrations that involve long-dis distance locomotion and tool manipulation a
robot, such quadrupedal robot with an arm mounted on its body. Secondly, we are interested in implementing method real-world robots for practical applications.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
This work is supported by National Science Foundation under Grant No.2222723. We would like to thank Yuxiang Yang and Jiahan Fan for valuable suggestions.
REFERENCES
Kfir Aberman, Peizhuo Li, Dani Lischinski, Olga Sorkine-Hornung, Daniel Cohen-Or, and Baoquan Chen. Skeleton- aware networks for deep motion retargeting. ACM Transactions Graphics (TOG), 39(4):62-1 1, 2020.
Shikhar Bahl, Abhinav Gupta, and Deepak Pathak. Human-to-ren imitation in the wild. 2022.
Aayush Bansal, Shugao Ma, Deva Ramanan, and Yaser Sheikh. Recycle-gan: Unsupervised video retargeting, In Proceedings of the European conference on computer vision (ECCV), pp. 119-135, 2018.
Kevin Bergamin, Simon Clavet, Daniel Holden, and James Richard Forbes. Drecon: data-driven responsive control of physics-basee boysic characters. ACM Transactions On Graphics (TOG), 38(6):
2019.
Konstantinos Bousmalis, Alex Irpan, Paul Wohlhart, Yunfei Bai, Matthew Kelcey, Mrinal Kalakrish- nan, Laura Downs, Julian Ibarz, Peter Pastor, Kurt Konolige, et Using simulation and domain adaptation to improve efficiency of deep robotic grasping. In 2018 IEEE international conference on robotics and automation (ICRA), pp. 4243 4250. IEEE, 2018.
Sungjoon Choi, Matt Pan, and Joohyung Kim. Nonparametric motion retargeting for humanoid robots on shared latent space. In 16th Robotics: Science and Systems, RSS 2020, MIT Press Journals, 2020.
Guillermo del Toro. Pacific rim, 2013.
Brian Delhaisse, Domingo Esteban, Leonel Rozo, and Darwin Caldwell. Transfer learning of shared latent spaces between robots with similar kinematic structure. In 2017 International Joint Con- ference on Neural Networks (IJCNN, pp. 4142-4149. IEEE, 2017.
Benjamin Eysenbach, Abhishek Gupta, Julian Ibarz, and Sergey Levine. Diversity is all you need: Learning skills without a reward function. arXiv preprint arXiv: MXXiv:8802.00770 2018.
Michael Gleicher. Retargetting motion to new characters. In Proceedings of the 25th annual con- ference on Computer graphics and interactive techniques, pp. 42, 1998.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde- Warde-Farley. Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
Ruben Grandia, Farbod Farshidian, Espen Knoop, Christian Schumacher, Marco Hutter, and Moritz Bächer. Doc: Differentiable optimal control for retargeting motions onto legged robots. ACM Transactions on Graphics, 42(4), 2023.
Karol Gregor, Danilo Jimenez Rezende, and Daan Wierstra. Variational intrinsic control. arXiv preprint arXiv:1611. 1.07501.,777,.. 2016.
Wen Guo, Yuming Du, Xi Shen, Vincent Lepetit, Xavier Alameda- Pineda, and Francese Moreno- Noguer. Back to mip: A simple baseline for human motion prediction. In Proceedings of the IEÉE/CVF Winter Conference on Applications of Computer Vision, pp. 4809-4819 2023.
Tuomas Haarnoja, Schoon Ha, Aurick Zhou, Jie Tan, George Tucker. and Sergey Levine. Learning to walk via deep reinforcement learning. arXiv preprint arXiv:1812.11003 2018.
Félix G. Harvey, Mike Yurick, Derek Nowrouzezahrai, and Christopher Pal. Robust motion in- betweening. 39(4), 2020.
10

---

Published as a conference paper at ICLR 2024
Donald Hejna, Lerrel Pinto, and Pieter Abbeel. Hierarchically decoupled imitation for morphologi- cal transfer. In International Conference on Machine Learning, pp.
PMLR, 2020.
Jonathan Ho and Stefano Ermon. Generative adversarial imitation learning. Advances in neural information processing systems, 29, 2016.
Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei Efros, and Trevor Darrell. Cycada: Cycle-consistent adversarial domain adaptation. In International conference on machine learning, pp. 1989-1998. Pmlr, 2018.
Daniel Holden, Oussama Kanoun, Maksym Perepichka, and Tiberiu Popa. Learned motion match- ing. ACM Transactions on Graphics (TOG), 19
Shūji Iuchi. Madö king granzört, 1989.
Stephen James, Paul Wohlhart, Mrinal Kalakrishnan, Dmitry Kalashnikov, Alex Irpan, Julian Ibarz, Sergey Levine, Raia Hadsell, and Konstantinos Bousmalis. Sim-to-real via sim-to-sim: Data- efficient robotic grasping via celoomized-t-caannnii adaptation networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12627-12637, 2019.
Hanyoung Jang, Byungjun Kwon, Moonwon Yu, Seong Uk Kim, and Jongmin Kim. A variational u-net for motion retargeting. In SIGGRAPH Asia 2018 Posters, pp. 1-2. 2018.
Kuno Kim, Yihong Gu, Jiaming Song, Shengjia Zhao, and Stefano Ermon. Domain adaptive imita- tion learning. In International Conference on Machine Learning, pp. 5286 5295. PMLR, 2020.
Tianyu Li, Hartmut Geyer, Christopher G Atkeson, and Akshara Rai. Using deep reinforcement learning learn high-level policies on the atrias biped. In 2019 International Conference on Robotics and Automation (ICRA), pp. 263-269. IEEE, 2019.
Tianyu Li, Jungdam Won, Jeongwoo Cho, Sehoon Ha, and Akshara Rai. Fastmimic: Model-based motion imitation for agile, diverse and generalizable quadrupedal locomotion. Robotics, 12(3): 90, 2023a.
Tianyu Li, Jungdam Won, Alexander Clegg, Jeonghwan Kim, Akshara Rai, and Sehoon Ha. Ace: Adversarial correspondence embedding for cross morphology motion retargeting from human to nonhuman characters. SIGGRAPH Asia, 2023b.
Hung Yu Ling, Fabio Zinno, George Cheng, and Michiel Van De Panne. Character controllers using motion vaes. ACM Transactions on Graphics (TOG), 4904):40-1, 2020.
Ming-Yo Yu Liu, Thomas Breuel, and Jan Kautz. Unsupervised image-to-image translation networks. Advances in neural information processing systems, 30, 2017.
Naureen Mahmood, Nima Ghorbani, Nikolaus F Troje, Gerard Pons-Moll, and Michael J Black. Amass: Archive of motion capture as surface shapes. In Proceedings of the IEEE/CVF interna- tional conference on computer vision, pp. 5442-5451, 2019.
Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, al. Isaac gym: High performance gau-based physics simulation for robot learning. arXiv preprint 10X1v:2108.10470, 2021.
Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel van de Panne. Deepmimic: Example- guided deep reinforcement learning of physics-basee character skills. ACM Trans. Graph., 37 URL. http:
(4):14311-143:1 July 2018a. ISSN 0730-0301. doi: 10. 10.1145/11951173221131.
//doi acm. org. 10. 1145/3 97517 30077711111117771 3201311
Xue Bin Peng, Angjoo Kanazawa, Jitendra Malik, Pieter Abbeel, and Sergey Levine. Sfv: Rein- forcement learning of physical skills from videos. ACM Trans. Graph., 37(6). November 2018b.
Xue Bin Peng, Erwin Coumans, Tingnan Zhang, Tsang- Wei Edward Lee, Jie Tan, and Sergey Levine. Learning agile robotic locomotion skills by imitating animals. In Robotics: Science and Systems, 07 2020. doi: 10. 1.5607/RSS.22200X1..666

---

Published as a conference paper at ICLR 2024
Xue Bin Peng, Yunrong Guo, Lina Halper, Sergey Levine, and Sanja Fidler. Ase: Large-scale reusable adversarial skill embeddings for physically simulated characters. ACM Trans. Graph. 41(4), July 2022.
Kanishka Rao, Chris Harris, Alex Irpan, Sergey Levine, Julian Ibarz, and Mohi Khansari. RI- cyclegan: Reinforcement learning aware simulation-to-real. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 11157-11166, 2020.
Helge Rhodin, James Tompkin, Kwang In Kim, Kiran Varanasi, Hans-Peter Seidel, and Christian Theobalt. Interactive motion mapping for real-time character control. In Computer Graphics Forum, volume 33, pp. 273-2822 Wiley Online Library, 2014.
Nikita Rudin, David Hoeller, Philipp Reist, and Marco Hutter. Learning walk in minutes using massively parallel deep reinforcement learning. In Conference on Robot Learning, pp. 91-100. PMLR, 2022.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint frXIv:1707.06347 2017.
Pierre Sermanet, Corey Lynch, Yevgen Chebotar, Jasmine Hsu, Eric Jang, Stefan Schaal, Sergey Levine, and Google Brain. Time-contrastive networks: Self-supervised learning from video. In 2018 IEEE international conference on robotics and automation (ICRA), pp. 1134-1141. IEEE, 2018.
Yonatan Shafir, Guy Tevet, Roy Kapon, and Amit H Bermano. Human motion diffusion as gener- ative prior. arXiv preprint arXiv:2303.01418 2023.
Tanmay Shankar, Yixin Lin, Aravind Rajeswaran, Vikash Kumar, Stuart Anderson, and Jean Oh. Translating robot skills: Learning unsupervised skill correspondences across robots. In Interna- fional Conference on Machine Learning, pp. 19626-19644. PMLR, 2022.
Pratyusha Sharma, Deepak Pathak, and Abhinav Gupta. Third-person visual imitation learning via decoupled hierarchical controller. Advances in Neural Information Processing Systems, 32, 2019.
Laura Smith, Nikita Dhawan, Marvin Zhang, Pieter Abbeel, and Sergey Levine. Avid: Learning multi-stage tasks via pixel-level translation of human videos. arXiv preprint arXiv:1912.0444; 2019.
Steven Spielberg. Ready player one, 2018.
Gregory J Stein and Nicholas Roy. Genesis-rt: Generating synthetic images for training secondary real-world tasks. 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 7151-7158. IEEE, 2018.
Seyoon Tak and Hyeong- Ko. physically-bee motion retargeting filter. ACM Transactions 117, 2005.
on Graphics (TOG),
Jie Tan, Tingnan Zhang, Erwin Coumans, Atil Iscen, Yunfei Bai, Danijar Hafner, Steven Bohez, and Vincent Vanhoucke. Sim-to-real: Learning agile locomotion for quadruped robots. arXiv preprint arXiv:1804 10332, 2018
Yujin Tang, Wenhao Yu, Jie Tan, Heiga Zen, Aleksandra Faust, and Tatsuya Harada. Saytap: Lan- guage quadrupedal locomotion. arXiv preprint arXiv:2306.07580. 2023.
Jonathan Tseng, Rodrigo Castellon, and Karen Liu. Edge: Editable dance generation from music. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 448-458, 2023.
unitree. Unitree robotics, 2023.
Ruben Villegas, Jimei Yang, Duygu Ceylan, and Honglak Lee. Neural kinematic networks for unsupervised motion retargetting. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8639-8648 2018
12

---

Published as a conference paper at ICLR 2024
Jungdam Won and Jehee Lee. Learning body shape variation in physics-based characters. ACM Transactions on Graphics (TOG), 38(6):1-12, 2019.
Zhaoming Xie, Glen Berseth, Patrick Clary, Jonathan Hurst, and Michiel van de Panne. Feedback control for cassie with deep reinforcement learning. In 2018 TEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1241-1246. IEEE, 2018.
Qiang Zhang, Tete Xiao, Alexci A Efros, Lerrel Pinto, and Xiaolong Wang. Learning cross-domain correspondence for control with dynamics cycle-consistency. arXiv preprint arXiv:2012.098/1, 2020.
Tinghui Zhou, Philipp Krahenbuhl, Mathieu Aubry. Qixing Huang, and Alexei A Efros. Learning dense correspondence via 3d 3d-guided cycle consistency. Proceedings of the IEEE conference computer vision and pattern recognition, pp. 117-126, 2016.
Jun Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE international conference computer vision, pp. 2223-2233 2017.
A TRAINING HYPERPARAMETERS
In this study, we trained the policy using Proximal Policy Optimization (PPO) Schulman et al. (017)) Additionally, both the H2R-Mapper and R2H-Mapper were trained utilizing supervised learning techniques. Below, we detail the hyperparameters employed in our training process.
Env number
1024
Batch size
2048
Policy Ir
le-4
Critie Ir
le-4
Mappers Ir
lc-4
Optimizer
Adam
Clip
0.2
Entropy Coefficient
5e-3
7
0.99
0.9
Table 2: Training hyperparameters.
B APPLICATIONS
Many research and work have explored the field of human motion synthesis using various input sources, including text (Bahl et al. 2022). music Tseng et al. (202)), and user inputs (Holden et al.) 2020 Our learned human2robot controller can be seamlessly integrated with these modules by using human motion as the interface for new applications. In this section, we present two examples: Rooo and Interactive Robot Control.
Rongu Our approach involves utilizing the texr2human module in combination with our human2robot controller. This allows for the generation of robot movements from language by first producing human motion using the text2human module, which is then transferred to the robot using the human2robot controller. Our method differs from recent language to quadrupedal robot motion angineering-intensse interface, such foot
work (Tang al. 2023) in that it does not require contact patterns, which could limit the range of possible generated robot motions.
In our implementation, utilize the Human Motion Diffusion Model (MDM) (Bahl et al., 2022 as our exx--tohhuman motion translator. MDM is a diffusion model-based lightweight model that achieves state-of-the-an results on leading benchmarks. However, MDM uses AMASS (Mahmood
13

---

Published as a conference paper at ICLR 2024
al. 2019) human model which is different from the LaFANI (Harvey et al. 2020) model we used for training the policy. Therefore, we retarget the outputs of the MDM to the skeleton model we use.
Our study involves testing the generation of robot motion based different input messages. The results our study are presented in Figure []] We show that this framework can generate robot motion according instructions. For instance, the message "strides swiftly in straight" results in a fast walking straight robot motion, while "squats down then jumps" triggers a robot squats and jumps motion.
Interactive Robot Control. Interactive control of robots in response to changing conditions or un- expected obstacles is significant challenge in robotics, which involves careful controller design and motion planning. Recent advances in character animation enable users to interactively control human characters using joysticks, automatically adapting their motion styles to the surrounding en- vironment, like crouching in confined spaces or leaping over obstacles. Our key idea for the second application, Interactive Robot Control, leverage the existing human animation techniques for robot control. Instead of retraining large-scale model from scratch, simply translate the output of the existing character controller to the robot's operational space using the proposed method.
Our implementation of this framework utilizes Learned Motion Matching (LMM) (Holden et al., 2020), which is a scalable neural network-based framework for interactive motion synthesis. We combine LMM with learned controller. During interactive robot control, LMM takes input user commands for human motion generation, and our controller converts the generated human motion to robot control commands.
We evaluated our implementation by controlling the robot using a joystick. Figure 5 presents the results of using LMM2robot. The experiment demonstrated that the robot can actively adjust its motion based on the user's s commands. These results provide evidence of the effectiveness of our learned controller for interactive robot control.
14


Reviews:
[{"review_id": "6IsDuuZo0Q", "reviewer": "Reviewer_TXup", "summary": "The paper proposes the CrossLoco framework, a guided unsupervised reinforcement learning framework that simultaneously learns robot skills and their correspondence to human motions. This is achieved thanks to the use of a \"cycle-consistency-based\" reward, inspired in generative vision system such as Cycle-GAN. The reward function has components related to the learning of a policy, components related to the human-robot correspondence problem, and terms to regulate the training and preserve the high-level semantics. The proposed framework is validated in the task of transferring a set of human motions to the Aliengo quadrupedal robot, and its performance compared against manually engineered and unsupervised base-line algorithms. CrossLoco obtains better results than the other algorithms in this comparison.", "questions": "No questions", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The main contribution of this paper is the CrossLoco framework which uses a \"cycle-consistency-based\" reward. The reward function has components related to the learning of a policy, components related to the human-robot correspondence, and additional rewards terms to regulate the training and preserve the high-level semantics.\n\n2. The proposed framework is correctly validated by using it for transferring a set of human motions to the Aliengo quadrupedal robot.\n\n3. The paper is clear, well-organized and well-written.", "weaknesses": "I do not see any clear weakness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "De1W7slhyY", "reviewer": "Reviewer_QSkw", "summary": "This paper presents CrossLoco, an unsupervised reinforcement learning framework designed to translate human motions into robot controls, addressing the challenge of establishing correspondence between humans and quadrupeds. This framework introduces a cycle-consistency-based reward term and maximize mutual information between human motions and robot states. CrossLoco outperforms baseline algorithms demonstrating its effectiveness. Additionally, the framework shows use in applications like language2robot and interactive robot control.", "questions": "It seems the authors didn't provide details about the human motion dataset, also lack RL training and supervised training hyperparameters. These details would be beneficial for researchers to follow.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The implementation of both R2H-Mapper and H2R-Mapper to reconstruct human and robot motion is a novel approach that ensures cycle consistency.\nThe inclusion of a supplementary video provides a more intuitive showcase of the qualitative results.\nThe authors conducted a user study to demonstrate the effectiveness of their motion retargeting control results.\nThe potential applications for language-driven motion control and interactive motion control for quadrupeds are promising. Scaling up to larger human more datasets to enable quadrupeds to perform more tasks could be very promising.", "weaknesses": "The baseline used for retargeting seems weak. Why not employ stronger learning-based retargeting baselines combined with motion imitation algorithms?\n\nThe definition of the root tracking reward is unclear, particularly the definition of $s_{root}$ and $\\bar s_{root}$. I assume $s_{root}$ refer to robot root states including global translation and orientation but what is $\\bar s_{root}$? The paper claims this reward minimizes deviation between the normalized base trajectory of human and robot, is  $\\bar s_{root}$ the root states of human? A clearer definition would be beneficial.\n\nThe evaluation metrics could be improved. For instance, introducing Frechet Inception Distance (FID), used in ACE, to measure human motions and robot motions. \n\nAdditionally, the use of the averaged correspondence reward is questionable; as the author mentioned, a human forward walking motion could be mapped into a robot’s lateral movements. This metric may not accurately reflect correspondence between motions and seems more like a reflection of the training reward (loss) rather than a robust evaluation metric. A suggestion would be for the author to use paired data to train the two mappers independently and apply these mappers for testing all methods or directly use existing SOTA motion retargeting models.\n\nThere is only one sequence for qualitative comparison with the baselines, I encourage authors to provide more and include the reference motion for deepmimic (retargeting results).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "K8tQcGCxjd", "reviewer": "Reviewer_Xm7A", "summary": "This paper proposes learning cross-morphology human motion-driven control to map from human motion to robot control signals in a guided unsupervised learning fashion. Without using any pre-collected dataset from the robot domain, this work proposes using a cycle-consistency reward based on translation from human motion to robot motion and then from robot motion to human motion to train such a control network. By formulating the problem as a skill-discovery problem, where the skill is specified through direct human motion input and the reward is based on the mutual information between the robot state and human motion (specified by the cycle-consistency reward), the method shows interesting results in translating human motion to semantically meaningful and natural robot motion.", "questions": "- I think providing results on the two Mappers, either a reconstruction metric or visualizations, could really help in understanding the capability of the framework. Does the algorithm learn some novel mapping?\n- Showing results on simple and semantically meaningful sequences, such as raising hands, moving hands, and hands and feet, could really demonstrate the capabilities of the method. Showing results on locomotion does not really add much value since root tracking reward already exists.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- This paper provides a convincing approach for automatic human motion to robot motion translation framework that does not rely on any predefined robot motion dataset. Since human motion is relatively accessible, using human motion to directly control robot motion and try to create semantically similar motion is an intuitive and useful approach.\n- I find the use of cycle consistency to translate between human motion and robot motion a well-principled approach for this task. As far as I know, this is a novel approach. The R2H-Mapper and H2R-Mapper provide a scalable approach for human motion-guided robot control.\n- The provided qualitative result shows that the method can translate human motion into relatively realistic human motion. In the handful of motion with semantic meaning (dancing motion), the robot generates similarly semantically meaningful motion.\n- Compared to prior methods that perform retargeting in the kinematic space (e.g. ACE [1]), this paper tackles the more challenging simulated control task. \n\n[1] Li, Tianyu et al. “ACE: Adversarial Correspondence Embedding for Cross Morphology Motion Retargeting from Human to Nonhuman Characters.” ArXiv abs/2305.14792 (2023): n. pag.", "weaknesses": "- Missing evaluation on the R2H-Mapper and H2R-Mapper. As the main reward provider, it is important to visualize how well these learned networks can translate between the two different modalities of motion. Do certain correspondences emerge (e.g. robot forward foot mapping to hands)? Are there failure modes that the mapper didn't learn?\n    - I think the whole application section could be replaced with a more detailed analysis and results from the two mappers. If the method can really handle human motion, applying it to language control and interactive control could be explored to supplement or just provide video results. If applications are studied, I think the robustness of the method to different modes of human motion should be investigated: e.g. can the learned motion mapper handle unseen motion like breakdancing and cartwheeling?\n    - Since the mapping between poses $p^h$ and $p^r$ is done locally, I think showing the reconstructed result together with the ground truth root would be beneficial.\n- Though the qualitative results provide a number of motion sequences, most of them are locomotion sequences (walking and running) which I think a Task-only controller should solve relatively easily. The only semantically meaningful sequences are dancing, of which there are only a handful of examples. There is a glimpse of good semantic mapping, but it is not super conclusive. Instead of focusing on dancing, maybe simpler motions like raising hands, reaching for objects, and more complex hand movements could be more suitable for evaluating this system's capabilities. Similar to the previous point about evaluating the mappers, it's not clear how well each part of the human body is mapped.\n    - The results on Deepmimic show that an expert-designed mapping may not be plausible and not easy to learn. However, since there is no study on what kind of mapping CrossLoco learned, it is hard to see how CrossLoco is better. Does a good mapping exist between bipedal and quadruped robots? Is this task too ill-posed? Or have we just not found a good mapping yet?\n- Need to discuss the limitations of the current framework. On what human motion does the matching score the lowest? Which human motion can be best matched? What happens if more or fewer human motion sequences are used for learning the mapping?\n    - I think the entire application section could be replaced with a more detailed analysis and results from the two mappers. If the method can truly handle human motion, it could be applied to language control and interactive control, which could be supported by supplementary materials or video results. If applications are studied, I believe the robustness of the method to different modes of human motion should be examined. For example, can the learned motion mapper handle unseen motions such as breakdancing and cartwheeling?\n    - Since the mapping between poses $p^h$ and $p^r$ is done locally, I believe showing the reconstructed result together with the ground truth (GT) root would be beneficial.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)