Team JANUS Humanoid Avatar:
A cybernetic avatar to embody human telepresence
R. Cisneros1,∗ , M. Benallegue1 , K. Kaneko1 , H. Kaminaga1 , G. Caron1 , A. Tanguy2 , R. Singh1 , L. Sun1 ,A. Dallard1,2 , C. Fournier2 , M. Tsuru1 , C. Yang1 , Y. Osawa3 , G. Lorthioir1 , F. Kanehiro1 and A. Kheddar1,2
Abstract— We present the avatar system developed by Team
JANUS, which is finalist in the ANA Avatar XPrize competition.
We briefly describe our avatar robot, the operator interface
system, and the robot operation system.

I. I NTRODUCTION
We recently experienced a very unusual situation worldwide. Our lifestyle has changed dramatically since the start of
the COVID-19 pandemic. The latter impacted lifestyles andhabits of most daily tasks that were usually done in physicalpresence. Before, experts were easily able to travel to a very
far remote location where their specific knowhow is needed.
The ordinary population was able to visit their loved ones
at will. Now, the situation is no longer the same. Medicaldoctors, engineers, researchers. . . are now in need to find analternative to deal more efficiently with such situations thatare highly likely to show-up again in the future. High fidelity
telepresence and beaming of robotic avatars is envisioned
as one of the technologies to mitigate the impact on ourdaily life, should such a catastrophic situation occur again. InMarch 2018, XPRIZE foundation launched a new challenge:
the ANA Avatar XPRIZE. The purpose of the competition
is to develop an avatar system to deploy senses, actions andpresence to a remote location in real time 1 .
Team Janus is among the 20 finalists of the ANA Avatar
XPRIZE competition. It is a bi-located team that gathers
expertise from the National Institute of Advanced Industrial
Science and Technology (AIST), Japan and the French
National Centre for Scientific Research (CNRS), France. Particularly, it is formed by members of the CNRS-AIST Joint
Robotics Laboratory (JRL) in Tsukuba, and the Interactive
Digital Human group at LIRMM in Montpellier.
II. O UR VISIONIn Roman mythology, Janus represents the transition fromthe past to the future, like the one we are seeing today withnew technologies. It also represents bridges and connections,like the ones we are building between humans and robots:
two entities into the same body.
We are using an adult-sized humanoid avatar: HRP-4CR,
which is the only one with a close-to-human look thatcan walk, manipulate objects, and realize facial expressions.
Therefore, we didn’t want to sacrifice the bipedal challenge
1 CNRS-AIST JRL (Joint Robotics Laboratory), IRL, AIST, Japan.
2 CNRS-UM, LIRMM, France.
3 Industrial Cyber-Physical Systems Research Center, AIST, Japan.
∗ Corresponding author E-mail: rafael.cisneros@aist.go.jp
1 https://www.xprize.org/prizes/avatarFig. 1.

HRP-4CR synchronized with the operator

nor its shape. Instead, we wanted to demonstrate a humanoidavatar that (i) can be easily controllable, (ii) can provide richsensory feedback, and (iii) can be an elegant solution in many
applications. See Figure 1.
It is worth to mention that among the finalists of the ANA
Avatar XPRIZE competition, there are only 4 teams that use
humanoid robots: Avatar-Hubo [1], Team SNU [2], iCub [3]and us.
III. AVATAR ROBOT
HRP-4CR is a 38-dof geminoid of 160 cm height, weighting 47 kg. It is the updated version of HRP-4C [4]. HRP-4C
had 6 dof arms, no force/torque sensors at the wrists and thehands were not dexterous. This is because it was originallydesigned for entertainment to imitate human-like motion.
As HRP-4C is already more than ten years old, some ofits components became deprecated and the wires became
unreliable. Therefore, to develop HRP-4CR we changed thewiring, the electronics, the computing system, the low-level
field-bus technology into EtherCAT and the cooling system.
Additionally, we improved the reachability of the humanoid
robot – by adding one dof at each arm (to have 7 dofarms), as well as the payload capability. We also mountedforce/torque sensors at the wrists, speakers, and a helmet
featuring a stereo camera (Zed Mini) and a microphone.
The head of HRP-4CR is mounted on a 3-dof neck,allowing for head motion close to that of a human. HRP-4C
originally had 8 dof to realize facial expressions; however, as
we mounted a small PC with GPU inside (Jetson NANO), it
took the place of 4 motor drivers. As a consequence, HRP4CR can only control 4 face dof: the eyelids, the horizontal
and vertical motion of both eyes, and the mouth aperture.
To realize dexterous grasping, our partner DOUBLE
R&D2 developed underactuated dexterous hands capable of2 https://j-d.co.jp

power and precision grasps. These hands feature a 1-dof
actuated flexion-extension of fingers and thumb, allowing thelatter to adapt to the shape of the grasped objects.
IV. O PERATOR INTERFACE SYSTEM
The human interface system consists of a PC running
Unity connecting an HTC Vive ProEye head-mount display(HMD) with facial tracker, two Valve Index controllers in thehands, and three Vive trackers: two on the forearms and one
on the torso (a setup similar to [2]). Each of these elementsallows to track in real-time the position and the orientation of
the following body parts respectively: head, wrists, elbows
and chest. The hands tracking allows an easy completionof object grasping tasks whereas the elbow tracking helps
to maintain a natural posture between the operator and therobot during teleoperation. The chest tracking is used as
the reference frame to guarantee posture integrity between
the robot and the operator if the latter moves during theteleoperation. The head tracking can be activated/deactivatedwith a button of the controller. The hand tracking can beactivated with variable stiffness depending on how hard theindex triggers are pressed. In that way, the operator can “rest”
and move freely without unintentionally operating the robot.
The system in Unity sends these postures as well as thecontroller commands to the robot operating system.
To transmit the facial expressions of the operator, we use
the eye-motion tracking system embedded into the HMD
and the facial tracker. As the eyes of the robot cannot
move independently (same pitch and yaw for both), the gaze
direction is calculated for a “virtual” eye placed between theoperator’s eyes, and the resulting angles are sent. As for themouth, the facial tracker detects the lip motion, from which
the aperture of the mouth is extracted and also sent.
The operator can also receive the stereo images from therobot’s stereo camera and hear the sounds from the robot’s
microphone by using the HMD and its speaker. The HMD
also integrates a Heads-Up-Display (HUD) that allows theoperator to visualize which end-effectors are currently being
tracked, as well as if the robot is walking or not. Additionally,
the HUD displays the weight of the objects as measured
by the wrist force/torque sensors, and the trackpad of theValve Index controllers transmits vibration whose amplitude
is proportional to that weight. See Figure 2.
The communication between Unity and the robot operating system is made using ROS# (provided by Siemens)
which can establish a bridge between ROS and the Unity
environment by using WebSocket communication.
V. ROBOT OPERATING SYSTEM
The robot is controlled using the open-source mc rtc
framework3 co-developed by CNRS and AIST [5]. Thecontroller consists of a task-space quadratic programming(QP) controller that computes joint commands that realize tothe robot capabilities a set of tasks under constraints (selfcollision avoidance, joint torque and kinematic limits, etc).
3 https://jrl-umi3218.github.io/mc_rtc

Fig. 2.

HUD showing tracked end-effectors and detected weight

The QP tasks and constraints are managed by a finite statemachine (FSM) that receives inputs from the operator side(the information sent by Unity) and accordingly executes
the appropriate states to achieve the desired behavior. For
example, when the operator moves the analog thumb joystick, the FSM switches to a walking mode that computes afeasible footstep plan and corresponding dynamic trajectoryof the robot’s CoM, and executes the walking (using thewalking controller of [6]). Multiple states can be executed inparallel, allowing complex behaviors. For instance, when theoperator enables the tracking of head and hands, end-effector
task targets are computed, therefore allowing the operator tocontrol them, even while walking.
Contrary to other solutions that use tablets to realize facial
expressions [7], we use the 4 dof of the face. The eyelids
are programmed to blink at random intervals automatically,
whereas the desired angles for the corresponding joints ofthe eyes and the mouth are received from Unity.
VI. S AFE INTERACTION
During the semifinals, the robot was required to interact
with objects on a table. In the lack of haptic feedback,an operator controlling the robot hand pose could easily
overforce on contacts, which could lead the robot to losingbalance. Supposing that the robot hand end-effector can only
hit the table from above, we derived an admittance control
scheme to constraint the effort of the robot hand end effector
in one direction such as: in the case of a potential collision
with the table, the admittance control will limit the effort
applied on the table whereas if the robot’s hand is lifting anobject, the effort on the hand would not be regulated.
VII. E XPERIMENTAL RESULTS
The accompanying video4 shows a compilation of shortclips that demonstrate the operation of our system, as described in this short paper. These clips correspond to thesemifinals video, as well as further demonstrations showing
some of the capabilities required for the finals.
VIII. F UTURE WORK
We are currently working on making the robot tetherless,by embedding appropriate batteries and improving our wireless communication. Additionally, we are integrating haptic
feedback and devising a suitable haptic interface for theoperator. Concurrently, we are improving the locomotion, theretargeting and the visual feedback.
4 https://www.youtube.com/watch?v=GnGmWgzANWUR EFERENCES[1] J.C. Vaz, A. Dave, N. Kassai, N. Kosanovic, and P.Y. Oh. Immersive
Auditory-Visual Real-Time Avatar System of ANA Avatar XPRIZE
Finalist Avatar-Hubo. In IEEE International Conference on Advanced
Robotics and its Social Impacts (ARSO), 2022.
[2] D. Lim, D. Kim, and J. Park. Online Telemanipulation Frameworkon Humanoid for both Manipulation and Imitation. In International
Conference on Ubiquitous Robots (UR), 2022.
[3] S. Dafarra, K. Darvish, R. Grieco, G. Milani, U. Pattacini, L. Rapetti,G. Romualdi, M. Salvi, A. Scalzo, I. Sorrentino, D. Tomè, S. Traversaro,E. Valli, P. Viceconte, G. Metta, M. Maggiali, and D. Pucci. iCub3
Avatar System, 2022.
[4] K. Kaneko, F. Kanehiro, M. Morisawa, K. Miura, S. Nakaoka, andS. Kajita. Cybernetic Human HRP-4C. In IEEE-RAS International
Conference on Humanoid Robots (Humanoids), 2009.
[5] K. Bouyarmane, K. Chappellet, J. Vaillant, and A. Kheddar. Quadratic
Programming for Multirobot and Task-Space Force Control. IEEE
Transactions on Robotics, 35(1):64–77, 2019.
[6] S. Caron, A. Kheddar, and O. Tempier. Stair Climbing Stabilizationof the HRP-4 Humanoid Robot using Whole-body Admittance Control.
In IEEE International Conference on Robotics and Automation (ICRA),
2019.
[7] M. Schwarz, C. Lenz, A. Rochow, M. Schreiber, and S. Behnke. NimbRo Avatar: Interactive Immersive Telepresence with Force-Feedback
Telemanipulation. In IEEE/RSJ International Conference on Intelligent
Robots and Systems (IROS), 2021.