# AnyPlace: Automatic Gaze Alignment of the Teleported Avatar for MR Collaborative Environments
## 4 Implementation  
Figure 5 describes our automatic gaze alignment process of the teleported avatar in MR tele-conference systems. To adapt the gaze of the teleported avatar into the local site, first we installed the sensing device such as a depth camera (KINECT [23]), at each local site, and for the 3D MR display, each participant wore a see-through head-mounted display (MS HoloLens 2 [24]). At this time, with the offline process, the participant chose where the teleported avatar would be placed at each site. Then, the participant determined the preferred position where he/she wanted to sit for the MR tele-conference. The participant's  
Figure 4: The gaze alignment approach of the teleported avatar using a reference direction in the left image and an adaptive control vector in the right image. The reference direction vector (**left**) is defined by the position of the sensor, \(\mathrm{P_{s}}\), and the position of the participant in the MR tele-conference environment, \(\mathrm{P_{P}}\). Here, \(\mathrm{P_{P1}}\), \(\mathrm{P_{P2}}\), and \(\mathrm{P_{P3}}\) are the position of candidates where the participant can sit. The adaptive control vector (**right**) is defined by the position of the participant, \(\mathrm{P_{P}}\), and the position of the teleported avatar, \(\mathrm{P_{a}}\). Here, \(\mathrm{P_{a1}}\), \(\mathrm{P_{a2}}\), \(\mathrm{P_{a3}}\), and \(\mathrm{P_{a4}}\) are the position of candidates where the teleported avatar can sit.  
Figure 3: The system configuration of our adaptive control for gaze alignment of the teleported avatar. A participant is wearing an MR see-through headset, and a sensing device is used for detecting the gaze direction of the participant. At this time, the virtual teleported avatar is made to match the gaze of the participant depending on the avatar’s position, associated with available real objects on which the avatar can sit during the conversation.  
specific transformation information was extracted from the installed depth camera, and the participant's gaze information was estimated for the corresponding teleported avatar. In our case, we used a rigged character as the avatar, and allowed movement of the gaze.  
Our implementation ran on a 64-bit Windows 11 Pro with Unity3D (Unity Technologies [25]) applying the adaptive gaze control of the avatar. We used Unity3D C# scripting language for interactions to create the gaze control of the avatar with a Mixamo character [26]. Additionally, the motion datasets of Mixamo were configured so that the avatar acted similar to a real person. In order to provide a better tele-conference experience, the teleported avatar should express behavior that resembles that of the remote participant. Future work should aim to achieve a more accurate full-body motion expression of the teleported avatar; in this paper, we focused primarily on the gaze motion.  
Figure 6 shows examples of the results of automatic gaze alignment with adaptation depending on the position of the participant and the avatar. Here, the avatar's position was designated off-line manually, and the avatar's gaze adjusted with the participant's position was estimated from the depth camera. The participant may freely move and sit in a desired place. According to our implementation results, we can express natural looking situations of the avatar's gaze behavior to adapt the position of the participant. In the next section, we present experiments to validate our results in terms of gaze matching accuracy and co-presence with the teleported avatar.  
Figure 5: Automatic gaze alignment process of the teleported avatar.