DOI: 10.82270/robotrio/v1
title: RoboTrio
title: RoboTrio

description: Le corpus a été enregistré en 2018 (stage de M1 de Nathan Loudjani). Il est constitué de 23 enregistrements, où un robot téléopéré par la même personne (pilote) interagit à chaque fois avec 2 joueurs humains différents.

Ont été enregistrés : 3 canaux audio, 2 canaux vidéo + la vision stéréo subjective du robot 
(yeux et têtes mobiles), ainsi que tous les paramètres de mouvement (3 
degrés de liberté pour le cou) et d'articulation (6 degrés de liberté 
pour la mâchoire et les lèvres) du robot. Cela représente plus de 8h d'enregistrement multimodal. 

Les échanges verbaux de chaque paire de sujets ont été retranscrits en 2019 au LPL
description: 
The RoboTrio corpus:

These resources -- the RoboTrio corpus -- were recorded at the GIPSA laboratory during the 2018 Summer, in French.
They belong to the RoboTrio project (funded by a CNRS PEPS). The project
 involves GIPSA-Lab (Grenoble), LPL (Aix-en-Provence) and INT 
(Marseille). 

More details can be found here: http://www.gipsa-lab.grenoble-inp.fr/projet/RoboTrio/home.html

The corpus involves a collaborative game, played simultaneously by two humans. They sit in front of a social robot that plays as a game animator and referee. This robot is teleoperated by a human pilot:
 Gaze for the robot, eye vergence, head orientation, lip and jaw 
articulations, speech are captured on the human pilot in real-time and 
drive the robot. In this immersive teleoperation setup, the pilot
 sees through the robot cameras (stereo) and perceives audio through the
 ears of the robot, leading to a high level of embodiment. What is 
demonstrated by the pilot is a viable solution for the robot 
sensors/actuators to conduct a natural interaction with humans and 
successfully perform the intended task (social interaction with gaze and speech turnovers
 in a gaming scenario). Data streams and events that link to the 
perception as well as to the action are logged. These were primarily 
intended to build  autonomous behaviour models for a social robot (Nina robot, a modified iCub).

The recording conditions and the live corpus can be judged in photo or in video from these two files:
Example_Interaction-Photo.jpg
Example_interaction-From_expe_13.mp4

23 experiments were recorded (around 20 minutes each). The
 pilot is always the same, the human players are always different. The 
two players in an experiment are either both male or both female.

Content of the directories:

Example_Interaction-Photo.jpg:shows the setup, with the two 
humans facing the robot. One can also see the two cameras in the mobile 
eyes of the robot, and it's human-like ears (rich HRTF to support audio 
localization). In front of the robot, on the table, the support (Bamp;W
 square marker) allows for an augmented reality display for the pilot to
 conduct the game, announce the themes, and validate the player answers.
 One can also see 2 fixed cameras, directed toward one human player 
each. Not used by the robot, they are used afterward to better analyze 
the corpus (head tracking, human gaze, prephonatory gestures, face 
analysis).Example_Interaction-From_expe_13.mp4:this video excerpt 
illustrates the nature of the in-game interaction. The montage depicts 
what was recorded by the two fixed cameras (top line), what the pilot 
sees when he directs his head/gaze (bottom line : stereo view, virtual 
tablet) and what humans see (center: robot face with gaze and blinks, 
neck movements, and speech articulation).Audio_Humans:23 wav files, with stereo recording of the human players. 1 file per experimentTranscriptions_Humans:46 Praat files, one for every human player. 2 files per experimentAudio_Pilot:23 wav mono files. Recording used an ambient 
microphone near the robot, capturing mainly the pilot but also the robot
 motors, the robot power supplies, and the human players.Transcriptions_Pilot:2 ELAN files and 2 video files (mkv 
format), for experiments 19 and 23. These illustrate the full 
transcription of the corpus, including the pilot speech and his 
actions/keywords/references to users, direction, theme of the game, 
answers of the players.Please contact us if you need more of these data.

Contact address:

mailto:/frederic.elisei@gipsa-lab.grenoble-inp.fr