CHARACTERIZING THE
DESIGN SPACE OF
RENDERED ROBOT FACESALISA KALEGINA, GRACE SCHROEDER, AIDAN ALLCHIN, KEARA BERLIN, MAYA CAKMAKHRI2018
工学院 応用物理学専攻
26193030 法花翼

INTRO• 人間にとって顔というものは重要。

• 最近は画面上に顔をレンダリングするロボットが普及。
（タブレットの汎用・低コスト・表現の柔軟性）• レンダリングされた顔をもつロボットの普及に対して、顔の目的別に適したバリエーションの研究は進んでいないロボットの顔に対して人々がどのように思うかを研究する

3つの実験
• ランダムに選んだ157のロボットの顔のパーツを76の項目について評価する実験• 12個のロボットサンプルからどんな印象を受け、どんな仕事に向いているかをアンケートで調査する実験• あるロボットに17種類の表情をさせて、それぞれからどんな印象を受け、どんな仕事に向いてるかをアンケートで調査する実験

実験１. 顔の寸法
• “robots with screens,” “robot
screen faces,” “touchscreen
robot,” “smartphone robot,”“telepresence robot.” で画像検索しサンプリング。
• データセット内のすべての顔は、76次元にわたってコーディング
される。

検索

実験１. 結果

• それぞれEVE・BAXTER SAWYER に似た特徴• ユニークな特徴

実験2. ロボット別表情、職種• 右の12サンプルを選択。
• 以下の6項目5ポイントの評価を依頼した。
• 機械的or人間的、友好的or非友好的、知的or
非知的、信頼できるor信頼できない、子供っ
ぽいor大人っぽい、男性的or女性的• 気に入ったかどうかを5段階評価、短い名前をつけよ、最適なジョブは？（選択性）• 参加者ごと顔の順番がランダム（公平性）

実験２．結果• 親しみやすさ、子供っぽさ→教育• 娯楽≒教育• 非友好的→警備実験３．表情×職種• 以下の6項目5ポイントの評価を依頼した。
• 機械的or人間的、友好的or非友好的、知的
or非知的、信頼できるor信頼できない、子
供っぽいor大人っぽい、男性的or女性的• 気に入ったかどうかを5段階評価、短い名
前をつけよ、最適なジョブは？（選択性）• 参加者ごと顔の順番がランダム（公平性）

実験3. 結果

DISCUSSION• 瞳孔・口がない→非友好的、機械的、好意的→警備• ピンク色または漫画風の頬→女風• 漫画風の頬、リアリティの低い→子供のよう・友好的→娯楽、教育• 笑顔の形の口→娯楽、教育

まとめ• 76項目の顔の特徴• ロボットの顔のセット• ロボットのここの顔の特徴の３つに関して、ロボットの顔の人々の印象を特徴付けることができた。

Publication
HCI lab Hayato Kurosawa

Paper• Title: Can a Humanoid Robot Engage in HeartwarmingInteraction Service at a Hotel?
• Authors: Junya Nakanishi, Itaru Kuramoto, Jun Baba, Ogawa
Kohei,Yuichiro Yoshikawa,Hiroshi IshiguroSummary• Today more and more robots are used in the industry. And
there is an open question about human-robot social
interaction on a heartwarming interaction service.

• The authors mention some research questions about
heartwarming interaction of robots and answer them throughpractical research.

Questions about heartwarming interactionof robots• Q1 Can interaction with a humanoid robot provide
heartwarming experience to a customer at a hotel?
• Q2 Can interaction with a humanoid robot enhance thecustomer’s satisfaction of the whole service at a hotel?
• Q3 Does an impression of the system differ between single
and two robots, male and female customer, or single andgroup customer?

Questions about heartwarming interactionof robots• Q4 Is an impression of the system affected by the number ofnights, frequency in use of a hotel, or the amount ofexperience in interaction with a humanoid robot or a voicecontrolled speaker?
• Q5 Does a customer follow the recommendation from ahumanoid robot engaging in a heartwarming interaction
service?

About practical research• The authors made prototype single/double robots system
which provide heartwarming service to customer at hotel.
• They gathered participants and executed questionnaire after
staying at hotel.
• Then they analyzed that questionnaire and answer above
questions.

About robot system• The system consists of humanoid robot and depth sensor.

• The depth sensor measure distance and moving of customer
and the robot react.

Sota
https://www.vstone.co.jp/products/sota/

Comm Uhttps://www.vstone.co.jp/produ
cts/commu/index.html

About questionnaire• Questions about robot system (“Should the robot be set at a
hotel?” or “Did you enjoy interaction with the robot?” etc.)• Questions about satisfaction of service (“Did feel comfort toyour stay?” or “Are you aroused to stay here again?” etc.)• Question about customer itself (“How often do you use ahotel?” or “How much have you interacted with a humanoid
robot?” etc.)

Answer of questions• Q1 Interaction with robot could provide heartwarming
experience to customer. However it couldn’t force customer
to feel strong necessity of robot system.
• Q2 Interaction with robot could enhance satisfaction forservice.
• Q3 The impression by robot system tend to depend on sex.
Female customer tend to feel comfortable than malecustomer.

Answer of question• Q4 The impression of robot system tend to be enhanced
depending on the how long or how much times customer
interact with robot (both positive effect and negative effect).

• Q5 Many customers thought to follow the information
received by robot.

◆
◆

◆
•
•

◆
•

◆
•

◆
•

◆
•
•

◆
•

◆
•

Calibrate My Smile:
Robot Learning Its Facial Expressions
Through Interactive Play with Humans
DINO ILIĆ, IVANA ŽUŽIĆ , DRAŽEN BRŠČIĆ
UNIVERSITY OF RIJEKA, RIJEKA, CROATIA
46193048
M U K A I N A K A N O RYO
1

Introduction
Robots have expressive faces⚫Static face

⚫Actuate parts⚫Display→ How to learn a emotional model2

Aisoy3Collection data
Variable⚫Eyebrows⚫Eyelids⚫Mouth curvature⚫Mouth openness4Expressive faces

Default

Learned model

Anger Disgust Fear Happiness
Sadness Surprise Indifference5Experimentimitation game⚫participants imitate robot⚫capturing participants’ facial features⚫monitor screen which the participants checktheir own facial expression

6

Learning model
Ten faces for each emotion

7

Result

Default

Learned model

8

Conclusion
Happiness and Sadness successfully recognize
Some emotions confused

Only 2 parameters
Different faces for each emotions
Learned model has a low margin

Skill of participants’ imitations
9

Analyzing Eye Movements in Interview Communication with
Virtual Reality Agents

Wang Han
46193130

Process ofreport·Introduction·VR Interview Virtual Agent System Architecture·Interview Evaluation·Results of Interview Evaluation·Conclusion

Introduction
In human-agent interactions, human emotions and gestures ex- pressed when interacting with agents is ahigh-level personally trait that quantifies human attitudes, intentions, motivations, and behaviors. The
virtual reality space provides a chance to interact with virtual agents in a more immersive way.
In the highly competitive globalized economy, it is becoming increasingly important to make accurate
assessments during job inter- views. It is still difficult to assess interview performance because of thecomplicated experimental settings required for the human interviewer

Introduction
This research develops a VR-based interview system using virtual agents and analyzes human eye
movements. This would provide insights into human gaze and explore their relationship with interview
performance. Empirically, eye contact reveals the effectiveness and delivery of communication and theemotional traits of the interviewee, which play an important role in job interview.
Firstly, a VR headset with an eye-tracking system called FOVE was used to render the interview scene.
Secondly, used the Unity 3D engine to build the interview scenario and demonstrated how our system
calculates the interviewee’s gazed targets by performing collision detection.

VR Interview Virtual Agent System Architecture

Figure 1: Flow chart of the system image

VR Interview Virtual Agent System Architecture
VR Device

With eye tracking.
Two cameras render images for the
person’s eyes.

Without eye tracking.
Customize the eye cameras and add infrared LEDs
to illuminate the eyes.

VR Interview Virtual Agent System Architecture
Virtual Environment by Unity

Figure 2: Scene setup in the Unity engine

Figure 3: Collision detection and gaze calculation

Figure 3 shows how this mechanism works. The positions of cameras in the VR scene represents
the positions of the eyes; the gaze directions were captured by FOVE. The two spheres in red andgreen indicate the left eye and the right eye respectively.

VR Interview Virtual Agent System Architecture
Virtual Environment

For example, if the head part of the agent was being looked at and the bodypart was not, the record would be [True, False] (“TF”).
TT，TF，FT，FF

Neck, Head, Body ,Somewhere else

VR Interview Virtual Agent System Architecture
Virtual Environment
While giving scores to each data clip, to the raters were asked questions to evaluate the interviewee’s
performance. These questions were divided into five categories: Total, Engagement, Eye Contact,
Friendliness, and Logical and Clear Presentation.
Total: An assessment of the overall interview performance: To what extent would you want tohire this person
Engagement: Did the interviewee show a positive attitude to- wards the question? Did he/shelook encouraged?
Eye Contact: Did the interviewee use proper eye contact to express himself/herself? Did he/shewatch the interviewer during interact without looking away?
Friendliness: Did the interviewee show a responsive attitude, and did you feel comfortable withthis interviewee?
Logical and Clear presentation: Did the interviewee effectively delivered his/her feelings?
Did the interviewee speak neither too fast nor too slow? Were you persuaded by his/her words?

Interview Evaluation
Result found that the participants gazed at the interviewer’s
head 16% of the total time. For another 51% of the total time,
the participants preferred to look at the interviewer’s body. No
targets were gazed at for 33% of the interview time.

Table 1: Features Computed from Raw Data

TT，TF，FT，FF

Neck, Head, Body ,Somewhere elseResults of Interview Evaluation
Inter-rater agreement

Predictions using gaze features

Table 2: Inter-rater agreement between raters

Table 3: Prediction accuracy and correlation using gazefeaturesResults of Interview Evaluation

Figure 5: Predictor importance for the field: “Eye Contact”

Figure 6: Estimated means of threesignifcant features

Therefore, we can conclude that if the participants did not gaze at the interviewer for a while, then theparticipants were not good at using eye contact communication, and their score of eye contact would
below.

Conclusion
This study developed a VR-based interview system capable of recording human eye movements.
Then used regression models to perform the automatic prediction of the interview performance.
Finally computed the importance of each of our features to determine the kinds of features that
affected interview scores the most. The experimental results suggest that this method is not only
conceptually easy to understand but also shows its consistency with real-world job offeringexperience. Also, the automatic evaluation experiments on evaluating interview performance showthat gazing at the interviewer and do not look down or look away is an important cue for improving
the interview outcomes.
problem
In real life, not only in the field of job interviews, eye contact plays a very important role in many aspects (suchas the field of public safety), how to use this technology in these fields to achieve the analysis of the relationshipbetween eye movement and the actual psychological state of the target is still in need of further research.

LEARNING FROM USERS:AN ELICITATION STUDY AND TAXONOMY FOR COMMUNICATINGSMALL UNMANNED AERIAL SYSTEM STATES THROUGH GESTURESJUSTIN W. FIRESTONE, RUBI QUINONES, BRITTANY A. DUNCAN
46193132
HE HANBACKGROUND
the general public will increasingly come across small Unmanned Aerial Systems (sUAS) in
everyday life.
It is important for sUAS to communicate common states quickly and intuitively withbystanders, because not all users are experts with sUAS.
• Not all sUAS will have hardware to communicate through sound or light due to cost or
battery limitations ,• Should be able to indicate key states through motions in space (gestures).

• A well-defined set of gestures can improve sUAS user experiences and ultimately increase
comfort with their greater prevalence in everyday life.

STUDY AND METHODOLOGY• Let participants construct a preliminary gesture set for sUAS states, all of which areimportant to communicate common conditions.
• Participants:• In this paper, the researcher asked users who recruited from the general public (N=20) withvarying levels of experience with sUAS to create their own gesture set for seven distinct
sUAS states.

• Experiment Materials• Ascending Technologies (AscTec) Hummingbird• A palm-sized model of the Hummingbird

STUDY AND METHODOLOGY• Experiment Procedure• The study took approximately one hour to complete three parts:• 1) Pre-interaction;• 2) Flight Path Design;

15 min• 3) Flight Path Observation.

• Each part included a survey• Concluded with an interview• Result:• Elicited 140 gestures

45 minSTUDY AND METHODOLOGY• Classification and Taxonomy for User-Designed Flight Paths• Create an objective classification and taxonomy• To group the elicited gestures according to specific common characteristics.

• Classified each gesture along six categories:• Categories from Related Work• Complexity• Space• Cyclicity• Additional Categories• Command• Altitude• Motion

STUDY AND METHODOLOGY• User-Defined Gestures for sUAS Communications
1.

Group those gestures with common features according to taxonomy
2.

Use groupings to calculate agreement scores.

Choose the most common gesture for each stateas the representative gesture for that state.
• An Agreement score is designed to determine the level of consensus.

STUDY AND METHODOLOGY• Inter-rater Reliability for Taxonomy• Purpose:• assess the usefulness of the taxonomy categories• classify the individual states according to common subcategories• Two raters were obtained to independently assign each of the 140 user-generated flightpaths to a single subcategory within each taxonomy category.

STUDY AND METHODOLOGY• After their independent assessments, their results were compared in order to calculate
Cohen’s Kappa coefficient and assess their agreement according to previous work.
• Result:• “Almost Perfect” agreement• Complexity (0.881), Motion (0.907), Command (0.92), and Altitude (0.914)• “Substantial Agreement.”• Space (0.79) and Cyclicity (0.641)

Cohen’s Kappa coefficient : https://en.wikipedia.org/wiki/Cohen%27s_kappa

RESULTS• The results indicate relatively strong agreement scores for three sUAS states:• Landing (0.455), Area of Interest (0.265), and Low Battery (0.245).

• The other four states have lower agreement scores, however even they show someconsensus for all seven states.

CONCLUSION• This paper presented an elicitation study to elicit gestures from participants recruitedfrom the general public to communicate seven key sUAS states to operators andespecially bystanders.
• The agreement scores showed promise that a common gesture set can be created andimplemented for current sUAS.