Emotional Gripping
Expression of a Robotic
Hand as Physical Contact
46193148
趙 哲宇

⚫ Abstract⚫ System Design

Contents⚫ Discussion⚫ Conclusion

Abstract
This research aims at the emotional expression of a robotic handthrough various gripping manners on the user’s hand. The proposedsystem is implemented with a robotic hand’s various haptic actuators
to realize the change of the fingers’ gripping force and the robotic
hand’s holding duration so that the user can haptically estimate theemotion of the robot. The system is expected to provide stress relief
or emotional stability, especially for elderly or challenged people.

⚫ Abstract⚫ System Design

Contents⚫ Discussion⚫ Conclusion

System Overview
The system consists of a PC, a robotic hand, an AVR
controller, and a servomotor. The gripping strengthof the robotic hand was simply changed by aservomotor which is controlled by the PC via the
AVR controller. The timings of the gripping /release action were decided by the hand-holding
duration. Currently, the two parameters of thegripping manner were directly controlled by thedesigned patterns. To automatically control theexpressions corresponding to the robot’s internal
state and the user’s demand, They verified therelationship between the gripping manner and theemotional expression of the robot in this paper.

System Evaluations
Factor A: The strength with which the robotic hand grips the user’s hand(A):Weak(40degrees)ordinary(60degrees)

strong (80 degrees)

Factor B: The duration for which the robotic hand grips the user’s hand(B):Short(0.8seconds)

normal(2.5seconds)

long (4.5 seconds).

ProcedureProcedure
The dialog between the bear and A-chan wasstarted after the participant held A-chan’s hand.
The content of the dialog simulated a scare scene
for the robotic hand to express strong emotion as
follows.
Bear: “Hey, hey, hey, can you see IT?”A-chan: “What do you mean IT?”
Bear: “Behind you....There is an ogre.”

After the dialogue, A-chan gripped the participant’s
hand based on the experimental conditions andreleases the participant’s hand after the decided
period for each condition.

Evaluation Statements
The participant evaluated the adjective pairs described in Table1 in afive-point-scale rating using the SD method as impression valuations
for factor analysis.
Table1 shows the commonality and factor loads of each item after a
Varimax rotation and give the explanation rate of variance for eachfactor. Each factor was interpreted based on the whose absolute valueof factor loading was 0.50 or more.
First, factor 1 was made to be hypersensitive based on “sensitive,”“fast,” “clear” and so on. Factor2 was based on affinity: “human,”“natural,” “friendly” and “accessible.” Factor3 was made to be
comfortable and was based on“cheerful,” “pleasant,” “warm” and soon. Factor4 was made to be quiet:“calm” and “equable.” Factor 5 wasmade to be complex :“complex.”

To compare the impression of each condition when
encountering the tactile sense of grasping the robotic hand, thestandard factor scores were calculated as an impression
evaluation of gripping expressiveness.
Figure4 shows the averages and standard deviations of thestandard factor scores by each conditions.
Here Table2 shows the result of analysis of variance(ANOVA)
based on the standard factor scores.

⚫ Abstract⚫ System Design

Contents⚫ Discussion⚫ Conclusion

Conducted multiple comparisons ofmain effect among three levels of thefactor A(Figure5). There were significant
differences between the Strong leveland other levels while the scores were
gradually increased corresponding to
the gripping force.

Discussion

As shown in Figure6, there weresignificant differences between the
Weak level and other levels while
the scores were gradually increased
corresponding to the gripping force.

Figure 7 also showed a significant
difference between the Short andLong levels, while the averagescore of the Normal level was
about an intermediat evalue of the
Short and Long levels.

Further more, there were several significant differences by the gripping manners (strength andduration). The standard factor scores for the five extracted factors as impression of the robotic
hand were processed by the two-factor ANOVA and the result showed significant differences of
the hypersensitivity and affinity; the difference in gripping strength seemed to affecthypersensitivity and affinity and the difference in holding duration seemed to affect affinity.
It is conjectured that the score of
hypersensitivity elevated by the stronger
grip.
It is also presumed that the strength of
the Strong conditions were perceived as
human-like or natural grip. They should
continue their verification on
naturalness of the gripping manner to bepositively accepted.

In regard to the gripping duration of therobotic hand, it was shown that thelonger gripping duration made the users
feel higher affinity.
The set of the holding duration in theexperimental configuration had alimitation, so we should verify the effectof holding duration with a wider range ofthe levels in the experiment setting.

Discussion

From the ANOVA for other threefactors of the impression, there wasno significant result.
These factors are still expected to berelated to the elements of thegripping manner except the strengthand duration, such as the gripping
position and direction on the user’s
hand.

⚫ Abstract⚫ System Design

Contents⚫ Discussion⚫ Conclusion

Conclusion
In this study, it proposed a robotic hand to provide tactile interaction with users who have physicalor psychological difficulty in daily life such as bedridden patients for lessening their loneliness andstabilizing their minds.
In this paper, they especially focused on the gripping manner of the robotic hand holding on theuser’s hand as a physical contact. The effect of the expression of the robotic hand of the gripping
manner based on based on the holding duration and gripping strength on the user’s impression
was examined. As a result, five factors (hypersensitivity, affinity, comfortable, quiet, and complex)
were extracted from the results of the factor analysis. In addition, the ANOVA results of thestandard factor scores of the five factors showed that the hypersensitive and affinity increase as
the gripping power strengthens, and that the longer holding duration increases affinity.
In the future, they consider that it is necessary to design the movement of the robotic handcombined to the physiological phenomenon on the skin to realize more realistic physical contact.

Thank you!
46193148
趙 哲宇

Ryuji Yamazaki, Hiroko Kase, Shuichi Nishio, Hiroshi Ishiguro

Hokkaido University
Intelligent Robot System Laboratory
Katsumasa Segawa.01
/*$,4&'32"(+



!%)

#

⇨ 

1





(*







"#0-.2/'…
$%





1,

 !"#0- )+&
2


1

2

c
)
)

01

3

5



4

4



4
,

2

1

4

4

4



4



2

5

4

0

4

4

0

3
5

.0
.0


 ',
3# & ' 
%+* -)!"( ($  ' !

5" ≤ 0.05

6

NPI-NHMean±SD



-

7

H
1I

H

5

0

-

0

1

8

I

H

H

-

10

9

-

N

P

0

0

I

1

H
10

“Compact Real-time avoidance ona Humanoid Robot for
Human-robot Interaction”
D. Nguyen, M. Hoffmann, A. Roncone, U. Pattacini, G. Metta,
HRI 2018

システム情報科学コース 知能ロボットシステム研究室

修士1年 鶴園 卓也 (46193192)
0/17

背景

1

既存のロボットは工場などで決められたタスクを処理
今後
図 産業用ロボット[1]

未知の環境でより自律的に動作
人間と空間を共有する
人間との衝突を避け安全な動作が求められる[1]産業用ロボットとは, https://www.sk-solution.co.jp/robotics/industrial_robot/[2] ASIMOの歴史, https://www.honda.co.jp/ASIMO/history/asimo/index.html

図 二足歩行ロボット[2]目的2

人間とロボットの物理的な相互作用
pHRI(physical human-robot interaction )を安全にする

実現するフレームワークの提案
周囲の人間の動きを把握し• 目標動作を達成すること（グローバルな目標）• 反応性障害物回避（ローカルな目標）
ヒューマノイドロボットiCubを対象にシステムを開発

図 iCub[1][1] iCub, https://en.wikipedia.org/wiki/ICub

オープンソースロボット
高さ 1[m], 重要 22 [kg]
3自由度の頭部
7自由度の腕 (接触センサ)
ステレオカメラ(頭部)

提案手法

3

iCub頭部に搭載されたステレオカメラ①③②④

図 システムの概要①人間の姿勢推定
左カメラを用いて人間の姿勢推定③身体近傍空間PPSによる衝突判定
ロボットと人間の接触危険性を判定②姿勢情報の3次元変換
左右のカメラを用いて深度を計測④制御
トリガが発生した場合，回避動作

姿勢推定

4

DeeperCut [1]画像から複数人の姿勢を同時に推定

図 動作の概要① カラー画像の入力② CNN型のモデルを用いて，キーポイント(肘や顔などの人物の部位) を抽出③ キーポイント同士の全ての繋がりを組み合わせ④ 人物の組み合わせを抽出⑤ キーポイントの代表部分を出力

キーポイントをロボットが回避する障害物として設定[1] EldarInsafutdinov,LeonidPishchulin,BjoernAndres,MykhayloAndriluka,and Bernt Schiele. 2016. Deepercut: A deeper,
stronger, and faster multi-person pose estimation model. In European Conference on Computer Vision. Springer, 34–50.

姿勢情報の3次元変換

5

生体力学的制約を適用

図 DeeperCutによる姿勢推定

近傍7×7ピクセルの3D位置を平均
図 ステレオ視による深度画像

図 推定された3次元姿勢

身体近傍空間 (peripersonal space : PPS) による障害物回避6

身体近傍空間PPS[1]による障害物回避
生物学を基に提案された手法．受容野RF (receptive field) に存在する障害物に対して衝突するかどうか，どれだけ危険かどうかを視覚的に把握する
手のひら : 5個のRF
腕: 24個のRF- 50%の応答減衰 (例 : 手)
- 100%の応答減衰 (例 : 頭)- Parzenウィンドウ補間手法← 回避動作のトリガー
図 RFの概要

図 距離と活性化の関係[1]A. Roncone, M. Hoffmann, U. Pattacini, and G. Metta. 2015. Learning periper- sonal space representation through artificial skin for avoidance
and reaching with whole body surface. In Intelligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on. 3366–3373.

実験1

7

静止状態での回避動作

動画 静止状態での回避動作

実験2

8

円軌道を描いた状態での回避動作

動画 円軌道を描いた状態での回避動作

実験3

9

頭部の回避動作

動画 頭部の回避動作

まとめ• 人間の行動を把握し，衝突を回避するフレームワークの開発を行った．
• 実験により，動作途中に人間の介入があった場合でも安全に回避することができた．

課題
1. 事前衝突のみを考慮している
衝突した場合の動作を実装していないため，接触センサなどを用いて安全に停止するなどの工夫が必要

2. 速度を利用していない
提案手法は、静的な位置状態で回避動作
対象物やロボットアームの速度を考慮することで，より柔軟な回避行動を実現できる

10Effects of Capability and Context on
Indirect Speech Act Use in TaskBased Human-Robot Dialogue
Kazuma TateiriIntroduction• Humans often use “indirect speech acts” (ISAs) to other humans.
• For example, “Could you open the door?” is a ISA.
• This sentence is literally questionnaire.
• But the actual intention of this speech is request to open the door.
• ISAs are used to achieve socio-cultural goals (e.g. politeness).
• The use of ISAs differs individually and cross-culturally.
• But their use is generally accepted feature of natural human dialogue.

Motivations of this research• The authors suspect that not handling ISAs might be one of thelargest stumbling blocks preventing successful natural language-based
human-robot interaction outside of the laboratory.
• Only recently have word error rates on speech recognition fallen into
the single digits (6.9%), and yet this rate is considered to be too high.
• But if ISA use rates is considerably higher than 6.9% in human-robot
dialogues, it deserves more attention from the research community.
• It is important to investigate the extent to which ISAs will be used.

The seven hypotheses they created:
1. ISAs are central to task-based human-robot dialogue regardless oftask context. ISAs will be used with sufficient frequency that nothandling them would yield an unacceptably high utterance error
rate greater than or equal to the current word error rate of 6.9%.
2. This high frequency of ISA use will occur in both conventionalized
and unconventionalized task contexts.
3. Human social conventions will carry over into human-robot
interactions.
4. ISAs will be more often used in conventionalized scenarios.

The seven hypotheses they created: (2)5. Even if a robot demonstrates itself to be fundamentally incapable ofunderstanding ISAs, humans will prefer to continue using ISAs
rather than direct commands.
6. If the hypothesis 1 holds, a human interacting with a robot unable
to understand ISAs should be less efficient in accomplishing a task
than a human interacting with a robot able to understand ISAs.
7. If the hypothesis 1 holds, a robot unable to understand ISAs shouldbe perceived less favorably than a robot able to understand ISAs.

Methodology• They conducted a Wizard-of-Oz between-subjects experiment.
• It employed two scenarios: restaurant and demolition.
• In the restaurant scenario, the participants were provided with a listof three “courses” which they could request to be delivered.
• In the demolition (解体) scenario, the participants were provided witha list of three towers which they could request to be knocked down.
• Each scenario had two conditions: understood and misunderstood.
• Each participant was randomly assigned to one of four conditions:• Such as (restaurant, understood), (demolition, misunderstood), etc.

Room setups• In the restaurant scenario, the room was
empty.

• In the demolition scenario, the room contained
three colored towers of aluminum cans, as
shown in figure on the left:

Behavioral results• The majority of participants (69%) used at least one
ISA.

• 46% of task-relevant utterances were coded as ISAs.
• ISAs were much more frequently used in therestaurant condition (0.75±0.39) than in the
demolition condition (0.16±0.34). (left figure)• ISA use rates was far above their threshold (6.9%)
in both the understood (1.0±0.49) andmisunderstood (0.4±0.27) conditions.

Implications• The first hypothesis (H1) was that ISAs would be consistently used,even after repeated demonstration of an inability to understand them.
As seen in the results section, ISAs were used by the majority ofparticipants and constituted the majority of task-relevant utterances.
• The second hypothesis (H2) was that this high frequency of ISA use
would occur across both conventionalized and unconventionalizedtask contexts. While ISAs were observed in both conditions, ISAs wereused far less frequently in our unconventionalized task context, at a
rate which did not clearly support this hypothesis.

Implications (3)• The results suggest a significant need for robots engaging in taskbased human-robot dialogue interactions to be able to understandISAs.
• Specifically, the results suggest that failing to understand ISAs could
result in an expected utterance error rate as high as 46% (the meanfrequency of ISAs among task relevant utterances) – a number that isclearly unacceptably high for task-based interactions.

Understanding Affective
Experiences With
Conversational AgentsXi Yang, Marco Aurisicchio, Weston Baxter
Imperial College London
MINJIE
79183054

Background• There are a lot of researches about technologies ofConversational Agents.
• voice interface and the dialogue system• human-like interaction• intelligence• Understanding Affective Experiences for better interaction
experience.

Implementation• Survey research about experience on Google Assistant• Critical incident method, which requires users to report anexperience that they have had for reliability of survey.

• mathematical analysis• PCA• Correlation analysis

Experiment• Random context and scenarios for better reliability of survey on
171 participantsAffective Responses in the Four Scenarios• They found that users’ overall experience was positive with interest
being the most salient positive emotion. And affective responses
differed depending on the scenarios.

interpretation of correlation• In positive affect, the hedonic quality was higher than that of thepragmatic quality.
• Pragmatic quality was found to significantly influence negative affect.

Summary• Contribution: help designers better understand users’
expectations across different scenarios and contexts, andtherefore design for a positive user experience.
• Limitation: recalled memories may be inconsistent withinteractions observed in process.

Second Language Tutoring using
Social Robots: A Large-Scale
Study

Paul Vogt (Tilburg University), Rianne van den Berghe (Utrecht University),
Mirjam de Haas (Tilburg University), Laura Hoﬀmann (Bielefeld University),
Junko Kanero, Ezgi Mamus (Koç University), Jean-Marc Montanier (SoftBank Robotics Europe),
Cansu Oranç (Koç University), Ora Oudgenoeg-Paz (Utrecht University),outline• course of 7 lessons designed to help children learn
English as a foreign language using a social robot• multi-staged experiment conducted to measure theeﬀectiveness of a social robot in teaching children:➡ comparing the eﬀect of learning from a robot tutor

accompanied by a tablet vs learning from a tablet
application alone!2

experiment; overview• pre-test (checking if target vocabulary of 34 words isalready known)• 7 lessons series (with 3 diﬀernt settings)• post-tests (immediate and delayed)!3

participants• 194 children (5-6 years old, Dutch native speakers)•!4

target vocabulary!5lessons; environmental setup!6lessons; plan
1) introduction where the robot would greet the child by name, andpresent the new virtual environment (e.g. forest) that set thecontext of the lesson2) words presentation and teaching/learning:- robot's narration (e.g. "Look, elephants!")- robot's verbal feedback (e.g. "Good job!", "Nice try, but you need
to touch the monkey in the cage, try again!")- robot's gestural feedback (in 1 out of 4 conditions)- tasks for children within tablet-based game (e.g. release themonkey from the cage)3) short test in which knowledge of each target word was tested
twice in a random order (no feedback from robot during this stage)!7

gestures; examples!8

gestures; examples#2!9

conditions
1) robot with iconic gestures + tablet
2) robot without iconic gestures + tablet
3) tablet-only without the robot4) control condition where children danced with the robot
but were not exposed to the educational material!10

post-experimential tests• immediate post-test (max. 2 days after the final lesson)• delayed post-test, (between 2 and 4 weeks after the final
lesson; M = 2 weeks 5 days, SD = 2.70 days)

1) translation from English to Dutch2) translation from Dutch to English
3) comprehension test of English target words!11

results!12

conclusions• children in the experimental conditions scored higher than children
in the control condition on all tasks• no significant diﬀerences between groups with diﬀerent conditions:➡ children learn equally well from the robot and the tablet as fromjust the tablet➡ children learn equally well from a robot producing iconic gesturesand from one that does not produce such gestures• scores of the delayed post-test were significantly higher than thoseof the immediate post-test!13

considerations & future work
• tablet's presence in conditions #1 & #2 may have limited the importance of
the interaction between child and robot• in condition #3 children could focus their attention solely on the tablet

game; in 1# & #2 attention had to be divided between the two devices(robot & tablet)➡ future trial without tablet• gestures might have been ambiguous➡ gestures redesign• learning sessions with robots might have been too long➡ getting rid of potentially redundant comments!14YouTube video-presentation

https://www.youtube.com/watch?v=IS8CbzJZX4k!15

thank you
for your attention.

What is Human-like?:
Decomposing Robotsʼ
Human-like Appearance
Using the AnthropomorphicroBOT (ABOT) Database
情報科学院メディアネットワークコース
M1 川幡知孝
http://www.abotdatabase.info/collection

Background• The appearance of a robot can have a significant impact on
people's perception of intelligence, sociability, favorability,
reliability, and compliance.
• Researchers warn about certain risks related to the humanlike appearance of robots.
• It is necessary to deepen the systematic understanding ofrobots that look like humans.

Figure 1: Robots characterized
as “humanoid” in (a) Stenzel etal. , (b) Wiese et al. , and (c)Meltzoff et al. .
Robots that share the samelabel across different
studies may actually differ
dramatically in their degreeof human-likeness.

ABOT(Anthropomorphic roBOT )• The largest repository of robots with human-like features todate.
• Identify distinct dimensions of robot appearance.
• Report two empirical studies• Offer the Human-Likeness Estimator̶a web-based linear
equation.
269 images
gathering that variedin both number andtype of human-likeappearance features

200 images

200 images

Carefulreview ofimagesImage Selection
and Editing

Study1 Investigate the appearance of the robot
via Amazonʼs Mechanical Turk crowdsourcing website (mTurk).

採⽤⼈数

1132⼈
(man:501,women:6
19 不明:12)

年齢18歳から81歳（M =36.07、SD = 11.68、4⼈未報告)

報酬$0.5

Yes or NO ?
66 images• Definition for each feature

N = 1,140 (15 raters x 19 features x 4 blocks of robots).

Study1 Result• The PCA in Study 1 yielded four
appearance dimensions (i.e.,feature bundles) .
• The “subscale scores,” (the boldfaced items in Table 2).
• (1) Surface Look,(2) Body-Manipulators,(3) Facial Features,(4) Mechanical Locomotion.
Together, these four dimensions
accounted for three-fourths of thetotal variance among the 18
individual features.

Study2 PREDICTING PHYSICAL HUMAN-LIKENESS• Identified which appearance dimensions best characterize
general human-likeness impressions.
participants

100(males:48,females:
50, lost:2)

ages ranging

19 to 64 （M = 33.42,
SD = 9.75 )

報酬

$1.00

66 images

25 judges to each robot in each block and thus predetermineda total sample of N = 100.

Study2 Result
1. These average human-likeness scores
across all the robots in our databaseranged from 1.44 to 96.46, with M =33.26, SD = 18.97.
2. A regression model using all 18
features explained 88.8% of the total
variance of overall human-like scores(R = .94, F (18, 179) = 78.5, p <.001 ).

torso

r_(semi−partial) = .31

genderedne r_sp = .44
ssskinr_sp = .23

These three 78.4% of thetotal r_sp
2. A stepwise forward
regression analysis with the
18 features as predictors.

Study2 Result
3 Predicting physical human-likeness from appearance dimensions.
Four regression-based principal component scores aspredictor variables.

This model explained 82.5% of the variance in humanlikeness, F (4, 193) = 227.0, p < .001.

Surface Look
(37.2%)

r_sp = .61

Body-Manipulators(36.0%)

r_sp = .60

Facial Features(5.7%)

r_sp = .24

MechanicalLocomotion(3.6%)

r_sp = -.19

four subscale scores as predictors. This model explained 81.5%of the variance in human-likeness, F (4, 193) = 212.4, p < .001.
BodyManipulators(28%)

r_sp = .53 p < .001

Surface Look(19%)

r_sp = .44 p < .001

MechanicalLocomotion(1.7%)

r_sp = -.13 p < .001

Facial Features(0.5%)

r_sp = .07, p = .025

Conclusion & Future work
• A more systematic, generalizable and reproducible study on
the anthropomorphic appearance of robots and their impact
on human-robot interaction.
• Robot integration into the ABOT database is constrained byboth knowledge of existing robots and search procedures.
• The number and type of features need to be refined.
• How the robot's static appearance features interact withdynamic properties.