Subtle Reaction and Response Time Effects in HumanRobot Touch Interaction
Masahiro Shiomi1[0000-0003-4338-801X] Takashi Minato,1 and Hiroshi Ishiguro1,2
1ATR-IRC, Kyoto, Japan
2Osaka Univ., Toyonaka, Japanm-shiomi@atr.jp

Abstract: This paper reports subtle reactions and response time effects in human-robot touch interaction with an android named ERICA whose has a human-like appearance. People usually prefer a quick response from interaction
targets, regardless whether the targets are computer systems or robots. One famous guideline to design response timing is called the “two second rule,” whichargues that a system should not take more than two seconds to respond to input.
To investigate whether such a response time design is applicable to humanrobot touch interactions, we investigated several response times when a robot is
touched by people. We also implemented subtle reactions for a robot to being
touched and investigated whether they increased the robot’s human-likeness.
Our experimental results with 12 participants showed that the robot’s perceived
human-likeness increased because it showed a subtle reaction to being touched.
The results also showed that people would prefer quick responses to their touch
interactions. In this study, zero seconds showed significantly higher preference
than two seconds, but one second did not show significant differences with two
seconds.
Keywords: Human-Robot Interaction, Touch, Subtle reaction, Reaction Time1Introduction

In human-robot and human-computer interactions, people prefer a quick response
from interaction targets. Several studies reported that the preferred length of system
response times is less than a second [1, 2]. Another study described guidelines for theacceptable length of system response times, and the two second rule is one wellknown guideline for designing system response times [3]. It can also be applied tohuman-robot interactions [4].
However, these studies mainly focused on verbal interaction or such machine interfaces as a mouse and keyboard; in other words, they downplayed response times intouch interactions. Even though using such machine interfaces is a kind of touch interaction, directly touching a robot is quite different. A past study addressed appropriate response times in touch interaction with a robot, but it focused on the relationshipbetween mistrust feelings and non-reaction time lengths [5]. This is different from the2preferred response time to a touch. To design appropriate response time for touch
interaction, we are interested in whether the two second rule is applicable in humanrobot touch interactions, but it is not well investigated yet as described above.
We are also interested in a robot’s subtle reactions to being touched by a person,
since in human-human interaction, we often subtly react to being a touched before
response to the touched person. Since such human reactions would be unconscious,
we can obviously eliminate them from robot beings. However, since such reactions
might increase a robot’s human-likeness in touch interactions, they are also importantin the design of human-robot touch interactions. In fact, several studies reported thatthe subtle facial/gaze expressions of robots contributed to more natural behaviors orcreated positive impressions [6] [7] [8] [9], even if these works did not focus on human-robot touch interactions.
In this study, we investigated the subtle reactions and response time effects in human-robot touch interaction with an android named ERICA whose has a human-likeappearance. Our robot can react to being touched on its shoulder by a person using atouch sensor. In this study, we address the following research questions:- Is the two second rule applicable in human-robot touch interactions?
- Do subtle reactions to a touch increase human-likeness?

(a)

(b)

(c)

(d)

Fig. 1 ERICA reacts subtly to being touched by a participant and turns toward her

32

System overview

Figure 2 overviews our system and the experimental environment in which we installed the system, which consists of the following five components: a human tracker,
a touch sensor, a motion controller, an android (ERICA), and a social robot (Sota). In
a 5.1 x 2.2 m experimental room, ERICA sat on a chair, and Sota was placed on adesk (Fig. 1). ERICA turns its back to the entrance by facing to Sota, to express that
ERICA does not notice people who entered the room. The details of each component
are described as follows.
2.1

Human tracker

As a human tracker, we used a Kinect V2, which sends the position information of theparticipants when they enter the room to the motion controller to initiate conversation
behaviors between Sota and ERICA.
2.2

Touch sensor

We used a touch sensor, ShokacCube by Touchence (Fig. 3), which can measure theheight changes on the top surface of a soft material with 16 measurement points. Thissensor is 36 x 20 x 30 mm and sends information with 100 Hz at maximum. We installed it on ERICA’s left shoulder that is tapped by participants in our experiment.
When the sensor detects a certain amount of pressure, ERICA reacts to the participants who touched it.

HumantrackerTouchsensorE ERICA

ERICA
S

Behavior
controller

S SotaE

Sota
Fig. 2 System overview (left) and experimental environment (right)

Fig. 3 Touch sensor

Fig. 4 Sota

4

2.3

Behavior controller

The behavior controller has two functions: starting conversations between ERICA andSota and controlling the former’s reaction behavior to being touched. The first function controls the start timing of the conversational behaviors of Sota and ERICA;
when the behavior controller receives a signal from the human tracker about a participant who has entered the room, the behavior controller sends signals to Sota and
ERICA to start their conversations. Their speech contents and motions are predefined. In this study, only Sota talks and ERICA just listens.
The other function, which manages ERICA’s reaction behavior, controls both thesubtle reactions and the times of ERICA’s reaction behaviors, which are determinedby the experimental conditions.
2.4

ERICA

Figure 1 shows ERICA, an intelligent conversational android characterized by its
human-like appearance [10]. ERICA has 44 DOFs for its torso and face as well as
both a network connection function and a voice synthesis function. Its mouth is automatically controlled by its voice features. As described above, we installed a touch
sensor in its left shoulder. ERICA’s motions and timings are controlled by the motion
controller.
2.5

Sota

Figure 4 shows Sota, an interactive social robot characterized by its humanlike physical mannerisms. It has eight DOFs: three in its head, two for its shoulders, two for its
elbows, and one in its base. It is 28 cm tall with a network connection function and avoice synthesis function. The LED on its mouth blinks based on the sound level toindicate speaking. Sota’s motions and timings are also controlled by the motion controller.

3

Experiment

3.1

Hypothesis and predictions

Past studies reported that the two second rule is applicable not only to humancomputer interactions [1, 2] but also to human-robot interactions [4]. Even thoughthese studies focused less on touch interactions with a robot, we still believe thisknowledge is applicable even if the interaction modalities are different.
Moreover, a subtle reaction to being touched might increase a robot’s humanlikeness because people also often showed such reaction to being touched, even if
they are functionally unnecessary for robots. Based on these considerations, we madethe following predictions:5

Prediction 1: People’s preference ratings will not decrease monotonically with system response time until one second. The rating will level off when the system response time is two seconds.
Prediction 2: People’s human-likeness ratings will increase more when the robot uses
subtle reactions to being touched than when the robot did not.
3.2

Participants

Twelve Japanese people (6 women and 6 men, whose average ages were 36.00, S.D
11.12) were paid for their participation.
3.3

Conditions

The study had a within-participant design with the following two factors: response
times and subtle reactions. The order of the conditions was counterbalanced. Eachparticipant joined six sessions.
Response time factor: For this factor, we prepared three conditions: zero, one, andtwo seconds. This time period indicates the duration between being touched by a participant and responsive speech to that touch. Zero seconds indicate that ERICA immediately turned to the participant when the touch sensor detected a certain pressure.
Subtle reaction factor: For this factor, we prepared two conditions: with a subtle
reaction and without a subtle reaction. In the with a subtle reaction condition, ERICA
slightly looks up within 0.5 seconds when the participant touches its left shoulder andthen it turns to the participant. In the without a subtle reaction condition, ERICA does
not show any subtle behaviors; it just turns to the participants. Before doing so,
ERICA waits for a certain time based on the response time factor.
Note that in the with a subtle reaction condition and a response time of zero seconds,
ERICA immediately turns to the participant and talks after finishing the subtle behavior. If the response time is a second, ERICA waits 0.5 seconds after the subtle behavior, turns to the participant, and talks for the same duration using the without subtle
reaction condition. Thus, in both conditions, the duration is identical between being
touched by a participant and the responsive speech to the touch, except for the zerosecond condition.
3.4

Procedure

Before the first session, the participants were given a brief description of our experiment’s purpose and procedure. In this explanation, we showed our robots and explained their interaction with them and literally demonstrated how to touch ERICA’s
shoulder. The participants joined six sessions based on combinations of response time

6factors (three conditions) and subtle reaction factors (two conditions). After eachsession, they filled out questionnaires.
This research was approved by our institution’s ethics committee for studies involving human participants. Written, informed consent was obtained from all of them.
3.5

Measurement

We investigated whether the response times and the subtle reactions changed the impressions of ERICA’s human-likeness and the preference of the reaction timing. We
prepared two questionnaire items: ERICA’S human-likeness and the reaction timing
preference. The items were evaluated on a 1-to-7 point scale, where 7 is the mostpositive.

4

Results

Figure 5 show the questionnaire results about preferences. We conducted a two-way
repeated measures ANOVA for the response time and subtle reaction factors, and theresults showed significant differences in the response time factor (F(2,22)=7.435,
p=.003, η2 = 0.403). But we found no significant differences in the subtle reaction
factor (F(1, 11)=0.022, p=.886, η2 = 0.002) or in interaction effect (F(2,22)=1.485,
p=.248, η2 = 0.119). Multiple comparisons about the response time factor with the
Bonferroni method showed significant differences between zero and two seconds(p=.030). We found a significant trend between zero and one seconds (p=0.087) andno significant difference between one and two seconds (p=0.246). Thus, prediction 1
was partially supported; the rating did level off compared to just zero and two seconds.
Figure 6 shows the questionnaire results about human-likeness. We conducted a
two-way repeated measures ANOVA for the response time and subtle reaction factors, and the results showed significant differences in the subtle reaction factor (F(1,11)=7.541, p=.019, η2 = 0.407). But we found no significant differences in the response time factor (F(2, 22)=1.989, p=.161, η2 = 0.153) or in the interaction effect(F(2, 22)=1.321, p=.287, η2 = 0.107). Thus, prediction 2 was supported. The humanlikeness rating increased more when the robot used a subtle reaction to being touchedthan when it did not.

7

Without subtle expression

With subtle expression

*

7
6
5
4
3
2
1

+

0

1
Seconds2+: p<.10 * : p < .05

Fig. 5 Response time preferences

Without subtle expression
7
6
5
4
3
2
1

With subtle expression*

0

1
Seconds
Fig. 6 Perceived human-likeness2* : p < .05

85

Discussion

5.1

Design implications

Our experimental results revealed that the user preference ratings showed a similar
phenomenon with a machine interface without interacting with a robot, where thepeople’s preference rating decreased monotonically with the system response times,
although the comparison showed a significant trend between zero and one seconds.
From these results, we believe that the two second rules is applicable for human-robot
touch interactions even though it showed a different phenomenon from a past study
about human-robot interaction; responding as fast as possible to a touch is preferred.
Our experimental results also showed that subtle reactions by the robot to being
touched increased its human-likeness. In this study, showing the subtle reaction did
not decrease the preference rating even if it needed 0.5 seconds; using subtle reactions
will probably achieve more natural behaviors for social robots.
However, we must carefully design such subtle reactions based on the touch interaction contexts. For example, if a person directly approaches a robot from the front
before touching it, the person would probably assume that the robot has already noticed him/her. In such a context, subtle reactions to a touch might provide strange
impressions. In other words, the behavior design of subtle reactions that depend on
the touch context is important.
5.2

Subtle reactions in active touch interactions

In this study, we only focused on subtle reactions in situations where a robot is
touched by a person, i.e., passive touch situations. However, in human-robot touch
interactions, robots often actively touch people, i.e., active touch situations [11] [12][13] [14]. In such situations, what kinds of subtle reactions might contribute to human-likeness or more natural interactions?
One possible subtle reaction is a combination of other modalities, such as gaze behavior. Already few studies have focused on eye-contact behaviors and gaze cues in
active touch or handing situations [9] [15], and reported both positive/negative effectsof eye-contact before touching/handing behaviors. Still it is unclear what kinds ofsubtle gaze expressions would be better for touching behaviors yet. Therefore, investigations of subtle gaze expressions in active touch situation would be one interesting
future work.
Another possible expression is verbal cues. Similar to our study, if a person did not
notice the robot’s existence, an unobserved touch was negative. Before touching, if
the robot uses verbal ques to be noticed by the person, its active touch would not be
negatively perceived. For example, a past research work investigated the effects ofverbal ques before being touched by a robot [12]; even if it is not subtle, verbal information changes the impressions of touch interactions.

9

5.3

Limitations

Since our experiment was conducted with our developed robots and specific situations, where participants interrupt the talking of robots by touching, we cannot ensurethat our findings can be applied to all human-robot touch interaction situations. To
generalize the effects of subtle reactions, we must investigate them with different
situations, such as where a robot has already noticed the participants before interrupting. Moreover, this study only used an android with a female appearance/voice. Since
gender differences are one essential factor to change the impressions of touch interactions [16] [17], using a different android with a male appearance/voice is critical forinvestigating the effects of subtle reactions on being touched. However, we believethat our setting offers essential knowledge for researchers who are interested in human-robot touch interactions.

6

Conclusion

We focused on the effects of subtle reactions and reaction times in a robot’s reaction
behaviors to being touched by a person. We placed a touch sensor in an android’s leftshoulder to detect being touched in a situation where the android and a desktop-sized
robot are chatting. We experimentally compared the impressions of participants to theandroid with/without subtle reactions and different response times to being touchedby participants. They felt more human-likeness when the android showed subtle reactions and preferred quick response times.

Acknowledgements
This research work was supported by JST ERATO Ishiguro Symbiotic Human Robot
Interaction Project (Grant Number: JPMJER1401), and JSPS KAKENHI Grant Number JP15H05322 and JP16K12505.

References
1. Goodman, T. and Spence, R., “The effect of system response time on interactive computer
aided problem solving,” in ACM SIGGRAPH Computer Graphics, pp. 100-104, 1978.
2. Guynes, J. L., “Impact of system response time on state anxiety,” Communications of theACM, vol. 31, no. 3, pp. 342-347 (1988).
3. Miller, R. B., “Response time in man-computer conversational transactions,” inProceedings of the December 9-11, 1968, fall joint computer conference, part I, pp. 267277, 1968.

10
4. Shiwa, T., Kanda, T., Imai, M., Ishiguro, H., and Hagita, N., “How Quickly Should a
Communication Robot Respond? Delaying Strategies and Habituation Effects,”
International Journal of Social Robotics, vol. 1, no. 2, pp. 141-155 (2009).
5. Wada, K. and Asano, Y., “Investigation of appropriate response time for an animal typerobot,” in Robot and Human Interactive Communication (RO-MAN), 2016 25th IEEE
International Symposium on, pp. 331-334, 2016.
6. Funakoshi, K., Kobayashi, K., Nakano, M., Yamada, S., Kitamura, Y., and Tsujino, H.,“Smoothing human-robot speech interactions by using a blinking-light as subtle
expression,” in Proceedings of the 10th international conference on Multimodal interfaces,
pp. 293-296, 2008.
7. Mathur, M. B. and Reichling, D. B., “Navigating a social world with robot partners: A
quantitative cartography of the Uncanny Valley,” Cognition, vol. 146, pp. 22-32 (2016).
8. Shiomi, M., Nakagawa, K., and Hagita, N., “Design of a gaze behavior at a small mistake
moment for a robot,” Interaction Studies, vol. 14, no. 3, pp. 317-328 (2013).
9. Gharbi, M., Paubel, P. V., Clodic, A., Carreras, O., Alami, R., and Cellier, J. M., “Toward
a better understanding of the communication cues involved in a human-robot object
transfer,” in Robot and Human Interactive Communication (RO-MAN), 2015 24th IEEE
International Symposium on, pp. 319-324, 2015.
10. Glas, D. F., Minato, T., Ishi, C. T., Kawahara, T., and Ishiguro, H., “Erica: The erato
intelligent conversational android,” in Robot and Human Interactive Communication (ROMAN), 2016 25th IEEE International Symposium on, pp. 22-29, 2016.
11. van Erp, J. B. F., and Toet, A., “Social Touch in Human–Computer Interaction,” Frontiersin Digital Humanities, vol. 2, no. 2 (2015).
12. Chen, T. L., King, C.-H. A., Thomaz, A. L., and Kemp, C. C., “An Investigation of
Responses to Robot-Initiated Touch in a Nursing Context,” International Journal of Social
Robotics, vol. 6, no. 1, pp. 141-161 (2013).
13. Shiomi, M., Nakagawa, K., Shinozawa, K., Matsumura, R., Ishiguro, H., and Hagita, N.,“Does A Robot’s Touch Encourage Human Effort?,” International Journal of Social
Robotics, pp. 1-11 (2016).
14. Shiomi, M., Nakata, A., Kanbara, M., and Hagita, N., “A Hug from a Robot Encourages
Prosocial Behavior,” in Robot and Human Interactive Communication (RO-MAN), 2017
26th IEEE International Symposium on, pp. to appear, 2017.
15. Hirano, T., Shiomi, M., Iio, T., Kimoto, M., Nagashio, T., Tanev, I., Shimohara, K., andHagita, N., “Communication Cues in a Human-Robot Touch Interaction,” in Proceedingsof the Fourth International Conference on Human Agent Interaction, Biopolis, Singapore,pp. 201-206. (2016).
16. Stier, D. S. and Hall, J. A., “Gender differences in touch: An empirical and theoreticalreview,” Journal of personality and social psychology, vol. 47, no. 2, pp. 440 (1984).
17. Ebesu Hubbard, A. S., Tsuji, A. A., Williams, C., and Seatriz, V., “Effects of Touch on
Gratuities Received in Same‐Gender and Cross‐Gender Dyads,” Journal of Applied Social
Psychology, vol. 33, no. 11, pp. 2427-2438 (2003).