Proceedings of the 17th IEEE International Symposium on Robot and Human Interactive Communication, Technische
Universität München, Munich, Germany, August 1-3, 2008

Smoothing Human-Robot Speech Interaction
with Blinking-Light Expressions
Kazuki Kobayashi, Kotaro Funakoshi, Seiji Yamada,
Mikio Nakano, Yasuhiko Kitamura, and Hiroshi Tsujino
is technically difficult and uneconomical. Another approach
uses speech such as interjections (such as “well” and “uh”).
Unfortunately, it is not easy to use interjections appropriately,
either [4], [5]. Once the system fails to use them appropriately, users prefer systems not using them to the system using
them [6].
This research verifies to what extent a method using
blinking-lights can ameliorate this situation. In contrast to
the above approaches that try to make machines resemble
humans, there are other approaches that utilize expressions
characteristic of artifacts [7]. Following these approaches, we
believe there is a way that we can avoid the difficulties inusing human-like expressions but still gain the desired effectfrom a simple and economical implementation.
In this study, we devised a robot that expresses its internal
state by using a blinking LED and conducted experiments inwhich users and the robot engaged in last-and-first games(shiritori) through speech. The smoothness of the resulting
dialogues was analyzed from the number of user repetitionsof last utterances and user feedback. In concrete terms, weshow that, by blinking an LED from the end of the user’s
speech until the robot’s speech, the number of repetitions
decreases and participants’ familiarity and attentiveness withthe communication increase. The experimental results suggest that the blinking-light can prevent utterance collisions
between a user and a robot.

Abstract— We propose a method to enable smooth speechinteractions between a user and a robot. Our method is basedon subtle expression whereby a robot blinks a small LED
attached to its chest. We performed experiments in which
participants played a last-and-first games and counted thenumber of repetitions made by the participants and analyzed
their impression of the game and the robot. The experimental
results suggested that the blinking-light could prevent utterance
collisions between a user and a robot and could create familiar
and attentive impressions about the game on users.

I. INTRODUCTION
Failures of turn-taking, especially speech overlaps between
systems and users, harm smooth communication and degrade
the usability of systems. When speech overlaps occur, users
tend to interrupt their own speech. This prevents systems
from responding adequately because interrupted speech is
hard to recognize automatically [1].
In dialogues between a system and a user, undesired
speech overlaps or utterance collisions arise from two situations: (a) the system misrecognizes a speech pause as theend of the user’s turn and starts speaking, and (b) the user
repeats her/his last utterance because of the lack of response
from the system. One approach to avoid situation (a) is tomake a long interval from the end of the user’s speech signal.
This approach, however, deteriorates system responsivenessand leads to situation (b).
To handle situation (a), one can use linguistic information
to detect turn-ends [2], but this approach is affected byspeech misrecognition. Although para-linguistic informationsuch as F0 (fundamental frequency) can be used [3], it
is not available for fragmented short utterances because areliable F0 contour requires a substantial length of speech.
Situation (b) might be handled by equipping the system
to convey its intention of turn-taking to users by using
body/eye movements as humans do. However, this approachII. BLINKING LED AS SUBTLE EXPRESSION
Although human communication is explicitly achieved
through verbal utterances, non-verbal facial expressions,gaze, gestures, etc., also play an important role [8], [9]. Such
non-verbal communication often influences the accuracy ofutterance understanding [10].
Furthermore, researchers have reported that very smallchanges (called subtle expression) in facial expressions andgestures might influence human communication. We believethat we can utilize such subtle expression to make humans
easily understand a robot’s internal state because human canintuitively understand subtle expression. Some studies havebeen done on applying subtle expression to human-agent
interaction [11], [12], [13]. However, since they tried toenable subtle expression on real faces and with real arms,
their implementations were considerably expensive.
In contrast with such approaches, subtle expression has
been studied for artifacts like a robot or PC [7]. Komatsu
and Yamada reported that an agent’s subtle expression of
simple beeping sounds with decreasing/increasing frequency

Kazuki Kobayashi is with Research Center for Human Media, Kwansei Gakuin University, 2-1 Gakuen, Sanda, Hyogo 669-1337, Japan

kby@kwansei.ac.jp
Kotaro Funakoshi is with Honda Research InstituteCo., Ltd., 8-1 Honcho, Wako, Saitama 351-0188,

Japan
Japan

funakoshi@jp.honda-ri.com
Seiji Yamada is with National Institute of Informatics, 2-1-2 Hitotsubashi,
Chiyoda, Tokyo 101-8430, Japan seiji@nii.ac.jp
Mikio Nakano is with Honda Research Institute Japan Co., Ltd., 8-1 Honcho, Wako, Saitama 351-0188, Japan nakano@jp.honda-ri.com
Yasuhiko Kitamura is with Department of Informatics, School of Scienceand Technology, Kwansei Gakuin University, 2-1 Gakuen, Sanda, Hyogo
669-1337, Japan ykitamura@kwansei.ac.jp
Hiroshi Tsujino is with Honda Research Institute Japan
Co., Ltd., 8-1 Honcho, Wako, Saitama 351-0188, Japantsujino@jp.honda-ri.com

978-1-4244-2213-5/08/$25.00 ©2008 IEEE47

enabled humans to interpret the agent’s positive/negative
states. Their work indicated the effectiveness of subtle expressions such as varying beeping sounds for a robot oragent.
In this work, we propose the use of a blinking LED asa means of subtle expression to intuitively notify a user
about robot’s internal states (such as processing or busy). We
implemented the subtle expression on a robot that engage indialogue with a user and conducted preliminary experiments
on participants to verify the effectiveness of the subtle
expression.
III. EXPERIMENTS
We conducted experiments in which participants played alast-and-first game with a robot. A last-and-first game wasan appropriate task to investigate speech overlaps because
it involved a lot of turn-taking. We counted the numberof times the participants repeated their last utterances andestimated their impression of the dialogue. The dialoguesystem, robot, light-blinking expression, and experimental
method are explained as follows.

Fig. 1.

Operational Interface

A. Last-and-First Dialogue System
We chose a Wizard-of-Oz method to avoid speech recognition errors. The operator listened to the participant andoperated the robot using the interface shown in Figure 1.
The robot’s utterances were voiced by a commercial speechsynthesizer (NTT-IT FineVoice). The operator performed thefollowing operations when needed.
1) Input participant’s answer: The operator inputs theparticipant’s answer immediately after the participant
utters it. If the answer violates rules, the robot claims
a foul and restarts another game. Otherwise, it chooses
its next answer from a predefined lexicon.
2) Order to re-utter: The robot’s speech is sometimes hardto listen to. Upon a participant’s request, the operator
directs the robot to re-utter its last utterance.
3) Order to clarify: Sometimes the operator cannot catchthe participant’s answer. The operator directs the robot
to utter “Say it again.”4) Order to restart: Dialogue can break down for a varietyof reasons. In such cases, the operator directs the robot
to restart another game by uttering “Let’s start again.”5) Record the participant’s repetition: If the participant
repeats her/his last utterance, the operator records itby pressing the ESC key.
We controlled the experiments as follows so that the target
phenomena could be clearly observed. First, the intervals
between the users’ answers and the robot’s answers wererandomly varied between 0 and 15 seconds by insertinga waiting time. In actual spoken dialogue systems, theseintervals correspond to delays due to internal processing.
Second, the robot neglected the participants’ answers witha probability of 1/4. In actual dialogue systems, this corresponds to rejecting speech as noise by mistake. Third, toresolve a standoff, the robot automatically re-uttered its lastutterances after 20 seconds if the participant said nothing.

Fig. 2.

RS Media and position of embedded LED

B. Robot and Blinking-Light Expression
The participants talked to the small human-like robot “RSMedia (WowWee)” shown in Figure 2. In our experiments,
the robot did not drive any actuators. We embedded a red
LED (diameter: 4 mm) in its chest. The LED started blinking
when an operator began to input a participant’s utterance
and stopped blinking when the robot began to utter. In
other words, the LED blinked while the robot processed theparticipant’s utterance to prepare an answer. When the robot
rejected utterances, the LED stayed off. The blinking pattern
was based on 1/f in order to express an emotional state [14].
The 1/f based generator provided random numbers in a range
from −0.5 to 0.5. If a number was more than 0, the LED
was turned on. If a number was less than 0, the LED was
turned off. Figure 3 shows examples of 1/f random numbers
and LED blinking pattern. A random number was generated
every 1/30 seconds and then the LED status was updated.
C. Participants
Twelve participants were divided into two experimentalgroups: For six men (mean age: 22.8, S.D. = 1.2), LED
blinked as explained above (the blinking-light condition), and48

TABLE II
N UMBER OF REPETITIONS IN REJECTING
condition
blinking
non-blinking
total

repetitions
37
20
57

no repetition
26
33
59

total
63
53
116

TABLE IIIN UMBER OF REPETITIONS DURING PROCESSING AN UTTERANCE

Fig. 3.

condition
blinking
non-blinking
total

LED Blinking Patternrepetitions
1
36
37

no repetition
204
174
378

total
205
210
415

TABLE IRULES OF THE JAPANESE LAST- AND - FIRST GAME ( SHIRITORI )1

Say a word which begins with the final kana (Japanesealphabet) of the previous word.

2

You lose if you say a work whose last character is “n( ).”

3
4
5
6

robot rejected 63 utterances in the blinking condition and
53 utterances in the non-blinking condition. The participants
repeated their last utterances 37 times (58.7%) in the blinking
condition and 20 times (37.7%) in the non-blinking condition. This represents a 21.0 point improvement (increase)of repetition during rejecting in the blinking condition. We
performed Fisher’s exact test and obtained a significant
difference between conditions (p < .05).
Table III shows the number of participants’ repetitions
for the case that the robot processed their utterances andprepared an answer. The robot processed 205 utterances inthe blinking condition and 210 utterances in the non-blinking
condition. The participants repeated their last utterance once
(0.5 %) in the blinking condition and 37 times (17.1%)
in the non-blinking condition. This represents a 16.6 point
improvement (decrease) of repetition during processing in
the blinking condition. We performed Fisher’s exact test
and obtained a significant difference between conditions(p < .01).
These results showed that, in the blinking condition, theparticipants adequately repeated their last utterance when
the robot rejected them, and they did not repeat when
it processed them. This suggests that a blinking-light can
prevent speech overlaps between a user and a robot.

Don’t use same words.
You can remove a voiced/semivoiced sound mark.
ex.) ga ( ) → ka ( )Remove a prolonged sound symbol.
ex.) ko: () → ko ( )Use the last character of a palatalized consonant symbol.
ex.) kya ( ) → ya ( )for five men and one woman (mean age: 23.8, S.D. =1.7),
LED did not blink at all (the non blinking condition).
D. Experimental Method
The experiments were conducted in a small room (W:256 D: 205 H: 215 cm) at Kwansei Gakuin University.
Participants entered the room, sat on a chair in front of adesk. They answered a questionnaire after they had been
given instructions about the experiments.
After answering the questionnaire, they were asked toplay a last-and-first game with the robot for 10 minutes andexplained the rules shown in Table I. They could answer
as they liked if they forgot the rules. We told them thatthe robot sometimes replied quickly, sometimes leisurely,
and sometimes rejected their utterances. Moreover, they wererequested to continue the game as long as they could. The
meaning of the light-blinking expression was not explained
to them. After giving the instructions, the experimenter left
the room, and the participants began to play a game when
the robot started to talk to them. The game finished after ten
minutes, when the robot said that it was over.
After finishing the game, the participants answered questionnaires about their impression of the game and the robot’s
behavior. To analyze the blinking-light effect, we counted thenumber of repetitions and assessed the impressions.

B. Impression on the Last-and-First Game
Table IV shows the results of participants’ ratings forthe last and first game. The adjective pairs in the table
are translated from Japanese words that we used in thequestionnaire. The ratings are based on the 1-to-7 scales,
where “1” equals strong agreement with a negative adjectiveand “7” equals strong agreement with a positive adjective.
The questionnaire included nine reversal items.
Table V shows the factor loadings under the varimax
rotation. We used 11 adjective pairs that had larger S.D.
values than the omitted 7 pairs. The extraction method usedunweighted least squares. We obtained three factors from thescree plot.
We interpreted the factors according to the adjective pairsthat had loadings greater than 0.4 in Table V. The first factor(factor I) was named the familiarity factor and was based on
the adjective pairs in the factor analysis of previous work[15]. The second factor (factor II ) was named the loosenessIV. EXPERIMENTAL RESULTS AND DISCUSSION
A. Number of Participants’ Repetitions
Table II shows the number of participants’ repetitions forthe case that the robot rejected their utterances. In total, the49TABLE VI
FACTOR SCORES FOR IMPRESSION OF THE GAMETABLE IV
R ATED ADJECTIVE PAIRS FOR IMPRESSION OF THE GAME
adjective pairs
lightexciting
comfortable
comprehensible
easy
informalspiritedrelaxedcasualpleasant
leisurely
likablepeacefulsmoothwarm
decentgoodinteresting

dark
dull
uncomfortable
incomprehensible
uneasyformalspiritlesstensional
grave
unpleasanthurrieddislikableannoyingroughcold
indecent
poor
boring

mean
3.33
5.33
3.42
3.92
3.33
3.08
3.33
2.67
3.83
3.08
2.92
3.33
2.92
3.42
2.83
3.33
3.50
2.58

S.D.
1.23
1.37
1.16
1.56
1.07
1.08
1.30
1.30
1.27
1.24
1.08
1.07
1.00
0.90
0.83
0.78
0.67
0.67

factorfactor I (familiarity)
factor II (looseness)factor III (pleasantness)

TABLE VII
adjective pairs
patient
friendlykindcarefulaccessible
excitedaggressive
activeimpressive
sociable
confident
respectful
responsible
seriousshy
decentinnocent
discreetbroad-mindedprettyTABLE Vlight
exciting
comfortable
comprehensible
easy
informalspiritedrelaxedcasualpleasant
leisurely
contribution(%)

factor I
0.90
0.77
0.69
0.44
0.02
0.64
0.40
0.44
0.25
−0.04
0.29
27.06

factor loadings
factor II
factor III
0.13
0.12
0.27
0.29
0.48
−0.06
0.34
0.38
0.81
0.52
0.69
−0.10
0.68
0.21
0.48
0.42
0.42
−0.21
−0.09
0.88
0.20
0.68
22.69
18.41

non-blinking(n=6)
mean
S.D.
−0.49
1.12
0.60
1.13
−0.42
1.28

R ATED ADJECTIVE PAIRS FOR IMPRESSION OF THE ROBOTR ESULTS OF FACTOR ANALYSIS OF IMPRESSION OF THE GAME( VARIMAX ROTATION , FACTOR LOADING MATRIX )itemblinking(n=6)
mean
S.D.
0.49
0.77
−0.60
0.53
0.42
0.67

communality
0.83
0.75
0.71
0.28
0.67
0.90
0.60
0.92
0.59
0.46
0.79

irritable
unfriendly
unkindcareless
inaccessible
cooldefensive
inactive
unimpressive
unsociable
unconfident
impudent
irresponsible
frivolousshamelessindecent
wickedindiscreet
narrow-minded
provoking

mean
4.17
3.42
3.08
5.17
3.42
3.50
4.33
3.67
4.92
3.75
5.00
3.42
4.08
4.58
3.50
3.50
3.58
4.08
3.50
3.58

S.D.
1.53
1.24
1.16
1.34
1.16
1.09
1.44
1.56
1.08
1.29
1.13
1.00
1.00
0.90
0.80
0.80
0.79
0.79
0.67
0.67

We interpreted the factors according to the adjective pairsthat had loadings greater than 0.4 in Table VIII. The first
factor (factor I) was named the friendliness factor since the“patient” and “friendly” had high loads. The second factor(factor II ) was named the optimism factor since “careless”
and “accessible” had high loads. The third factor (factor
III ) was named the activeness factor since “active” and“impressive” had a high load.
We obtained the factor scores by using Bartlett’s method
and compared blinking condition with the non-blinking condition. Table IX shows the factor scores for the impressionof the robot. We performed the Mann Whitney test andfound no significant difference between conditions (factor I:
U = 16.00, Z = −0.32, p = .75, factor II : U = 15.00, Z =−0.48, p = .63, factor III : U = 18.00, Z = 0.00, p = 1.00).
These results suggest that the blinking-light caused no effectto users’ impression about the robot which they talked to.

factor, since “easy” and “informal” had high loads. The third
factor (factor III ) was named the pleasantness factor, since
the “pleasant” had a high load.
We obtained the factor scores by using Bartlett’s method
and compared blinking condition with the non-blinking condition. Table VI shows the factor scores for the impression of
the game. We performed the Mann Whitney test and found asignificant difference between conditions in the factor I score(U = 7.00, Z = −1.76, p = .05) and 10% level differencein the factor II score (U = 6.00, Z = −1.92, p = .08).
There was no significant difference in the factor III score(U = 10.00, Z = −1.28, p = .20). These results suggest thatthe blinking-light created familiar and attentive impression
about the game on users.

D. Waiting TimeIn the questionnaire, we asked participants to answer theamount of time that the robot spent in replying. Table X
shows their reported waiting time and actual waiting time.
We performed Mann-Whitney test and found the significant
difference between the conditions in reported time (U =3.00, Z = −2.42, p < .05). There was no significant
difference in actual time (U = 15.00, Z = −.48, p = .63).
These results suggest that users felt their waiting time short
by observing the blinking-light.

C. Impression on the Robot
Table VII and Table VIII show the results of participants’ ratings for the robot and the factor loadings underthe varimax rotation, respectively. We used 11 adjective
pairs that had larger S.D. values than the omitted 7 pairs.
The extraction method used unweighted least squares. We
obtained three factors from the scree plot. The questionnaire
included ten reversal items.

50

TABLE X
R EPORTED WAITING TIME AND ACTUAL WAITING TIMETABLE VIIIR ESULTS OF FACTOR ANALYSIS OF IMPRESSION OF THE ROBOT( VARIMAX ROTATION , FACTOR LOADING MATRIX )itempatient
friendlykindcarefulaccessible
excitedaggressive
activeimpressive
sociable
confident
contribution (%)

factor loadings
factor I
factor II
factor III
0.95
−0.32
−0.07
0.79
0.27
0.28
0.73
−0.05
0.19
0.28
−0.92
0.13
0.26
0.74
−0.10
−0.38
0.65
0.08
0.04
0.44
0.31
0.30
−0.20
0.78
0.22
−0.05
0.78
0.07
−0.13
−0.64
−0.58
0.15
0.62
25.86
20.77
20.46

waiting timecommunality

reported time

1.00
0.78
0.58
0.95
0.63
0.58
0.29
0.74
0.66
0.43
0.74

actual time

factor I (friendliness)factor II (optimism)
factor III (activeness)

S.D.
2.16
3.76
1.04
1.61

TABLE XINo

FACTOR SCORES FOR IMPRESSION OF THE ROBOT
blinking(n=6)
mean
S.D.
−0.07
0.83
0.14
1.36
0.00
1.29

mean [sec]
3.67
8.83
7.21
6.63

R EPORTED BEHAVIOR AND MEANING IN BLINKING CONDITION

1

Robot’s behavior
The LED turned on when the
robot answered.

Meaning
The robot recognized a participant’s word when the LED
turned on.

2

The LED turned on.

The robot said a next word.

3(1) The LED turned on.
(2) The LED stayed off.

4

The red LED blinked.

(1) The robot was thinking.
(2) The robot did not listen.
The robot recognized a participant’s word.

5

Nothing.

(no response)

6

The LED on its chest blinked.

The robot thought and understood a participant’s word.

TABLE IX

factorcondition (n=6)
blinking
non-blinking
blinking
non-blinking

non-blinking(n=6)
mean
S.D.
0.07
1.23
−0.14
0.66
0.00
0.91

TABLE XIIR EPORTED BEHAVIOR AND MEANING IN NON - BLINKING

E. Participants’ Interpretation of Blinking-Light
No

Participants were asked to answer robot’s behavior andits meaning that they noticed. Table XI and XII show theresults of the questionnaire in the blinking and non-blinking
condition, respectively. In the blinking condition, five of thesix participants noticed the LED blinking and reported thatits meaning was thinking or recognizing. In the non-blinking
condition, four of the six participants noticed the behavior
about last-and-first game and only one participant reported
that changing answer time meant thinking. These results
suggest that LED blinking was interpreted as thinking.

1

2

3

4

V. CONCLUSION

5

We proposed a method to enable smooth speech interactions between a user and a robot. Our method was basedon subtle expression whereby a robot blinks a small LED
attached to its chest. We performed experiments in which
participants played a last-and-first games. The number ofrepetitions made by the participants and their impression of
the game and the robot were analyzed.
The analysis of the number of participants’ repetitions
showed that the participants adequately repeated their lastutterance when the robot rejected them and they did not
repeat when it processed them in the blinking condition.
The analysis of participants’ impression of the last-and-firstgame showed that the blinking-light created familiar andattentive impressions about the game on users. The blinkinglight caused no effect to user’s impression about the robot.
Participants felt their waiting time short by observing theblinking-light. Most of participants interpreted the blinking
as thinking. These results supported the effectiveness of theblinking-light expression. Although these experiments were
small-scale and preliminary, we think they indicate that LED-

6

Robot’s behavior
The robot recognized the
speech and answered a word.
The robot said the participant
lost when it was not able torecognize a his/her word or
he/she was quiet for a certain
time.

Meaning

Nothing.

(no response)(1) The robot changed answer
time.
(2) The robot uttered “Say it
again.”The robot asked again, repeated an word, and spoke ina monotone voice.

(1) The robot was thinking.

Nothing.

(no response)(no response)(no response)(2) The robot failed to recognize participant’s speech(no response)

based subtle expression is helpful for smoothing humanrobot speech interaction.
R EFERENCES[1] M. Nakano, Y. Nagano, K. Funakoshi, T. Ito, K. Araki, Y. Hasegawa,and H. Tsujino, “Analysis of user reactions to turn-taking failures inspoken dialogue systems,” in Proc. of SIGdial Workshop on Discourseand Dialogue, 2007.
[2] L. Bell, J. Boye, and J. Gustafson, “Real-time handling of fragmented
utterances,” in Proc. of NAACL-2001 workshop on Adaptation in
Dialogue Systems, 2001.
[3] T. Ohsuga, M. Nishida, Y. Horiuchi, and A. Ichikawa, “Investigationof the relationship between turn-taking and prosodic features inspontaneous dialogue,” in Proc. of European Conference on SpeechCommunication and Technology, 2005, pp. 33–36.
[4] W. Tsukahara and N. Ward, “Evaluating responsiveness in spoken
dialog systems,” in Proc. of International Conference on Spoken
Language Processing, 2000.

51

[5] N. Ward, “On the expressive competencies needed for responsive
systems,” in Proc. of CHI2003 workshop on Subtle Expressivity for
Character s and Robots, 2003.
[6] N. Kitaoka, M. Takeuchi, R. Nishimura, and S. N. a, “Response timing
detection using prosodic and linguistic information for human- friendly
spoken dialog systems,” Journal of The Japanese Society for Artificial
Intelligence, vol. 20, no. 3, pp. 220–228, 2005.
[7] T. Komatsu and S. Yamada, “How do robotic agents’ appearances
affect people’s interpretations of the agents’ attitudes?” in Proc. of
CHI-2007, 2007, pp. 2519–2524.
[8] A. Kendon, “Do gestures communicate?” A Review. Research in
Language and Social Interaction, vol. 27, no. 3, pp. 175–200, 1994.
[9] ——, “Some functions of gaze direction in social interaction,” Acta
Psychologica, vol. 26, pp. 1–47, 1967.
[10] W. Rogers, “The contribution of kinesic illustrators towards thecomprehension of verbal behavior within utterances,” Human Communication Research, vol. 5, pp. 54–62, 1978.
[11] C. Bartneck and J. Reichenbach, “Subtle emotional expressions of synthetic characters,” International Journal of Human-Computer Studies,vol. 62, no. 2, pp. 179–192, 2005.
[12] S. Brave, C. Nass, and K. Hutchinson, “Computers that care: investigating the effects of orientation of emotion exhibited by an embodied
computer agent,” International Journal of Human-Computer Studies,vol. 62, no. 2, pp. 161–178, 2005.
[13] H. Prendinger, J. Mori, and M. Ishizuka, “Using human physiology toevaluate subtle expressivity of a virtual quizmaster in a mathematicalgame,” International Journal of Human-Computer Studies, vol. 62,
no. 2, pp. 231–245, 2005.
[14] S. Hosomi, M. Tsukamoto, and S. Nishio, “A system for controlling
led blink in wearable fashion,” in Proc. of International Conference
on Wireless Communications and Mobile Computing. New York, NY,
USA: ACM, 2007, pp. 665–670.
[15] T. Kanda, H. Ishiguro, and T. Ishida, “Psychological analysis on
human-robot interaction,” in Proc. of IEEE International Conferenceon Robotics and Automation, 2001, pp. 4166–4173.

52