電気学会論文誌 C（電子・情報・システム部門誌）
IEEJ Transactions on Electronics, Information and Systems
Vol.137 No.12 pp.1625-1632
DOI: 10.1541/ieejeiss.137.1625

Paper

Gender Effects on Lexical Alignment in Human-Robot Interaction
Mitsuhiko Kimoto＊,＊＊a) Non-member,
Masahiro Shiomi＊
Non-member,＊＊
Katsunori Shimohara
Member,Takamasa Iio＊,＊＊＊ Non-member
Ivan Tanev＊＊
Non-member＊
Norihiro Hagita
Non-member(Manuscript received March 1, 2017, revised July 31, 2017)
Humans tend to change their lexical expressions to resemble those used by their interlocutors to achieve smooth conversations.
Such phenomena, called “lexical alignments,” are affected by gender. Even though lexical alignment is observed not only inhuman-human interaction but also in human-robot interaction, the gender effects on it in human-robot interaction haven’t been
investigated yet. Identifying whether gender affects lexical alignment in human-robot interaction would contribute to the designof conversational strategies for interactive robots for more natural interaction. This paper reveals that gender affects lexicalalignment in human-robot interaction. We conducted an experiment with twenty participants who interacted with a robot in objectreference conversations and referred to an object whose identity was confirmed by a robot. We developed a robotic system thatengaged in object reference conversations with two interaction strategies and measured the gender effects on lexical alignment in
human-robot interaction. Our experimental results showed that female participants were lexically more aligned with the robot
than males; female participants used more references that were useful to uniquely identify objects in environments than males.
Keywords : human-robot interaction, alignment, gender effects, interaction strategyalignment of word choices occur in conversations between
humans and a robot. Their experiment’s participants were more
likely to use the same words as the robot in conversations. In thehuman-robot interaction field, lexical alignment findings havealso used to improve a robot’s performance. For example, Kimoto
et al.(16) proposed a strategy that incorporates lexical alignment
findings in the robot’s behavior model to improve the recognition
performance of objects indicated by a user.
In alignment between humans and artificial media/robots,other works cited gender differences(13)(15)(17). Thomason et al.(17)concluded that males aligned more than females for vocal loudness
features. Strupka et al.(15) reported that even though humans alignedwith robot voices in relation to acoustic energy level, gender had
no effect on their voice adjustments.
Past research works reported that the degree of alignment isaffected by gender. However, since no study has addressed thegender-based differences of lexical alignment in human-robot
interaction, we focus on gender effects on lexical alignment in
human-robot interaction. Gender effects on lexical alignment in
human-robot interaction is worth investigating for the following
two reasons. First, to investigate gender effects is important forunderstanding human activity and many researchers haveinvestigated gender differences on various psychologicalattributes(18)(19). Hyde(19) stated the reason for the importance ofthe research on gender differences and similarities in his review
article as follows: “… stereotypes about psychological genderdifferences abound, influencing people’s behavior, and it isimportant to evaluate whether they are accurate”. In the research
fields of alignment, gender differences have also been investigated.
However, gender differences on lexical alignment between people
and robots have not been well investigated and not been revealed.
Interaction between people and robots is the new interaction stylecompared to the interaction between people, and the gender effects

1. Introduction
In communication, humans tend to repeat lexical expressions
that resemble those of their interlocutor(2)(3). This phenomenon iscalled lexical alignment. Its importance is located in the fact that
it is often associated with successful dialogues. Nenkova et al.(4)found that alignment in the use of high-frequency words wascorrelated with task success and turn-taking in dialogues. Lee etal.(5) reported that the alignment measures of two prosodic
features, pitch and energy, were higher in positive interactions
between married couples than in negative interactions. According
to Pickering and Garrod(6), alignment is a critical element forsuccessful communication.
Alignment has often been researched in terms of gender effects.
Some past research works on human-human interaction observed
differences in the degree of alignment by gender(7)-(9). Namy etal.(8) reported females aligned in relation to word pronunciation
more than males. Levitan et al.(7) found that alignment in acoustic/
prosodic features was most prevalent for female-male conversation
pairs.
Alignment is also observed in interactions between a human andartificial media, for example, in spoken dialogue systems(10)-(12)and robots(13)-(15). Iio et al.(14) found that lexical alignment and theThis paper is an extended version of a previous work of Kimoto et al.(1)and contained additional experiment results about gender effects andmore detailed discussions.
a) Correspondence to: Mitsuhiko Kimoto. E-mail: kimoto@atr.jp＊
Intelligent Robotics and Communication Laboratories, ATR
2-2-2, Hikaridai, Seikacho, Sourakugun, Kyoto 619-0288, Japan＊＊
Graduate School of Science and Engineering, Doshisha
University
1-3, Tatara Miyakodani, Kyotanabe, Kyoto 610-0321, Japan＊＊＊
Graduate School of Engineering Science, Osaka University
1-3, Machikaneyama-cho, Toyonaka, Osaka 560-8531, Japan© 2017 The Institute of Electrical Engineers of Japan.

1625

Gender Effects on Lexical Alignment in Human-Robot Interaction (Mitsuhiko Kimoto et al.)Fig. 1.

the gender differences in the degree of alignment reported by past
research lacks consistency. For example, Namy et al.(8) argued thatfemales are more likely to align than males, in contrast to Pardo’s
results, where males were more likely to align than females(9).
2.2 Gender Effects on Alignment between Humans andArtificial Media/Robots
Gender differences in the degree ofalignment have also been mentioned by past research on humanartificial media/robot interaction(13)(15)(17). Thomason et al.(17)
investigated the relationships between acoustic/prosodic alignment
to a tutoring dialogue system and concluded that males aligned to
loudness features more than females. Strupka et al.(15) investigatedacoustic/prosodic alignment in human-robot dialogues. Their results
showed that the gender of the robot’s voice marginally affected theacoustic/prosodic alignment, but they found no human gendereffect on it. Iio et al.(13) experimented with a remotely operated
robot and investigated whether human pointing was aligned to
the robot’s gestures. They analyzed their experimental results onparticipant genders and concluded there is no effect of humangender differences on the alignment of pointing gestures.
In the field of human-artificial media/robot interaction ofalignment, gender differences for the degree of alignment havealso been discussed. However, to the best of our knowledge, sinceno past research has treated lexical alignment, we focus on whether
gender differences affect the degrees of lexical alignment.

Recognition of indicated object by an interlocutor

on human-robot interaction would be different from human-humaninteraction. Revealing gender-based differences of lexical alignmentthat occur in human-robot interaction has importance itself. Next,
recently lexical alignment is used to design human-robot interaction.
If we can identify the gender effects on lexical alignment between
humans and robots, such understanding would help design
human-robot interaction. For example, Kimoto et al.(20) focusedon strategies to clarify the lexical expressions of users’ references
when they refer to the objects in an environment and compared
two strategies. In one strategy, robots implicitly elicited the lexical
expressions contained in the robot’s database from users throughlexical alignment (implicit alignment). In their second strategy,
robots explicitly instructed a user how to refer to objects (explicit
instruction). They concluded that implicit alignment is more
effective than explicit instruction. However, they did not compare
the gender effects of the two strategies, and if lexical alignment isaffected by gender, appropriate strategies may differ based on
gender.
This paper investigated whether gender-based differences oflexical alignment occur in human-robot interactions and discusses
a robot’s interaction strategies based on gender effects on lexicalalignment. We conducted an experiment using a robotic systemthat interacted with humans in situations where a human referred
to an object in an environment and a robot confirmed that indicatedobject (Fig. 1). Our work is based on past research works aboutalignment between humans and robots(13)(14)(16)(20).

3. Interaction Design

2. Related WorkIn this chapter, we summarize the related works about gendereffects on alignment.
2.1 Gender Effects on Alignment between Humans
Some past research has observed differences in the degree ofalignment by gender(7)-(9)(21). Namy et al.(8) gave a group of maleand female participants a single-word shadowing task to investigate
gender differences in vocal alignment. Their experimental results
suggest that female shadowers are more likely to align than maleshadowers. Levitan et al.(7) measured alignment in three acoustic/
prosodic features (intensity, pitch, and jitter) that were extracted
from the speech of subjects playing a cooperative computer gameand found that alignment is most prevalent for female-male pairs,
followed by female-female pairs. The alignment of the male-malepairs was the lowest. Pardo(9) investigated the alignment ofpronunciation in task-oriented conversations and found that maletalkers overall aligned more than females.
Gender differences in the degree of alignment between humans
has also been observed in past work on alignment related to vocal
interaction between humans. However, to the best of our knowledge,
no research work has investigated lexical alignment. Furthermore,1626

To investigate the gender effects on lexical alignment between
humans and robots, we used an interaction called object reference
conversations (Fig. 2). Such conversations focus on confirmation
behavior, which is often observed in human-human communication.
If a person cannot confidently understand which object was beingreferenced, she is likely to ask for confirmation. Furthermore,
people sometimes confirm the referenced object even in case of
the referenced object is clear to avoid discrepancies in theinterpretation. Such conversations are already being used inhuman-robot interaction research fields to explore lexicalalignment as well as the alignment of pointing gestures inhuman-robot interaction(13)(14)(16)(20).
Object reference conversations consist of four parts: Ask, Refer,
Confirm, and Answer. First, a robot asks an interlocutor to refer toan object in an environment (Ask). Next the interlocutor refers toan object (Refer), and the robot confirms the object to which theinterlocutor referred (Confirm). Then the interlocutor answers
whether the object confirmed by the robot is correct (Answer).
In this paper, based on past research works(16)(20), we employ
two interaction strategies for a robot. The past research works
investigated whether lexical alignment occurs in human-robot
interaction and its degree. Both strategies are multi-modal
interaction considering speech and gestures together. We used themulti-modal strategies rather than strategies that use specificmodalities because basically human-human interaction ismulti-modal and multi-modal interaction is need for naturalinteraction between people and robots. A use of only partial
modality is also unnatural for interaction with people. Especially,
for lexical alignment multi-modal interaction is important because
human’s gestures are reported to affect lexical alignment. Holler
and Wilkin(22) reported that lexical alignment became suppressed
when human aligns with their interlocutor’s gestures. Iio et al.(14)also suggested that lexical alignment about objects’ attributes
became suppressed when human align with robot’s pointing gesture.
IEEJ Trans. EIS, Vol.137, No.12, 2017

Gender Effects on Lexical Alignment in Human-Robot Interaction (Mitsuhiko Kimoto et al.)Therefore, we did not investigate the effects of each modality (e.g.,
gesture only) or different interaction style (e.g., typing) partially;rather we are interested in the gender effects under human-likeconversation style, because this style would be common style forsocial robots which act in real environments. To investigate thegender effects in human-robot interaction research field, we focusedon the two major conversation strategies in human-robot interaction
under object-reference conversations: explicit and implicit. These
conversation strategies are already used to investigate the degreeof alignment in human-robot interaction(16)(20), therefore using
these two strategies would be appropriate for our purpose. The
details of each strategy are described as follows.
3.1 Implicit Alignment Strategy
One approach is theimplicit alignment strategy proposed by Kimoto et al.(16) In thisstrategy, a robot makes confirmations that contain minimum
information for distinguishing objects. Fig. 3 shows an example ofan object reference conversation with an implicit alignment
strategy.
This strategy exploits alignment in object reference conversations.
Based on these three alignment phenomena, lexical alignment,
gestural alignment, and alignment inhibition, past work designed
robot behavior as follows. A robot should use minimum information

for distinguishing among objects in the environment. Alignment
inhibition is a formation phenomenon that decreases in someconversations. Through this design, people learn to make referencesthat include sufficient information to identify the objects by
reducing the alignment inhibitions. Past work of Kimoto et al.(16)
implemented this design in the Confirm part in the object reference
conversation.
3.2 Explicit Instruction Strategy
Another scheme isthe explicit instruction strategy. In it, a robot provides instructions
about how to refer to objects in a way of instructions that asks theinterlocutors to make a reference that includes as much informationas possible and requests that they use the information that was
missing from the previous references. Fig. 4 shows an example ofan object reference conversation with an explicit instruction
strategy.
This strategy is based on the following considerations. If aninterlocutor refers to an object, as prodded by the robot, it willprobably recognize it with high performance. If the interlocutor
fails to follow the robot’s instructions, the robot should also requestthat the interlocutor uses all of the instructed information for thefollowing object references. This suggestion encourages theinterlocutor to “obey” in subsequent conversations.

4. System

Fig. 3. Example of object reference conversation with
implicit alignment approach: robot confirmed indicatedobject using minimum information for distinguishingobjects (thick line denotes the part implicit alignment
strategy is implemented in)

We developed a system based on past works that implemented
implicit alignment and/or explicit instructions for object reference
conversations(16)(20). The system consists of four parts: sensors,
an indicated object recognition function, an object information
database, and a robot behavior control function. When a user refers
to an object, the indicated object recognition function identifies
the user’s reference behavior and estimates the indicated object.
The robot behavior control function chooses a robot behavior that
corresponds to the implemented strategy and sends a behavior
command to the robot, which confirms the indicated object andasks a user to refer to it in the next conversation in a manner
decided by the robot behavior control function. Fig. 5 shows thearchitecture of our developed system.
The system can also have object reference conversations as abasic function. In its Ask and Confirm parts, the robot performs a
behavior that corresponds to whichever approach was used by the
robot.
4.1 Robot
In this study, we used Robovie-R ver.2, ahumanoid robot developed by the Intelligent Robotics andCommunication Labs, ATR, which has a human-like upper bodydesigned for communication with humans. It has three DOFs for
its neck and four for each arm. Its body has sufficient expressive
ability for object reference conversations. We used XIMERA for

Fig. 4. Example of explicit robot instructions: robot
explicitly instructs how to refer to objects (thick line
denotes the part explicit instruction strategy isimplemented in)

Fig. 5. System architecture to recognize objects
indicated by interlocutor

Fig. 2. Object reference conversation: black and whiteboxes respectively denote robot and human turns

1627

IEEJ Trans. EIS, Vol.137, No.12, 2017

Gender Effects on Lexical Alignment in Human-Robot Interaction (Mitsuhiko Kimoto et al.)speech synthesis(23). It is 1100-mm tall, 560-mm wide, 500-mm
deep, and weighs about 57 kg.
4.2 Indicated Object Recognition Function
To develop
this function, we implemented an algorithm(16)(20) that combines
the speech recognition, pointing gesture recognition, and face
direction recognition results.
4.2.1 Speech Recognition Module
The speech
recognition module receives human speech that refers to an object
and outputs the normalized reference likelihood of each object
based on speech recognition. To calculate the likelihood, we used
the number of attributes in human speech(16), which were captured
by a microphone attached to a human’s collar. In this system, weused a speech recognition engine called Julius, which gives good
performance for Japanese(24).
4.2.2 Pointing Gesture Recognition ModuleThe
pointing gesture recognition module obtains the body frame datafrom a depth image sensor called Kinect for Windows v2 andoutputs the normalized reference likelihood of each object basedon pointing gesture recognition. We modeled the likelihood as thedifference from the pointing vector (between the human head andthe tip of the human hand) to a vector between the human head
and an object with a normal distribution function N(0, 1).
4.2.3 Face Direction Recognition Module
The face
direction recognition module obtains the face direction vector
from the depth image sensor and outputs the reference likelihood
based on the face direction recognition. We modeled the likelihood
based on an angle parallel to the plane of the floor between theface direction vector and a vector between a human head and an
object. If the vector is less than 110º, the person is considered to
be viewing the object; its likelihood is 1, and otherwise 0. This isbecause a human’s field of view is 110º at most(25). The likelihoods
are finally normalized from 0 to 1.
4.2.4 Integration Module
The integration module merges
the reference likelihoods of the speech and both the pointing gestureand face direction recognitions. These three likelihoods are summed
and normalized(16). The object with the highest likelihood is
estimated to be the one indicated by the interlocutor.
4.3 Robot Behavior Control Function
The robot
behavior control function determines how the robot confirms theindicated object (Confirm behavior) and how it asks an interlocutor
to refer to it (Ask behavior) in subsequent conversations. The
conversation contents of the Confirm and Ask behaviors reflect
whether the implicit alignment strategy or the explicit instruction
strategy is used.
When using the implicit alignment strategy, this function
chooses the Confirm behavior and adopts the implicit alignment
strategy, and the robot confirms the indicated object withminimum information for distinguishing among objects. The Ask
behavior does not adopt a particular strategy, and the robot does
not explicitly instruct the participants how to make references.
When using the explicit instruction strategy, this function
chooses the Ask behavior and adopts the explicit instruction
approach, and the robot explicitly provides instructions about
how to refer to objects. The Confirm behavior does not adopt a
particular strategy, and the robot confirms the indicated object bypointing and verifying all of the information about it.

gender differences reported by past research are inconsistent. One
reported tendency is that females align more than males(7)(8), butanother argues the opposite(9)(17). Since predicting the tendency
related to gender effects on alignment is difficult, we made twocontradictory hypotheses about gender effects on lexical alignment
in human-robot interaction.
Hypothesis about female-dominant effects on lexicalalignment in human-robot interaction
Past research identified female-dominant effects on alignment.
Namy et al.(8) reported that females are more likely to align thanmales even when social interaction is severely limited. Their
participants did a shadowing task in which they sat alone in aroom and repeated single words uttered by various speakers overheadphones. Levitan et al.(7) measured alignment on acoustic/
prosodic by analyzing the speech of participants who were playing
cooperative computer games for female-female, female-male, andmale-male dyads and found that alignment is most prevalent for
female-male pairs, followed by female-female pairs. Male-malepairs aligned the least. Although their results didn’t show thatfemale-female pairs aligned the most, the least aligned pairs werethe male-male pairs. These results suggest that females align morethan males. Therefore, we believe that females will align morewith interlocutors than males. Based on these considerations, wemade the following hypothesis:
Prediction 1-a: Females will lexically align more with a robot
interlocutor than males.
Hypothesis about male-dominant effects on lexical alignment
in human-robot interaction
Past research identified male-dominant effects on alignment.
Pardo(9) concluded that the speech of talkers became more similar
to the pronunciation of their partner’s speech during conversational
interactions. She reported that overall, male talkers were morealigned than female talkers. Thomason et al.(17) investigated whether
students acoustically/prosodically aligned with a tutoring dialoguesystem. Each student verbally responded to either pre-recorded orsynthesized tutor questions. Their results suggested that males
were significantly more aligned than females to minimum andmaximum features of loudness. Therefore, we believe that males
will align more with interlocutors than females. Based on these
considerations, we made the following hypothesis:
Prediction 1-b: Males will lexically align more with a robot
interlocutor than females.
5.2 Environment
Our participants sat in front of the robot.
We arranged books as objects in the environment by following pastworks that used object reference conversations(13)(14)(16)(20). Fivebooks were placed in a 1.5 m by 3.3 m rectangular area between
the robot and the participant and grouped close together without
overlapping approximately 0.6 – 2.6 m from the participants. Fig. 6
shows the experimental environment.
We controlled the attributes of the books based on past research
work that focused on object reference conversations(16)(20). All
the books were 21 cm by 27.5 cm, and their attributes were color,
symbol, and a letter on the cover. There were three colors: red,
blue, or yellow. Three symbols were placed on the book covers: acircle, a triangle, or a square. There were two letters: Q and B. We
prepared 18 books to satisfy all combinations of attributes.
5.3 Conditions
We controlled the strategy that wasapplied to our developed system (applied strategy factor). The
applied strategy factor had two levels: implicit alignment andexplicit instruction. Both were applied to the Confirm and Ask

5. Experiment
5.1 Hypotheses and Predictions
Some past researchworks reported a gender effect on alignment. However, such
1628

IEEJ Trans. EIS, Vol.137, No.12, 2017

Gender Effects on Lexical Alignment in Human-Robot Interaction (Mitsuhiko Kimoto et al.)Fig. 7. Example of book arrangement
Fig. 6. Experimental environment
subjects simply is difficult, both number of female and malesubjects is less than or equal to 10(7)-(9)(13)(15).
5.5 Procedure
We conducted our experiment as follows.
First, we explained it to the participants who signed consent forms.
Next, we orally gave them the following instructions: “The robot
can recognize human speech, pointing gestures, and face directions.
It will ask you to indicate a book. Do so as if you were dealing
with a person”.
After the instructions, the participants selected five books fromamong the 18 and arranged them based on the experimenter’s
instruction: “Please arrange the books in one place”. An exampleof the arrangement is shown in Fig. 7. After that the participant
repeated the object reference conversations ten times. We call thisset of ten object references the conversation sessions, which wereconducted in both the applied approach conditions: explicit
instruction and implicit alignment. The participants answeredquestionnaires about their impressions of the conversations after
each conversation session. We counterbalanced the order of theinteractive strategy conditions.
5.6 Measurement
5.6.1 Information Amount of Reference
To investigate
the change of reference styles, we measured the mean number ofobject attributes (color, symbol, and letter) in the participantreferences per sessions.
According to lexical alignment findings between humans androbots, humans tend to use the same word as the robot in
conversations(14). This finding suggests, for example, if a robot
uses the word “blue” as a color attribute, humans will avoid theword “cyan” and use blue instead. Therefore, we measured themean number of object attributes contained in the robot’s object
information database and used in the Confirm part.
5.6.2 Reference Redundancy of UtterancesThe
reference redundancy of an utterance is defined as the difference
between the numbers of object attributes in the participant’sreferences and the minimum number of attributes for uniquely
identifying the referenced objects in the environment per sessions.
Our objects have three attributes (color, symbol, and letter), andthe number of object attributes in the participant’s references was
defined as 0 to 3. For example, if a participant’s reference has no
attributes, the numbers of object attributes is 0. If a participant’sreference has all three attributes (color, symbol, and letter), thenumbers of object attributes is 3. The minimum number of attributes
to uniquely identify the indicated object in the environment ranges
from 1 to 3. For these reasons, reference redundancy ranges from
−3 to 2. For example, if a participant refers to a book with no
attributes (i.e., “that book” or “this book”) in the environment
where all the attributes are needed to uniquely identify objects(minimum number of attributes is 3), the reference redundancy ofutterances is −3. We measured the reference redundancy of theparts of the object reference conversations. The applied strategy
factor had a within-participant condition. There was no differencein the manner of recognizing the interlocutor’s reference behavior
or estimating the indicated object.
5.3.1 Implicit Alignment Condition
In the implicit
alignment condition, unlike the explicit instruction condition, therobot did not explicitly provide instructions about the reference
style; it just said, “Please choose a book” in the Ask part.
On the other hand, in the Confirm part the robot said a different
sentence. For this purpose, we implemented an implicit alignment
design for the reference behavior. In this condition, the robot
confirmed the object with minimum information for distinguishingamong objects; confirmations were based on the implicit alignment
strategy of references. This approach determined the robot’s objectreference behaviors, i.e., with or without a pointing behavior andspeech contents, by considering the objects’ position relationships
and characteristics. The robot pointed to reduce the number ofcandidates of the referenced objects. The speech format of theconfirmations is the sequence of object attributes. For example,
the robot asks, “That yellow book with a triangle on its cover?” or“That blue book?”
5.3.2 Explicit Instruction Condition
In the explicit
instruction condition, the robot gives instructions about how torefer to objects in a way that asks interlocutors to make a reference
that includes as much information as possible in the Ask part.
The speech format of the explicit instructions includes two
sentences. The first is used every time; the second is only used
when a participant failed to use all of the information requested in
the first sentence in the previous reference.
For example, the robot asks, “Can you refer to the book bycolor, the symbol and the letter on its cover as well as by pointing
and looking at it? Please refer to a letter and point.”
In the Confirm part of this condition, since the robot verified theobjects with all of the information, it gave every attribute of an
object and pointed during the confirmations.
5.4 Participants
Twenty people (ten females and ten
males who averaged 35.5 years of age, SD = 9.9) participated inour experiment. We decided the number of subjects based on past
research works about alignment. Five seventh of past researchworks we cited in chapter “2. Related Work” investigated gendereffects on alignment by less than 20 subjects except the number ofsubjects who assesses alignment: Levitan et al.(7), Namy et al.(8),Pardo(9), Iio et al.(13) and Strupka et al.(15) For example, in the
Namy et al.’s shadowing task 8 female and 8 male shadowers
repeated words sounded from headphones(7). In the Iio et al.’stask 10 female and eight male participated the conversation withthe robot(13). Although experimental procedures of the five past
research works are all different and comparing the number of
1629

IEEJ Trans. EIS, Vol.137, No.12, 2017

Gender Effects on Lexical Alignment in Human-Robot Interaction (Mitsuhiko Kimoto et al.)in the applied strategy factor (F(1,18) = 6.616, p = .019, partial η2= .269). We found no significance in the gender factor (F(1,18) =2.646, p = .121, partial η2 = .128) and no significant interaction(F(1,18) = .952, p = .342, partial η2 = .050). These results showedthat the number of object attributes in the references with explicit
instruction was significantly larger than the number of object
attributes in references with implicit alignment. On the other hand,these results showed no gender-based differences in the information
amount of the references.
Fig. 9 shows the results of the reference redundancy of utterances.
We conducted a two-factor mixed ANOVA for both applied strategy
and gender factors and found significant main effects in the applied
strategy factor (F(1,18) = 4.485, p = .048, partial η2 = .199) andgender factor (F(1,18) = 4.423, p = .050, partial η2 = .197); we
found no significant interaction (F(1,18) = 2.382, p = .140, partialη2 = .117). These results showed that reference redundancy with
explicit instruction was significantly larger than reference
redundancy with implicit alignment. They also showed that thereference redundancy of females exceeds that of males.
From these results on the amount of information about references
and the reference redundancy of utterances, prediction 1-a ispartially supported, but prediction 1-b is not supported.

utterances for the following two reasons.
First, lexical alignment does not just increase the use of theword contained in the robot’s attributes; it also leads to alignmentof word selection/combination. For example, if a robot uses “blueand B” when it refers to objects, humans tend to use the same
selection/combination of words (color and symbol). Such alignment,
called word selection/combination, is also observed in human-robot
interaction(14)(26). For our experiment, the robot selects words basedon two strategies: implicit alignment and explicit instruction. As
mentioned in Section 5.3, in both strategies, the robot uses a word
combination that can uniquely identify the objects referenced in
the environment. With such lexical alignment, humans tend to use
words that can identify objects. For example, in an environmentthat only has red books, humans rarely use “That red book” as thereference, but instead they say “That red book with a circle and a
B on its cover” because the second reference way clearly identifies
the object in the environment.
Second, objects in environments differ with respect toconversation sessions and participants, and value of the numberof object attributes in a participant’s references depends on theenvironment. For example, the value of “red” as a color attributein an environment that only has red books is much less than its
value in an environment that has red, blue, and yellow books.

7. Discussion

6. Results: Verification of Prediction 1
Fig. 8 shows the results of the information amount of references.
We conducted a two-factor mixed ANOVA for both applied
strategy and gender factors and identified significant main effects

Mean number of objects' attributes

3
2.5
2
1.5
1
0.5
0
Explicit instruction
FemalesFig. 8.

Implicit alignment
Males

Information amount of reference

Mean of reference redundacy of utterances

2
1.8
1.6
1.4
1.2
1
0.8
0.6
0.4
0.2
0
Explicit instruction
FemalesFig. 9.

Implicit alignment
Males

Reference redundancy of utterance
1630

7.1 Implication
Our experimental results showed thatfemales refer to objects with references that have higher reference
redundancy of utterances than males. The reference redundancy ofutterances reflects how useful references are for identifying objects,
and our experimental results suggest that robots need to change
their interaction strategies for effective alignment with humanreferences to useful references to identify objects in object reference
conversations. For example, when a robot uses the implicit
alignment strategy, the reference redundancy of male utterances isrelatively lower than that of females, and therefore robots shouldchoose the explicit instruction strategy to obtain useful information
to identify objects from males. Since the overall conversation
impressions did not differ by gender or applied strategy factors,
explicit instruction to males by robots would have fewer
disadvantages. For these reasons, considering gender effects on
lexical alignment is important for designing conversation strategies
for robots. Our findings might be integrated not only for objectreference conversation contexts but also for other conversation
contexts, since lexical alignment is not a phenomenon that is only
observed in object reference conversations.
For the information amount, there were no gender effects on
lexical alignment, although there were gender differences for thereference redundancy of utterances. This result suggests that eventhough males aligned as many words as females, they aligned withfewer word combinations that can uniquely identify the referenced
object in the environment than females. Namy et al.(8) found thatin shadowing tasks, females were vocally more likely to align thanmales. They discussed why females aligned more than males, andposited females might be more sensitive to the indexical featuresof interlocutors. If sensitivity to conversational features differs bygender based on their discussions, the difference of sensitivity
might explain the discrepancy of our results between information
amount and reference redundancy.
We found significant main effects in the gender factor about
the reference redundancy of utterances (F(1,18) = 4.423, p = .050,
partial η2 = .197). Although the interpretation of effect size varies
IEEJ Trans. EIS, Vol.137, No.12, 2017

Gender Effects on Lexical Alignment in Human-Robot Interaction (Mitsuhiko Kimoto et al.)environment. We believe that our findings of the female-dominant
effects on lexical alignment in human-robot interaction will help
robotics researchers design conversation strategies between
humans and robots.
Acknowledgements
This research work was supported by JSPS KAKENHI Grant
Numbers JP15H05322, JP16K12505, and JP15K16075.

Mean of overall conversation impressions

7
6
5
4
3
2

References

1
Explicit instruction
FemalesFig. 10.

(1)

Implicit alignment
Males

Overall conversation impression(2)by experiments, Cohen(27) offers standard interpretation of partial
η2 as benchmarks: small, medium and large effects would bereflected in partial η2 = .0099, .0588 and .1379 respectively.
Compared to the Cohen’s benchmark, the gender factor has large
effects on the reference redundancy of utterances.
7.2 Conversation Impressions
To investigate theparticipant’s impressions of the conversations, we measured aquestionnaire item, overall conversation impression of the robot,
and evaluated it on a 1-to-7 point scale. Fig. 10 shows thequestionnaire results about the overall conversation impressions.
We conducted a two-factor mixed ANOVA for both factors, applied
strategy and gender, and found no significance in the applied
strategy factor (F(1,18) = 2.751, p = .115, partial η2 = .133), no
significance in the gender factor (F(1,18) = 1.254, p = .278, partialη2 = .065) and no significant interaction (F(1,18) = .306, p = .587
partial η2 = .017).
These results show that overall conversation impressions did
not differ based on the applied strategies and genders.
7.3 Limitations
We conducted our experiment’s study
in a limited situation. The participants referred to objects with only
three features: color, a symbol, and a letter. In real environments,
the features of objects are not limited and obviously influence thereference ways. But since the interaction manner between a robot
and an interlocutor does not depend on features, our findings are
general for other objects.
Since our experiment was conducted with an existing robot
named Robovie-R ver.2, robot generality is also limited. Some past
research works on gender effects on lexical alignment investigatedfrom the viewpoint of gender pairs(7)(9)(15). Robovie-R ver.2 and its
synthesized speech have no intended gender. If a robot and/or its
speech are designed to represent a specific gender, the robot’s
gender will undoubtedly also influence lexical alignment inconversations.

(3)
(4)

(5)

(6)
(7)

(8)

(9)

(10)

(11)

(12)

(13)

(14)

(15)

8. Conclusion
We investigated gender effects on lexical alignment in objectreference conversation contexts between humans and robots byemploying two interaction strategies based on related works:
implicit alignment and explicit instruction. We developed a systemthat recognized the indicated objects and did object reference
conversations with humans. Experimental results indicated thatfemales lexically align more with a robot interlocutor than males
in terms of reference redundancy of utterances. Our female
participants aligned more than males and used more referencesthat are useful to uniquely identify referenced objects in the

(16)

(17)

(18)
(19)

1631

M. Kimoto, T. Iio, M. Shiomi, I. Tanev, K. Shimohara, and N. Hagita :“Relationship between Personality and Robots' Interaction Strategies in
Object Reference Conversations”, The Second International Conference on
Electronics and Software Science, pp.128-136, Takamatsu, Japan (2016)
S. E. Brennan and H. H. Clark : “Conceptual pacts and lexical choice inconversation”, Journal of Experimental Psychology: Learning, Memory,
and Cognition, Vol.22, No.6, pp.1482-1493 (1996)
S. E. Brennan : “Lexical entrainment in spontaneous dialog”, 1996
International Symposium on Spoken Dialogue, pp.41-44 (1996)
A. Nenkova, A. Gravano, and J. Hirschberg : “High frequency word
entrainment in spoken dialogue”, The 46th Annual Meeting of the Association
for Computational Linguistics on Human Language Technologies: Short
Papers, pp.169-172, Columbus, Ohio, USA (2008)
C. C. Lee, M. Black, A. Katsamanis, A. C. Lammert, B. R. Baucom, A.
Christensen, P. G. Georgiou, and S. S. Narayanan : “Quantification of prosodic
entrainment in affective spontaneous spoken interactions of married couples”,
INTERSPEECH 2010, pp.793-796 (2010)
M. J. Pickering and S. Garrod : “Toward a mechanistic psychology of
dialogue”, Behavioral and Brain Sciences, Vol.27, No.2, pp.169-190 (2004)
R. Levitan, A. Gravano, L. Willson, S. Benus, J. Hirschberg, and A. Nenkova :“Acoustic-prosodic entrainment and social behavior”, The 2012 Conferenceof the North American Chapter of the Association for Computational
Linguistics: Human Language Technologies, pp.11-19, Montréal, Canada(2012)
L. L. Namy, L. C. Nygaard, and D. Sauerteig : “Gender Differences in
Vocal Accommodation”, Journal of Language and Social Psychology,
Vol.21, No.4, pp.422-432 (2002)
J. S. Pardo : “On phonetic convergence during conversational interaction”,
The Journal of the Acoustical Society of America, Vol.119, No.4,
pp.2382-2393 (2006)
J. Pearson, J. Hu, H. P. Branigan, M. J. Pickering, and C. I. Nass :“Adaptive language behavior in HCI: how expectations and beliefs about asystem affect users' word choice”, SIGCHI Conference on Human Factors
in Computing Systems 2006, pp.1177-1180, Montréal, Canada (2006)
H. P. Branigan, M. J. Pickering, J. Pearson, and J. F. McLean : “Linguisticalignment between people and computers”, Journal of Pragmatics, Vol.42,
No.9, pp.2355-2368 (2010)
H. P. Branigan, M. J. Pickering, J. Pearson, J. F. McLean, and C. Nass :“Syntactic alignment between computers and people: The role of belief
about mental states”, 25th Annual Conference of the Cognitive Science
Society, pp.186-191 (2003)
T. Iio, M. Shiomi, K. Shinozawa, T. Akimoto, K. Shimohara, and N. Hagita :“Investigating Entrainment of People’s Pointing Gestures by Robot’s
Gestures Using a WOZ Method”, International Journal of Social Robotics,
Vol.3, No.4, pp.405-414 (2011)
T. Iio, M. Shiomi, K. Shinozawa, K. Shimohara, M. Miki, and N. Hagita :“Lexical Entrainment in Human Robot Interaction”, International Journalof Social Robotics, Vol.7, No.2, pp.253-263 (2015)
E. Strupka, O. Niebuhr, and K. Fischer : “Influence of Robot Gender andSpeaker Gender on Prosodic Entrainment in HRI”, Interactive Session at2016 25th IEEE International Symposium on Robot and Human Interactive
Communication, New York City, USA (2016)
M. Kimoto, T. Iio, M. Shiomi, I. Tanev, K. Shimohara, and N. Hagita :“Improvement of object reference recognition through human robotalignment”, 2015 24th IEEE International Symposium on Robot and Human
Interactive Communication, pp.337-342 (2015)
J. Thomason, H. V. Nguyen, and D. Litman : Prosodic Entrainment andTutoring Dialogue Success, In: H. C. Lane, K. Yacef, J. Mostow, and P.
Pavlik, editors, Artificial Intelligence in Education, pp.750-753, Springer
Berlin Heidelberg, Berlin, Heidelberg (2013)
J. A. Hall : “Gender effects in decoding nonverbal cues”, Psychologicalbulletin, Vol.85, No.4, pp.845-857 (1978)
J. S. Hyde : “Gender Similarities and Differences”, Annual Review of
Psychology, Vol.65, pp.373-398 (2014)

IEEJ Trans. EIS, Vol.137, No.12, 2017

Gender Effects on Lexical Alignment in Human-Robot Interaction (Mitsuhiko Kimoto et al.)(20) M. Kimoto, T. Iio, M. Shiomi, I. Tanev, K. Shimohara, and N. Hagita :“Alignment Approach Comparison between Implicit and Explicit Suggestions
in Object Reference Conversations”, The Fourth International Conferenceon Human Agent Interaction, pp.193-200, Biopolis, Singapore (2016)(21) Z. Xia, R. Levitan, and J. Hirschberg : “Prosodic entrainment in Mandarin
and English: A cross-linguistic comparison”, 7th Speech Prosody (2014)(22) J. Holler and K. Wilkin : “Co-Speech Gesture Mimicry in the Process ofCollaborative Referring During Face-to-Face Dialogue”, Journal of Nonverbal
Behavior, Vol.35, No.2, pp.133-153 (2011)(23) H. Kawai, T. Toda, J. Ni, M. Tsuzaki, and K. Tokuda : “XIMERA: A new
TTS from ATR based on corpus-based technologies”, Fifth ISCA Workshop
on Speech Synthesis (2004)(24) A. Lee, T. Kawahara, and K. Shikano : “Julius — an open source real-timelarge vocabulary recognition engine”, 7th European Conference on SpeechCommunication and Technology, pp.1691-1694, Aalborg, Denmark (2001)(25) S. J. Ryan, A. P. Schachat, C. P. Wilkinson, D. R. Hinton, S. V. R. Sadda,
and P. Wiedemann : Retina, Elsevier Health Sciences (2012)(26) S. Kazuhiko, M. Takahiro, K. Masayuki, and N. Hagita : “User specification
method and humanoid confirmation behavior”, 2007 7th IEEE-RAS
International Conference on Humanoid Robots, pp.366-370 (2007)(27) J. Cohen : Statistical power analysis for the behavioral sciences, Academic
Press, New York (1969)

Mitsuhiko Kimoto(Non-member) received his M. Eng. degree from
Doshisha University, Kyoto, Japan, in 2016. He iscurrently a Ph.D. student at Doshisha University anda Student Intern at the Intelligent Robotics andCommunication Laboratories (IRC), Advanced
Telecommunications Research Institute International(ATR), Kyoto, Japan. His research interests include
human-robot interaction.

Takamasa Iio(Non-member) received his M. Eng. and Ph.D.
degree from Doshisha University, Kyoto, Japan, in2009 and 2012. He is currently an assistant professor
at Osaka University, Osaka, Japan. His researchinterests include human-robot interaction, groupconversation between humans and multiple robots
and social behaviors of robots.

Ivan Tanev(Non-member) was born in 1964 in Simeonovgrad,
Bulgaria. He earned M.S. (with honors) and Ph.D.
degrees from Saint-Petersburg State Electrotechnical
University, Russia in 1987 and 1993 respectively,
and Dr.Eng. degree from Muroran Institute ofTechnology, Japan in 2001. He has been with the
Bulgarian Space Research Institute (1987), Bulgarian
Central Institute of Computer Engineering andComputer Technologies (1988-1989), Bulgarian National Electricity
Company (1994-1997), Synthetic Planning Industry Co. Ltd., Japan(2001-2002), and ATR Human Information Science Laboratories(2002-2004), Japan. Since April 2013 he has been a professor at Doshisha
University, Japan. Dr. Tanev’s research interests include evolutionary
computations, evolutionary robotics and multi-agent systems.
Katsunori Shimohara (Member) received the B.E. and M.E. degrees in
Computer Science and Communication Engineeringand the Doctor of Engineering degree from Kyushu
University, Fukuoka, Japan, in 1976, 1978 and 2000,
respectively. He was Director of the Network
Informatics Laboratories and the Human
Information Science Laboratories, Advanced
Telecommunications Research Institute (ATR)
International, Kyoto, Japan. He is currently a Professor at the Department
of Information Systems Design, Faculty of Science and Engineering, ant
the Graduate School of Science and Engineering, Doshisha University,
Kyoto, Japan. His research interests include human communication
mechanisms, evolutionary systems, human-system interactions, andsocio-informatics.
Norihiro Hagita(Non-member) received B.E., M.E., and Ph.D.
degrees in electrical engineering from Keio
University in 1976, 1978, and 1986. In 1978, hejoined Nippon Telegraph and Telephone Public
Corporation (now NTT), where he developed
handwritten character recognition. He was a visiting
researcher in the Department of Psychology,
University of California, Berkeley from 1989-90. He
is currently a Board Director of ATR and an ATR Fellow, a director of theSocial Media Research Laboratory Group and Intelligent Robotics andCommunication laboratories. His research interests include networked
robotics, human-robot interaction, pattern recognition, and data-mining
technology. He is a member of IEEE, the Information Processing Societyof Japan, and the Japanese Society for Artificial Intelligence, The Instituteof Electronics, Information and Communication Engineers (IEICE) and
The Robotics Society of Japan.

Masahiro Shiomi(Non-member) received M. Eng. and Ph.D. degrees
in engineering from Osaka University in 2004 and2007. From 2004 to 2007, he was an intern researcher
at the Intelligent Robotics and Communication
Laboratories (IRC). He is currently a group leader inthe Agent Interaction Design department at IRC,
Advanced Telecommunications Research InstituteInternational (ATR). His research interests include
human-robot interaction, robotics for child-care, networked robots, andfield trials.

1632

IEEJ Trans. EIS, Vol.137, No.12, 2017