INTERSPEECH 2011

Evaluation of Listening-oriented Dialogue Control Rules
based on the Analysis of HMMs
Toyomi Meguro†, Yasuhiro Minami†, Ryuichiro Higashinaka‡, Kohji Dohsaka†† NTT Communication Science Laboratories, NTT Corporation‡ NTT Cyber Space Laboratories, NTT Corporation{meguro.toyomi, minami.yasuhiro, higashinaka.ryuichiro, dohsaka.kohji}@lab.ntt.co.jp

Abstract㽲

We have been working on listening-oriented dialogues for the
purpose of building listening agents. In our previous work[1], we trained hidden Markov models (HMMs) from listeningoriented dialogues (LoDs) between humans, and by analyzing
them, discovered a distinguishing dialogue ﬂow of LoD. For
example, listeners suppress their information giving and selfdisclosure, and instead, increase acknowledgments and questions to elicit speakers’ utterances. As an initial step for building listening agents, we decided to create dialogue control rules
based on our analysis of the HMMs. We built our rule-based
system and compared it with three other systems by a Wizard ofOz (WoZ) experiment. As a result, we found that our rule-based
system achieved as much user satisfaction as human listeners.
Index Terms: Listening-oriented dialogue, Dialogue system,
Wizard of Oz㽴㽶>͗'ƌĞĞƚŝŶŐ͗Ϭ͘ϴϴϴ
0.47>͗^ĞůĨͲĚŝƐĐůŽƐƵƌĞ͗Ϭ͘ϰϰϱ>͗YƵĞƐƚŝŽŶ͗Ϭ͘ϰϵϮ
0.18

0.63

>͗^ĞůĨͲĚŝƐĐůŽƐƵƌĞ͗Ϭ͘ϱϱϲ>͗ĐŬ͗Ϭ͘Ϯϳ

㽳

0.37

0.43

0.56

^͗'ƌĞĞƚŝŶŐ͗Ϭ͘ϵϴ
0.11

㽵^͗^ĞůĨͲĚŝƐĐůŽƐƵƌĞ͗Ϭ͘ϴϯϱ

0.38

㽷

0.51

0.92

^͗^ĞůĨͲĚŝƐĐůŽƐƵƌĞ͗Ϭ͘ϭϮϱ^͗ĐŬ͗Ϭ͘ϲϲϭ

0.25

Figure 1: HMM trained from LoDs between humans [1].
• It is safe in terms of system development because a completely wrong action cannot be chosen.
• It is easy to make a system perform certain actions, to modify system actions, and to maintain the system.

1. Introduction

The disadvantage may be that it is often difﬁcult to write down
rules for all possible dialogue states, which may make system
actions seem rigid and inﬂexible, although this can be overcome
by adopting stochastic models with large dialogue data.
Acknowledging that both handcrafted rules and stochastic
models have advantages and disadvantages, we weigh highly
the advantages of handcrafted rules and build our prototype system using handcrafted rules, although we have also begun working on using stochastic models [6]. To verify the effectivenessof our rules, we create three other systems and compare themwith our rule-based prototype by a Wizard of Oz (WoZ) experiment.
Note that this paper deals with the dialogue control moduleof a listening agent, which receives the user’s dialogue act (i.e.,
meaning representation of an utterance) as input, and outputs
the system’s next dialogue act. The module does not deal withsurface utterances. Although other modules such as language
understanding and language generation are important, we consider dialogue control to be the most important module for listening agents because it decides the ﬂow of dialogue and that,
according to our analysis, is what characterizes LoDs.

We have been working on listening-oriented dialogue (LoD) in
which one conversational participant listens attentively to theother (see Fig. 5 for a typical LoD). Our aim is to build listening
agents that can implement a listening process in which users cansatisfy their desire to speak and have themselves heard.
For this purpose, we have been analyzing LoDs conducted
between humans by comparing them with casual conversationin which conversational participants have no predeﬁned roles.
For this analysis, we used hidden Markov models (HMMs) tomodel LoDs and casual conversation, and compared state transitions that represent dialogue ﬂows. Figure 1 shows an example
of a trained HMM. By examining such transitions, we foundthat LoD and casual conversation have signiﬁcantly different
dialogue ﬂows. For example, in LoD:
• Listeners ask questions actively with a frequent insertionof self-disclosure, which seems in concordance with social
penetration theory [2], which states that one needs to selfdisclose to prompt others to speak about themselves.
• Listeners suppress information giving and self-disclosure,and instead, increase acknowledgments and questions toelicit speakers’ utterances.

2. Related work

Since such insights would help us build automated listening
agents, we decided to encode them as dialogue control rules andbuild a dialogue control module. Here, we create rules by handbecause this is a typical and basic way of building less taskoriented dialogue systems [3, 4, 5], and handcrafted rules can be
ﬂexible at the initial stage of research. In addition, handcrafted
rules have the following advantages:

There is emerging work on building listening agents. Shitaoka
et al. [3] investigated the functions of listening agents, focusing especially on their response generation component. Theirsystem takes the conﬁdence score of speech recognition into
account and changes the system response accordingly; that is,
it repeats the user utterance or utters an empathic utterance forhigh-conﬁdence user utterances, and it makes a back-channel
when the conﬁdence is low. Here, the system’s emphatic utterances can be “I’m happy” or “It’s a pity”, depending on whether
a positive or negative expression is included in user utterances.

• It is not necessary to collect a large number of dialogues
generally needed for training stochastic models. Note thatthe state space can be very large for less task-oriented dialogues.

Copyright © 2011 ISCA

0.42

809

28- 31 August 2011, Florence, Italy

Their system’s response generation only uses the speech recognition conﬁdence and the polarity of user utterances as cues tochoose its actions. Currently, their system does not take into
account the content of an utterance, such as dialogue acts.
To create listening agents that achieve high smoothness,a switching mechanism between “active listening mode”, inwhich a system is a listener and actively utters back-channelingutterances, and “topic presenting mode”, in which the system
is a speaker, has been proposed [4, 5]. Here, the system uses
a heuristic function to maintain a high user interest level andto keep the system in the active listening mode. Dialogue control is done by handcrafted rules. Our motivation is similar totheirs, but differs in that we want to build a listening agent thatgives the user a sense of being heard not only by maintaining theuser’s interest by back-channeling but also by various linguistic
actions.
Maatman et al. [7] have also been working on building
listening agents, but they focus only on non-linguistic actions,such as gestures. We consider linguistic actions to be equallyimportant in LoD because they concern the content of a dialogue.

Table 1: Deﬁnitions of dialogue acts with sample generation
templates. The templates were used by a WoZ experimenter togenerate utterances for the dialogue acts. The templates wereadapted for the topic of “food” for our experiment. Except for
REPEAT , PARAPHRASE and OTHER , each dialogue act has several generation templates. Templates are shown in square brackets. (time) can be replaced with lunch, breakfast or dinner and(day) with tomorrow, today etc. (food) is ﬁlled with arbitrary
food by a WoZ experimenter according to a dialogue context.
Dialogue act
G REETING

Deﬁnition and sample generation templates
Greeting and conﬁrmation of a dialogue theme.
e.g.[Hello.],[The theme of dialog is XX.]
Delivery of objective information. [(placeI NFORMATION
name) is famous for (food).]
S ELF - DISCLOSURE Disclosure of one’s preferences and feelings.
[I ate (food) for (time) (day).]
sub: fact[I have eaten (food).]sub: experience
sub: habit[I always go out to eat.][I like (food).]sub: preference[ (food) is delicious.](positive)[I don’t like (food).]sub: preference[(food) is not delicious.](negative)[(food) is (adjective: either positive
sub: preferenceor negative).](neutral)[I want to eat (food) for (time) (day).]
sub: desire[I will eat (food) for (time) (day).]
sub: planA CKNOWLEDGMENT Encourages the conversational partner to speakby a backchannel. [Well.] [Aha.]
Utterances that expect answers.
Q UESTION[Please tell me about (food).]sub: information[What did you eat for (time) (day)?]
sub: fact[Have you eaten (food) before?]
sub: experience[Do you usually go out to eat?]sub: habit[Do you like (food)?]sub: preference[What do you want to eat for (time) (day)?]
sub: desire[What will you eat for (time) (day)?]
sub: plan
Sympathetic utterances and praises. [Me, too]S YMPATHY
N ON - SYMPATHY
Negative utterances. [I don’t think so.]Conﬁrm what the conversation partner said.
C ONFIRMATION[Really?]
Encourage the partner to act. [Please eat
P ROPOSAL(food).]
Repeat the partner’s most recent utterance.
R EPEAT
Paraphrase the partner’s most recent utterance.
PARAPHRASE
Bring up or show goodwill toward the partner.
A PPROVAL[Absolutely!]
Express one’s thanks [Thank you.]
T HANKS
Express one’s regret [I’m sorry.]
A POLOGY
Filler between utterances. [Uh.],[Let me see.]F ILLER
Express one’s affection. [A-ha-ha.]
A DMIRATIONOther utterances.
O THER

3. Rule-based LoD control module
We made our dialogue control rules based on the analysis of
our HMMs [1]. The rules were made to output dialogue acts
in response to user dialogue acts. Table 1 lists the dialogue
acts that can be exchanged between a listening agent and a user.
The table also shows generation templates corresponding to the
dialogue acts that were used by WoZ experimenters to create
utterances. Note that, to enable more ﬁne-grained exchanges,
the dialogue act set here has been extended from when we performed our analysis [1].
Figure 2 shows all of the rules that we created. One dialogue act corresponds to one sentence and a user can utter multiple sentences in one utterance. The rules respond to the last
dialogue act of the user utterance and generate the next system
dialogue act. The rules basically encode the results of our analysis although we needed to insert some rules using insight fromprevious studies. In the rules, after a user’s self-disclosure, thesystem self-discloses and then asks a question. This is because,in Fig. 1, the typical ﬂow of LoD is that when the speaker selfdiscloses, the listener ﬁrst self-discloses and then asks a question. Also, the rules sparingly output INFORMATION, which isonly returned when requested by the user or in accordance withthe user’s information giving.
On the basis of previous studies, the rules return the selfdisclosure with the same sub-tag for a question, which is reasonable in cooperative dialogue. The rules also have some mirroring actions [8]; for example, for a user’s sympathy, the system
also sympathizes with the user. Also, in the rules, SYMPATHY
is output or inserted frequently. For example, after two or moreuser’s actions of REPEAT , PARAPHRASE , etc., the system sympathizes with the user. This is because, in previous research [9],sympathy has been found effective for keeping the user engaged
in conversation. For active listening practice, paraphrasing andrepeating have been found effective [10] because they show ahigh level of understanding by the listener. Therefore, after
two or more user’s actions of INFORMATION, APOLOGY, CON FIRMATION , etc., we made the rules return REPEAT or PARA PHRASE .

systems in all. Four of the systems were those that we wanted
to compare: one with our rules and three other systems for comparison; and the other three systems were the experimental systems that we wanted to test for future development. We only
deal with the former four systems and discuss their results inthis report, but note that we made sure that each of the foursystems was evaluated under the same condition in terms of experimental order. We describe our four systems in detail in thenext section. The WoZ experimenters did not have any knowledge about the systems except that they needed to convert user
utterances into dialogue acts, input them to the system, receive
system dialogue acts, and ﬁnally generate utterances from thesystem dialogue acts to communicate with the user. Here, generation is done using the generation templates (see Fig. 2). The
topic of all dialogues was “food”.
We recruited 18 participants (nine males; nine females) as
speakers and three participants (two females; one male) as wizards (i.e. listeners). The participants did not include the authors.
The WoZ experimenters and the participants were kept in different rooms and they only used text to communicate. Eachparticipant talked with two WoZ experimenters. Each partici-

4. Experiment4.1. Experimental setup
To verify the effectiveness of our rules, we performed a WoZ
experiment. In the experiment, the wizards manipulated seven810

function generate action(user action) returns system action
num ← random(1..3)
system action ← {}if user action = G REETING thenreturn G REETING × num
else if user action = INFORMATION thensystem action ← SYMPATHY
system action ← INFORMATION × num
return system action
else if user action = S - DISC thensystem action ← user action × num
system action ← QUESTION(sub:(random.sub))return system action
else if user action = ACKNOWLEDGMENT thenreturn S - DISC(sub: fact) × num
else if user action = QUESTION thenif user action.sub = information thenreturn INFORMATION × num
else if user action.sub = preference thenreturn S - DISC(sub:pref+) × num
else
return S - DISC (sub: user action.sub) × num
else if user action = SYMPATHY or NON - SYMPATHY thensystem action ← REPEAT
system action ← user action × num
return system actionelse
if user action = REPEAT or PARAPHRASE
or THANKS or FILLER thensystem action ← SYMPATHY
else if user action = PROPOSAL or APPROVAL thensystem action ← THANKS
else if user action = APOLOGY or OTHER thensystem action ← REPEAT
else if user action = CONFIRMATION thensystem action ← PARAPHRASE
else if user action = ADMIRATION thensystem action ← ADMIRATION
system action ← S - DISC (sub: fact) × num
return system actionend

㽲 >͗'ƌĞĞƚŝŶŐ͗Ϭ͘ϴϭϮ

Ϭ͘ϯϮ
Ϭ͘ϲϲ

㽳 ^͗'ƌĞĞƚŝŶŐ͗Ϭ͘ϴϵϴϬ͘ϯ㽴>͗^ͲŝƐĐ;ƐƵď͗ĨĂĐƚͿ͗Ϭ͘ϭϭϵ>͗^ͲŝƐĐ;ƐƵď͗ƉƌĞĨнͿ͗Ϭ͘ϭϯϰ>͗YƵĞƐƚŝŽŶ;ƐƵď͗ĨĂĐƚͿ͗Ϭ͘ϭϯϵ
Ϭ͘ϱϴ

Ϭ͘ϯϭ

Ϭ͘ϱϲ

㽵 ^͗^ͲŝƐĐ;ƐƵď͗ĨĂĐƚͿ͗Ϭ͘Ϯϭϳ^͗/ŶĨŽƌŵĂƚŝŽŶ͗Ϭ͘ϭϯϲ^͗^ͲŝƐĐ;ƐƵď͗ƉƌĞĨнͿ͗Ϭ͘ϭϯϵ

Ϭ͘ϰϵ

Ϭ͘ϲϴ

㽷 ^͗^ͲŝƐĐ;ƐƵď͗ĨĂĐƚͿ͗Ϭ͘ϭϮϯ^͗^ǇŵƉĂƚŚǇ͗Ϭ͘ϯϮϱ

㽶 >͗^ͲŝƐĐ;ƐƵď͗ƉƌĞĨнͿ͗Ϭ͘ϭϭϯ>͗^ǇŵƉŚĂƚŚǇ͗Ϭ͘ϮϮϰ

6
2

3

4

5

All
WoZ:A
WoZ:B
WoZ:C

1

Averaged Satisfaction Rate

Figure 3: HMM used for HMM System. For brevity, only output probabilities over 0.1 and transition probabilities over 0.3
are shown.

Human

Rule

RandomHMM

Figure 4: Averaged user satisfaction ratings.
logue acts by greedily choosing the most probable dialogue act
at each step. The number of states for the HMM was 16. Figure
3 shows the graphical representation of the HMM we used. The
ﬁgure shows only six states to make it easy to understand the
dialogue ﬂow.
4.2.3. Human dialogue control

Figure 2: Our dialogue control rules written as a function. S DISC is shorthand for SELF - DISCLOSURE and PREF + for preference positive. “←” means adding a dialogue act to a list, “×num” a repetition of a preceding dialogue act “num” times, and“random.sub” means a randomly chosen sub tag.

In this system, the WoZ experimenters choose the system dialogue acts (both the dialogue acts and the number of dialogueacts) at their discretion. We regard this system as an upperbound. We call this system the Human system.
4.2.4. Random dialogue control

pant talked with a WoZ using the four different systems, whichyielded eight dialogues per participant. Each dialogue lasted approximately 15 minutes. After each dialogue, the participants
ﬁlled out questionnaires to rate their user satisfaction level byanswering the question, “Did you feel that you were listened
to?” on a 7-point Likert scale.

This system chooses the system dialogue acts randomly. The
number of dialogue acts to be output is also random from one
to three. We call this system the Random system.
4.3. Experimental results
Figure 4 shows the averaged user satisfaction ratings for thefour systems. Except between the Human system and Rulebased system, there are signiﬁcant differences between all systems (p<0.01) by a statistical test (t-test). Surprisingly, the user
satisfaction rating of the Rule-based system almost equaled thatof the Human System, and it greatly outperformed the HMM
system, the stochastic model. In addition, the Rule-based system worked preferably for a person who is not a good listener;
WoZ:B’s score of the Human system was not that high compared to other WoZ experimenters, but WoZ:B’s score of theRule-based system was higher than the Human system and comparable to other WoZ experimenters, suggesting that our rules
could be used as possible aids for listeners.
Figures 5 and 6 show the example dialogues conducted using the Rule-based system and HMM system. In the dialogueof the Rule-based system, WoZ experimenters asked questions
after their self-disclosures, which closely follows our analysis,
and thus obtained high user satisfaction ratings. On the other
hand, the HMM system always chose the same action, SELF -4.2. Systems
4.2.1. Rule-based dialogue control
This system is our proposed system and uses the rules in Fig. 2.
We call this system the Rule-based system.
4.2.2. HMM dialogue control
This system uses for dialogue control an HMM trained from
dialogue-act annotated LoD data between humans that we collected [6]. The annotation was done using the tag set in Table
1. The data include 1260 LoDs. We have this system to compare our rules with a stochastic model. The system (hereafter,
HMM system) chooses the most probable dialogue act at eachstep. The system makes a prediction as to whether it should
continue its utterance or pass the turn to the user. If the mostprobable action at the next step is that of the user, the systemstops generating dialogue acts. If the most probable dialogue
act is that of the system, the system continues to generate dia-811

Utterance
Dialogue act
L: Nice to meet you.
G REETING
S: Nice to meet you, too.
G REETING
L: The theme of dialogue is “Most impres- G REETING
sive food”.
S: OK.
SYMPATHY
L: OK.
R EPEATS: My most impressive food is the spit- S-D ISC(sub: fact)roasted silkworms at a Chinese restaurant
in Ikebukuro.
L: I ate Chinese food last night.
S-D ISC(sub: fact)
I ate it in Ikebukuro.
S-D ISC(sub: fact)

Table 2: Kullback-Leibler divergence between HMMs.
Human-human LoD [6]Human System

may be attributable to the templates used by the WoZ experimenters; the templates could have restricted the experimenters’
freedom slightly in the Human system.

5. Conclusions and Future work
In this paper, we built a dialogue control module for a listening agent using handcrafted rules based on our analysis of the
HMMs trained from human-human LoDs. Then, we evaluatedthe rules by comparing them with three other systems by a WoZ
experiment. As a result, we found that our Rule-based system
performed as well as the Human system. We also found thatthe HMM-based system is worse than other systems because it
repeatedly generated the same dialogue acts that appeared frequently in the training data, showing some limitation of stochastic models.
As future work, we plan to continue improving and augmenting our rules. However, it would be difﬁcult to write down
rules for all possible dialogue states. For this reason, we havealready started examining a method to apply partially observable Markov decision processes to automate dialogue control
from data [6]. Whether rule-based or statistic-based, we would
like to investigate ways to create listening agents that maximize
user satisfaction.

Figure 5: Excerpt of LoD using the Human system. S stands
for speaker, and L for listener. The dialogue was originally inJapanese and was translated by the authors.
Utterance
L: Nice to meet you.
S: Nice to meet you, too.
L: The theme of dialogue is “Today’s
food”.
S: Yes, I ate spaghetti.
L: The theme of dialogue is “Today’s
food”.
S: Yes, I understand.
L: I ate grilled ﬁsh for dinner yesterday.
S: It sounds delicious.
L: I can cook grilled ﬁsh.

Human HMM Rule Random
0.9
4.4 2.8
2.4
N/A
4.7 1.4
1.7

Dialogue act
G REETING
G REETING
G REETING
S-D ISC(sub: fact)
G REETINGS YMPATHY
S-D ISC(sub: fact)
S-D ISC(sub: pref+)S-D ISC(sub: fact)

Figure 6: Excerpt of LoD using the HMM system.
DISCLOSURE (sub: fact), repeatedly because it is the most probable. Even if the users showed a variety of dialogue acts, theHMM system chose the same dialogue act and that repetition
made dialogues monotonous, which led to its poor performance.
An appropriate amount of self-disclosure would make social relationships between systems and users stronger. However, if thesystem always self-discloses, this would make users feel thatthey are not being listened to attentively.
The Random system generated all kinds of dialogue acts.
Although there was no tangible dialogue ﬂow, because of therandomness, the dialogue was not monotonous, which probably
led to higher user satisfaction ratings than the HMM system.
To further investigate the quality of dialogues by each system, we calculated the Kullback-Leibler (KL) divergence between (a) an HMM trained from human-human LoDs and (b)HMMs trained from the data of each system; that is, the moresimilar an HMM is to that of an HMM trained from humanhuman dialogue or from the data of the Human system, the better the system should be.
Table 2 shows the KL divergence of HMMs for each system. Low scores mean that the distributions of the HMMs (i.e.,
output distribution of dialogue acts) are similar. The ﬁrst row
shows the divergence between (a) and (b). The second row
shows the KL divergence between an HMM trained from thedata of the Human system and (b). From the ﬁrst row, we cansee that the distribution of human-human data and that of the
Human system are very similar. This suggests that there wereno unrealistic restrictions in the experiment and the participants
could talk as usual, conﬁrming that the dialogues in the WoZ
experiment were realistic. The dialogue distribution of the Random system is closer to that of human-human dialogues than theRule-based system, meaning that the dialogue of the Rule-based
system could still be monotonous and the number of our rules
could still be small. We may need more randomness in the Rulebased system. However, notice that the user satisfaction ratingsof the Random system are low. Therefore, too much randomness would not be effective. Focusing on the second row, thedialogue act distribution of the Rule-based system is more similar to that of the Human system than the Random system. This

6. Acknowledgements
This work was partially supported by a Grant-in-Aid for Scientiﬁc Research on Innovative Areas, “Founding a creative society via collaboration between humans and robots” (21118004),
from MEXT, Japan.

7. References[1] T. Meguro, R. Higashinaka, K. Dohsaka, Y. Minami, and
H. Isozaki, “Analysis of listening-oriented dialogue for building
listening agents,” in Proc. SIGDIAL, 2009, pp. 124–127.
[2] I. Altman and D. Taylor, Social Penetration: The Development ofInterpersonal Relationships. Holt Rinehart and Winston, 1973.
[3] K. Shitaoka, R. Tokuhisa, T. Yoshimura, H. Hoshino, andN. Watanabe, “Active listening system for dialogue robot,” inProc. SIG-SLUD, JSAI, vol. 58, 2010, pp. 61–66, (in Japanese).
[4] S. Yokoyama, D. Yamamoto, Y. Kobayashi, and M. Doi, “Development of dialogue interface for elderly people—switching the
topic presenting mode and the attentive listening mode to keep
chatting—,” in IPSJ SIG Technical Report, vol. 2010-SLP-80,
no. 4, 2010, pp. 1–6, (in Japanese).
[5] Y. Kobayashi, D. Yamamoto, T. Koga, S. Yokoyama, and M. Doi,
“Design targeting voice interface robot capable of active listening,” in Proc. HRI, 2010, pp. 161–162.
[6] T. Meguro, R. Higashinaka, Y. Minami, and K. Dohsaka, “Controlling listening-oriented dialogue using partially observable
Markov decision processes,” in Proc. COLING, 2010, pp. 761–
769.
[7] R. M. Maatman, J. Gratch, and S. Marsella, “Natural behavior ofa listening agent,” Lecture Notes in Computer Science, vol. 3661,
pp. 25–36, 2005.
[8] R. B. Cialdini, Inﬂuence: Science and Practice. Allyn & Bacon,
2000.
[9] R. Higashinaka, K. Dohsaka, and H. Isozaki, “Effects of selfdisclosure and empathy in human-computer dialogue,” in Proc.
SLT, 2008, pp. 108–112.
[10] G. Buck, Assessing listening. Cambridge University Press, 2001.

812