INTERSPEECH 2011

User Study of Spoken Decision Support System
Teruhisa Misu, Kiyonori Ohtake, Chiori Hori, Hisashi Kawai and Satoshi Nakamura*
National Institute of Information and Communications Technology (NICT), Kyoto, Japanteruhisa.misu@nict.go.jp

Abstract
This paper presents the results of the user evaluation of spoken decision support dialogue systems, which help users selectfrom a set of alternatives. Thus far, we have modeled this decision support dialogue as a partially observable Markov decision
process (POMDP), and optimized its dialogue strategy to maximize the value of the user’s decision. In this paper, we present a
comparative evaluation of the optimized dialogue strategy withseveral baseline strategies, and demonstrate that the optimized
dialogue strategy that was effective in user simulation experiments works well in an evaluation by real users.
Index Terms: Spoken dialogue systems, user modeling,
POMDP, decision support systems

Criteria

Alternatives(choices)p1

p2

1. Cherry
Blossoms

2. Japanese
Garden

v11 …

v12 …

KinkakujiTemple

RyoanjiTemple

p3
3. Easy
Access

䊶䊶䊶䊶䊶

v13 …
NanzenjiTemple

䊶䊶䊶䊶䊶

Figure 1: Hierarchy structure for sightseeing guidance dialogue

2. Spoken Decision Support Dialogue
We assume a situation in which a user selects from a given setof alternatives. In the real world this is highly likely, such as
when a user selects a restaurant from a list of candidates presented by a car navigation system. This work considers a sightseeing planning task where the user determines the sightseeing
destination, with little prior knowledge of the target domain.
Consulting dialogues such as these are also regarded as a typeof decision-making problem. That is, the user selects from agiven set of alternatives based on certain criteria. Several studies in the operations research field have addressed decision support systems, with the Analytic Hierarchy Process [5] (AHP)
being the typically employed method. In the AHP, the problem
is modeled as a hierarchy consisting of the decision goal, thealternatives for achieving it, and the criteria for evaluating thesealternatives. For the sightseeing planning task that we focus onin this paper, the goal is to decide on an optimal spot that aligns
with the user’s preference. The alternatives are comprised ofall sightseeing spots that can be proposed and explained by
the system. We adopt the decision criteria defined in our tagging scheme [6]. These include various deciding factors used inplanning sightseeing activities, such as the presence of “cherry
blossoms” or a “Japanese garden.” Figure 1 shows an example
hierarchy using these criteria.
In this model, the user’s problem of making an optimal decision can be solved by fixing a weight vector
Puser = (p1 , p2 , . . . , pM ) for criteria and a local weight matrix Vuser = (v11 , v12 , . . . , v1M , . . . , vN M ) for alternatives
in terms of the criteria. The optimal alternative is then identified by selecting the spot k that maximizes the priority ofM
p v . In typical AHP methods, the procedure of fixm=1 m km
ing these weights is often conducted through pairwise comparisons for all possible combinations of criteria and spots in termsof the criteria, followed by weight tuning based on the resultsof the comparisons [5]. However, the methodology cannot bedirectly applied to spoken dialogue systems. To users, the information about a spot in terms of the criteria is not known, andis obtained only through the system’s information navigation;
thus, it is difficult to evaluate and compare the spots without
this navigation. Spoken dialogue systems also usually handle
several candidates and criteria, which makes pairwise comparison expensive. We therefore consider a spoken dialogue framework that estimates the weights for the user’s preference (poten-

1. Introduction
In many situations where spoken dialogue interfaces are installed, information access by the user is not itself a goal, but
a means for decision-making [1]. For example, in using a
restaurant retrieval system, the user’s goal may not be obtaining price information but instead making a decision based on
the retrieved information about the restaurants. In these situations, users try to extract information from the system thatwill aid their decision-making. Yet users, often unaware not
only of what kind of information the system can provide but
also their own preferences or factors in which they place value,
can ultimately retrieve insufficient information. Systems themselves have little knowledge of the users, or where their interests
lie; thus a system must bridge these gaps by sensing (potential)user preferences and recommending information in which theywould be interested, considering the trade-off with the length ofthe dialogue.
To handle such use cases, we have proposed a user modeland dialogue state representation, which consider user preferences as well as their knowledge about the domain changing through a decision support dialogue by modeling as a partially observable Markov decision process (POMDP). A dialogue strategy for information recommendation was optimized,
and its effectiveness was confirmed by user simulation [2]. Unfortunately, as mentioned in several studies, an improvement in
the simulation environment does not necessarily mean an improvement in a real user experiment [3, 4]. The main point of
this paper is thus to demonstrate that the optimized dialogue
strategy is also effective with real users.
The paper is organized as follows. Section 2 gives an
overview of spoken decision support dialogue. Section 3 covers
an overview of the system and task domain. Section 4 reportson the proposed user model, dialogue state expression and theirevaluation by user simulation. Section 5 gives user study of thesystem and Section 6 concludes the paper.
* currently with Nara Institute of Science and Technology (NAIST)

Copyright © 2011 ISCA

Choose the optimal spot

Goal

797

28- 31 August 2011, Florence, Italy

4. Optimization of dialogue strategy

tial preferences) as well as the user’s knowledge of the domain
through interactions of information retrieval and navigation.

4.1. User modelingWe introduce a user model that consists of a tuple of knowledge vector Kuser , preference vector Puser , and local weightmatrix Vuser . In this paper, for simplicity, a user’s preference
vector or weight for criteria Puser = (p1 , p2 , . . . , pM ) is assumed to consist of binary parameters. That is, if the user isinterested in (or potentially interested in) the criterion m andplaces value on it when making a decision, the preference pm
is set to “1”. Otherwise, it is set to “0”. In order to represent a
state in which the user has potential preferences, we introduce
a knowledge parameter Kuser = (k1 , k2 , . . . , kM ). This parameter represents whether the user perceives that the system
can accommodate his/her preferences. It is also interpreted asa parameter to indicate if the user is aware that he/she is interested in the criteria. km is set to “1” if the user knows that thesystem can handle criterion m and “0” when he/she does not.
For example, the state that the criterion m is the potential preference of a user (but he/she is unaware of that) is represented by(km = 0, pm = 1). km is updated to “1” when the system informs the user of the recommendation through Methods 1, 2, 4,
and 5. A user’s local weight vnm for spot n in terms of criterion
m works as the user’s knowledge on whether the preference pm
is satisfied by visiting the spot n. vnm is set to “1”, when thesystem lets the user know that the evaluation of spots are “1”through recommendation Methods 1 and 2.

3. Decision support system with spoken
dialogue interface
Our dialogue system has two functions: answering users’ information requests and making recommendations. When asked to
explain certain spots or their properties (decision criteria), thesystem provides an explanation in terms of the requested property. After providing the requested information, it then provides
information to aid in making a decision (e.g., instructing what
the system can explain or recommending details on the current
topic that the user might be interested in). Note that the latter is
optimized via reinforcement learning.
Our back-end database consists of 15 sightseeing spots asalternatives and 10 decision criteria described for each spot 1 .
We select decision criteria that frequently appear in our corpus [6]. Our candidate spots are evaluated and annotated interms of these criteria if they apply to them. The value of theevaluation enm is “1” when the spot n applies to the criterion
m and “0” when it does not.
The content of the recommendation is determined by oneof six possible methods. The dialogue act (or action) of systemrecommendation asys consists of a communicative act ca (orrecommendation method) and semantic content sc. The semantic content includes spots and/or criteria, which are determinedby the heuristic rules defined for each method.

4.2. Dialogue state expression
In above section we presented the user state representation. However the problem is that for the system, the state(Puser , Kuser , Vuser ) is not observable, but is only estimatedfrom the interactions with the user. Therefore this model
can be seen as a partially observable Markov decision process(POMDP) [7]. In order to estimate unobservable properties of
the POMDP, we introduce the system’s inferential user knowledge vector Ksys or probability distribution (estimate value)
Ksys = (P r(k1 = 1), P r(k2 = 1), . . . , P r(kM = 1))
and that of preference Psys = (P r(p1 = 1), P r(p2 =1), . . . , P r(pM = 1)). In this work, we do not estimate theweight, because vnm is assumed to be set to “1” only when
the system lets the user know that the evaluation of the criterion m of the spot n is “1” through recommendations, thus
Vsys = Vuser . This consequently means that Vuser is observable.
The dialogue state DS t+1 or estimated user’s dialogue stateof the step t+1 is assumed to be dependent only on the previous
state DS t , as well as the interactions I t = (atsys , atuser ). Thisapproximation is often adopted in spoken dialogue management
systems using a dynamic Bayesian network (DBN) representation [4]. The relation of the parameters used in our model isillustrated as DBN in Figure 2. The estimated user’s preference
Psys is updated when the system observes the interaction I t .
The update is conducted using Bayes’ Theorem, with the previous state DS t as a prior. The posterior of the estimated user’s
knowledge of criteria km is updated to “1” when the systemtells or the user requests the criteria. An example of this update
is illustrated in Figure 3.

1. Recommendation of criteria based on the currently focused on spot (Method 1)
This method is structured on the basis of the user’s current
focus on a particular spot. Specifically, the system selects
several criteria related to the current spot whose evaluationis “1” and presents them to the user.
2. Recommendation of spots based on the currently focused on criterion (Method 2)
This method functions on the basis of focus on a certain criterion.
3. Open prompt (Method 3)
The system does not make a recommendation, and presents
an open prompt.
4. Listing of criteria 1 (Method 4)This method lists several decision criteria to inform the userof the criteria that the system can handle, regardless of thecurrent focus spot and criterion. The system lists the criteria
in ascending order starting from what it estimates as lowlevel.
5. Listing of criteria 2 (Method 5)
In this method, the system lists in the order of what it estimates as user’s higher preference.
6. Recommendation of user’s possibly preferred spot
(Method 6)
The system recommends spots that the users would be interested in based on the estimated preferences. It selects several spots that matches to the user’s preferences based on
the estimated user preferences. This collaborative filteringlike method will be helpful to users if the system successfully estimates the user’s preference; but it irrelevant if it
does not.

4.3. Reward function
The reward function that we use is based on the number of attributes agreed upon between the user preference and the decided spot. The reward R is then calculated based on the improvement in the number of agreed attributes between the user’s
actual (potential) preferences and the decided spot k over the

1 The number of alternatives is small compared to systems dealing

with information retrieval, but note that this work focuses on the processof comparing and evaluating candidates that meet “essential condition”(e.g., Famous temple easily accessible on foot from Kyoto station).

798

User
unobservablevaluables
for thesystemSystemUser state

Vuser

a userDS t+1

Vuser

scuser

Puser

Kuser

causer

Kuser

Ksys

ca sys

Psys

scsys
asys

Table 1: Evaluation by user simulation

User state

Puser

Estimated
state



DSt

decision

Ksys
Psys
Estimated
state

Figure 2: Dynamic Bayesian network of the model

Priors of the estimated state:- Knowledge: Ksys = (0.22, 0.31, 0.02, 0.18, . . .)- Preference: Psys = (0.37, 0.19, 0.48, 0.38, . . .)Interactions (observation):- System recommendation:asys = M ethod1{(Spot5 ), (Det1 , Det3 , Det4)}“Japanese garden (Det1 ), World heritage (Det3 ) andfall foliage (Det4 ) are some of the areas about which information is available on Ninnaji (Spot5 ).”- User query: auser = Accept{(Spot5 ), (Det3 )}“Tell me about world heritage sites. (Det3 )”Posterior of the estimated state:- Knowledge: Ksys = (1.00, 0.31, 1.00, 1.00, . . .)- Preference: Psys = (0.26, 0.19, 0.65, 0.22, . . .)Strategy
optimized
B1
B2

T=5
0.43 (0.59)
0.22 (0.52)
0.02 (0.41)

Reward (±std)
T = 10
T = 150.68 (0.64)
0.84 (0.62)
0.46 (0.63)
0.65 (0.63)
0.13 (0.55)
0.26 (0.58)

derstanding (SLU) errors [8], assuming that the semantic content is deleted at 16.7%, which is the probability that the system
was not able to handle in-domain queries in the experiment using the trial system. The average reward that the system would
have received when the simulated users made the decision after
5, 10, and 15 turns of interaction is listed in Table 1.
The following baseline strategies were compared with the trained policy:
1. Random recommendation (B1): The system randomly
chooses a recommendation from six methods.
2. No recommendation (B2): The system only provides therequested information and does not generate any recommendations.
The comparison of the average reward between the baseline
strategies is also listed in Table 1. The reward by the optimized
strategy was significantly better than that of baseline strategies3 .

5. User Evaluation

The three dialogue strategies discussed above were evaluatedby 40 subjects who had not previously used spoken dialoguesystems. Subjects were requested to use the system to select one sightseeing spot from among 15 alternatives. No instructions or scenarios were given. They were also requested to use
Figure 3: Example of state update
the phrase “I’ve decided to go to (spot name),” signifying their
expected agreement by random spot selection.
commitment once they had reached a decision.
We asked 20 users to use the optimized system first and to
M
N
M

1 
carry
out a dialogue session of selecting one spot. After the diaR=pm · ekm −pm · en,mlogue session using the optimized system, 13 out of the 20 usersN
m=1
n=1 m=1
were asked to use the baseline system 1 and seven users thebaseline system 2, and another dialogue session was conducted.
For example, if the decided spot satisfies three preferences and
The other 20 users are asked to use the systems in reverse order.
the average by random selection is 1.3, then the reward is 1.7.
That is, 13 users used the baseline system 1, then the optimized
system, and seven users used the baseline system 2, then the
4.4. Experiment by simulated users
optimized system. In total, 785 user utterances were collected.
The dialogue strategy of the system was optimized using theNote that only the first dialogue session for each subject was aabove dialogue state expression, reward function, and user simtruly valid dialogue episode for our experiment, since the first
ulator using the statics of our trial system via reinforcement
such dialogue would very likely alter the state of user knowllearning so that the manager selects the optimal recommendaedge. After the dialogue sessions, users were asked to selecttion method matched to the dialogue state [2]. In the typical ditheir preferences (four out of 10 criteria) they place value on
alogue strategy of the optimized policy, the system first bridges
when selecting sightseeing spots through questionnaires. Since
the knowledge gap with the user and estimates the user’s prefersubjects were asked to select from among all criteria, their seences (Method 4 and 5), then recommends specific informationlections are considered to be their preferences within the fullthat would be useful to the user (Method 6). This flow is simknowledge of the system. We assumed that the values of p forilar to the strategy of a human guide collected in our dialogue
selected criteria are “1”.
corpus [6].
Before comparing the dialogue strategies,we examined if
Prior to the user study, we examined the performance by
the user decisions are based on the priority of m kk · pk · vkm
user simulation. Simulated users are assumed to continue a dias assumed in the user simulator. We regarded the km and vnm
alogue for 5, 10, and 15 turns, and episodes are sampled using
as “1”, when the system lets the user know the information
the optimized policy. They are also assumed to have four prefthrough recommendations, and examined the order of the seerences2 , and to determine the spot based on their preference
lected spot in the priority. The average order was 1.4, and more
Puser under their knowledge Kuser
than half of users selected the spot with the first priority. These
at that time, and select the
spot k with a maximum priority of m kk · pk · vkm . The sysresults suggest that user decisions are related to the priority, altem parameters, Psys and Ksys are initialized using the statisthough users may be unconscious of the priority.
tics obtained by the trial system [2]. The system is rewarded
We then focused on the users’ first sessions and examinedby the reward function of section 4.3. We simulated possible
the obtained reward, length of the dialogue (number of interacautomatic speech recognition (ASR) and spoken language untions before user’s commitment), out-of-domain (OOD) query
2 As a result, four parameters in P
user are “1” and the others are

3 The maximum possible reward, which is achieved when the user
selects the spot that best satisfies their preferences, was 1.47.

“0”.

799

Table 2: Evaluation by human subjects (first session)
Strategy
optimized
B1
B2

Reward
0.85
0.07
0.09

Length
10.1
9.3
11.8

% OOD
18.2
21.4
21.1

% Error
1.0
1.8
1.4

Table 3: Comparison of first and second sessions
Strategy
optimized
B1
B2% Accept
52.5
27.0
-

㪊
㪉㪅㪌
㪉
㪈㪅㪌
㪈
㪇㪅㪌
㪇
㪄㪇㪅㪌 㪇
㪄㪈
㪄㪈㪅㪌
㪄㪉
㪄㪉㪅㪌

㪩㪼㫎㪸㫉㪻

rate, ASR/SLU error rate, and user’s acceptance rate of systemrecommendation4 . The results are listed in Table 2. As withthe result of the simulation, the system with the optimized dialogue strategy obtained a much higher reward. The averagereward of the optimized strategy 0.85 was significantly higher
than that of 0.07 and 0.09 by baseline strategies (p < .05). The
user’s acceptance rate of the system recommendation generated
by the optimized strategy was 52.5%, and the percentage of the
OOD utterance rate was lower than that of the baseline strategies. These figures suggest that the optimized dialogue strategy
can recommend appropriate information matched to the user’s
dialogue state, resulting in better decisions by users.
For reference, we then evaluated the result of the second
sessions and compared the results with that from the first sessions. The rewards of the sessions are listed in Table 3. Interestingly, the average reward of the optimized strategy of users
in the second session (= users who used the system with the
optimized dialogue strategy after using the baseline system)
was much smaller than that of the users in the first session.
The primary reason for this would be that users selected theirsecond-best spots, since all users selected a different spot withthat which they selected in the first session. Another reason forthis would stem from the fact that the user’s acceptance rate
for system recommendations was lower, resulting in a failurein estimating user preferences. Actually, the acceptance rateof users of the system with an optimized strategy in the second session was 30.0%, which was much lower than that of
52.5% by users in their first session. The relationship between
the average acceptance rate and reward is plotted in Figure 4.
Although there was no strong correlation, the rewards of thesystem by the users with a low acceptance rate were very likely
to be low. While about half of the users who used the optimized
system in the first session accepted system recommendations, atmore than 50%, more than half of users who used the optimized
system after using the baseline system did not accept the system recommendation at all. The reason for the low acceptance
rate in the second session would be that the optimized strategy
was intended for users (and trained by simulated users) withlittle knowledge about the system; consequently, several recommended items of information were known information for
the users in the second session. In the trained strategy, the system usually estimates user preference via methods 4 and 5 at an
early stage. And since many users in the second trial did not
accept the recommendation, the system seemed to fail to estimate user’s preferences5 and recommending appropriate spots
matched to the user. These results indicate the importance ofthe prior information for the system parameters in making appropriate recommendations.
Nonetheless, we believe that the optimized strategy and the
dialogue state representation are effective because every user
will experience the first use, and it is important to enable users
to succeed at the first use, as this will likely influence the user’s

Reward (1st session)
0.85
0.07
0.09

Reward (2nd session)
-0.17
-0.06
0.28

㪽㫀㫉㫊㫋㩷㫊㪼㫊㫊㫀㫆㫅㫊㪼㪺㫆㫅㪻㩷㫊㪼㫊㫊㫀㫆㫅

㪉㪇

㪍㪇

㪋㪇

㪏㪇

㪈㪇㪇

㪘㪺㪺㪼㫇㫋㪸㫅㪺㪼㩷㫉㪸㫋㪼㩷㩿㩼㪀

Figure 4: Relationship between recommendation acceptance
rate and reward
beliefs and their impressions of the system. In addition, theproblem is considered to be solved by maintaining the systemuse history of the user (e.g. by using phone number [9]) andcoordinating prior information.

6. Conclusion
In this paper, we addressed the results of user evaluations of several dialogue strategies in spoken decision support dialogue systems. We compared the optimized dialogue strategy based on
POMDP-based dialogue state expression with several baseline
strategies. We confirmed that the optimized strategy that was
effective in user simulation experiments worked well in evaluations by real users. However, the results also demonstrated
the importance of prior information on users. Future work thus
involves estimating user profiles, for instance through the system use history, as well as training in dialogue strategies using
multiple profiles of the users.

7. References[1] J. Polifroni and M. Walker, “Intensional Summaries as Cooperative Responses in Dialogue: Automation and Evaluation,” in Proc.
ACL/HLT, 2008, pp. 479–487.
[2] T. Misu, K. Sugiura, K. Ohtake, C. Hori, H. Kashioka, H. Kawai,and S. Nakamura, “Dialogue Strategy Optimization to Assist
User’s Decision for Spoken Consulting Dialogue Systems,” inProc. IEEE-SLT, 2010, pp. 342–347.
[3] J. Schatzmann, M. Stuttle, K., and S. Young, “Effects of theuser model on simulation-based learning of dialogue strategies,” inProc. Automatic Speech Recognition and Understanding Workshop(ASRU), pp. 220–225.
[4] B. Thomson, K. Yu, M. Gasic, S. Keizer, F. Mairesse, J. Schatzmann, and S. Young, “Evaluating semantic-level confidence scoreswith multiple hypotheses,” in Proc. Interspeech, 2008, pp. 1153–
1156.
[5] T. Saaty, The Analytic Hierarchy Process: Planning, Priority Setting, Resource Allocation. Mcgraw-Hill, 1980.
[6] K. Ohtake, T. Misu, C. Hori, H. Kashioka, and S. Nakamura, “Annotating Dialogue Acts to Construct Dialogue Systems for Consulting,” in Proc. The 7th Workshop on Asian Language Resources,
2009, pp. 32–39.
[7] R. Sutton and A. Barto, Reinforcement Learning: An Introduction.
MIT Press, 1998.
[8] O. Pietquin, S. Rossignol, and M. Ianotto, “Training Bayesian networks for realistic man-machine spoken dialogue simulation,” inProc. First International Workshop on Spoken Dialogue Systems
Technology (IWSDS), 2009.
[9] K. Komatani and H. Okuno, “Online Error Detection of Barge-In
Utterances by Using Individual Users’ Utterance Histories in Spoken Dialogue System,” in Proc. SIGDIAL, 2010, pp. 289–296.

4 Note, that Method 1, 4 and 5 are regarded as accepted when the user
requests information on either of the recommended criteria. Method 2
and 6 are regarded as accepted when the user requests information oneither of the recommended spots.
5 The system could detect only 1.1 out of four preferences of theusers who used the baseline system in advance. The figure was muchworse than that of 1.8 for the user who used the optimized system first.

800