Construction and Analysis of a Persuasive
Dialogue Corpus
Takuya Hiraoka† , Graham Neubig† , Sakriani Sakti† , Tomoki Toda† , Satoshi
Nakamura†
Abstract Persuasive dialogue systems, systems which are not passive actors, but
actually try to change the thoughts or actions of dialogue participants, have gainedsome interest in recent dialogue literature. In order to construct more effective persuasive dialogue systems, it is important to understand how the system’s humancounterparts perform persuasion. In this paper, we describe the construction of a
corpus of persuasive dialogues between real humans, and an analysis of the factorsthat contribute to the persuasiveness of the speaker. Specifically, we collect dialogue
between 3 professional salespeople and 19 subjects, where the salesperson is trying
to convince a customer to buy a particular product. We annotate dialogue acts of thecollected corpus, and based on this annotated corpus, perform an analysis of factorsthat influence persuasion. The results of the analysis indicate that most common dialog acts are information exchange, and about 30% of the persuader’s utterances areargumentation with framing aiming at making listener select a particular alternative.
Finally, we perform a regression analysis of factors contributing to the satisfaction of
the customer and persuasive power of the salesperson. We find that factors derived
from dialogue acts are particularly effective predictor of satisfaction, and factors
regarding framing are particularly effective predictors of persuasive power.

1 Introduction
In traditional dialogue systems, the main abstract goal is to increase user satisfaction, and this is achieved by helping users perform a specific task [1], helping users
with uncertain needs discover the information they are interested in [2], or entertaining users through chat [3]. On the other hand, there has also been a focus in recent
years on persuasive technology and computational deception, where the computer is
not simply a passive actor, but actively tries to change the thoughts or habits of theusers [4, 5]. This persuasive technology has been widely used as an indirect means
to improve user satisfaction by helping to improve bad habits [6], and also has beenused to identify factors of user decisions [7], for selling items, and for interactive
advertisement [4]. There is also some related research in dialogue on optimizing
policies of dialogue systems for argumentation [8] or for persuading users to makea choice that satisfies both the user’s goal and that of the system [9].

T. Hiraoka, G. Neubig, S. Sakti, T. Toda, S. Nakamura† Nara Institute of Science and Technology (Japan)
e-mail: {takuya-h,neubig,ssakti,tomoki,s-nakamura}@is.naist.jp

213

Proceedings of 5th International Workshop on Spoken Dialog Systems
Napa, January 17-20, 2014

Takuya Hiraoka† , Graham Neubig† , Sakriani Sakti† , Tomoki Toda† , Satoshi Nakamura†

However, these persuasive dialogue systems are in their first stages of development, and are far from the abilities of their human counterparts, both in terms of
persuasive ability, and also ability to achieve user satisfaction. Thus, in this paper
our aim is to elucidate which factors contribute to persuasive power and user satisfaction in the context of human persuasion. To do so, we focus on a common realworld situation in which persuasion is a factor: sales conversation. In this case, thesalesperson (persuader) tries to convince the customer (persuadee) to purchase acertain product while maintaining customer satisfaction. By focusing on this typeof dialogue, we hope to elucidate features of dialogue where both the persuader andthe persuadee are satisfied.
In particular, in this paper we make the following contributions.
• We collect and annotate a corpus of nearly five half hours of dialogue between
subjects and professional salespeople, who are trying to convince the subjects tobuy a camera. We describe data collection and tagging of dialogue acts over two
dimensions of analysis of the recorded dialogue.
• We perform an analysis of the major dialog acts constituting the corpus, including
differences between the persuader and persuadee.
• We perform a regression analysis to identify the relationship between persuasive
success, persuadee satisfaction, and a number of salient factors. Knowledge obtained by the analysis is a guide for not only dialogue system construction, but
may also provide insights from the perspective of human persuasion.

2 Related Work
There has been some previous work on corpus collection and annotation in the context of persuasive dialogue. For example, Georgila et al. [10] proposed a tag scheme
for persuasive dialogue and argumentation. In the tag scheme that was proposed inthis research, argumentation tags were sorted by their role. For example, tags are
given to roles such as, “invalidate argument,” “accept argument,” etc. On the other
hand, we design argumentation tags that focus on the user’s preference information and particularly framing [11]. This information is known to be important for
persuasion, and is not captured by a purely argumentative tagging scheme.
Nguyen et al. [12] also examined the relationship between boredom of persuadees, success of persuasion and other salient factors in persuasive dialogue.
This research analyzed persuasive dialogue between an embodied agent and human,
while we analyze dialogue between two humans. In addition, the study examined theeffects of the persuaders message style, persuadees participation (conversing withthe agent or only listening), and the number of persuaders. In contrast, we analyze
factors based on the user’s preference and framing.
In addition, our research proposes a predictive model of the achievement of persuasion and user satisfaction, and we can evaluate other persuasive dialogues semiautomatically by using this model. This is also an additional contribution over theprevious works.

214

Proceedings of 5th International Workshop on Spoken Dialog Systems
Napa, January 17-20, 2014

Construction and Analysis of a Persuasive Dialogue Corpus
Table 1 Details of the scope of the sales dialogue corpus.
Salesperson Experience AgeA
4 years
40’s
B
3 years
30’s
C
2 years
30’s
Total

Dialogues Minutes Salesperson Words Customer words
10
127
33,3306,451
12
10632,8357,544
12
104
24,821
7,675
34
337
90,986
22,626

Table 2 The beginning of a dialogue from the corpus (translated from Japanese)
Speaker
CustomerTranscription
GPF TagWell, I am looking for a camera,
P ROPOSITIONAL Q
do you have camera B?
Salesperson Yes, we have camera B.
C ONFIRM
Salesperson Did you already take a look at it somewhere? P ROPOSITIONAL Q
Customer Yes. On the Internet.
C ONFIRM
Salesperson It is very nice. Don’t you think?
C HECK Q
Customer Yes, that right, yes.
AGREEMENT

3 Collection of a Camera Sales Dialogue Corpus
3.1 Data Collection
As a typical example of persuasive dialogue, we choose dialogue between a salesperson (persuader) and customer (persuadee), in which the salesperson attempts toconvince the customer to purchase a particular product (decision) from a number ofalternatives (decision candidates). We will define this type of dialogue as “sales dialogue.” More concretely, we assume the customer is in an appliance store looking
for a camera, and the customer must decide which camera to purchase from 5 alternatives. Prior to recording, the salesperson is given the description of the 5 cameras
and instructed to try to convince the customer to purchase a specific camera (thepersuasive target). This persuasive target is invariant over all subjects. The customer
is also instructed to select one preferred camera from the catalog of the cameras, andchoose one aspect of the camera that is particularly important in making their decision (the determinant). During recording, the customer and the salesperson converse
and refer to the information in the camera catalog as support for their arguments. The
customer can close the dialogue whenever they want, and choose to buy a camera,
not buy a camera, or reserve their decision for a later date.
We collect a role-playing corpus with participants consisting of 3 salespeople
from 30 to 40 years of age and 19 customers from 20 to 40 years of age. All salespeople have experience working in an appliance store. The total number of dialogues
is 34, and the total time is about 340 minutes. Table 1 shows the scope of the corpus,
and Table 2 show an example transcript of the beginning of one dialogue.

215

Proceedings of 5th International Workshop on Spoken Dialog Systems
Napa, January 17-20, 2014

Takuya Hiraoka† , Graham Neubig† , Sakriani Sakti† , Tomoki Toda† , Satoshi Nakamura†

3.2 Annotation of Dialogue Acts
3.2.1 Dialogue Act Scheme
In order to perform an in-depth analysis of the recorded dialogues, we annotate eachutterance with three varieties of tags, the first covering dialogue acts in general,and the rest being specifically defined for analyzing persuasion (argumentation andframing). Formally, the relationship between collected dialogues and annotated tags
is defined as follows:
U = {u1 , u2 , ..., uK }uk = ⟨r, g, A, F⟩

(1)
(2)

where U represents a dialogue, and is composed of sequences of utterances uk . uk
is annotated with four varieties of tags, a role tag r which takes the value SALES
for the salesperson or CUST for the customer, a dialogue act tag g, argumentation
tags A, and framing tags F. Each of these are introduced in later paragraphs in thissection.
As a tag set to represent traditional dialogue acts, we use the general-purpose
functions (GPF) defined by the ISO international standard for dialogue act annotation [13]. Annotated GPF tag g is defined to be one of the tags in this set. In order toassign only one GPF tag for each utterance, we first annotate the GPF tags, and if asingle turn would be assigned multiple tags, we split the turn into multiple utterance
units. Table 2 shows examples of GPF tags. For example, “P ROPOSITIONAL Q” isused to annotate utterances intended to confirm that an opinion or fact is correct.
To annotate information regarding the aspects of each utterance particularly relevant to persuasion, we devise a separate tag set based on knowledge of persuasion
and attitudes in psychological research [14]. In this research, it has been suggested
that humans generally evaluate decision candidates by selecting based on several determinants weighted by the user’s preference. In particular, it has been suggested thatthe framing method is an effective way of increasing persuasive power. In this work,
we focus on negative/positive framing [11, 15], which uses emotionally charged
words to explain particular alternatives, with negative framing using negative words
and positive framing using positive words. Through a preliminary analysis of oursales dialogue data, we built a hypothesis that argumentation using framing plays
an important role in sales dialogue and decided to pursue this hypothesis further
through annotation of the data.
The annotated argumentation tag A is defined as follows:
A = {a1 , a2 , ..., aJ }(3)

a j ∈ ALT

(4)

where variable a j is selected from the set ALT of possible alternative (in this case,the five cameras). In the annotated corpus the argumentation tag is described by thefollowing format, similar to XML:

216

Proceedings of 5th International Workshop on Spoken Dialog Systems
Napa, January 17-20, 2014

Construction and Analysis of a Persuasive Dialogue Corpus
Table 3 A argumentation tag annotation of a salesperson’s utterance<arg alt=A><fra alt=A,polarity=POS,pref=NO>(Camera A is)able to achieve performance of comparable single-lens cameras and can fit in your pocket</fra>, this is a point.</arg><arg alt=a j > ... </arg>(5)

We also define annotated framing tags F as follows:
F = { f1 , f2 , ..., fI }(6)

fi = ⟨ai , pi , ri ⟩ai ∈ ALT(7)(8)

pi ∈ {POS, NEG}ri ∈ {YES, NO}

(9)
(10)

where ai represents the target alternative, pi takes value NEG if the framing is negative, and POS if the framing is positive, and ri represents whether the arguments
contain a reference to the persuadees preferred determinant, taking the value TRUE
if contained, and FALSE is not contained. The user’s preferred determinant is annotated on the basis of the results of questionnaire. In the annotated corpus, fi is
described by the following format:<fra alt=ai ,polarity=pi ,pref=ri > ... </fra>(11)

Table 3 shows an example of annotation of positive framing (p=POS) about theperformance of Camera A (a=A). In this example, the customer answered that his
preference is the price of camera, and this utterance does not contain any descriptionof price. Thus, r=NO is annotated. Finally, we annotate <arg alt=A> around theentire utterance because at least one fra tag exists.
3.2.2 Reliability of Annotation
To evaluate the reliability of the annotation, we randomly selected 10% of the collected data and evaluated the data for inter-annotator agreement. The GPF and argumentation tags were evaluated on the basis of the agreement between two annotators. The description section and the variables of the fra tag were evaluated bya second annotator regarding whether the annotation result of the primary annotator was acceptable or not. The acceptability rate is calculated as the percentage oftags judged as appropriate by the second annotator out of the tags annotated by the
primary annotator.
Initially, the agreement of the 18 annotated GPF tags was only 30%. As this is too
low to achieve reliable results in our analysis, we merged tags with low agreement,
resulting in a total of 6 tags and an agreement of 76% (see Table 4). This agreement
is comparable to other research in a different task [16]. We use these merged GPF

217

Proceedings of 5th International Workshop on Spoken Dialog Systems
Napa, January 17-20, 2014

Takuya Hiraoka† , Graham Neubig† , Sakriani Sakti† , Tomoki Toda† , Satoshi Nakamura†
Table 4 Result of the merging GPF tags
GPF tag P ROPOSITIONAL Q, S ET Q, I NFORM , A NSWER , C OMMISSIVE , D IRECTIVE

tags in the analysis of later sections. The agreement of the argumentation tag was
94%, and, the acceptability rate of the description section and target candidate of theframing tag was 94%, polarity was 100%, and of preference information was 82%.

4 Success Measures and Dialogue Factors
Given the corpus described in the previous section, we would now like to elucidate
the factors that contribute to persuasive power and user satisfaction.

4.1 Success Measures for Persuasive Dialogue
First we define our measures for success of persuasive dialogue. As the dialogue
consists of two interlocutors, we define a successful dialogue as a dialogue where
both participants achieve their goal. As with dialogue systems, simply using satisfaction as measure of dialogue success for the persuadee seems appropriate [17].
However, as far as we are aware there is no widely shared evaluation criterion in therelatively young field of persuasive technology. Thus we propose two measures forthe success for the persuader: 1) Whether the persuadee finally chooses the persuasive target at the end of the dialogue, and 2) the amount the persuadees intention
changed about the persuasive target as the result of the dialogue.
We measured these values by conducting a questionnaire of the persuadees tomeasure satisfaction, intention change about the persuasive target, and success of
persuasion, as described below:
Satisfaction (Sat): The persuadee’s subjective satisfaction with the persuader defined as a 5 level score of customer satisfaction (1: Not satisfied，3: Neutral，5:Satisfied).
Intention change (∆ In): The amount the persuadees intention to buy the persuasive target changed as a result of the dialogue. We conducted a questionnaire
about intention to buy persuasive target (1: Don’t want to buy，3: Neutral，5:Want to buy) before (Inbe f ore ) and after (Ina f ter ) the dialogue. ∆ In is measured
as follows:∆ In = Ina f ter − Inbe f ore(12)

Persuasive success (Suc): Suc takes the value 1 when the customer decides to purchase the persuasive target at the end of dialogue, and 0 otherwise.

218

Proceedings of 5th International Workshop on Spoken Dialog Systems
Napa, January 17-20, 2014

Construction and Analysis of a Persuasive Dialogue Corpus

4.2 Dialogue Factors
In this section, we describe several measurable characteristics of the dialogue that
may contribute to persuasive power and user satisfaction. These include factors regarding negative/positive framing, original preference of the persuadee, and dialogue acts.
4.2.1 Factors Regarding Negative/Positive Framing
Two dialogue factors to measure negative-positive framing are defined as follows:
Negative framing ratio for non-target (RNEG,a̸=t ): The ratio of utterances stating
negative facts about alternatives other than the persuasive target, where we definet as the persuasive target:
RNEG,a̸=t =∑Kk=1 δ (∃ f ∈uk .F ( f .a ̸= t ∧ f .p = NEG))
,
K

(13)

where δ is Kronecker’s delta, 1 when the condition is true, and 0 otherwise.
Positive framing ratio for target (RPOS,a=t ): Likewise the ratio of utterances by
the persuader positively framing the persuasive target:
RPOS,a=t =∑Kk=1 δ (∃ f ∈uk ,F ( f .a = t ∧ f .p = POS))
.
K

(14)

4.2.2 Factors Regarding the Persuadees Original Preference
We also define 3 kinds of factors to measure the persuadees attitude change.
Conveyed preferred determinant (CPDa ): Whether the persuadee has been toldby the persuader that alternative a satisfies the determinant that the persuadee
has mentioned as important in the pre-dialogue questionnaire
CPDa = δ (∃ f f =< a, POS, YES >).

(15)

Prior candidate evaluation (PCEa ): The persuadees evaluation of alternative a at
the beginning of dialogue. In this paper, we calculated one feature for each alternative that is 1 if that alternative is selected by the persuadee as preferred before
the dialogue and 0 otherwise.
Prior persuasive target evaluation (PPTA): The persuadees evaluation of the persuasive target at the beginning of the dialogue as measured by questionnaire.
4.2.3 Other Factors
In addition to the above factors, we defined factors based frequency of traditional
dialogue acts and argumentation, and total time.
Number of argumentation events (I): The total number of occurrences of argumentation tags during the dialogue I.

219

Proceedings of 5th International Workshop on Spoken Dialog Systems
Napa, January 17-20, 2014

Takuya Hiraoka† , Graham Neubig† , Sakriani Sakti† , Tomoki Toda† , Satoshi Nakamura†
Table 5 Distribution of general purpose function (GPF) and argumentation tags
GPF
Argument
PropQ SetQ Commisive Directive Answer Inform Tar NonTar BothSalesperson 14% 4%
6%8%
16% 45% 25% 3%
3%Customer
21% 2%
9%5%
17% 37%-Frequency of general purpose function (Rr,g ):role in the dialogue
Rr,g =
Total time (T T ):The ratio of each GPF tag for each∑Ii δ (ui=1 = ⟨r, g, •, •⟩).
∑Ii δ (ui=1 = ⟨r, •, •, •⟩)(16)

Total dialogue time in seconds.

5 Analysis
In this section, we present a manual analysis of the dialogue acts included in the
corpus, and a linear regression analysis of the factors that contribute to persuasion.

5.1 Analysis of Dialog Acts
First, in order to perform a general analysis of the main dialogue acts comprising
persuasive dialogue, we show the proportion of argumentation tags of all utterancesof the salesperson and the GPF distribution for both the customer and salespersonin Table 5. From the result, we can see information presentation (Answer, Inform)tags cover more than half of both of the customer and salesperson utterances. In
addition, when considering information seeking tags (PropQ, SetQ), the percentage
reaches about 80%.
31% of all dialogue acts of the salesperson are arguments. This indicates thatthe argumentation tag proposed in Section 3 is highly relevant in this situation. A
more detailed breakdown is that 25% of arguments target only the persuasive target,
3% of arguments target only an alternative other than the persuasive target, and
3% of arguments target both the persuasive target and a non-persuasive target. This
indicates that, in persuasive dialogue, the persuader rarely suggests arguments forselecting alternatives other than persuasive target, but does occasionally mention
other options.
Table 6 shows mean persuadee satisfaction categorized by initial and final choiceof alternative. The results seem to indicate that it is possible to achieve satisfaction
and persuasion simultaneously when the customer has initially chosen the persuasive target or doesn’t have an initial choice, but it is harder when the customer has
initially chosen an alternative other than the persuasive target. However, the data is
still somewhat small to make conclusions about this fact.

220

Proceedings of 5th International Workshop on Spoken Dialog Systems
Napa, January 17-20, 2014

Construction and Analysis of a Persuasive Dialogue Corpus
Table 6 Average satisfaction (and number of dialogues) for each initial and final choice.
Final Choice
PT Not PT None
PT 4.0(3)5.0(5)
Initial Choice Not PT 2.0(2) 2.0(1) 4.4(7)None 4.0(3) 2.0(2) 3.4(7)Table 7 Linear regression for satisfaction and persuadees intension change, and logistic regression
for success of persuasion with selected factors. All factors are normalized.
w0 +w1 x1 +...wn xn
Sat +3.56 Bias+.501 RS ALES ,P ROP Q-.509 RS ALES ,C OMMISIVE∆ In +.920 Bias-.475 RNEG,a̸=t+.625 I-.303 CPDE+.429 PPTA+.295 PCEC+.422 RC UST,A NSWER +.464 RC UST,I NFO - PROV +.276 RC UST,C OMMISIVE-.368 T T
w0 +w1 x1 +...wn xn
Suc -4.349 Bias
+2.00 CPDB
-8.14 PCEB
-2.12 T T

R2
.396
.640

Accuracy80%

5.2 Regression Analysis of Factors in Persuasion
To analyze the relationship between the success measures in Section 4.1 and factors
in Section 4.2, we performed a regression analysis to discover the important factors and measure accuracy of the prediction model. Factor selection is performed
using step-wise multinomial linear regression [18]. We repeatedly perform multinomial regression and exclude predictors that do not sufficiently contribute to themodel until we get a model for which all of the predictors are significant. In thisresearch, we excluded any predictor with a p-value above .25 at each iteration, andthe final model is comprised of predictors that are statistically significant (p < .05).
Prediction accuracy of the selected factors is evaluated through leave-one-out crossvalidation after the selection.
Table 7 shows the results. First focusing on the factors for satisfaction, we cansee that predictors account for 39% of the variance of satisfaction. Focusing on
the variables selected as useful in the linear regression results, we can see thatboth of the two features come from the salesperson’s GPF tags. The weight of
RS ALES ,P ROP Q is high, which indicates that by asking many questions, the salesperson can make the customer feel more satisfied with the conversation. The reason
why the weight of RS ALES ,C OMMISIVE is assigned a large negative value is that
RS ALES ,C OMMISIVE represents the degree of failure in answering the customer’squestions. For example, most of the utterances such as “Sorry, I don’t know. I’ll
take a look” are annotated C OMMISIVE. This result is interesting, as it shows that
customer satisfaction is largely dependent on the salesperson, a fact that may guideour implementation.
Next, focusing on the weight of factors in the linear regression results for opinionchange, factors derived from argumentation tags account for 46% of total weight,
making the largest contribution to prediction. The highest weight is I, indicating that
more argumentation for the persuasive target results in a larger change in the opin-

221

Proceedings of 5th International Workshop on Spoken Dialog Systems
Napa, January 17-20, 2014

Takuya Hiraoka† , Graham Neubig† , Sakriani Sakti† , Tomoki Toda† , Satoshi Nakamura†ion of the persuadee. On the other hand, PPTA is assigned a large negative weight,
indicating the persuader does not change the opinion of a persuadee who alreadywanted to select the persuasive target a priori, a natural result as the persuader will
not want to change an already favorable result. The weight of factors derived fromthe GPF tag account for 33% of the total weight. Especially, the ratio of informationexchange (RC UST,A NSWER ，RC UST,I NFO - PROV ) assumes a high weight, indicating that making the customer speak more contributes to opinion change.
Finally, looking at the result for logistic regression over persuasive success, we
can see that 80% of the data are correctly predicted, compared to a chance rate of
68% when predicting only failure of persuasion. Focusing on the weights of thevariables in the logistic regression result, the weight of PCEB is relatively high,indicating that if customers select camera B pre-dialogue, the persuasion becomes
more difficult. CPDB is the only variable with positive weight, indicating that informing the persuadee about alternatives other than the persuasive target that match
the persuadee’s preference increases the persuasive power for the persuasive target.
We hypothesize the reason why only camera B appeared in predictors is that camera B was chosen many times compared to other alternatives, and appeared as thealternative for comparison to the persuasive target in many dialogues.
Combining all these results together, we can see that the persuader is required touse a sophisticated dialogue strategy, as different factors contribute to the achievement of successful persuasion and persuadee satisfaction. However in Table 7,
we can also see that no predictor influences both successful persuasion and persuadee satisfaction. Therefore, the persuader could potentially perform dialogue toachieve both goals simultaneously. For example, the persuader would perform alarge amount of argumentation to achieve persuasion, and ask many questions toincrease user satisfaction. However, as observed by the negative weight for T T ,intention change of the persuadee also tends to decrease as time passes. Thus, thepersuader must achieve both goals in a short time, considering interaction efficiently
and accurately predicting the persuadees interest in each of the alternatives.

6 Conclusion
In this paper, we analyzed persuasive dialogue between humans, focusing on thefactors that contribute to persuasion and satisfaction. In order to do so, we collecteda corpus of dialogues between salespeople and customers, and defined an argumentation tag scheme and dialogue factors for predicting dialogue goals.
The experimental results indicate that the main dialog acts that compose the dialogue are information exchange and argumentation. A regression analysis demonstrated that argumentation contributes effectively to the achievement of persuasion,and factors derived from GPF were effective for predicting satisfaction.
Our next step in this research is to incorporate these observations into the persuasive dialogue framework of [9]. In addition, this experiment result is still limitedin the corpus we collected. We will investigate the flexibility of the proposed tag
scheme and persuasive factors on other persuasion tasks.
Acknowledgements
ATR-Trek.

Part of this work was supported by the collaborative research with I2R and222

Proceedings of 5th International Workshop on Spoken Dialog Systems
Napa, January 17-20, 2014

Construction and Analysis of a Persuasive Dialogue Corpus

References
1. Wayne Ward, “The CMU air travel information service: Understanding spontaneous speech,”
Proceedings of the Workshop on Speech and Natural Language, 1990.
2. Teruhisa Misu, Komei Sugiura, Kiyonori Ohtake, Chiori Hori, Hideki Kashioka, Hisashi
Kawai, and Satoshi Nakamura, “Modeling spoken decision support dialogue and optimizationof its dialogue strategy,” ACM Transactions on Speech and Language Processing, 2011.
3. Joseph Weizenbaum, “ELIZA – a computer program for the study of natural langage communication between man and machine,” Communications of the Association for Computing
Machinery, 1966.
4. B.J.Fogg, Persuasive Technology, Morgan Kaufman, 2003.
5. Anton Nijholt, Ronald C Arkin, Sebastien Brault, Richard Kulpa, Frank Multon, Benet
Bideau, David Traum, Hayley Hung, Eugene Santos Jr, Dequing Li, Fei Yu, Lina Zhou, andDonsong Zhang, “Computational deception and noncooperation,” IEEE Intelligent Systems,
2012.
6. Stephen Purpura, Victoria Schwanda, Kaiton Williams, William Stubler, and Phoebe Sengers,“Fit4life: the design of a persuasive technology promoting healthy behavior and ideal weight,”
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2011.
7. Marco Guerini, Oliviero Stock, and Massimo Zancanaro, “Persuasion model for intelligent interfaces,” Proceedings of the IJCAI Workshop on Computational Models of Natural Argument,
2003.
8. Kallirroi Georgila and David Traum, “Reinforcement learning of argumentation dialogue
policies in negotiation,” Proceedings of INTERSPEEECH, 2011.
9. Takuya Hiraoka, Yuki Yamauchi, Graham Neubig, Sakriani Sakti, Tomoki Toda, and Satoshi
Nakamura, “Dialogue management for leading the conversation in persuasive dialogue systems,” Proceedings of ASRU, 2013.
10. Kallirroi Georgila, Ron Arstein, Angela Nazarian, Michael Rushforth, David Traum, and Katia
Sycara, “An annotation scheme for cross-cultural argumentation and persuasion dialogues,”
Proceedings of the SIGDIAL, 2011.
11. Sandra L. Schneider Levin Irwin and Gary J. Gaeth, “All frames are not created equal: A typology and critical analysis of framing effects,” Organizational behavior and human decision
processes 76.2, 2013.
12. Hien Nguyen, Judith Masthoff, and Pete Edwards, “Persuasive effects of embodied conversational agent teams,” Proceedings of Human-Computer Interaction, 2007.
13. ISO 24617-2, Language resource management-Semantic annotation frame work (SemAF),
Part2: Dialogue acts. ISO, 2010.
14. Martin Fishbein, “An investigation of the relationship between beliefs about an object and theattitude toward that object,” Human relations, 1963.
15. Irene Mazzotta and Fiorella de Rosis, “Artifices for persuading to improve eating habits,”
AAAI Spring Symposium: Argumentation for Consumers of Healthcare, 2006.
16. Kiyonori Ohtake, Teruhisa Misu, Chiori Hori, Hideki Kashioka, and Satoshi Nakamura, “Annotating dialogue acts to construct dialogue systems for consulting,” Proceedings of the 7thWorkshop on Asian Language Resources. Association for Computational Linguistics, 2009.
17. Marilyn A Walker, Diane J. Litman, Candace A. Kamm, and Alicia Abella, “Paradise: A
framework for evaluating spoken dialogue agents,” Proceedings of the eighth conference on
European chapter of the Association for Computational Linguistics, 1997.
18. Allison Terrell and Mutlu Bilge, “A regression-based approach to modeling addressee
backchannels,” Proceedings of the 13th Annual Meeting of SIGDIAL, 2012.

223

Proceedings of 5th International Workshop on Spoken Dialog Systems
Napa, January 17-20, 2014