Utilizing deception information for dialog managementof doctor-patient conversations.
Nguyen The Tung∗1 , Koichiro Yoshino, Sakriani Sakti, Satoshi Nakamura
Graduate School of Information Science, Nara Institute of Science and Technology
Almost all of existing negotiation systems assume that their interlocutors (the user) are telling the truth. However,in negotiations, participants can tell lies to earn a profit. In this research, we proposed a negotiation dialog
management system that detects user’s lies and designed a dialog behavior on how should the system react with.
As a typical case, we built a dialog model of doctor-patient conversation on living habits domain. We showed that
we can use partially observable Markov decision process (POMDP) to model this conversation and use reinforcement
learning to train the system’s policy.

1.

Introduction

Recently, negotiation dialog has attracted the attention of many
researchers around the world. Other than improving the efficiency
of dialog management, the focus of these studies have switchedfrom passive role systems (eg. restaurant, tourist information
provider) to become more actively systems that can influence theuser’s decision (persuasive technology) [5, 10]. Most of current researches mainly deal with cooperative dialog, in which the systemand the user work together to reach a balanced point . However,there are situations that the mutual goal cannot be reached (noncooperative dialog) [11, 3].
Existing works [3] did not cover the situation in which participants use “false promises” tactics - which can directly impact theoutcome of negotiation. For example, someone pretends to give up
unhealthy custom (eg. smoking/drinking) after receiving advices
from friends or family but he actually does not have intention tofollow the advice. [12] put a learning agent that can tell lies against
an adversary who consider lies with contradiction in simulation.
In this paper, we focus on a typical example of negotiation, thedoctor-patient conversation on living habits domain. In this typeof dialog, the patient has their own perspective and opinions about
their health condition. The doctor needs to consider these opinions
when making a treatment plan (recommend a new habit) and negotiates with users to reach a plan that satisfy both user’s demand andrequirements of the treatment. Patients sometimes tell lies because
they do not want to change their habit, thus considering deceptive
information will improve the negotiation strategy of the doctor.

2.

Scenario and Modeling

2.1

Dialog scenario

This work considers a dialog scenario between a system (doctor) and a user (patient). They discuss about user’s living habit,
which can be about: sleeping, food, working/studying, exercise,social media usage and leisure activities. The system tries to convince the user that they need to change to a more healthy living
habit. The system persuades the user by giving them information
Contact: Nara Institute of Science and Technology;
8916-5 Takayama, Ikoma, Nara;{nguyen.tung.np5,koichiro,ssakti,s-nakamura}@is.naist.jp

about the new habit (system’s recommendation), health benefitsof the new habit and negative effects of user’s current habit. Thisaction is denoted as Framing in this research. On the other hand,the user wants to continue the current habit and gives reasons toshow that it is too difficult for him to change. The system behaves
cooperatively; if user’s reason is honest, system will give an easierrecommendation.
To make the conversation simpler, only the system can propose
recommendations, the user cannot suggest what habit they shouldchange to. However, the users are allowed to use dishonest reasons
to make the system offer an easier recommendation. The user canalso pretend to accept the system’s offer while they actually do notintend to change their current habit.
Start

Offer
user’saction?

Hesitate

Offer

Framing

RejectAccept

Question
Yes

Lie?

Framing

Yes

No

Framing

End

Lie?
No

End

Offer

Figure 1: Flow chart of the conversation.
Figure 1 describes the proposed dialog behavior, which considers deceptions of the user. In this flowchart, rectangles indicate
system actions. The set of dialog acts for the system includes:
Offer: the system offers a recommendation to the user, whichmeans the system suggests the user should change to a new
habit.
Framing: the system provides information about the new living habit, the necessity and health benefits of the habit and consequences if the user keep the current habit.
End: the system ends the conversation.

For persuasive dialog, framing technique was shown to be a very
effective way to convince users to accept the system’s goal ([5]).
On the other hand, the user can react with different actions as described below:µ represent the rounded probability for every 0.1 that the observation comes from SLU and Deception Detection equal to actualuser state and deception information.

Accept: the user accepts the system’s recommendation (offer).
The user can pretend that they accept the system’s offer.

Question: the user asks the system about the benefits of new
habit or asks the system to give a clear explanation it.
According to a study by [7], when the patient is lying, the doctor should tell the patient about the necessity and benefits of thetreatment plan and consequences if patient refuses to follow it.
Applying to the “living habit” scenario, the most logical reaction
when the user is telling lie (uses fake reasons or pretend to agree)
is Framing. In contrast, a conventional negotiation system that
does not consider user’s deception always offer a new recommendation when the user rejects and end the conversation when theuser agrees regardless whether they are telling the truth or not.

2.2

Policy Management using POMDP

To find the best strategy of dialog system against the user deception, considering errors of dialog act classification and deception
detection for user utterance is necessary, because these model do
not have 100% accuracy. Partially observable Markov decision
process (POMDP) is widely used to learn the best strategy of dialog systems for such error containing cases [13].
The equation for updating belief state of a POMDP can be written as:
bt+1 ∝ P(ot+1 |st+1 ) ∑ P(st+1 |st , ât )bt

(1)

si

Apart from user’s action st , the proposed dialog system also uses
deception information for dialog management. To solve this problem, we used method in a similar work for user focus by [13]. By
extending Equation (1) with deception information of the current
turn d t and next turn d t+1 , we have the belief update of the proposed system:
bt+1
s,d ∝

t+1 t+1 t+1P(ot+1,d )∑∑P(st+1 ,d t+1 |st ,d t ,ât )bts,ds ,od |ss d

(2)With the observation result come from SLU and Deception Detection modules being denoted as os and od respectively.
In this research, we use Q-learning, a popular method to train the
optimal policy π ∗ . We utilize Grid-based Value Iteration method
proposed by [2] . The belief is calculated by:
(
bsi =

µ

if s = o

1−µ
|S|−1

otherwise

Offer

Rewards
Framing

End

0
1
0
1
0
0

-10
-10
+10
-10
-10
+10

-10
+10
+10
+10
+10
+10

+100
-100
-100
-100
-100
-100

AcceptReject: the user rejects the recommendation given by the system and gives reason why they reject it. In order to make thesystem to give an easier habit, the user can reject with a deceptive reason.
Hesitate: the user says he/she is unsure about whether to accept
the offer or not.

Dialog state
User DA (s) dReject
Question
Hesitate

Table 1: Rewards in each turn.
In table 1, we show the reward received for each turn.

2.3

Deception Detection using multi-modal approach

There are various clues that can help us to detect lies, which
include lexical modal, acoustic modal, gestures and facial expressions. As a result, multi-modal approach that combines thosemodalities was proved to be very efficient in detecting deception[8]. Thus, we also utilized multi-modal approach with acoustic
and facial features to build the deception detection module.
To extract facial features, we used OpenFace toolkit developedby [1]. From this tool, we were able to extract 14 face AU (Action Unit) regression and 6 AU classification values as well as head
position, head direction parameters. All these values are then normalized and discretized into 5 different levels of intensity.
Acoustic features are extracted from audio files using OpenSMILE tool [4]. The acoustic feature template was taken from thework by [6]. From the pitch and loudness values, we calculated
maximum (max), minimum (min), mean (mean) and standard deviation (std). The duration-related features include: percentage offrame with voice, percentage of frame with lower pitch than previous frame (falling pitch), percentage of frame with higher pitch
than previous frame (raising pitch).
The classification model that we used for deception detectionis Multi Layer Perceptron with hierarchical structure to combine
acoustic and facial features. This method was proposed by [9] foremotion detection task. In particular, for our research, we put facial features directly into the first layer of the network while acoustic features are incorporated at the second hidden layer.
To test the effectiveness of the deception detection module, weused data from recorded conversations between 2 participants.
One of them plays the role of a doctor and the other one’s roleis a patient. In this conversation, the “patient” will try to get aprescription from the “doctor” by telling lies about his health condition. Deception label for each utterance was manually annotatedby themselves. The total number of utterances in this data is 146.
We took 34 of them (17 honest, 17 deceptive) Results of the experiment are shown in Table 2.

Precision
RecallFeaturescombineDecision
combineHierarchical

61.77%
35.29%

67.65%
41.17%

64.71%
58.23%(3)
Table 2: Deception detection accuracy.

3.

Experiments of dialog management system

3.1

Data

Due to the difficulties of recruiting actual doctors and patients
to collect the data, the participants who took part in data collection
are students who have good level of English fluency. All participants are working at the same academic environment at the timeof data collection.
The data used for the experiments are recorded using the “living habits” dialog scenario in Wizard-of-Oz (WoZ) setup. Each
recording session is carried out by 2 participants who play the roleof the system and the user respectively. Each session consists of
6 dialogs for each of the living habit topic. The participants whoplay the role of the user (patient) are given payment as reward forthe outcome of the conversation. If they pretend to agree with thesystem’s offer, they will receive lower payment. On the other hand,
if they choose to truly agree with the system’s offer they will gethigher payment with a condition that they will need to change tothe new habit for one week. The payment is to create the situation
where the user has to choose between an easy activity (continue
current habit) with low reward and an activity that is difficult but
has higher reward in return (change to new habit) to observe more
lies.
There are total of 7 participants who take part in the recordings.
4 of them played the role of the system and 6 played the role ofthe user (participants who played the role of the user can switch
their role to doctor in later recordings.) In the end, the recorded
training data are about 3 hours 20 minutes long with a total of 883
utterances. Labels of DA are annotated by one expert and labels of
deception are provided by the participant who made the deception.
With test corpus, recordings are done as direct conversations between participants. The recording setup is similar to WoZ scenario
but without the help of TTS since the participants are now talking
directly with each other. This type of evaluation data contains 936
utterances and about 2 hours 35 minutes in length. Table 3 shows
statistics of the collected data.
Data
Train
Test

End

System DA
Framing

Offer

Hesitate

User DA
Question
AcceptReject% lie in user’sutterances

14.43%
17.54%

43.30%
36.26%

42.27%
46.20%

21.69%
17.64%

3.61%
9.86%

22.89%
52.82%

18.07%
14.08%

51.81%
19.72%

Table 3: Deception and dialog acts statistic.

3.2

Results
Dialog policy

Success rate

Avg. offer

Conventional
Proposed policy

21.83%
29.82%

2.472
2.447

Table 4: Success rate and average offer per succeeded dialogs.
First, we test the negotiation efficiency of the learned policy bycomparing it with a baseline negotiation policy (denoted as Conventional in Table 4) that treats user’s utterances as honest. With
this baseline policy, the system receives reward similar to Table 1
but only for the case when user is being honest (d = 0); becausewith this policy, deception is not taken into account. We let thesystem interacted with a simulated user created from test data for
100,000 dialogs. The simulator is created with the same method

as described in [13]. The transition probabilities are given by:
P(st+1 ,d t+1 |st ,d t ,ât ) = P(st+1 |d t+1 ,st ,d t ,ât )P(d t+1 |st ,d t ,ât )
{z
}|
{z
}
|
intention model

deception model(4)
Probabilities for intention model and deception model are calculated from test data using maximum likelihood.
Performance is evaluated with success rate and average offer
per succeeded dialog. Success rate is the percentage of dialogs inwhich user truly accepts the system’s offer. For the second evaluation metric, average offer; since every time the system makes anOffer action, the new habit will be easier but gives less health benefit, thus it is less favorable for the system. Therefore, using fewer
offers to successfully persuade the user is better. From the results
shown in Table 4, it is clear that our proposed system outperforms
the baseline.
Dialog system

DA
Deceptionaccuracy handling

Conventional
Proposed policy + gold-label deception
Proposed policy + predicted deception

68.15%
80.45%
79.32%

35.00%
80.00%
55.00%

Table 5: Dialog selection acts accuracy.

Next, we evaluate the performance of the system’s dialog acts
decision. The accuracy are measured with 2 metrics. DA accuracy
refers to the accuracy of the system’s chosen dialog acts againstreference actions that were chosen by human. Deception handlingindicates the accuracy of dialog acts decision when user is lying.
From the results of Table 5, we can see that our proposed systemoutperformed the baseline again.

4.

Conclusions

In this paper, we present a negotiation strategy that counter deception in doctor-patient conversation. We also propose a dialog
system that utilize this strategy and showed the construction andperformance of each module. Experiment results proved that the
proposed strategy outperformed normal negotiation strategy significantly, beating it by more than 8% in chance of successful persuasion. DA accuracy experiments indicated that the learned policy achieved a good level of naturalness when compared to humanbehavior in the same scenario. In future, we would like to combine
the modules and conduct experiments of the system with human.

References[1] Tadas Baltrušaitis, Peter Robinson, and Louis-Philippe
Morency. 2016. Openface: an open source facial behavior analysis toolkit. In Applications of Computer Vision(WACV), 2016 IEEE Winter Conference on. IEEE, pages 1–
10.
[2] Blai Bonet. 2002. An e-optimal grid-based algorithm for partially observable markov decision processes. In Proc. of the
19th Int. Conf. on Machine Learning (ICML-02).
[3] Ioannis Efstathiou and Oliver Lemon. 2014a. Learning noncooperative dialogue behaviours. In Prooceedings of the15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL). Association for Computational Linguistics, Philadelphia, PA, U.S.A., pages 60–68.
http://www.aclweb.org/anthology/W14-4308.

[16] http://articles.latimes.com/2009/jun/08/health/he-lying8[4] Florian Eyben, Martin Wollmer, and Bjorn Schuller. 2010.
Opensmile: the munich versatile and fast open-source audio
feature extractor. In Proceedings of the 18th ACM international conference on Multimedia. ACM, pages 1459–1462.
[5] Takuya Hiraoka, Graham Neubig, Sakriani Sakti, Tomoki
Toda, and Satoshi Nakamura. 2014. Reinforcement learningof cooperative persuasive dialogue policies using framing. In
COLING. pages 1706–1717.
[6] Julia Hirschberg, Stefan Benus, JasonMBrenier, Frank Enos,Sarah Friedman, Sarah Gilman, Cynthia Girand, Martin Graciarena, Andreas Kathol, Laura Michaelis, et al. 2005. Distinguishing deceptive from non-deceptive speech. In Interspeech. pages 1833–1836.
[7] Karin I Kjellgren, Johan Ahlner, and Roger Säljö. Taking
antihypertensive medication controlling or co-operating withpatients? International journal of cardiology, 47(3):257–268,
1995.
https://doi.org/10.1023/A:1005719122519.
[8] Verónica Pérez-Rosas, Mohamed Abouelenien, Rada Mihalcea, and Mihai Burzo. 2015a. Deception detection using reallife trial data. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. ACM, pages
59–66.
[9] Leimin Tian, Johanna Moore, and Catherine Lai. 2016. Recognizing emotions in spoken dialogue with hierarchically
fused acoustic and lexical features. In Spoken Language
Technology Workshop (SLT), 2016 IEEE. IEEE, pages 565–
572.
[10] Kristian Torning and Harri Oinas-Kukkonen. 2009. Persuasive system design: state of the art and future directions. In
Proceedings of the 4th international conference on persuasive technology. ACM, page 30.
[11] David R Traum. 2008. Computational models of noncooperative dialogue .
[12] Aimilios Vourliotakis, Ioannis Efstathiou, and Verena Rieser.
2014. Detecting deception in noncooperative dialogue: A
smarter adversary cannot be fooled that easily. In Proceedings of the 18th Workshop on the Semantics and Pragmatics
of Dialogue (SemDIAL). Edinburgh, Scotland, U.K., pages
252–254.
[13] Koichiro Yoshino and Tatsuya Kawahara. 2015. Conversational system for information navigation based on pomdp
with user focus tracking. Computer Speech & Language
34(1):275–291.
[14] https://www.wsj.com/articles/SB10001424127887323478004578306510461212692[15] https://www.zocdoc.com/about/news/new-zocdoc-studyreveals-women-are-more-likely-than-men-to-lie-to-doctors/