A Tool for Animated Agents in Network-Based Negotiation
Masahide Yuasa, Yoshiaki Yasumura, Katsumi Nitta
Department of Computational Intelligence and Systems Science,
Tokyo Institute of Technology,
Nagatsuta 4259, Yokohama, Japan
Email:{yuasa, yasumura, nitta}@ntt.dis.titech.ac.jp
Abstract
In this paper, we describe a tool for developing animated agents with facial expressions in negotiation
through a computer network. The tool learns a user’s
tendency to select facial expressions of the animatedagent, and generates facial expressions instead of human. In order to estimate facial expressions, the tool
has an emotional model constructed by Bayesian Network. We can easily develop animated agents if we
use this tool as a component. And we describe the estimation of an opponent’s emotional state, based on
observed data, by using the Bayesian Network.

1

Introduction

As computer networks have spread into our lives,
the opportunity that people communicate throughcomputer networks has increased. As current userinterfaces are not convenient for ordinary people tonegotiate with each other, it is required to develop
more user-friendly interfaces. For this purpose, several
researchers have developed applications of animatedagents with facial expressions, synthesized speech andgestures [1, 2]. However, they have not developed
practical application systems using animated agentsyet.
Our target is to develop animated agents which participate in the negotiation instead of a human. During
the negotiation, the agent generates next proposal andcontrols its facial expression. We also focused on the
facial expression of the animated agent in the negotiation, and we have conducted the research of controllingfacial expressions in the negotiation [3, 4].
We should pay attention that people use two kindsof facial expressions in negotiation [5]. The one is an
unintentional facial expression, which reflects a human’s emotional state directly just after he receives
the opponent’s proposal. The other is an intentional
facial expression, which is controlled intentionally according to some negotiation strategy when a user
sends the user’s proposal to the opponent.

Therefore, when we apply animated agents to the
network-based negotiation, we need a mechanism in
the agents which observes the situation of the negotiation and controls both kinds of facial expressions
according to the situation.
However, it is troublesome to implement animatedagents that control two facial expressions. We assume
that the unintentional facial expression depends on
only the each personality of a human and it is independent of the application domain. And we assume that
we can make the model of the personality by machinelearning technologies. Then, by learning the personality, the tool can generate the unintentional expression.
By implementing this part beforehand, we can reducea cost of developing the agent.
In this paper, we describe such a tool, named TAA(a Tool for Animated Agent), which constructs themodel of human personality and generates the unintentional facial expression. The model is used as apart of an animated-agent that negotiates instead ofa human. TAA uses an emotional model, which is anextension of ABX model which describes psychologicalrelationship between people [6].
In Section 2, we introduce an animated negotiation
agent. The emotional model in the architecture andexperimental results are shown in Section 3. Section
4 presents the estimation of an opponent’s emotional
states. Finally, Section 5 is the concluding remarks.

2
2.1

Animated Negotiation Agent
Functions

In order to negotiate with an opponent through a
computer network, a user interface of negotiation has
been developed. In this interface, the user exchanges
not only a proposal but also animated facial expressions, because the facial expression has an importantrole in negotiation. By using the interface, the user
inputs a proposal and selects a facial expression in apanel. Then, they are sent to the opponent (Fig. 1).
Fig. 2 shows facial expressions that the user can selectFigure 1: User Interface of network-based negotiation

Figure 2: Facial Expressions(i.e., COOL, ANGRY, HAPPY, SAD, SURPRISED).
Usually, participants of the online negotiation exchange their proposals and their facial expressions using this user interface. However, if the participant isbusy, the user may use the animated negotiation agent
temporarily. The agent generates new proposal andnew facial expression and sends them to the opponent
instead of the user.
Examples of the application areas of an animatednegotiation agent are a plaintiff or a defendant of an
online court, a trader of e-commerce, and a bidder ofan online auction.

2.2

Architecture and Modules

Fig. 3 shows the architecture of an animated negotiation agent. The lower part of this figure is a userinterface. The middle part and the upper part of thisfigure show the sub-modules of the animated negotiation agent. The middle part shows TAA, whichfunction is to generate unintentional expressions. It
depends on the user’s personality and is independentof the domain knowledge. The upper part shows modules for each application modules. It generates a newproposal and a new intentional facial expression.
(1) User Interface• Message Management ModuleThis module transmits messages (a proposal
and two types of facial expressions) among an

Figure 3: Architecture of agent
opponent, sub-modules of the animated thisagent, and ”Operating Panel”. When a proposal and facial expressions are come from the
opponent, they are sent to ”Proposal Evaluation Module”, ”Next Move and Intentional Facial Expression Generation Module”, and ”Operating Panel”. And the messages made by”Unintentional Facial Expression Generation
Module”, ”Next Move and Intentional Facial
Expression Generation Module”, and ”Operating Panel” is sent to the opponent.
• Operating Panel
An ”Operating Panel” is used when the user negotiates with the opponent without the agent.
This panel displays the opponent’s animationon the screen. The animation talks the opponent’s proposal with changes of the facial expressions. At first, the user inputs the user’s
unintentional facial expression using this panel,and then the user inputs the user’s new proposal and the intentional facial expression (Fig.
1). This information is sent to ”Message Management Module” and ”Learning Module”.
(2) TAATAA constructs the model of the personality of
the user, and generates unintentional facial expressions. According to these two functions, TAA
has two modes, a learning mode and an automaticmode. In the learning mode, TAA learns the human’s tendency when the user uses the unintentional facial expression after the user receives a
proposal from the opponent. In the automaticmode, TAA works as a part of an agent and generates the unintentional facial expression insteadof the user.
• Emotional Model
This model represents the user’s personality. It
consists of 9 emotional states. During the negotiation, when a user receives new proposal
and new facial expression from the opponent,
the user’s emotion moves from one state tothe other state, and the user’s unintentional
facial expression is generated when the statetransition occurs. The personality of a user isreflected as the state transition probabilities.
Using these probabilities, Bayesian Network is
constructed which is used to estimate the facial
expressions of the user. We explain a detail ofthe emotional model in the next section.
• Learning ModuleThis module observes the user’s actions (theuser’s proposal and facial expressions), andthen it generates the state transition probabilities. The result is used for constructing theemotional model.
• Unintentional Facial Expression Generation
ModuleThis module generates the unintentional facial
expression, based on the emotional model. The
generated facial expression is sent to ”Message
Management Module”.
(3) Application
The modules of this part are application programs
which generates the new proposal and the newintentional facial expressions. These modules areused only in automatic mode.
• Proposal Evaluation ModuleThis module evaluates a proposal, based on an
evaluation function, and outputs an evaluation
value as a real number between 0 and 10. The
user needs to prepare the evaluation function inadvance. The evaluation value is sent to ”Unintentional Facial Expression Generation Module”.

• Next Move and Intentional Facial Expression
Generation ModuleThis module makes a new proposal and a new
intentional facial expression and sends them to”Message Management Module”.

2.3

Flow of Message

During negotiation, several messages are transmitted among sub-modules. Fig. 4 shows example messages among the modules. (M1) - (M4) in Fig. 4 correspond them in Fig. 3. In this example, the agent receives (M1) from an opponent, makes (M3) and (M4),and sent them to the opponent. (M1) means a proposal and a facial expression from the opponent. (M3)
is a new message for the opponent with an unintentional facial expression from Unintentional Facial Expressions Generation Module. (M4) is a new messageof an intentional facial expression and a next proposal
for the opponent. (M2) is a message with an evaluation value of the opponent’s proposal from Proposal
Evaluation Module.
In a message, the ”action” means an action in negotiation. It takes one of ”propose”, ”concede”, ”deny”,”ask”, and ”answer”. The ”character” is a name ofcharacter. Users can select one from some characters. In this example, Taka uses ”zen” and Yuasa
uses ”jony”. The ”evaluation value” is an evaluation
value between 0 and 10, which is calculated by Proposal Evaluation Module. The ”face” is a facial expression against a receiver (i.e., HAPPY, ANGRY,
COOL, etc.). In this message, the emotional model
selects the ”COOL” as an unintentional facial expression and it is sent. The proposal and ”HAPPY” areselected by the user’s module as an intentional expression. The ”sender” and the ”receiver” are a sender anda receiver of the message.

3
3.1

Generation of Unintentional Facial
Expressions
Emotional Model

The emotional model in TAA is based on Newcomb’s ABX model [6] (Fig. 5). The ABX model describes emotional states among two participants andone proposal. The relation among two of these entities is indicated by a plus sign, minus sign, or 0. The
plus sign means a person has a positive feeling. The
minus sign means a person has a negative feeling. The
0 means intermediate state.
In this model, each state is categorized into unbalanced states and balanced states. If one person
has both positive relation for one entity and negative
relation for another entity, then the state is unbalanced. Otherwise, the state is balanced. The unbal-

(M1) <MESSAGE action=”propose” character=”jony.acf”
face=”HAPPY” sender=”yuasa” receiver=”taka” >I want to sell it at $ 50.
</MESSAGE>(M2) <MESSAGE character=”jony.acf”
face=”HAPPY” sender=”yuasa” receiver=”taka”
evaluation value=”5” ></MESSAGE>(M3) <MESSAGE character=”zen.acf”
face=”COOL” sender=”taka” receiver=”yuasa” ></MESSAGE>(M4) <MESSAGE action=”propose” character=”zen.acf”
face=”HAPPY” sender=”taka” receiver=”yuasa” >I want to buy it at $ 35.
</MESSAGE>

Figure 5: Emotional States for B

Figure 4: Messages among modules
anced states are unstable and tend to move to thebalanced states.
In the case of negotiation, we regards A and B as
participants, and X as a current proposal. And weview arrows from participants to X as evaluation values of a proposal by participants [3, 4].
During the negotiation, when a participant receives
a proposal from the opponent, the state transition occurs. When a participant B offers a proposal X, B
must like X. Therefore, just after A receives B’s proposal X, A and B’s emotional state must be one of 9
states of Fig. 5. The state (S1) and state (S9) areunbalanced states and the other balanced states.
We assume that the participant’s emotional statetransits from one state to another state with someprobability, when the participant receives a new proposal and a facial expression. And one of unintentional
facial expressions is generated with some probability,
when the transition occurs. We use Bayesian Networkin order to learn the probabilistic relations (Fig. 6).
The network shows that the transition from a previous emotional state (P3) to a current emotional state(P4) when the user receives a new proposal (P1) and afacial expression (P2). And a facial expression is generated when the transition from (P3) to (P4) occurs
(P5).

3.2

Learning Emotional Model

The probabilities among the emotional states depend on a personality of a human. An emotionalmodel is defined as probabilities of state transitions
and generating unintentional facial expressions. To

Figure 6: The generation of facial expressions by
Bayesian Network
construct the emotional model, TAA needs to learn
these probabilities from a participant.
We carried out the experiment of learning the emotional model in TAA. At first, we acquired probabilities by observing users’ selections in an experimentusing price negotiation. Next, we compared a selection of the users’ and that of TAA.
The interface of learning is shown in Fig. 1. Fig.
2 shows facial expressions in the experiment (i.e.,
COOL, ANGRY, HAPPY, SAD, SURPRISED). We
used MFACE[7] to display facial expressions. In thisnegotiation, users play a role of buyer and a program
plays a role of seller, but users don’t know whether theseller is human or not. And users must input an evaluation (+, 0, or -) of a feeling against the opponent
on the panel.
Eleven students conducted the negotiation game.
Fig. 7 shows a sample result after the tool learned

Table 2: The accuracy of estimating facial expressions
by using the emotional model of X and Y

Test data

X
Y

Figure 7: The case of (+) and (HAPPY)
Table 1: The accuracy of estimating facial expressions by using emotional model of the first-half andthe second-half
XY(1)First - Second
62.3 %
52.8 %(2) Second - First
73.0 %
68.5 %

the probabilities of transitions among emotional statesand generations. Probabilities in Fig. 7 means existence probabilities of the each states and facial expressions. We used a Bayesian Network Tool ”Netica” tocalculate existence probabilities [8].
The upper right part of Fig. 7 shows the existence
probabilities of ”current emotional states” when theuser received (+) in proposals and (HAPPY) in the facial expressions, at the state was ”previous emotional
states”. And the lower part shows the existence probabilities of facial expressions at that time. In that case,the probability of S3 has an 86.7% and the probabilityof HAPPY has a 79.7%. This means that when theuser receives a good proposal and good facial expressions, the state tends to transit a better one, and theuser selected the HAPPY in many times. As above,
our emotional model is a good model, which can consider the feelings for the opponent and the transitionsof the emotional states.
Next, we tested an accuracy of the estimating theuser’s facial expression by the tool. We selected two
students, X and Y, in this experiment. And we divided the students’ data into two parts, and used theone of them as Training data and the other of themas Test data. We compared the following two cases.
First case is that we use first-half as Training data

Training data
X
Y
X-X
Y-X
73.0 % 19.4 %X-Y
Y-Y
48.8 %
68.5%

and second-half as Test data. Second case is that we
use second-half as Training data and first-half as Test
data. Table 1 is the result of the accuracy of the estimation. The table shows the accuracy by the tool has
a high percent in both cases and the users’ selectionof facial expressions is stable.
Table 2 shows an accuracy of estimating the facial
expressions by using the emotional model of X and Y.
We compared two cases; one case is that X as Training
data and Y as Test data. The other case is that we use
Y as Training data and X as Test data. In this table,”X-Y” means that we compared the TAA’s selection
with Y’s selection, after TAA learned the pattern of
X. In this table, the accuracy by TAA has a 73.0%
in case of X-X. In the same way, the accuracy has a
68.5% in case of Y-Y. However, in case of X-Y, it theaccuracy has a 48.8% and in case of Y-X, the accuracy
has only 19.4%. These result shows that TAA learns
the user’s pattern for each individuals and generates
the unintentional facial expression appropriately.

4

Estimation of an Opponent’s Emotional StateIn the previous section, we showed that if the opponent’s proposal and facial expression are given, thefacial expression selected by the user could be estimated using Bayesian Network. In the same way, if
the user’s proposal and facial expression and the opponent’s facial expression are given, we can estimate
the opponent’s emotional state by the Bayesian Network of the opponent. If the opponent’s emotional
state is estimated, the user will succeed in the negotiation because the user can select a proposal and afacial expression to take more advantage.

4.1

Method of Estimating an Opponent’s
Emotional State

In order to estimate the opponent’s emotional state,
we use Bayesian Network in Section 3. In the network,
the opponent’s emotional state is estimated from theuser’s proposal and facial expressions to the opponent,
and the opponent’s facial expressions.
For example, Fig. 8 shows existence probabilitiesof the opponent’s emotional state, when (P1), (P2)The experimental result showed that the agent
could express the unintentional facial expression appropriately and the tool could estimate the opponent’s
emotional state using Bayesian Network.
As another application of the animated agent, wehave been making an online ADR (Alternative Dispute
Resolution) system. This is a mediation system fortwo participants to solve a dispute through computer
network. We have developed an animated mediator
agent which mediates participants in the ADR system.

References
Figure 8: The example of the estimation of an opponent emotional state
Table 3: The estimation of an opponent’s emotional
state.

Opponent’s data
Average data

Facial Expressions
use
don’t use
71.0 %
48.3 %
45.5 %
42.3 %and (P5) are given. (P1) is a proposal (+), (P2) is(HAPPY) and (P5) is (HAPPY). In this case, thestates have the probabilities; (S3) has an 88.3% and(S2) has an 11.7%.

4.2

Experiment

We tested an accuracy of estimating the opponent’s
emotional state. We compared two cases. First caseis that we know the opponent’s Bayesian Network accurately. Second case is that as we don’t know the
opponent’s Bayesian Network, we use the Bayesian
Network of average people instead of the opponent’s
one. For each case, at first, we experimented the performance of estimation using the opponent’s facial expression, and then we investigated the performancenot using opponent’s facial expression.
Table 3 is the result of the accuracy of the estimation. In the case of using the opponent data and facial
expressions as a hint of the estimation, the accuracy of
estimation has a 71.0%. However, in the other cases,
the accuracy has a low percent.

5

Conclusion

We developed a tool to make an animated agent
in negotiation, based on the emotional model constructed by Bayesian Network. And this tool can select the user’s facial expressions instead of the user.
The user could make the agent easily and reduce a
cost of developing the agent.

[1] Justine Cassell, Hannes Vilhjalmsson and Tim
Bickmore, “BEAT: the Behavior Expression Animation Toolkit”, In Proceeding of SIGGRAPH 01,
2001.
[2] Tsutsui, Saeyor and Ishizuka, “MPML: A Multimodal Presentation Markup Language with Character Agent Control Functions”, In Proceeding of
WebNet 2000 World Conference. on the WWW
and Internet, San Antonio, Texas, USA, 2000.
[3] Masahide Yuasa, Yoshiaki Yasumura and Katsumi
Nitta, “Negotiation Support System Using Physiological Information”, In Proceeding of IEEESMC2000, pp. 248-253, 2000.
[4] Masahide Yuasa, Yoshiaki Yasumura and Katsumi
Nitta, “Negotiation Support Tool Using Emotional
Factors”, In Proceeding of IFSA-NAFIPS2001, pp.
2096-2911, 2001.
[5] Ekman, P., Friesen, W.V, “Unmasking the face”,
Prentice-Hall, 1975.
[6] Theodore M. Newcomb, “An Approach to the
Study of Communicative Acts”, In Psychological
Review, pp. 393-404, No. 60, 1953.
[7] Osamu Hasegawa and Katsuhiko Sakaue, “CG
Tool for Constructing Anthropomorphic Interface
Agents”, Proc. IJCAI-97 WS (W5) , ANIMATEDINTERFACE AGENTS, pp.23-26, 1997-8[8] Netica, http://www.norsys.com/