multi-issu negoti
crisi manag
decis make
multilater negoti
model
negoti protocol
multi-agent system
autom negoti
cooper agent
myriad
autom negoti
negoti strategi
multi-criterion decis make

A Multilateral Multi-issue Negotiation Protocol Miniar Hemaissia THALES Research & Technology France RD 128 F-91767 Palaiseau Cedex, France miniar.hemaissia@lip6.fr Amal El Fallah Seghrouchni LIP6, University of Paris 6 8 rue du Capitaine Scott F-75015 Paris, France amal.elfallah@lip6.fr Christophe Labreuche and Juliette Mattioli THALES Research & Technology France RD 128 F-91767 Palaiseau Cedex, France ABSTRACT In this paper, we present a new protocol to address multilateral multi-issue negotiation in a cooperative context.
We consider complex dependencies between multiple issues by modelling the preferences of the agents with a multi-criteria decision aid tool, also enabling us to extract relevant information on a proposal assessment.
This information is used in the protocol to help in accelerating the search for a consensus between the cooperative agents.
In addition, the negotiation procedure is defined in a crisis management context where the common objective of our agents is also considered in the preferences of a mediator agent.
Categories and Subject Descriptors I.2.11 [Distributed Artificial Intelligence]: Intelligent agents, Multiagent systems General Terms Theory, Design, Experimentation 1.
INTRODUCTION Multi-issue negotiation protocols represent an important field of study since negotiation problems in the real world are often complex ones involving multiple issues.
To date, most of previous work in this area ([2, 3, 19, 13]) dealt almost exclusively with simple negotiations involving independent issues.
However, real-world negotiation problems involve complex dependencies between multiple issues.
When one wants to buy a car, for example, the value of a given car is highly dependent on its price, consumption, comfort and so on.
The addition of such interdependencies greatly complicates the agents utility functions and classical utility functions, such as the weighted sum, are not sufficient to model this kind of preferences.
In [10, 9, 17, 14, 20], the authors consider inter-dependencies between issues, most often defined with boolean values, except for [9], while we can deal with continuous and discrete dependent issues thanks to the modelling power of the Choquet integral.
In [17], the authors deal with bilateral negotiation while we are interested in a multilateral negotiation setting.
Klein et al. [10] present an approach similar to ours, using a mediator too and information about the strength of the approval or rejection that an agent makes during the negotiation.
In our protocol, we use more precise information to improve the proposals thanks to the multi-criteria methodology and tools used to model the preferences of our agents.
Lin, in [14, 20], also presents a mediation service but using an evolutionary algorithm to reach optimal solutions and as explained in [4], players in the evolutionary models need to repeatedly interact with each other until the stable state is reached.
As the population size increases, the time it takes for the population to stabilize also increases, resulting in excessive computation, communication, and time overheads that can become prohibitive, and for one-to-many and many-to-many negotiations, the overheads become higher as the number of players increases.
In [9], the authors consider a non-linear utility function by using constraints on the domain of the issues and a mediation service to find a combination of bids maximizing the social welfare.
Our preference model, a nonlinear utility function too, is more complex than [9] one since the Choquet integral takes into account the interactions and the importance of each decision criteria/issue, not only the dependencies between the values of the issues, to determine the utility.
We also use an iterative protocol enabling us to find a solution even when no bid combination is possible.
In this paper, we propose a negotiation protocol suited for multiple agents with complex preferences and taking into account, at the same time, multiple interdependent issues and recommendations made by the agents to improve a proposal.
Moreover, the preferences of our agents are modelled using a multi-criteria methodology and tools enabling us to take into account information about the improvements that can be made to a proposal, in order to help in accelerating the search for a consensus between the agents.
Therefore, we propose a negotiation protocol consisting of solving our decision problem using a MAS with a multi-criteria decision aiding modelling at the agent level and a cooperation-based multilateral multi-issue negotiation protocol.
This protocol is studied under a non-cooperative approach and it is shown 943 978-81-904262-7-5 (RPS) c 2007 IFAAMAS that it has subgame perfect equilibria, provided that agents behave rationally in the sense of von Neumann and Morgenstern.
The approach proposed in this paper has been first introduced and presented in [8].
In this paper, we present our first experiments, with some noteworthy results, and a more complex multi-agent system with representatives to enable us to have a more robust system.
In Section 2, we present our application, a crisis management problem.
Section 3 deals with the general aspect of the proposed approach.
The preference modelling is described in sect.
4, whereas the motivations of our protocol are considered in sect.
5 and the agent/multiagent modelling in sect.
6.
Section 7 presents the formal modelling and properties of our protocol before presenting our first experiments in sect.
8.
Finally, in Section 9, we conclude and present the future work.
2.
CASE STUDY This protocol is applied to a crisis management problem.
Crisis management is a relatively new field of management and is composed of three types of activities: crisis prevention, operational preparedness and management of declared crisis.
The crisis prevention aims to bring the risk of crisis to an acceptable level and, when possible, avoid that the crisis actually happens.
The operational preparedness includes strategic advanced planning, training and simulation to ensure availability, rapid mobilisation and deployment of resources to deal with possible emergencies.
The management of declared crisis is the response to - including the evacuation, search and rescue - and the recovery from the crisis by minimising the effects of the crises, limiting the impact on the community and environment and, on a longer term, by bringing the community``s systems back to normal.
In this paper, we focus on the response part of the management of declared crisis activity, and particularly on the evacuation of the injured people in disaster situations.
When a crisis is declared, the plans defined during the operational preparedness activity are executed.
For disasters, master plans are executed.
These plans are elaborated by the authorities with the collaboration of civil protection agencies, police, health services, non-governmental organizations, etc..
When a victim is found, several actions follow.
First, a rescue party is assigned to the victim who is examined and is given first aid on the spot.
Then, the victims can be placed in an emergency centre on the ground called the medical advanced post.
For all victims, a sorter physician - generally a hospital physician - examines the seriousness of their injuries and classifies the victims by pathology.
The evacuation by emergency health transport if necessary can take place after these clinical examinations and classifications.
Nowadays, to evacuate the injured people, the physicians contact the emergency call centre to pass on the medical assessments of the most urgent cases.
The emergency call centre then searches for available and appropriate spaces in the hospitals to care for these victims.
The physicians are informed of the allocations, so they can proceed to the evacuations choosing the emergency health transports according to the pathologies and the transport modes provided.
In this context, we can observe that the evacuation is based on three important elements: the examination and classification of the victims, the search for an allocation and the transport.
In the case of the 11 March 2004 Madrid attacks, for instance, some injured people did not receive the appropriate health care because, during the search for space, the emergency call centre did not consider the transport constraints and, in particular, the traffic.
Therefore, for a large scale crisis management problem, there is a need to support the emergency call centre and the physicians in the dispatching to take into account the hospitals and the transport constraints and availabilities.
3.
PROPOSED APPROACH To accept a proposal, an agent has to consider several issues such as, in the case of the crisis management problem, the availabilities in terms of number of beds by unit, medical and surgical staffs, theatres and so on.
Therefore, each agent has its own preferences in correlation with its resource constraints and other decision criteria such as, for the case study, the level of congestion of a hospital.
All the agents also make decisions by taking into account the dependencies between these decision criteria.
The first hypothesis of our approach is that there are several parties involved in and impacted by the decision, and so they have to decide together according to their own constraints and decision criteria.
Negotiation is the process by which a group facing a conflict communicates with one another to try and come to a mutually acceptable agreement or decision and so, the agents have to negotiate.
The conflict we have to resolve is finding an acceptable solution for all the parties by using a particular protocol.
In our context, multilateral negotiation is a negotiation protocol type that is the best suited for this type of problem : this type of protocol enables the hospitals and the physicians to negotiate together.
The negotiation also deals with multiple issues.
Moreover, an other hypothesis is that we are in a cooperative context where all the parties have a common objective which is to provide the best possible solution for everyone.
This implies the use of a negotiation protocol encouraging the parties involved to cooperate as satisfying its preferences.
Taking into account these aspects, a Multi-Agent System (MAS) seems to be a reliable method in the case of a distributed decision making process.
Indeed, a MAS is a suitable answer when the solution has to combine, at least, distribution features and reasoning capabilities.
Another motivation for using MAS lies in the fact that MAS is well known for facilitating automated negotiation at the operative decision making level in various applications.
Therefore, our approach consists of solving a multiparty decision problem using a MAS with • The preferences of the agents are modelled using a multi-criteria decision aid tool, MYRIAD, also enabling us to consider multi-issue problems by evaluating proposals on several criteria.
• A cooperation-based multilateral and multi-issue negotiation protocol.
4.
THE PREFERENCE MODEL We consider a problem where an agent has several decision criteria, a set Nk = {1, ... , nk} of criteria for each agent k involved in the negotiation protocol.
These decision criteria enable the agents to evaluate the set of issues that are negotiated.
The issues correspond directly or not to the decision criteria.
However, for the example of the crisis management 944 The Sixth Intl..
Joint Conf.
on Autonomous Agents and Multi-Agent Systems (AAMAS 07) problem, the issues are the set of victims to dispatch between the hospitals.
These issues are translated to decision criteria enabling the hospital to evaluate its congestion and so to an updated number of available beds, medical teams and so on.
In order to take into account the complexity that exists between the criteria/issues, we use a multi-criteria decision aiding (MCDA) tool named MYRIAD [12] developed at Thales for MCDA applications based on a two-additive Choquet integral which is a good compromise between versatility and ease to understand and model the interactions between decision criteria [6].
The set of the attributes of Nk is denoted by Xk 1 , ... , Xk nk .
All the attributes are made commensurate thanks to the introduction of partial utility functions uk i : Xk i → [0, 1].
The [0, 1] scale depicts the satisfaction of the agent k regarding the values of the attributes.
An option x is identified to an element of Xk = Xk 1 × · · · × Xk nk , with x = (x1, ... , xnk ).
Then the overall assessment of x is given by Uk(x) = Hk(uk 1 (x1), ... , uh nk (xnk )) (1) where Hk : [0, 1]nk → [0, 1] is the aggregation function.
The overall preference relation over Xk is then x y ⇐⇒ Uk(x) ≥ Uk(y) .
The two-additive Choquet integral is defined for (z1, ... , znk ) ∈ [0, 1]nk by [7] Hk(z1, ... , znk ) = X i∈Nk 0 @vk i − 1 2 X j=i |Ik i,j| 1 A zi + X Ik i,j >0 Ik i,j zi ∧ zj + X Ii,j <0 |Ii,j| zi ∨ zj (2) where vk i is the relative importance of criterion i for agent k and Ik i,j is the interaction between criteria i and j, ∧ and ∨ denote the min and max functions respectively.
Assume that zi < zj.
A positive interaction between criteria i and j depicts complementarity between these criteria (positive synergy) [7].
Hence, the lower score of z on criterion i conceals the positive effect of the better score on criterion j to a larger extent on the overall evaluation than the impact of the relative importance of the criteria taken independently of the other ones.
In other words, the score of z on criterion j is penalized by the lower score on criterion i. Conversely, a negative interaction between criteria i and j depicts substitutability between these criteria (negative synergy) [7].
The score of z on criterion i is then saved by a better score on criterion j.
In MYRIAD, we can also obtain some recommendations corresponding to an indicator ωC (H, x) measuring the worth to improve option x w.r.t. Hk on some criteria C ⊆ Nk as follows ωC (Hk, x)= Z 1 0 Hk ` (1 − τ)xC + τ, xNk\C ´ − Hk(x) EC (τ, x) dτ where ((1−τ)xC +τ, xNk\C ) is the compound act that equals (1 − τ)xi + τ if i ∈ C and equals xi if i ∈ Nk \ C. Moreover, EC (τ, x) is the effort to go from the profile x to the profile ((1 − τ)xC + τ, xNk\C ).
Function ωC (Hk, x) depicts the average improvement of Hk when the criteria of coalition A range from xC to 1C divided by the average effort needed for this improvement.
We generally assume that EC is of order 1, that is EC (τ, x) = τ P i∈C (1 − xi).
The expression of ωC (Hk, x) when Hk is a Choquet integral, is given in [11].
The agent is then recommended to improve of coalition C for which ωC (Hk, x) is maximum.
This recommendation is very useful in a negotiation protocol since it helps the agents to know what to do if they want an offer to be accepted while not revealing their own preference model.
5.
PROTOCOL MOTIVATIONS For multi-issue problems, there are two approaches: a complete package approach where the issues are negotiated simultaneously in opposition to the sequential approach where the issues are negotiated one by one.
When the issues are dependant, then it is the best choice to bargain simultaneously over all issues [5].
Thus, the complete package is the adopted approach so that an offer will be on the overall set of injured people while taking into account the other decision criteria.
We have to consider that all the parties of the negotiation process have to agree on the decision since they are all involved in and impacted by this decision and so an unanimous agreement is required in the protocol.
In addition, no party can leave the process until an agreement is reached, i.e. a consensus achieved.
This makes sense since a proposal concerns all the parties.
Moreover, we have to guarantee the availability of the resources needed by the parties to ensure that a proposal is realistic.
To this end, the information about these availabilities are used to determine admissible proposals such that an offer cannot be made if one of the parties has not enough resources to execute/achieve it.
At the beginning of the negotiation, each party provides its maximum availabilities, this defining the constraints that have to be satisfied for each offer submitted.
The negotiation has also to converge quickly on an unanimous agreement.
We decided to introduce in the negotiation protocol an incentive to cooperate taking into account the passed negotiation time.
This incentive is defined on the basis of a time dependent penalty, the discounting factor as in [18] or a time-dependent threshold.
This penalty has to be used in the accept/reject stage of our consensus procedure.
In fact, in the case of a discounting factor, each party will accept or reject an offer by evaluating the proposal using its utility function deducted from the discounting factor.
In the case of a time-dependent threshold, if the evaluation is greater or equal to this threshold, the offer is accepted, otherwise, in the next period, its threshold is reduced.
The use of a penalty is not enough alone since it does not help in finding a solution.
Some information about the assessments of the parties involved in the negotiation are needed.
In particular, it would be helpful to know why an offer has been rejected and/or what can be done to make a proposal that would be accepted.
MYRIAD provides an analysis that determines the flaws an option, here a proposal.
In particular, it gives this type of information: which criteria of a proposal should be improved so as to reach the highest possible overall evaluation [11].
As we use this tool to model the parties involved in the negotiation, the information about the criteria to improve can be used by the mediator to elaborate the proposals.
We also consider that the dual function can be used to take into account another type of information: on which criteria of a proposal, no improvement is necessary so that the overall evaluation of a proposal is still acceptable, do not decrease.
Thus, all information is a constraint to be satisfied as much as possible by The Sixth Intl..
Joint Conf.
on Autonomous Agents and Multi-Agent Systems (AAMAS 07) 945 Figure 1: An illustration of some system.
the parties to make a new proposal.
We are in a cooperative context and revealing one``s opinion on what can be improved is not prohibited, on the contrary, it is useful and recommended here seeing that it helps in converging on an agreement.
Therefore, when one of the parties refuses an offer, some information will be communicated.
In order to facilitate and speed up the negotiation, we introduce a mediator.
This specific entity is in charge of making the proposals to the other parties in the system by taking into account their public constraints (e.g. their availabilities) and the recommendations they make.
This mediator can also be considered as the representative of the general interest we can have, in some applications, such as in the crisis management problem, the physician will be the mediator and will also have some more information to consider when making an offer (e.g. traffic state, transport mode and time).
Each party in a negotiation N, a negotiator, can also be a mediator of another negotiation N , this party becoming the representative of N in the negotiation N, as illustrated by fig. 1 what can also help in reducing the communication time.
6.
AGENTIFICATION How the problem is transposed in a MAS problem is a very important aspect when designing such a system.
The agentification has an influence upon the systems efficiency in solving the problem.
Therefore, in this section, we describe the elements and constraints taken into account during the modelling phase and for the model itself.
However, for this negotiation application, the modelling is quite natural when one observes the negotiation protocol motivations and main properties.
First of all, it seems obvious that there should be one agent for each player of our multilateral multi-issue negotiation protocol.
The agents have the involved parties'' information and preferences.
These agents are: • Autonomous: they decide for themselves what, when and under what conditions actions should be performed; • Rational: they have a means-ends competence to fit its decisions, according to its knowledge, preferences and goal; • Self-interested: they have their own interests which may conflict with the interests of other agents.
Moreover, their preferences are modelled and a proposal evaluated and analysed using MYRIAD.
Each agent has private information and can access public information as knowledge.
In fact, there are two types of agents: the mediator type for the agents corresponding to the mediator of our negotiation protocol, the delegated physician in our application, and the negotiator type for the agents corresponding to the other parties, the hospitals.
The main behaviours that an agent of type mediator needs to negotiate in our protocol are the following: • convert_improvements: converts the information given by the other agents involved in the negotiation about the improvements to be done, into constraints on the next proposal to be made; • convert_no_decrease: converts the information given by the other agents involved in the negotiation about the points that should not be changed into constraints on the next proposal to be made; • construct_proposal: constructs a new proposal according to the constraints obtained with convert_improvements, convert_no_decrease and the agent preferences; The main behaviours that an agent of type negotiator needs to negotiate in our protocol are the following: • convert_proposal: converts a proposal to a MYRIAD option of the agent according to its preferences model and its private data; • convert_improvements_wc: converts the agent recommendations for the improvements of a MYRIAD option into general information on the proposal; • convert_no_decrease_wc: converts the agent recommendations about the criteria that should not be changed in the MYRIAD option into general information on the proposal; In addition to these behaviours, there are, for the two types of agents, access behaviours to MYRIAD functionalities such as the evaluation and improvement functions: • evaluate_option: evaluates the MYRIAD option obtained using the agent behaviour convert_proposal; • improvements: gets the agent recommendations to improve a proposal from the MYRIAD option; • no_decrease: gets the agent recommendations to not change some criteria from the MYRIAD option; Of course, before running the system with such agents, we must have defined each party preferences model in MYRIAD.
This model has to be part of the agent so that it could be used to make the assessments and to retrieve the improvements.
In addition to these behaviours, the communication acts between the agents is as follows.
1.
mediator agent communication acts: 946 The Sixth Intl..
Joint Conf.
on Autonomous Agents and Multi-Agent Systems (AAMAS 07) m1 m 1 1 m inform1 m mediator negotiator accept−proposal l 1 accept−proposal m−l reject−proposal propose propose Figure 2: The protocol diagram in AUML, and where m is the number of negotiator agents and l is the number of agents refusing current proposal.
(a) propose: sends a message containing a proposal to all negotiator agents; (b) inform: sends a message to all negotiator agents to inform them that an agreement has been reached and containing the consensus outcome.
2.
negotiator agent communication acts: (a) accept-proposal: sends a message to the mediator agent containing the agent recommendations to improve the proposal and obtained with convert_improvements_wc; (b) reject-proposal: sends a message to the mediator agent containing the agent recommendations about the criteria that should not be changed and obtained with convert_no_decrease_wc.
Such agents are interchangeable, in a case of failure, since they all have the same properties and represent a user with his preference model, not depending on the agent, but on the model defined in MYRIAD.
When the issues and the decision criteria are different from each other, the information about the criteria improvement have to be pre-processed to give some instructions on the directions to take and about the negotiated issues.
It is the same for the evaluation of a proposal: each agent has to convert the information about the issues to update its private information and to obtain the values of each attribute of the decision criteria.
7.
OUR PROTOCOL Formally, we consider negotiations where a set of players A = {1, 2, ... , m} and a player a are negotiating over a set Q of size q.
The player a is the protocol mediator, the mediator agent of the agentification.
The utility/preference function of a player k ∈ A ∪ {a} is Uk, defined using MYRIAD, as presented in Section 4, with a set Nk of criteria, Xk an option, and so on.
An offer is a vector P = (P1, P2, · · · , Pm), a partition of Q, in which Pk is player k``s share of Q.
We have P ∈ P where P is the set of admissible proposals, a finite set.
Note that P is determined using all players general constraints on the proposals and Q. Moreover, let ˜P denote a particular proposal defined as a``s preferred proposal.
We also have the following notation: δk is the threshold decrease factor of player k, Φk : Pk → Xk is player k``s function to convert a proposal to an option and Ψk is the function indicating which points P has to be improved, with Ψk its dual function - on which points no improvement is necessary.
Ψk is obtained using the dual function of ωC (Hk, x): eωC (Hk, x)= Z 1 0 Hk(x) − Hk ` τ xC , xNk\C ´ eEC (τ, x) dτ Where eEC (τ, x) is the cost/effort to go from (τxC , xNk\C ) to x.
In period t of our consensus procedure, player a proposes an agreement P. All players k ∈ A respond to a by accepting or rejecting P.
The responses are made simultaneously.
If all players k ∈ A accept the offer, the game ends.
If any player k rejects P, then the next period t+1 begins: player a makes another proposal P by taking into account information provided by the players and the ones that have rejected P apply a penalty.
Therefore, our negotiation protocol can be as follows: Protocol P1.
• At the beginning, we set period t = 0 • a makes a proposal P ∈ P that has not been proposed before.
• Wait that all players of A give their opinion Yes or No to the player a.
If all players agree on P, this later is chosen.
Otherwise t is incremented and we go back to previous point.
• If there is no more offer left from P, the default offer ˜P will be chosen.
• The utility of players regarding a given offer decreases over time.
More precisely, the utility of player k ∈ A at period t regarding offer P is Uk(Φk(Pk), t) = ft(Uk(Φk(Pk))), where one can take for instance ft(x) = x.(δk)t or ft(x) = x − δk.t, as penalty function.
Lemma 1.
Protocol P1 has at least one subgame perfect equilibrium 1 .
Proof : Protocol P1 is first transformed in a game in extensive form.
To this end, one shall specify the order in which the responders A react to the offer P of a.
However the order in which the players answer has no influence on the course of the game and in particular on their personal utility.
Hence protocol P1 is strictly equivalent to a game in 1 A subgame perfect equilibrium is an equilibrium such that players'' strategies constitute a Nash equilibrium in every subgame of the original game [18, 16].
A Nash equilibrium is a set of strategies, one for each player, such that no player has incentive to unilaterally change his/her action [15].
The Sixth Intl..
Joint Conf.
on Autonomous Agents and Multi-Agent Systems (AAMAS 07) 947 extensive form, considering any order of the players A.
This game is clearly finite since P is finite and each offer can only be proposed once.
Finally P1 corresponds to a game with perfect information.
We end the proof by using a classical result stating that any finite game in extensive form with perfect information has at least one subgame perfect equilibrium (see e.g. [16]).
Rational players (in the sense of von Neumann and Morgenstern) involved in protocol P1 will necessarily come up with a subgame perfect equilibrium.
Example 1.
Consider an example with A = {1, 2} and P = {P1 , P2 , P3 } where the default offer is P1 .
Assume that ft(x) = x − 0.1 t. Consider the following table giving the utilities at t = 0.
P1 P2 P3 a 1 0.8 0.7 1 0.1 0.7 0.5 2 0.1 0.3 0.8 It is easy to see that there is one single subgame perfect equilibrium for protocol P1 corresponding to these values.
This equilibrium consists of the following choices: first a proposes P3 ; player 1 rejects this offer; a proposes then P2 and both players 1 and 2 accepts otherwise they are threatened to receive the worse offer P1 for them.
Finally offer P2 is chosen.
Option P1 is the best one for a but the two other players vetoed it.
It is interesting to point out that, even though a prefers P2 to P3 , offer P3 is first proposed and this make P2 being accepted.
If a proposes P2 first, then the subgame perfect equilibrium in this situation is P3 .
To sum up, the worse preferred options have to be proposed first in order to get finally the best one.
But this entails a waste of time.
Analysing the previous example, one sees that the game outcome at the equilibrium is P2 that is not very attractive for player 2.
Option P3 seems more balanced since no player judges it badly.
It could be seen as a better solution as a consensus among the agents.
In order to introduce this notion of balanceness in the protocol, we introduce a condition under which a player will be obliged to accept the proposal, reducing the autonomy of the agents but for increasing rationality and cooperation.
More precisely if the utility of a player is larger than a given threshold then acceptance is required.
The threshold decreases over time so that players have to make more and more concession.
Therefore, the protocol becomes as follows.
Protocol P2.
• At the beginning we set period t = 0 • a makes a proposal P ∈ P that has not been proposed before.
• Wait that all players of A give their opinion Yes or No to the player a.
A player k must accept the offer if Uk(Φk(Pk)) ≥ ρk(t) where ρk(t) tends to zero when t grows.
Moreover there exists T such that for all t ≥ T, ρk(t) = 0.
If all players agree on P, this later is chosen.
Otherwise t is incremented and we go back to previous point.
• If there is no more offer left from P, the default offer ˜P will be chosen.
One can show exactly as in Lemma 1 that protocol P2 has at least one subgame perfect equilibrium.
We expect that protocol P2 provides a solution not to far from P , so it favours fairness among the players.
Therefore, our cooperation-based multilateral multi-issue protocol is the following: Protocol P. • At the beginning we set period t = 0 • a makes a proposal P ∈ P that has not been proposed before, considering Ψk(Pt ) and Ψk(Pt ) for all players k ∈ A. • Wait that all players of A give their opinion (Yes , Ψk(Pt )) or (No , Ψk(Pt )) to the player a.
A player k must accept the offer if Uk(Φk(Pk)) ≥ ρk(t) where ρk(t) tends to zero when t grows.
Moreover there exists T such that for all t ≥ T, ρk(t) = 0.
If all players agree on P, this later is chosen.
Otherwise t is incremented and we go back to previous point.
• If there is no more offer left from P, the default offer ˜P will be chosen.
8.
EXPERIMENTS We developed a MAS using the widely used JADE agent platform [1].
This MAS is designed to be as general as possible (e.g. a general framework to specialise according to the application) and enable us to make some preliminary experiments.
The experiments aim at verifying that our approach gives solutions as close as possible to the Maximin solution and in a small number of rounds and hopefully in a short time since our context is highly cooperative.
We defined the two types of agents and their behaviours as introduced in section 6.
The agents and their behaviours correspond to the main classes of our prototype, NegotiatorAgent and NegotiatorBehaviour for the negotiator agents, and MediatorAgent and MediatorBehaviour for the mediator agent.
These classes extend JADE classes and integrate MYRIAD into the agents, reducing the amount of communications in the system.
Some functionalities depending on the application have to be implemented according to the application by extending these classes.
In particular, all conversion parts of the agents have to be specified according to the application since to convert a proposal into decision criteria, we need to know, first, this model and the correlations between the proposals and this model.
First, to illustrate our protocol, we present a simple example of our dispatch problem.
In this example, we have three hospitals, H1, H2 and H3.
Each hospital can receive victims having a particular pathology in such a way that H1 can receive patients with the pathology burn, surgery or orthopedic, H2 can receive patients with the pathology surgery, orthopedic or cardiology and H3 can receive patients with the pathology burn or cardiology.
All the hospitals have similar decision criteria reflecting their preferences on the level of congestion they can face for the overall hospital and the different services available, as briefly explained for hospital H1 hereafter.
For hospital H1, the preference model, fig. 3, is composed of five criteria.
These criteria correspond to the preferences on the pathologies the hospital can treat.
In the case of 948 The Sixth Intl..
Joint Conf.
on Autonomous Agents and Multi-Agent Systems (AAMAS 07) Figure 3: The H1 preference model in MYRIAD.
the pathology burn, the corresponding criterion, also named burn as shown in fig. 3, represents the preferences of H1 according to the value of Cburn which is the current capacity of burn.
Therefore, the utility function of this criterion represents a preference such that the more there are patients of this pathology in the hospital, the less the hospital may satisfy them, and this with an initial capacity.
In addition to reflecting this kind of viewpoint, the aggregation function as defined in MYRIAD introduces a veto on the criteria burn, surgery, orthopedic and EReceipt, where EReceipt is the criterion for the preferences about the capacity to receive a number of patients at the same time.
In this simplified example, the physician have no particular preferences on the dispatch and the mediator agent chooses a proposal randomly in a subset of the set of admissibility.
This subset have to satisfy as much as possible the recommendations made by the hospitals.
To solve this problem, for this example, we decided to solve a linear problem with the availability constraints and the recommendations as linear constraints on the dispatch values.
The set of admissibility is then obtained by solving this linear problem by the use of Prolog.
Moreover, only the recommendations on how to improve a proposal are taken into account.
The problem to solve is then to dispatch to hospital H1, H2 and H3, the set of victims composed of 5 victims with the pathology burn, 10 with surgery, 3 with orthopedic and 7 with cardiology.
The availabilities of the hospitals are as presented in the following table.
Available Overall burn surg.
orthop.
cardio.
H1 11 4 8 10H2 25 - 3 4 10 H3 7 10 - - 3 We obtain a multiagent system with the mediator agent and three agents of type negotiator for the three hospital in the problem.
The hospitals threshold are fixed approximatively to the level where an evaluation is considered as good.
To start, the negotiator agents send their availabilities.
The mediator agent makes a proposal chosen randomly in admissible set obtained with these availabilities as linear constraints.
This proposal is the vector P0 = [[H1,burn, 3], [H1, surgery, 8], [H1, orthopaedic, 0], [H2, surgery, 2], [H2, orthopaedic, 3], [H2, cardiology, 6], [H3, burn, 2], [H3, cardiology, 1]] and the mediator sends propose(P0) to H1, H2 and H3 for approval.
Each negotiator agent evaluates this proposal and answers back by accepting or rejecting P0: • Agent H1 rejects this offer since its evaluation is very far from the threshold (0.29, a bad score) and gives a recommendation to improve burn and surgery by sending the message reject_proposal([burn,surgery]); • Agent H2 accepts this offer by sending the message accept_proposal(), the proposal evaluation being good; • Agent H3 accepts P0 by sending the message accept_ proposal(), the proposal evaluation being good.
Just with the recommendations provided by agent H1, the mediator is able to make a new proposal by restricting the value of burn and surgery.
The new proposal obtained is then P1 = [[H1,burn, 0], [H1, surgery, 8], [H1, orthopaedic, 1], [H2, surgery, 2], [H2, orthopaedic, 2], [H2, cardiology, 6], [H3, burn, 5], [H3, cardiology, 1]].
The mediator sends propose(P1) the negotiator agents.
H1, H2 and H3 answer back by sending the message accept_proposal(), P1 being evaluated with a high enough score to be acceptable, and also considered as a good proposal when using the explanation function of MYRIAD.
An agreement is reached with P1.
Note that the evaluation of P1 by H3 has decreased in comparison with P0, but not enough to be rejected and that this solution is the Pareto one, P∗ .
Other examples have been tested with the same settings: issues in IN, three negotiator agents and the same mediator agent, with no preference model but selecting randomly the proposal.
We obtained solutions either equal or close to the Maximin solution, the distance from the standard deviation being less than 0.0829, the evaluations not far from the ones obtained with P∗ and with less than seven proposals made.
This shows us that we are able to solve this multi-issue multilateral negotiation problem in a simple and efficient way, with solutions close to the Pareto solution.
9.
CONCLUSION AND FUTURE WORK This paper presents a new protocol to address multilateral multi-issue negotiation in a cooperative context.
The first main contribution is that we take into account complex inter-dependencies between multiple issues with the use of a complex preference modelling.
This contribution is reinforced by the use of multi-issue negotiation in a multilateral context.
Our second contribution is the use of sharp recommendations in the protocol to help in accelerating the search of a consensus between the cooperative agents and in finding an optimal solution.
We have also shown that the protocol has subgame perfect equilibria and these equilibria converge to the usual maximum solution.
Moreover, we tested this protocol in a crisis management context where the negotiation aim is where to evacuate a whole set of injured people to predefined hospitals.
We have already developed a first MAS, in particular integrating MYRIAD, to test this protocol in order to know more about its efficiency in terms of solution quality and quickness in finding a consensus.
This prototype enabled us to solve some examples with our approach and the results we obtained are encouraging since we obtained quickly good agreements, close to the Pareto solution, in the light of the initial constraints of the problem: the availabilities.
We still have to improve our MAS by taking into account The Sixth Intl..
Joint Conf.
on Autonomous Agents and Multi-Agent Systems (AAMAS 07) 949 the two types of recommendations and by adding a preference model to the mediator of our system.
Moreover, a comparative study has to be done in order to evaluate the performance of our framework against the existing ones and against some variations on the protocol.
10.
ACKNOWLEDGEMENT This work is partly funded by the ICIS research project under the Dutch BSIK Program (BSIK 03024).
11.
REFERENCES [1] JADE.
http://jade.tilab.com/.
[2] P. Faratin, C. Sierra, and N. R. Jennings.
Using similarity criteria to make issue trade-offs in automated negotiations.
Artificial Intelligence, 142(2):205-237, 2003.
[3] S. S. Fatima, M. Wooldridge, and N. R. Jennings.
Optimal negotiation of multiple issues in incomplete information settings.
In 3rd International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS``04), pages 1080-1087, New York, USA, 2004.
[4] S. S. Fatima, M. Wooldridge, and N. R. Jennings.
A comparative study of game theoretic and evolutionary models of bargaining for software agents.
Artificial Intelligence Review, 23:185-203, 2005.
[5] S. S. Fatima, M. Wooldridge, and N. R. Jennings.
On efficient procedures for multi-issue negotiation.
In 8th International Workshop on Agent-Mediated Electronic Commerce(AMEC``06), pages 71-84, Hakodate, Japan, 2006.
[6] M. Grabisch.
The application of fuzzy integrals in multicriteria decision making.
European J. of Operational Research, 89:445-456, 1996.
[7] M. Grabisch, T. Murofushi, and M. Sugeno.
Fuzzy Measures and Integrals.
Theory and Applications (edited volume).
Studies in Fuzziness.
Physica Verlag, 2000.
[8] M. Hemaissia, A. El Fallah-Seghrouchni, C. Labreuche, and J. Mattioli.
Cooperation-based multilateral multi-issue negotiation for crisis management.
In 2th International Workshop on Rational, Robust and Secure Negotiation (RRS``06), pages 77-95, Hakodate, Japan, May 2006.
[9] T. Ito, M. Klein, and H. Hattori.
A negotiation protocol for agents with nonlinear utility functions.
In AAAI, 2006.
[10] M. Klein, P. Faratin, H. Sayama, and Y. Bar-Yam.
Negotiating complex contracts.
Group Decision and Negotiation, 12:111-125, March 2003.
[11] C. Labreuche.
Determination of the criteria to be improved first in order to improve as much as possible the overall evaluation.
In IPMU 2004, pages 609-616, Perugia, Italy, 2004.
[12] C. Labreuche and F. Le Hu´ed´e. MYRIAD: a tool suite for MCDA.
In EUSFLAT``05, pages 204-209, Barcelona, Spain, 2005.
[13] R. Y. K. Lau.
Towards genetically optimised multi-agent multi-issue negotiations.
In Proceedings of the 38th Annual Hawaii International Conference on System Sciences (HICSS``05), Big Island, Hawaii, 2005.
[14] R. J. Lin.
Bilateral multi-issue contract negotiation for task redistribution using a mediation service.
In Agent Mediated Electronic Commerce VI (AMEC``04), New York, USA, 2004.
[15] J. F. Nash.
Non cooperative games.
Annals of Mathematics, 54:286-295, 1951.
[16] G. Owen.
Game Theory.
Academic Press, New York, 1995.
[17] V. Robu, D. J. A. Somefun, and J. A. L. Poutr´e. Modeling complex multi-issue negotiations using utility graphs.
In 4th International Joint Conference on Autonomous agents and multiagent systems (AAMAS``05), pages 280-287, 2005.
[18] A. Rubinstein.
Perfect equilibrium in a bargaining model.
Econometrica, 50:97-109, jan 1982.
[19] L.-K.
Soh and X. Li.
Adaptive, confidence-based multiagent negotiation strategy.
In 3rd International Joint Conference on Autonomous agents and multiagent systems (AAMAS``04), pages 1048-1055, Los Alamitos, CA, USA, 2004.
[20] H.-W.
Tung and R. J. Lin.
Automated contract negotiation using a mediation service.
In 7th IEEE International Conference on E-Commerce Technology (CEC``05), pages 374-377, Munich, Germany, 2005.
950 The Sixth Intl..
Joint Conf.
on Autonomous Agents and Multi-Agent Systems (AAMAS 07)

A Multilateral Multi-issue Negotiation Protocol
ABSTRACT
In this paper, we present a new protocol to address multilateral multi-issue negotiation in a cooperative context.
We consider complex dependencies between multiple issues by modelling the preferences of the agents with a multi-criteria decision aid tool, also enabling us to extract relevant information on a proposal assessment.
This information is used in the protocol to help in accelerating the search for a consensus between the cooperative agents.
In addition, the negotiation procedure is defined in a crisis management context where the common objective of our agents is also considered in the preferences of a mediator agent.
1.
INTRODUCTION
Multi-issue negotiation protocols represent an important field of study since negotiation problems in the real world are often complex ones involving multiple issues.
To date, most of previous work in this area ([2, 3, 19, 13]) dealt almost exclusively with simple negotiations involving independent issues.
However, real-world negotiation problems involve complex dependencies between multiple issues.
When one wants to buy a car, for example, the value of a given car is highly dependent on its price, consumption, comfort and so on.
The addition of such interdependencies greatly
complicates the agents utility functions and classical utility functions, such as the weighted sum, are not sufficient to model this kind of preferences.
In [10, 9, 17, 14, 20], the authors consider inter-dependencies between issues, most often defined with boolean values, except for [9], while we can deal with continuous and discrete dependent issues thanks to the modelling power of the Choquet integral.
In [17], the authors deal with bilateral negotiation while we are interested in a multilateral negotiation setting.
Klein et al. [10] present an approach similar to ours, using a mediator too and information about the strength of the approval or rejection that an agent makes during the negotiation.
In our protocol, we use more precise information to improve the proposals thanks to the multi-criteria methodology and tools used to model the preferences of our agents.
Lin, in [14, 20], also presents a mediation service but using an evolutionary algorithm to reach optimal solutions and as explained in [4], players in the evolutionary models need to repeatedly interact with each other until the stable state is reached.
As the population size increases, the time it takes for the population to stabilize also increases, resulting in excessive computation, communication, and time overheads that can become prohibitive, and for one-to-many and many-to-many negotiations, the overheads become higher as the number of players increases.
In [9], the authors consider a non-linear utility function by using constraints on the domain of the issues and a mediation service to find a combination of bids maximizing the social welfare.
Our preference model, a nonlinear utility function too, is more complex than [9] one since the Choquet integral takes into account the interactions and the importance of each decision criteria/issue, not only the dependencies between the values of the issues, to determine the utility.
We also use an iterative protocol enabling us to find a solution even when no bid combination is possible.
In this paper, we propose a negotiation protocol suited for multiple agents with complex preferences and taking into account, at the same time, multiple interdependent issues and recommendations made by the agents to improve a proposal.
Moreover, the preferences of our agents are modelled using a multi-criteria methodology and tools enabling us to take into account information about the improvements that can be made to a proposal, in order to help in accelerating the search for a consensus between the agents.
Therefore, we propose a negotiation protocol consisting of solving our decision problem using a MAS with a multi-criteria decision aiding modelling at the agent level and a cooperation-based multilateral multi-issue negotiation protocol.
This protocol is studied under a non-cooperative approach and it is shown
that it has subgame perfect equilibria, provided that agents behave rationally in the sense of von Neumann and Morgenstern.
The approach proposed in this paper has been first introduced and presented in [8].
In this paper, we present our first experiments, with some noteworthy results, and a more complex multi-agent system with representatives to enable us to have a more robust system.
In Section 2, we present our application, a crisis management problem.
Section 3 deals with the general aspect of the proposed approach.
The preference modelling is described in sect.
4, whereas the motivations of our protocol are considered in sect.
5 and the agent/multiagent modelling in sect.
6.
Section 7 presents the formal modelling and properties of our protocol before presenting our first experiments in sect.
8.
Finally, in Section 9, we conclude and present the future work.
2.
CASE STUDY
3.
PROPOSED APPROACH
4.
THE PREFERENCE MODEL
944 The Sixth Intl. .
Joint Conf.
on Autonomous Agents and Multi-Agent Systems (AAMAS 07)
5.
PROTOCOL MOTIVATIONS
6.
AGENTIFICATION
946 The Sixth Intl. .
Joint Conf.
on Autonomous Agents and Multi-Agent Systems (AAMAS 07)
7.
OUR PROTOCOL
The Sixth Intl. .
Joint Conf.
on Autonomous Agents and Multi-Agent Systems (AAMAS 07) 947
8.
EXPERIMENTS
948 The Sixth Intl. .
Joint Conf.
on Autonomous Agents and Multi-Agent Systems (AAMAS 07)
The Sixth Intl. .
Joint Conf.
on Autonomous Agents and Multi-Agent Systems (AAMAS 07) 949
10.
ACKNOWLEDGEMENT
This work is partly funded by the ICIS research project under the Dutch BSIK Program (BSIK 03024).

A Multilateral Multi-issue Negotiation Protocol
ABSTRACT
In this paper, we present a new protocol to address multilateral multi-issue negotiation in a cooperative context.
We consider complex dependencies between multiple issues by modelling the preferences of the agents with a multi-criteria decision aid tool, also enabling us to extract relevant information on a proposal assessment.
This information is used in the protocol to help in accelerating the search for a consensus between the cooperative agents.
In addition, the negotiation procedure is defined in a crisis management context where the common objective of our agents is also considered in the preferences of a mediator agent.
1.
INTRODUCTION
Multi-issue negotiation protocols represent an important field of study since negotiation problems in the real world are often complex ones involving multiple issues.
To date, most of previous work in this area ([2, 3, 19, 13]) dealt almost exclusively with simple negotiations involving independent issues.
However, real-world negotiation problems involve complex dependencies between multiple issues.
The addition of such interdependencies greatly
complicates the agents utility functions and classical utility functions, such as the weighted sum, are not sufficient to model this kind of preferences.
In [17], the authors deal with bilateral negotiation while we are interested in a multilateral negotiation setting.
Klein et al. [10] present an approach similar to ours, using a mediator too and information about the strength of the approval or rejection that an agent makes during the negotiation.
In our protocol, we use more precise information to improve the proposals thanks to the multi-criteria methodology and tools used to model the preferences of our agents.
In [9], the authors consider a non-linear utility function by using constraints on the domain of the issues and a mediation service to find a combination of bids maximizing the social welfare.
Our preference model, a nonlinear utility function too, is more complex than [9] one since the Choquet integral takes into account the interactions and the importance of each decision criteria/issue, not only the dependencies between the values of the issues, to determine the utility.
We also use an iterative protocol enabling us to find a solution even when no bid combination is possible.
In this paper, we propose a negotiation protocol suited for multiple agents with complex preferences and taking into account, at the same time, multiple interdependent issues and recommendations made by the agents to improve a proposal.
Moreover, the preferences of our agents are modelled using a multi-criteria methodology and tools enabling us to take into account information about the improvements that can be made to a proposal, in order to help in accelerating the search for a consensus between the agents.
Therefore, we propose a negotiation protocol consisting of solving our decision problem using a MAS with a multi-criteria decision aiding modelling at the agent level and a cooperation-based multilateral multi-issue negotiation protocol.
This protocol is studied under a non-cooperative approach and it is shown
that it has subgame perfect equilibria, provided that agents behave rationally in the sense of von Neumann and Morgenstern.
The approach proposed in this paper has been first introduced and presented in [8].
In this paper, we present our first experiments, with some noteworthy results, and a more complex multi-agent system with representatives to enable us to have a more robust system.
In Section 2, we present our application, a crisis management problem.
Section 3 deals with the general aspect of the proposed approach.
The preference modelling is described in sect.
4, whereas the motivations of our protocol are considered in sect.
5 and the agent/multiagent modelling in sect.
6.
Section 7 presents the formal modelling and properties of our protocol before presenting our first experiments in sect.
8.
Finally, in Section 9, we conclude and present the future work.
10.
ACKNOWLEDGEMENT

A Multilateral Multi-issue Negotiation Protocol
ABSTRACT
In this paper, we present a new protocol to address multilateral multi-issue negotiation in a cooperative context.
We consider complex dependencies between multiple issues by modelling the preferences of the agents with a multi-criteria decision aid tool, also enabling us to extract relevant information on a proposal assessment.
This information is used in the protocol to help in accelerating the search for a consensus between the cooperative agents.
In addition, the negotiation procedure is defined in a crisis management context where the common objective of our agents is also considered in the preferences of a mediator agent.
1.
INTRODUCTION
Multi-issue negotiation protocols represent an important field of study since negotiation problems in the real world are often complex ones involving multiple issues.
To date, most of previous work in this area ([2, 3, 19, 13]) dealt almost exclusively with simple negotiations involving independent issues.
However, real-world negotiation problems involve complex dependencies between multiple issues.
When one wants to buy a car, for example, the value of a given car is highly dependent on its price, consumption, comfort and so on.
The addition of such interdependencies greatly
complicates the agents utility functions and classical utility functions, such as the weighted sum, are not sufficient to model this kind of preferences.
In [10, 9, 17, 14, 20], the authors consider inter-dependencies between issues, most often defined with boolean values, except for [9], while we can deal with continuous and discrete dependent issues thanks to the modelling power of the Choquet integral.
In [17], the authors deal with bilateral negotiation while we are interested in a multilateral negotiation setting.
Klein et al. [10] present an approach similar to ours, using a mediator too and information about the strength of the approval or rejection that an agent makes during the negotiation.
In our protocol, we use more precise information to improve the proposals thanks to the multi-criteria methodology and tools used to model the preferences of our agents.
Lin, in [14, 20], also presents a mediation service but using an evolutionary algorithm to reach optimal solutions and as explained in [4], players in the evolutionary models need to repeatedly interact with each other until the stable state is reached.
As the population size increases, the time it takes for the population to stabilize also increases, resulting in excessive computation, communication, and time overheads that can become prohibitive, and for one-to-many and many-to-many negotiations, the overheads become higher as the number of players increases.
In [9], the authors consider a non-linear utility function by using constraints on the domain of the issues and a mediation service to find a combination of bids maximizing the social welfare.
Our preference model, a nonlinear utility function too, is more complex than [9] one since the Choquet integral takes into account the interactions and the importance of each decision criteria/issue, not only the dependencies between the values of the issues, to determine the utility.
We also use an iterative protocol enabling us to find a solution even when no bid combination is possible.
In this paper, we propose a negotiation protocol suited for multiple agents with complex preferences and taking into account, at the same time, multiple interdependent issues and recommendations made by the agents to improve a proposal.
Moreover, the preferences of our agents are modelled using a multi-criteria methodology and tools enabling us to take into account information about the improvements that can be made to a proposal, in order to help in accelerating the search for a consensus between the agents.
Therefore, we propose a negotiation protocol consisting of solving our decision problem using a MAS with a multi-criteria decision aiding modelling at the agent level and a cooperation-based multilateral multi-issue negotiation protocol.
This protocol is studied under a non-cooperative approach and it is shown
that it has subgame perfect equilibria, provided that agents behave rationally in the sense of von Neumann and Morgenstern.
The approach proposed in this paper has been first introduced and presented in [8].
In this paper, we present our first experiments, with some noteworthy results, and a more complex multi-agent system with representatives to enable us to have a more robust system.
In Section 2, we present our application, a crisis management problem.
Section 3 deals with the general aspect of the proposed approach.
The preference modelling is described in sect.
4, whereas the motivations of our protocol are considered in sect.
5 and the agent/multiagent modelling in sect.
6.
Section 7 presents the formal modelling and properties of our protocol before presenting our first experiments in sect.
8.
Finally, in Section 9, we conclude and present the future work.
2.
CASE STUDY
This protocol is applied to a crisis management problem.
Crisis management is a relatively new field of management and is composed of three types of activities: crisis prevention, operational preparedness and management of declared crisis.
The crisis prevention aims to bring the risk of crisis to an acceptable level and, when possible, avoid that the crisis actually happens.
The operational preparedness includes strategic advanced planning, training and simulation to ensure availability, rapid mobilisation and deployment of resources to deal with possible emergencies.
The management of declared crisis is the response to--including the evacuation, search and rescue--and the recovery from the crisis by minimising the effects of the crises, limiting the impact on the community and environment and, on a longer term, by bringing the community's systems back to normal.
In this paper, we focus on the response part of the management of declared crisis activity, and particularly on the evacuation of the injured people in disaster situations.
When a crisis is declared, the plans defined during the operational preparedness activity are executed.
For disasters, master plans are executed.
These plans are elaborated by the authorities with the collaboration of civil protection agencies, police, health services, non-governmental organizations, etc. .
When a victim is found, several actions follow.
First, a rescue party is assigned to the victim who is examined and is given first aid on the spot.
Then, the victims can be placed in an emergency centre on the ground called the medical advanced post.
For all victims, a sorter physician--generally a hospital physician--examines the seriousness of their injuries and classifies the victims by pathology.
The evacuation by emergency health transport if necessary can take place after these clinical examinations and classifications.
Nowadays, to evacuate the injured people, the physicians contact the emergency call centre to pass on the medical assessments of the most urgent cases.
The emergency call centre then searches for available and appropriate spaces in the hospitals to care for these victims.
The physicians are informed of the allocations, so they can proceed to the evacuations choosing the emergency health transports according to the pathologies and the transport modes provided.
In this context, we can observe that the evacuation is based on three important elements: the examination and classification of the victims, the search for an allocation and the transport.
In the case of the 11 March 2004 Madrid attacks, for instance, some injured people did not receive the appropriate health care because, during the search for space, the emergency call centre did not consider the transport constraints and, in particular, the traffic.
Therefore, for a large scale crisis management problem, there is a need to support the emergency call centre and the physicians in the dispatching to take into account the hospitals and the transport constraints and availabilities.
3.
PROPOSED APPROACH
To accept a proposal, an agent has to consider several issues such as, in the case of the crisis management problem, the availabilities in terms of number of beds by unit, medical and surgical staffs, theatres and so on.
Therefore, each agent has its own preferences in correlation with its resource constraints and other decision criteria such as, for the case study, the level of congestion of a hospital.
All the agents also make decisions by taking into account the dependencies between these decision criteria.
The first hypothesis of our approach is that there are several parties involved in and impacted by the decision, and so they have to decide together according to their own constraints and decision criteria.
Negotiation is the process by which a group facing a conflict communicates with one another to try and come to a mutually acceptable agreement or decision and so, the agents have to negotiate.
The conflict we have to resolve is finding an acceptable solution for all the parties by using a particular protocol.
In our context, multilateral negotiation is a negotiation protocol type that is the best suited for this type of problem: this type of protocol enables the hospitals and the physicians to negotiate together.
The negotiation also deals with multiple issues.
Moreover, an other hypothesis is that we are in a cooperative context where all the parties have a common objective which is to provide the best possible solution for everyone.
This implies the use of a negotiation protocol encouraging the parties involved to cooperate as satisfying its preferences.
Taking into account these aspects, a Multi-Agent System (MAS) seems to be a reliable method in the case of a distributed decision making process.
Indeed, a MAS is a suitable answer when the solution has to combine, at least, distribution features and reasoning capabilities.
Another motivation for using MAS lies in the fact that MAS is well known for facilitating automated negotiation at the operative decision making level in various applications.
Therefore, our approach consists of solving a multiparty decision problem using a MAS with
• The preferences of the agents are modelled using a multi-criteria decision aid tool, MYRIAD, also enabling us to consider multi-issue problems by evaluating proposals on several criteria.
• A cooperation-based multilateral and multi-issue negotiation protocol.
4.
THE PREFERENCE MODEL
We consider a problem where an agent has several decision criteria, a set Nk = {1,..., nk} of criteria for each agent k involved in the negotiation protocol.
These decision criteria enable the agents to evaluate the set of issues that are negotiated.
The issues correspond directly or not to the decision criteria.
However, for the example of the crisis management
944 The Sixth Intl. .
Joint Conf.
on Autonomous Agents and Multi-Agent Systems (AAMAS 07)
problem, the issues are the set of victims to dispatch between the hospitals.
These issues are translated to decision criteria enabling the hospital to evaluate its congestion and so to an updated number of available beds, medical teams and so on.
In order to take into account the complexity that exists between the criteria/issues, we use a multi-criteria decision aiding (MCDA) tool named MYRIAD [12] developed at Thales for MCDA applications based on a two-additive Choquet integral which is a good compromise between versatility and ease to understand and model the interactions between decision criteria [6].
The set of the attributes of Nk is denoted by Xk1,..., Xknk.
All the attributes are made commensurate thanks to the introduction of partial utility functions uki: Xki → [0, 1].
The [0, 1] scale depicts the satisfaction of the agent k regarding the values of the attributes.
An option x is identified to an element of Xk = Xk1 × · · · × Xknk, with x = (x1,..., xnk).
Then the overall assessment of x is given by
where Hk: [0, 1] nk → [0, 1] is the aggregation function.
The overall preference relation ~ over Xk is then
where vki is the relative importance of criterion i for agent k and Iki, j is the interaction between criteria i and j, ∧ and ∨ denote the min and max functions respectively.
Assume that zi <zj.
A positive interaction between criteria i and j depicts complementarity between these criteria (positive synergy) [7].
Hence, the lower score of z on criterion i conceals the positive effect of the better score on criterion j to a larger extent on the overall evaluation than the impact of the relative importance of the criteria taken independently of the other ones.
In other words, the score of z on criterion j is penalized by the lower score on criterion i. Conversely, a negative interaction between criteria i and j depicts substitutability between these criteria (negative synergy) [7].
The score of z on criterion i is then saved by a better score on criterion j.
In MYRIAD, we can also obtain some recommendations corresponding to an indicator ωC (H, x) measuring the worth to improve option x w.r.t. Hk on some criteria C ⊆ Nk as follows
erage improvement of Hk when the criteria of coalition A range from xC to 1C divided by the average effort needed order 1, that is EC (τ, x) = τ E for this improvement.
We generally assume that EC is of i ∈ C (1 − xi).
The expression of ωC (Hk, x) when Hk is a Choquet integral, is given in [11].
The agent is then recommended to improve of coalition C for which ωC (Hk, x) is maximum.
This recommendation is very useful in a negotiation protocol since it helps the agents to know what to do if they want an offer to be accepted while not revealing their own preference model.
5.
PROTOCOL MOTIVATIONS
For multi-issue problems, there are two approaches: a complete package approach where the issues are negotiated simultaneously in opposition to the sequential approach where the issues are negotiated one by one.
When the issues are dependant, then it is the best choice to bargain simultaneously over all issues [5].
Thus, the complete package is the adopted approach so that an offer will be on the overall set of injured people while taking into account the other decision criteria.
We have to consider that all the parties of the negotiation process have to agree on the decision since they are all involved in and impacted by this decision and so an unanimous agreement is required in the protocol.
In addition, no party can leave the process until an agreement is reached, i.e. a consensus achieved.
This makes sense since a proposal concerns all the parties.
Moreover, we have to guarantee the availability of the resources needed by the parties to ensure that a proposal is realistic.
To this end, the information about these availabilities are used to determine admissible proposals such that an offer cannot be made if one of the parties has not enough resources to execute/achieve it.
At the beginning of the negotiation, each party provides its maximum availabilities, this defining the constraints that have to be satisfied for each offer submitted.
The negotiation has also to converge quickly on an unanimous agreement.
We decided to introduce in the negotiation protocol an incentive to cooperate taking into account the passed negotiation time.
This incentive is defined on the basis of a time dependent penalty, the discounting factor as in [18] or a time-dependent threshold.
This penalty has to be used in the accept/reject stage of our consensus procedure.
In fact, in the case of a discounting factor, each party will accept or reject an offer by evaluating the proposal using its utility function deducted from the discounting factor.
In the case of a time-dependent threshold, if the evaluation is greater or equal to this threshold, the offer is accepted, otherwise, in the next period, its threshold is reduced.
The use of a penalty is not enough alone since it does not help in finding a solution.
Some information about the assessments of the parties involved in the negotiation are needed.
In particular, it would be helpful to know why an offer has been rejected and/or what can be done to make a proposal that would be accepted.
MYRIAD provides an analysis that determines the flaws an option, here a proposal.
In particular, it gives this type of information: which criteria of a proposal should be improved so as to reach the highest possible overall evaluation [11].
As we use this tool to model the parties involved in the negotiation, the information about the criteria to improve can be used by the mediator to elaborate the proposals.
We also consider that the dual function can be used to take into account another type of information: on which criteria of a proposal, no improvement is necessary so that the overall evaluation of a proposal is still acceptable, do not decrease.
Thus, all information is a constraint to be satisfied as much as possible by
Figure 1: An illustration of some system.
the parties to make a new proposal.
We are in a cooperative context and revealing one's opinion on what can be improved is not prohibited, on the contrary, it is useful and recommended here seeing that it helps in converging on an agreement.
Therefore, when one of the parties refuses an offer, some information will be communicated.
In order to facilitate and speed up the negotiation, we introduce a mediator.
This specific entity is in charge of making the proposals to the other parties in the system by taking into account their public constraints (e.g. their availabilities) and the recommendations they make.
This mediator can also be considered as the representative of the general interest we can have, in some applications, such as in the crisis management problem, the physician will be the mediator and will also have some more information to consider when making an offer (e.g. traffic state, transport mode and time).
Each party in a negotiation N, a negotiator, can also be a mediator of another negotiation N', this party becoming the representative of N' in the negotiation N, as illustrated by fig. 1 what can also help in reducing the communication time.
6.
AGENTIFICATION
How the problem is transposed in a MAS problem is a very important aspect when designing such a system.
The agentification has an influence upon the systems efficiency in solving the problem.
Therefore, in this section, we describe the elements and constraints taken into account during the modelling phase and for the model itself.
However, for this negotiation application, the modelling is quite natural when one observes the negotiation protocol motivations and main properties.
First of all, it seems obvious that there should be one agent for each player of our multilateral multi-issue negotiation protocol.
The agents have the involved parties' information and preferences.
These agents are:
• Autonomous: they decide for themselves what, when and under what conditions actions should be performed; • Rational: they have a means-ends competence to fit its decisions, according to its knowledge, preferences and goal; • Self-interested: they have their own interests which may conflict with the interests of other agents.
Moreover, their preferences are modelled and a proposal evaluated and analysed using MYRIAD.
Each agent has private information and can access public information as knowledge.
In fact, there are two types of agents: the mediator type for the agents corresponding to the mediator of our negotiation protocol, the delegated physician in our application, and the negotiator type for the agents corresponding to the other parties, the hospitals.
The main behaviours that an agent of type mediator needs to negotiate in our protocol are the following:
• convert-improvements: converts the information given by the other agents involved in the negotiation about the improvements to be done, into constraints on the next proposal to be made; • convert-no-decrease: converts the information given by the other agents involved in the negotiation about the points that should not be changed into constraints on the next proposal to be made; • construct-proposal: constructs a new proposal according to the constraints obtained with convert-improvements, convert-no-decrease and the agent preferences;
The main behaviours that an agent of type negotiator needs to negotiate in our protocol are the following:
• convert-proposal: converts a proposal to a MYRIAD option of the agent according to its preferences model and its private data; • convert-improvements-wc: converts the agent recommendations for the improvements of a MYRIAD option into general information on the proposal; • convert-no-decrease-wc: converts the agent recom
mendations about the criteria that should not be changed in the MYRIAD option into general information on the proposal; In addition to these behaviours, there are, for the two types of agents, access behaviours to MYRIAD functionalities such as the evaluation and improvement functions:
• evaluate-option: evaluates the MYRIAD option obtained using the agent behaviour convert-proposal; • improvements: gets the agent recommendations to improve a proposal from the MYRIAD option; • no-decrease: gets the agent recommendations to not change some criteria from the MYRIAD option;
Of course, before running the system with such agents, we must have defined each party preferences model in MYRIAD.
This model has to be part of the agent so that it could be used to make the assessments and to retrieve the improvements.
In addition to these behaviours, the communication acts between the agents is as follows.
1.
mediator agent communication acts:
946 The Sixth Intl. .
Joint Conf.
on Autonomous Agents and Multi-Agent Systems (AAMAS 07)
Figure 2: The protocol diagram in AUML, and where m is the number of negotiator agents and l is the number of agents refusing current proposal.
(a) propose: sends a message containing a proposal to all negotiator agents; (b) inform: sends a message to all negotiator agents to inform them that an agreement has been reached and containing the consensus outcome.
2.
negotiator agent communication acts: (a) accept-proposal: sends a message to the mediator agent containing the agent recommendations to improve the proposal and obtained with convert_improvements_wc; (b) reject-proposal: sends a message to the mediator agent containing the agent recommendations about the criteria that should not be changed and obtained with convert_no_decrease_wc.
Such agents are interchangeable, in a case of failure, since they all have the same properties and represent a user with his preference model, not depending on the agent, but on the model defined in MYRIAD.
When the issues and the decision criteria are different from each other, the information about the criteria improvement have to be pre-processed to give some instructions on the directions to take and about the negotiated issues.
It is the same for the evaluation of a proposal: each agent has to convert the information about the issues to update its private information and to obtain the values of each attribute of the decision criteria.
7.
OUR PROTOCOL
Formally, we consider negotiations where a set of players A = {1, 2,..., m} and a player a are negotiating over a set Q of size q.
The player a is the protocol mediator, the mediator agent of the agentification.
The utility/preference function of a player k ∈ A ∪ {a} is Uk, defined using MYRIAD, as presented in Section 4, with a set Nk of criteria, Xk an option, and so on.
An offer is a vector P = (P1, P2, · · ·, Pm), a partition of Q, in which Pk is player k's share of Q.
We have P ∈ P where P is the set of admissible proposals, a finite set.
Note that P is determined using all players general constraints on the proposals and Q. Moreover, let P˜ denote a particular proposal defined as a's preferred proposal.
We also have the following notation: δk is the threshold decrease factor of player k, 4Dk: Pk → Xk is player k's function to convert a proposal to an option and Ψk is the function indicating which points P has to be improved, with Ψk its dual function--on which points no improvement is necessary.
Ψk is obtained using the dual function of ωC (Hk, x):
In period t of our consensus procedure, player a proposes an agreement P. All players k ∈ A respond to a by accepting or rejecting P.
The responses are made simultaneously.
If all players k ∈ A accept the offer, the game ends.
If any player k rejects P, then the next period t +1 begins: player a makes another proposal P' by taking into account information provided by the players and the ones that have rejected P apply a penalty.
Therefore, our negotiation protocol can be as follows:
• At the beginning, we set period t = 0 • a makes a proposal P ∈ P that has not been proposed before.
• Wait that all players of A give their opinion Yes or No to the player a.
If all players agree on P, this later is chosen.
Otherwise t is incremented and we go back to previous point.
• If there is no more offer left from P, the default offer P˜ will be chosen.
• The utility of players regarding a given of
fer decreases over time.
More precisely, the utility of player k ∈ A at period t regarding offer P is Uk (4Dk (Pk), t) = ft (Uk (4Dk (Pk))), where one can take for instance ft (x) = x. (δk) t or ft (x) = x − δk.t, as penalty function.
Proof: Protocol P1 is first transformed in a game in extensive form.
To this end, one shall specify the order in which the responders A react to the offer P of a.
However the order in which the players answer has no influence on the course of the game and in particular on their personal utility.
Hence protocol P1 is strictly equivalent to a game in 1A subgame perfect equilibrium is an equilibrium such that players' strategies constitute a Nash equilibrium in every subgame of the original game [18, 16].
A Nash equilibrium is a set of strategies, one for each player, such that no player has incentive to unilaterally change his/her action [15].
The Sixth Intl. .
Joint Conf.
on Autonomous Agents and Multi-Agent Systems (AAMAS 07) 947
extensive form, considering any order of the players A.
This game is clearly finite since P is finite and each offer can only be proposed once.
Finally P1 corresponds to a game with perfect information.
We end the proof by using a classical result stating that any finite game in extensive form with perfect information has at least one subgame perfect equilibrium (see e.g. [16]).
Rational players (in the sense of von Neumann and Morgenstern) involved in protocol P1 will necessarily come up with a subgame perfect equilibrium.
It is easy to see that there is one single subgame perfect equilibrium for protocol P1 corresponding to these values.
This equilibrium consists of the following choices: first a proposes P3; player 1 rejects this offer; a proposes then P2 and both players 1 and 2 accepts otherwise they are threatened to receive the worse offer P1 for them.
Finally offer P2 is chosen.
Option P1 is the best one for a but the two other players vetoed it.
It is interesting to point out that, even though a prefers P2 to P3, offer P3 is first proposed and this make P2 being accepted.
If a proposes P2 first, then the subgame perfect equilibrium in this situation is P3.
To sum up, the worse preferred options have to be proposed first in order to get finally the best one.
But this entails a waste of time.
Analysing the previous example, one sees that the game outcome at the equilibrium is P2 that is not very attractive for player 2.
Option P3 seems more balanced since no player judges it badly.
It could be seen as a better solution as a consensus among the agents.
In order to introduce this notion of balanceness in the protocol, we introduce a condition under which a player will be obliged to accept the proposal, reducing the autonomy of the agents but for increasing rationality and cooperation.
More precisely if the utility of a player is larger than a given threshold then acceptance is required.
The threshold decreases over time so that players have to make more and more concession.
Therefore, the protocol becomes as follows.
Protocol P2.
• At the beginning we set period t = 0 • a makes a proposal P ∈ P that has not been proposed before.
• Wait that all players of A give their opinion Yes or No to the player a.
A player k must accept the offer if Uk (Φk (Pk)) ≥ ρk (t) where ρk (t) tends to zero when t grows.
Moreover there exists T such that for all t ≥ T, ρk (t) = 0.
If all players agree on P, this later is chosen.
Otherwise t is incremented and we go back to previous point.
• If there is no more offer left from P, the default offer P˜ will be chosen.
One can show exactly as in Lemma 1 that protocol P2 has at least one subgame perfect equilibrium.
We expect that protocol P2 provides a solution not to far from P", so it favours fairness among the players.
Therefore, our cooperation-based multilateral multi-issue protocol is the following:
Protocol P. • At the beginning we set period t = 0 • a makes a proposal P ∈ P that has not been proposed before, considering' Pk (Pt) and' Pk (Pt) for all players k ∈ A. • Wait that all players of A give their opinion (Yes,' Pk (Pt)) or (No,' Pk (Pt)) to the player a.
A player k must accept the offer if Uk (Φk (Pk)) ≥ ρk (t) where ρk (t) tends to zero when t grows.
Moreover there exists T such that for all t ≥ T, ρk (t) = 0.
If all players agree on P, this later is chosen.
Otherwise t is incremented and we go back to previous point.
• If there is no more offer left from P, the default offer P˜ will be chosen.
8.
EXPERIMENTS
We developed a MAS using the widely used JADE agent platform [1].
This MAS is designed to be as general as possible (e.g. a general framework to specialise according to the application) and enable us to make some preliminary experiments.
The experiments aim at verifying that our approach gives solutions as close as possible to the Maximin solution and in a small number of rounds and hopefully in a short time since our context is highly cooperative.
We defined the two types of agents and their behaviours as introduced in section 6.
The agents and their behaviours correspond to the main classes of our prototype, NegotiatorAgent and NegotiatorBehaviour for the negotiator agents, and MediatorAgent and MediatorBehaviour for the mediator agent.
These classes extend JADE classes and integrate MYRIAD into the agents, reducing the amount of communications in the system.
Some functionalities depending on the application have to be implemented according to the application by extending these classes.
In particular, all conversion parts of the agents have to be specified according to the application since to convert a proposal into decision criteria, we need to know, first, this model and the correlations between the proposals and this model.
First, to illustrate our protocol, we present a simple example of our dispatch problem.
In this example, we have three hospitals, H1, H2 and H3.
Each hospital can receive victims having a particular pathology in such a way that H1 can receive patients with the pathology burn, surgery or orthopedic, H2 can receive patients with the pathology surgery, orthopedic or cardiology and H3 can receive patients with the pathology burn or cardiology.
All the hospitals have similar decision criteria reflecting their preferences on the level of congestion they can face for the overall hospital and the different services available, as briefly explained for hospital H1 hereafter.
For hospital H1, the preference model, fig. 3, is composed of five criteria.
These criteria correspond to the preferences on the pathologies the hospital can treat.
In the case of
948 The Sixth Intl. .
Joint Conf.
on Autonomous Agents and Multi-Agent Systems (AAMAS 07)
Figure 3: The H1 preference model in MYRIAD.
the pathology burn, the corresponding criterion, also named burn as shown in fig. 3, represents the preferences of H1 according to the value of Cburn which is the current capacity of burn.
Therefore, the utility function of this criterion represents a preference such that the more there are patients of this pathology in the hospital, the less the hospital may satisfy them, and this with an initial capacity.
In addition to reflecting this kind of viewpoint, the aggregation function as defined in MYRIAD introduces a veto on the criteria burn, surgery, orthopedic and EReceipt, where EReceipt is the criterion for the preferences about the capacity to receive a number of patients at the same time.
In this simplified example, the physician have no particular preferences on the dispatch and the mediator agent chooses a proposal randomly in a subset of the set of admissibility.
This subset have to satisfy as much as possible the recommendations made by the hospitals.
To solve this problem, for this example, we decided to solve a linear problem with the availability constraints and the recommendations as linear constraints on the dispatch values.
The set of admissibility is then obtained by solving this linear problem by the use of Prolog.
Moreover, only the recommendations on how to improve a proposal are taken into account.
The problem to solve is then to dispatch to hospital H1, H2 and H3, the set of victims composed of 5 victims with the pathology burn, 10 with surgery, 3 with orthopedic and 7 with cardiology.
The availabilities of the hospitals are as presented in the following table.
We obtain a multiagent system with the mediator agent and three agents of type negotiator for the three hospital in the problem.
The hospitals threshold are fixed approximatively to the level where an evaluation is considered as good.
To start, the negotiator agents send their availabilities.
The mediator agent makes a proposal chosen randomly in admissible set obtained with these availabilities as linear constraints.
This proposal is the vector Po = [[H1, burn, 3], [H1, surgery, 8], [H1, orthopaedic, 0], [H2, surgery, 2], [H2, orthopaedic, 3], [H2, cardiology, 6], [H3, burn, 2], [H3, cardiology, 1]] and the mediator sends propose (Po) to H1, H2 and H3 for approval.
Each negotiator agent evaluates this proposal and answers back by accepting or rejecting Po: • Agent H1 rejects this offer since its evaluation is very far from the threshold (0.29, a bad score) and gives a recommendation to improve burn and surgery by sending the message reject_proposal ([burn, surgery]);
• Agent H2 accepts this offer by sending the message accept_proposal (), the proposal evaluation being good; • Agent H3 accepts Po by sending the message accept _ proposal (), the proposal evaluation being good.
Just with the recommendations provided by agent H1, the mediator is able to make a new proposal by restricting the value of burn and surgery.
The new proposal obtained is then Pi = [[H1, burn, 0], [H1, surgery, 8], [H1, orthopaedic, 1], [H2, surgery, 2], [H2, orthopaedic, 2], [H2, cardiology, 6], [H3, burn, 5], [H3, cardiology, 1]].
The mediator sends propose (Pi) the negotiator agents.
H1, H2 and H3 answer back by sending the message accept_proposal (), Pi being evaluated with a high enough score to be acceptable, and also considered as a good proposal when using the explanation function of MYRIAD.
An agreement is reached with Pi.
Note that the evaluation of Pi by H3 has decreased in comparison with Po, but not enough to be rejected and that this solution is the Pareto one, P ∗.
Other examples have been tested with the same settings: issues in IN, three negotiator agents and the same mediator agent, with no preference model but selecting randomly the proposal.
We obtained solutions either equal or close to the Maximin solution, the distance from the standard deviation being less than 0.0829, the evaluations not far from the ones obtained with P ∗ and with less than seven proposals made.
This shows us that we are able to solve this multi-issue multilateral negotiation problem in a simple and efficient way, with solutions close to the Pareto solution.
9.
CONCLUSION AND FUTURE WORK This paper presents a new protocol to address multilateral multi-issue negotiation in a cooperative context.
The first main contribution is that we take into account complex inter-dependencies between multiple issues with the use of a complex preference modelling.
This contribution is reinforced by the use of multi-issue negotiation in a multilateral context.
Our second contribution is the use of sharp recommendations in the protocol to help in accelerating the search of a consensus between the cooperative agents and in finding an optimal solution.
We have also shown that the protocol has subgame perfect equilibria and these equilibria converge to the usual maximum solution.
Moreover, we tested this protocol in a crisis management context where the negotiation aim is where to evacuate a whole set of injured people to predefined hospitals.
We have already developed a first MAS, in particular integrating MYRIAD, to test this protocol in order to know more about its efficiency in terms of solution quality and quickness in finding a consensus.
This prototype enabled us to solve some examples with our approach and the results we obtained are encouraging since we obtained quickly good agreements, close to the Pareto solution, in the light of the initial constraints of the problem: the availabilities.
We still have to improve our MAS by taking into account
The Sixth Intl. .
Joint Conf.
on Autonomous Agents and Multi-Agent Systems (AAMAS 07) 949
the two types of recommendations and by adding a preference model to the mediator of our system.
Moreover, a comparative study has to be done in order to evaluate the performance of our framework against the existing ones and against some variations on the protocol.
10.
ACKNOWLEDGEMENT
This work is partly funded by the ICIS research project under the Dutch BSIK Program (BSIK 03024).