# Self-organization Combining Incentives and Risk Management for a Dynamic Service-Oriented Multi-agent System

Nelson Rodrigues, Eugénio Oliveira, Paulo Leitão

► To cite this version:

Nelson Rodrigues, Eugénio Oliveira, Paulo Leitão. Self-organization Combining Incentives and Risk Management for a Dynamic Service-Oriented Multi-agent System. 5th Doctoral Conference on Computing, Electrical and Industrial Systems (DoCEIS), Apr 2014, Costa de Caparica, Portugal. pp.101-108, 10.1007/978-3-642-54734-8_12. hal-01274758

HAL Id: hal-01274758

https://inria.hal.science/hal-01274758

Submitted on 16 Feb 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Self-organization Combining Incentives and Risk Management for a Dynamic Service-oriented Multi-agent System

Nelson Rodrigues¹,³, Eugénio Oliveira²,³, Paulo Leitão¹,³

¹ Polytechnic Institute of Bragança, Campus Sta Apolonia, Apartado 1134, 5301-857 Bragança, Portugal

² Faculty of Engineering - University of Porto, Rua Dr. Roberto Frias s/n, 4200-465 Porto,

³ LIACC - Artificial Intelligence and Computer Science Laboratory, R. Dr. Roberto Frias, 4200-465 Porto, Portugal {nrodrigues, pleitao}@ipb.pt, eco@fe.up.pt

**Abstract.** Companies are nowadays placed in very complex and dynamic environments, making their competitiveness mandatory. This competitiveness can also be achieved through the reconfiguration of systems' network. In this paper, high level of self-organization of service-oriented multi-agent systems is explored, aiming to achieve more trustworthy and automatic reconfiguration processes, in dynamic and open environments. The correct self-organization model directly impacts the success of agents' behaviours to actively change or create new appropriate services dynamically. This paper advocates on the influence, in a distributed and cooperative way, of risk management, similarities and incentives to work together in order to speed up the self-organization agent's network. By leveraging the SOA and MAS benefits it is possible to reduce time, effort and money. Based on identified benefits and needs, several research leads for my thesis plan are here proposed towards the realization of self-organization capabilities aiming to accomplish more trustworthy and automatic reconfiguration processes, in volatile and open environments.

**Keywords:** Multi-agent systems, Self-organization, Service-orientation, Risk Management.

## 1 Introduction

Innovative solutions and technologies were created to solve the companies' problems that are playing in very dynamic and competitive environments, subject to the markets pressures demanding highly customized and quality products at reduced price. Companies, to increase their competitiveness, need to implement more flexible and responsiveness systems [1] and increase their participation in cooperative networks, which request different solutions. Currently, decentralized approaches are being pointed to address this challenge [2], based on the distribution of control functions over a network of decision-making entities. Multi-agent systems (MAS) and Service Oriented Architectures (SOA) are two examples of such approaches, based on the same principles of distribution, which can be combined to extract benefits from the two worlds: the intelligence and autonomy provided by MAS solutions and the interoperability offered by SOA solutions [3]. The addition of dynamic and automatic
reconfiguration processes of a distributed system [4], e.g. through adapting or creating the services offered by the several intelligent entities to the new requirements, implies a deeply intelligent orchestration of services. The aim of this paper is to describe an innovative self-organization approach based on incentives mechanisms and risk management in service-oriented multi-agent systems, aiming to achieve more trustworthy and automatic reconfiguration processes, in volatile and open environments.

This paper is organized as follows. Section 2 describes the similarity of the work with the collective awareness systems. Section 3 presents the literature review and Section 4 briefly presents the paper’s contribution as well as the formalization of the proposed model. Section 5 overviews the critical analysis of this proposal and states forward the research directions. Finally, Section 6 wraps up the paper with the conclusions.

## 2 Relationship to Collective Awareness Systems

This contribution is related to collective awareness systems, since it uses a MAS approach to distribute the knowledge and decisions points, being necessary to perform the management of the agent’s colligations that are self-organized and to select the best service. This research needs to be driven to the collective awareness system, since the agents behaves as a global brain to offer better services; local knowledge about the environment allows the agents to take local decisions, which has a huge global impact. Additionally, the management becomes critical since too many agents communicating can imply the congestion of the agent’s network.

## 3 Literature Review

This section reviews the current state of the art in several related fields, providing the initial context around service-oriented domains, and how agents and risk management are being used. To determine the value of risk management, it is considered trust and reputation values to measure threats and predict uncertainty. In a quick perspective, agents are seen as intelligent entities that offer support for robustness, flexibility and adaptability, in case of management of services [3]. For a better autonomy process, they follow decentralized approaches [5] that require the cooperation between the agents. In some situations, this cooperation can take a form of competition, being sometimes necessary to request advices or make alliances, which require the consideration of the level of the risk management (based on trust and reputation).

Fig. 1 summarises the important works focusing individually each one of the referred domains. Once analysing Fig. 1 it is obvious the reasons that motivated this study. Some of these approaches are able to support SOA [14], [15], [16], however they do not introduce the self-organization paradigm. On other hand, the other works lack on components that covers this work’s interests (self-organization phenomena achieved by using incentives and risk management metrics supported by multi-agents). Whenever a network of agents follows an overall goal, and this network does not encourage the cooperation, incentive mechanisms can improve this cooperation
[6]. Additionally, is also considered works that look into the self-organization phenomena, meaning approaches that evolve by small cooperation for a greater goal.

<figure><img src="image_2.png" /><figcaption>Fig. 1. Summary of the literature review.</figcaption></figure>

Essentially there is lack of proposals with self-organization that have in common multi-agent systems and risk management models, which means, an opportunity to research in self-organized systems.

# 4 Research Contribution and Innovation

Complex systems may benefit from self-* characteristics since the agents should be able to change their own behaviours facing the overall system. The individual agents use an intelligent behaviour to actively dynamically change or create new services, without a request but instead in a response to a condition change. As previously stated, the literature review demonstrates some efforts in decentralized approaches to solve the dynamic and competitive environments. However, the existing works lack in some important points, as described previously. This work aims to propose self-organization and incentive mechanisms that can offer a dynamic and automatic self-* properties (e.g., reconfiguration, adaptation) based on agent networks, where each agent combines the learning and risk management about the interaction and cooperation to evolve and offer better services. The system, as a whole, must also find opportunities to evolve by analysing their dynamic environment, the agents should be able to self-configure and self-organize, allowing increase of the system performance on-the-fly. Additionally, this approach pretends to decompose the risk management in trust and reputation models that allow the agents to work with another level of confidence when faced by unexpected scenarios, being capable of evolving in its configuration to a state where there is a better state in order to offer new/modified services with highly customized and trustworthy services with better quality.
**4.1 Self-organization Mechanism**

The addressed research challenge is to move the system from weak connections to strongest ones, as illustrated in Fig. 2 (note that the strength of the connections represents the degree of similarities with the neighbours). By applying self-organization methods, the network self-organize, passing from one specific network topology to another one more consistent and structured. Reminding that the domain is dynamic, therefore it is covered the Open MAS (OMAS), where each agent belong to a specific open society over the network. A self-organized network is essential to create a consistent network, nonetheless some confidence is needed when the agents are being created and organized among themselves. Having this in mind, it is proposed a self-organization model based on incentives, risk management and homophily [8], where individuals share common characteristics (e.g. beliefs, values and education). Incentives, risk management and homophily enables to create a network based on the similarity and confidence of the agent's neighbours but also on the learning of individual players. In this approach, the self-organization behaviour takes into account incentive mechanisms that are used to stimulate this cooperation. A representative part of the proposed approach is shown in Fig. 2.

<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Self-organization network topology (note that each edge has a preferred binding, which corresponds to a stronger connection of each entity).</figcaption></figure>

After a self-organization phenomenon has occurred in a distributed and decentralized system, robustness has increased, allowing surviving without some entities or with incomplete information. Self-organization is structured (Def. 1), which represents the function to achieve or not society and the connections.

**Definition 1.** Being the *self-organization (SelfO)* working in an OMAS with the structure, $SelfO(\mathcal{Soc}, N_t, \mathcal{Oenv}, R_m, \mathcal{Simi}, \mathcal{In}, fSelfO) \to (\mathcal{Soc}', N_t')$, where:

* $\mathcal{Soc} = \{Ag_1, ..., Ag_n\}$ represents a, finite, society of agents $Ag$;

* $\mathcal{Oenv}$ characterises the environment accessible by the agents, where reside different societies of agents;

* $N_t \subseteq \mathcal{Soc} \times \mathcal{Soc}$, is a set of connections where $(Ag_i, Ag_j) \in \mathcal{Oenv}$ represents a direct relationship;

* $R_m$ represents the risk management function of a specific $N_t$;

* $\mathcal{Simi}$ represents the similarity degree function of the assigned $N_t$;

* $\mathcal{In}$ means the incentives function of the assigned $N_t$;
* $fSelfO: Soc \times Oenv \times Rm \times Simi \times In \rightarrow [0,1]$, represents the self-organization function between a threshold $\varepsilon$ (to be parameterized), [...,$\varepsilon$,...1] which corresponds to organize or not;

* $Soc'$ and $Nt'$ represents the next finite society and their connections, after self-organization, respectively.

The model of agents also represents the strength of connections, which can evolve to create a proper network towards the goal, depending on the network topology. In order to increase the cooperation, incentive mechanisms will allow the formation of more stable colligations. In practice, the incentives are given from a normative environment, meaning that there is no supervision entity obligating to accept these incentives.

**Definition 2.** *Mechanism of incentive (In)* based on the Centeno's work [6] is represented as a tuple $In(St,Oenv,Soc, fIn) \rightarrow [0,1]$, where the agent has incentives to create coalitions of the environment and can accept or ignore the incentive depending on the impact it will have on their utility.

* $St = \{St_1,...,St_m\}$ represents the internal states of an agent;

* $fIn: St \times Oenv \times Soc \rightarrow [0,1]$ represents the incentives mechanism function;

Taking into consideration that incentive mechanisms should be tuned for specific topologies, it is realized that to establish weak connections it is required greater incentives, and stronger connections do not require higher values. The self-organisation methodology has a direct relation with the strength of the connections, which means that stronger connections allow fastest and stronger reorganisation. Working in an OMAS has a certain amount of risk, and a stronger and reliable connection structure must be created. Gathering the concepts of trust and reputation models it will be possible to create a risk management model.

**Definition 3.** *Risk management* is defined as $Rm(Nt, St, Trt, Rp) \rightarrow [0,1]$, where:

* $Trt$ represents the trust measure adapted from [18], being defined by $Trt(Xtr,Ytr,Cxt,G,t, fm) \rightarrow [0,1]$ where $Xtr$ represents the trustor, $Ytr$ means the trustee, in a context $Cxt$, for a specific goal $G$ at the time $t$ with the function $fm: Xtr,Ytr,Cxt,G \rightarrow [0,1]$;

* $Rp(Tp, Xtr, Ytr, Cxt, G, t, fr) \rightarrow [0,1]$, where $tp = \{tp_1,...,tp_k\}$ represents a set of recommendations, given by third-party recommender $Tp$ of a trustor agent $Xtr$;

Following the principle that the level of risk depends on trust and reputation of certain agents and certain goals at given times for example, the risk shifts if there is no trust/reputation or only for certain goals. The risk level suffers alteration according to time, if the interaction occurs in the last minute or a week ago the risk level is not the same. Social structures allow to create better organizations without central decision points. One of the social structures characteristics is the homophily, based on the similarity degree of other entities [8], which allows to create stronger connections.

**Definition 4.** *Similarity* is defined as a tuple $Simi(Ro, Sv, Ont, fSim) \rightarrow [0,1]$ where:

* $Ro = \{Ro_1,...,Ro_\lambda\}$ represents a set of roles of each agent $Ag$;

* $Sv$ is a set of service descriptions defined by another structure $Sv(St,Sd,Isv,Osv,Psv,Esv)$, composed by the service name $St$, the description $Sd$, the configuration input $Isv$, output $Osv$, preconditions $Psv$ and effects $Esv$;

* $Ont$ represents a set of concepts $\{c_1,...,c_\mu\}$ and properties $\{p_0,...,p_\nu\}$. This variable is important to evaluate the similarities;
* $fSim: S_v \times S_v \times R_o \rightarrow [0,1]$ represents the degree function;

It is expected that after a self-organization procedure, the system continues to have weak connections, and others became stronger (see Fig. 2). This cooperation among the entities automatically increases the probability of achieving objectives more effectively as predict future events.

## 4.2 Agent's Internal Model

The OMAS structure has several agents with several behaviours, outstanding to the complexity, an autonomic process should be developed for the agents behaviours. Inheriting these characteristics from the computer science, an autonomic system should remain self-configured, self-optimized, self-healed and self-protected. The agents developed should be intelligent enough to trigger the self-healing and self-protection behaviours, this means, learning when non-intentional accidents could occur, or have the intelligence to detect, diagnose and repair such accidents.

This insight fulfils very well the system architecture in two separated layers: in one layer it is represented the intelligence layer, namely the agents, and in the second layer, each agent is responsible to offer the services.

**Definition 5.** The agent's internal model is specified as $Ag(St, Oenv, K, ft, hUT, Sv) \rightarrow St'$ where $St'$ represents the next state, K signifies the knowledge of agent of its set of direct neighbours inside the society, ft is the transition function $ft: Oenv \times St \rightarrow St'$, and $hUt$ represents the utility function of each agent for a specific state and knowledge $hUt: St \times Oenv \times K \rightarrow R$ (remember that not all topologies are motivated to reach its maximum utility [19]);

The agent tends to maximize their own utility, although this local maximization might sometimes not be the best for the global system. Without the supervision, the agents need to collaborate to achieve the global maximum. At this point, and since the objective is to provide services more trustworthy, the agents intend to increase their risk management accuracy $SelfO(Rm, Simi, In) \rightarrow (Soc', Nt')$ by $\max_{i=1...l}{Rm_i}$, where $l$ is the system interactions generated. Fig. 3 depicts the behaviour of decision support and composition, namely, what the agent performs after reasoning about the available services as well as the current network neighbour, to cooperate and owning the information that exists in the network configuration to compose trustworthy services.

<figure><img src="image_4.png" /><figcaption>**Fig. 3.** Dynamic and automatic reconfiguration process engine of each agent.</figcaption></figure>

Just like in the self-organization, the agents will try to compose the best service in
on-going loops. From a strategic perspective, the monitoring of services for planning, e.g. functionality and availability, is controlled. Other perspective is the focus on the operational task, which explore algorithms to compose services, to achieve the goals.

Summing up, the system tends to evolve and organize itself taking into account statistical fluctuations and internal agent's history. Self-(behavioural and structural) organization are applied, by changing the agent's behaviours and network structure.

# 5 Discussion of Results and Critical View

The insights of this work are to create a vision and to formalize concepts necessary to future lines of research. Aiming to verify the potentiality of this approach, two case studies are explored. The first in one business-to-business (B2B) and Virtual Enterprises (VE) domain, by exploring the benefits of using decentralized entities based on models of trust and reputation, by taking advantage of a framework that already provides a trust negotiation based on the normative environment called ANTE (Agreement Negotiation in Normative and Trust-enabled Environments) [20]. The second case study is related to the Smart Grids domain, aiming to achieve a consistent network based on some properties, such as reliability. The network can be smart enough to recognize when each entity is starting to fail or even reduce its performance and react appropriately. To validate the proposed approach more serious tests must be performed after this first attempt to identify the research opportunities. Beyond the chosen domain it is necessary to create the environment propitious to the evolution of the system and characterization of the entities feedback in order to be able to use previous equations in their overall, this mode is possible to show the advantages of this approach, which are beyond human capabilities to solve such complex.

# 6 Conclusions and Future Work

This work intends to propose one contribution focusing the dynamic offer, highly customized and trustworthy, of services with better configurability and responsiveness. An innovative self-organization approach for service management in service-oriented multi-agent systems is proposed, describing a reconfigurable architecture that considers risk management, incentives and homophily visions, to achieve self-* characteristics. In such self-organized systems several research questions may arise, namely how and when to evolve, what is the degree of the similarities to take into account in the trust model and finally, how all these combined pieces can offer a better configuration that modifies or creates more trustworthy services. To validate the hypothesis, two use cases focusing distributed environments will be considered, namely B2B and Smart Grids. The preliminary achievements show that the self-organization component can enhance services' selection by increasing the degree of connections, increasing the trustworthy. Future work will try to answer some remaining open questions, such as, "how to handle the correct management of the network's features", which directly impacts the success of the service governance.
## References

1. Colombo, A. W., Jammes, F., Smit, H., Harrison, R., Lastra, J. L. M., Delamer, I. M.: Service-oriented architectures for collaborative automation, in Industrial Electronics Society, in Proc. of the 31st Conf. IEEE Industrial Electronics Society (IECON'05), 2005

2. Leitão, P.: Agent-based distributed manufacturing control: A state-of-the-art survey, Eng. Appl. Artif. Intell., vol. 22, no. 7, pp. 979-991, 2009

3. M. N. Huhns.: Agents as Web services, Internet Computing, 6(4), pp. 93-95, 2002

4. Maximilien, E. M., Singh, M. P.: A Framework and Ontology for Dynamic Web Services Selection, IEEE Internet Computing, vol. 8, no. 5, pp. 84-93, 2004

5. Khan, S.P., Ismaeel, S., Ahmad, H.F., Suguri, H., Akbar, M., Elahi, A.: Enabling Negotiation Between Agents and Semantic Web Services, AWIC, 43, pp. 284-291, 2007

6. Centeno, R.: Mecanismos Incentivos para la Regulación de Sistemas MultiAgente Abiertos basados en Organizacióne, Universidad Rey Juan Carlos, 2012

7. Maximilien, E. M., Singh, M. P., Reputation and endorsement for web services, SIGecom Exch., vol. 3, no. 1, pp. 24-31, 2001

8. E. del Val.: Decentralized semantic service discovery based on homophily for self-adaptive service-oriented MAS, in Proc. if the 10th International Conference on Autonomous Agents and Multiagent Systems, vol. 3, pp. 1347-1348, 2011

9. Zhang, J., Cohen, R., Larson, K.: A Trust-Based Incentive Mechanism for E-Marketplaces, in Proc. of the 11th International Workshop on Trust in Agent Societies, (TRUST'08), Estoril, Portugal, May 12-13, vol. 5396, pp. 135-161, 2008

10. Pan, Z., Baik, J., A QOS enhanced framework and trust model for effective web services selection," J. Web Eng., vol. 9, no. 2, pp. 186-204, 2010

11. Raftopoulou, P. Petrakis, E. G. M.: iCluster: a self-organizing overlay network for P2P information retrieval, in Proc. of the IR research, 30th European conference on Advances in information retrieval, pp. 65-76, 2008

12. Yolum, P., Singh, M. P.: Engineering self-organizing referral networks for trustworthy service selection, Trans. Sys. Man Cyber. Part A, vol. 35, no. 3, pp. 396-407, 2005

13. Buhler, P. A., Vidal J. M.: Adaptive Workflow = Web Services + Agents, in Proc. of the International Conference on Web Services, pp. 131-137, 2003

14. Lai, K., Feldman, M., Stoica, I., Chuang, J.: Incentives for Cooperation in Peer-to-Peer Networks. 2003

15. Ye, D., Zhang, M., Sutanto, D.: Self-organization in an agent network: A mechanism and a potential application, Decision Support Systems, vol. 53, no. 3, pp. 406-417, 2012

16. Quang-Anh Nguyen, Vu, Hassasz, S., Armetta, F., Gaudou, B., Canal, R.: Combining Trust and Self-Organization for Robust Maintaining of Information Coherence in Disturbed MAS, Fifth IEEE International Conference on Self-Adaptive and Self-Organizing Systems (SASO'11), pp. 178-187, 2011.

17. Shin, J., Kim, T., S. Tak.: Incentive mechanism for service differentiation in P2P networks, in Proc. of the 2009 Int. Conf. Hybrid Information Techn., pp. 333-339, 2009.

18. Oliveira, E.: Software Agents: Can We Trust Them?, in Proceedings of IEEE 16th International Conference on Intelligent Engineering Systems (INES'12), pp. 15-20, 2012.

19. Barabási A. L., Albert, R.: Emergence of scaling in random networks, Science, vol. 286, no. 5439, pp. 509-512, 1999.

20. Cardoso, H., Urbano, J., Brandão, P., Rocha, A., Oliveira, E.: ANTE: Agreement Negotiation in Normative and Trust-Enabled Environments, PAAMS, pp. 261-264, 2012.

21. Menasce, D., Gomaa, H., Malek, S., Sousa, J.P.: SASSY: A Framework for Self-Architecting Service-Oriented Systems, Software, IEEE, vol.28, no.6, pp.78-85, 2011.