# Behavioral Norms in Virtual Organizations

Mahdieh Shadi, Hamideh Afsarmanesh

► To cite this version:

Mahdieh Shadi, Hamideh Afsarmanesh. Behavioral Norms in Virtual Organizations. 15th Working Conference on Virtual Enterprises (PROVE), Oct 2014, Amsterdam, Netherlands. pp.48-59, 10.1007/978-3-662-44745-1_5.hal-01392091

HAL Id: hal-01392091

https://inria.hal.science/hal-01392091

Submitted on 4 Nov 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Behavioral Norms in Virtual Organizations

Mahdieh Shadi¹ and Hamideh Afsarmanesh¹

¹Federated Collaborative Networks Group, Informatics Institute, Computer Science Department, University of Amsterdam {m.shadi, h.afsarmanesh}@uva.nl

**Abstract.** Virtual Organizations (VOs) consist of groups of agents that collaborate towards achieving their specified goals. VO Partners are independent, autonomous, and heterogeneous, thus often exhibiting complex behaviors in co-working. Frictional behavior demonstrated by even a few partners, may cause drastic results and total failure of the VO. Therefore, it is necessary to model and analyze VO partners' behavior. This paper introduces the VO Supervision Assistance Tool (VOSAT), developed based on leveraging partners' commitments/promises, to monitor partners' behavior against the synergetic norms in the VO. For this purpose, three kinds of behavioral norms are defined, including: socio-legal norms, functional norms, and activity-related norms. Additionally, a fuzzy norm is introduced to indicate agents' trustworthy behavior. The functionalities supported in VOSAT enable the VO coordinator with identifying the high risk tasks and the weak or weakest points in the flow of VO planned operations. It further assists the coordinator with finding suitable candidate partners for handling the exceptions that arise during the VO operation phase. These in turn improve the success rate of the VOs.

**Keywords:** Virtual Organizations; Behavioral Norms; Promise Formalization; Trust modeling; Behavior Monitoring.

## 1 Introduction

An agile Virtual Organization (VO) can be effectively launched within a Virtual organizations Breeding Environment (VBE), to respond to the emerging opportunities, either for innovation or to reflect on market changes [1]. Typically VOs consist of autonomous, heterogeneous, and geographically distributed organizations, which collaborate to achieve the specified set of VO goals. As reported in [2], the study of Laciyt on the construction, benefits and risks of Virtual Enterprises indicates that less than 50% of these established VOs were successful, while more than 30% of these VOs ended up either in total failure or having high risk to fail. Primarily, two categories of risks - exogenous and endogenous - are highlighted in the literature [3] in relation to the success rate of virtual organizations. The former refers to the risks caused by external factors such as the external political risks, technical risks, financial risks, etc. The latter refers to the risks that caused by factors internal to the VO, which rise through its business activities. Furthermore, the behavior of involved organizations is identified as a main element of the endogenous risks. In the approach, which is proposed in this paper, the behavior of organizations can be monitored through the time, such that weak
points can be identified and brought to the attention of the VO coordinators, to take
appropriate strategic actions.

For this purpose, a framework is introduced for monitoring partners’ behavior in the
VO. Within this framework, we introduce three kinds of behavioral norms, including
socio-legal norms, functional norms, and activity-related norms for the VOs [4], which
constrain the partners’ behavior and that can be monitored for predication of potential
related risks in the VO. Although these three kinds of norms are briefly described
below, please note that the main focus of the paper and the remaining sections is on the
activity-related norms.

The main norms that rule over the socio-legal aspects applied to VO partners are either those that are known and generally observed in the society, or those that are formalized as clauses within the consortium agreement prepared for the VO, and which are agreed and signed by all partners. We have adopted the approach proposed in [5] to categorize the socio-legal norms as *obligations*, *prohibitions*, and *permissions*, although formalization of these norms is not the main focus of the paper. For example, leadership rights are considered as socio-legal norms from the permission category.

The general terms of operational collaboration among agents involved in a VO are officially negotiated among them via the contracts signed during the VO formation stage, indicating that the partners together fulfill the VO's objectives. In our framework, based on the specifications provided in the VO contracts, a *responsibility template* is first assumed to be extracted and formalized partially as a time chart, and partially as textual terms specified in the contract, indicating the main roles and general responsibilities of the VO partners. However considering the dynamic and adaptive nature of the VO, these contract terms do not and/or cannot specify the details of day-to-day activities of the VO partners.

Functional norms in the VO correspond to and reflect on the assignment of coarse-grained tasks, but with partial responsibility for each partner, according to what is expressed and represented within the VO responsibility template. Furthermore, the functional norms together with the responsibility template provide the base for definition and assigning of day-to-day activities to each partner, throughout the VO operation phase. However, before such activity assignment is made, it needs to be planned and agreed among the VO partners. In fact such agreement must be reached between two parties, e.g. the task leader who suggests the sub-task, and the partner who commits to the sub-task. In our proposed approach, after reaching each agreement, a *promise* is made by the partner to the task leader, to perform the needed activity. Fulfillment of a promise made as the result of the agreement described above shall correspond to the *activity-related norms* in that VO. Therefore, the activity-related norms are also in conformance with the functional norms at the VO.

In this paper, the presented examples target the fine-grained behavior of VO partners. In other words, while contract-based obligations of partners are reflected within responsibility template and can be checked against the functional norms in the VO, the presented examples focus mainly on detailed promises given by partners, in relation to their daily activities. Through the proposed framework, activity-related behavior of partners can be monitored and their possible violations against activity-related norms are identified. The decision on how to deal with the violation of norms and which sanction to impose on violating partners is also usually specified in the
consortium agreement document. But the decision on applying a sanction to a partner is typically made by the VO coordinator and the management team.

As a consequence of supporting the monitoring of activity-related behavior of VO partners, it is also possible to reason about partner's trustworthiness level, based on their past behavior. For the purpose of reasoning on trust level of partners, a fuzzy norm is defined in the framework. We can also monitor and check for the violation of this defined fuzzy norm representing the partner's trustworthiness level. Using the information about partners' trustworthiness, it is possible to reason about the risk factors in the VO. For instance, considering the complete set of current promises at the VO, the least trustworthy promise(s) can be identified. Furthermore, establishing the trustworthiness of partners can also be used for making other decisions by the VO coordinator. For instance when and if an exception is raised for which VO partners need to volunteer to take over some tasks, the VO coordinator can select the most trustworthy among all volunteers.

The remaining structure of this paper is as follows: Section 2 addresses a high level formalization of behavior in Virtual Organization. Section 3 discusses monitoring the activity-related norms. Section 4 specifies how our tool enables the VO coordinators to monitor the trustworthiness of agents. Section 5 addresses some related research, and how our proposed framework compares against them, and Section 6 concludes the paper.

## 2 High Level Formalization of Behavior in Virtual Organizations

Virtual Organization is a temporary goal-oriented collaborative network, which is formed in response to the emergence of a business opportunity, e.g. a manufacturing project, and will be dissolved when its goals are achieved. To fulfill the VO's objectives, typically at its creation stage, a number of contracts and a VO Consortium agreement are prepared and signed. A Consortium agreement represents the base for socio-legal norms in the VO, consisting of authorizations, permissions, prohibitions, and obligations. Although socio-legal norms are very important to be defined, they do not directly relate to the operational goals of the VO, rather they support the collaboration atmosphere/infrastructure, which is required for fulfilling its operational goals. These norms include both the generic norms common to all VOs, as well as certain specific norms relevant only to each VO as a single virtual entity. The co-working norms, the norms for sharing data / knowledge / resource, and the prohibitive norms are some examples of these generic norms. The inheritance of the VO assets and responsibilities at the VO dissolution phase, among others are examples of domain specific norms. Socio-legal norms can be categorized using the approach proposed in [5], and are not the main focus of this paper.

As mentioned in the Introduction section, the general responsibilities of VO partners can be extracted from their contracts, which are established during the VO creation stage, and presented in a responsibility template. For example, Fig. 1 shows the responsibility template in the VO established for an R&D project.
<figure><img src="image_2.png" /><figcaption>**Fig. 1.** Example of responsibility template for R&D projects</figcaption></figure>

However, usually a task involves several partners, and pre-defining all details of the sub-tasks cannot be effectively done during the VO creation stage. It is rather during the VO operation stage when the responsibility template constituting the functional norms is further instantiated with the definition of more detailed activities for each sub-task, with their interdependencies and specific partner assignments.

We consider that all agreements made among agents for performing their day-to-day activities (sub-tasks) are specified as promises. As such, the promises made in a VO provide the definition of activity-related norms. One important and challenging issue in Virtual Organizations is how to deal with joint-responsibilities. Usually, there are activities that should be performed by a group of agents, for which all agents are jointly responsible. In VOSAT framework, these kinds of day-to-day activities are referred to as joint-promises [4]. Making a joint-promise by several agents leads to each of them obliging itself to performing its own responsibilities as well as contributing to the fulfillment of parts for which other partners are responsible. This is due to the fact that
with joint responsibilities, the reputation and financial interests of all involved partners are at risk. To support fairness in keeping joint-promises, the framework requires that agents involved in a joint-promise also rank their collaborators on the level of cooperativeness and performance toward the joint-responsibility.

The supervisory function of VOSAT provides an enforcement mechanism to detect when a norm is active as well as when and if it gets violated. In other words, to support the VO supervision, it is needed to monitor agents' behavior. This is done partially through checking the compliance of agents' behavior with the norms of the VO, and imposing corresponding sanctions on agents that violate norms. Additionally, since functional norms are in conformance with the activity-related norms, it is sufficient to monitor only the activity-related norms. Besides the activity-related norms, as a part of functionality for monitoring partners' behavior, the level of partners' trustworthiness is also monitored and analyzed against a fuzzy norm defined in the VO.

In a normative environment like a VO, norms can have different levels, norms at level zero are triggered by the external events, whereas a level k + 1 norm, with k > 0, is triggered in case of a violation of some norm(s) defined at level k [5]. In VOSAT framework, trust level is defined as a fuzzy norm for each agent. Activity-related norms are at level 0, because these norms are triggered by environment-related facts. However, the trust(worthiness) level, which is typically triggered by the violation of activity-related norms, is at level 1.

Imposing related sanctions against the norms' violations is very important in Virtual Organizations. For instance, if a promisor notifies the VO coordinator before reaching the deadline that it cannot fulfill its promise on time, it should then be punished less than if it were in the situation in which the deadline is passed and the promise if not fulfilled. But clearly, the sanction policies are not the same in different VOs. Usually, there are two kinds of sanctions that can be applied in order to incentivize the norm compliance and to discourage deviation from norms, one affects agent's resources (e.g. financial punishment), and the other one affects agent's reputation (e.g. black listing), and both leading to the need of re-assignment of the agent's tasks to others. Sanction rules are either mentioned in the consortium agreement or specified by the VO coordinator, and can be defined as response to the violation of norms at each of the mentioned levels.

## 3 Monitoring Activity-related Norms

In VOSAT framework, as originally introduced in [4] and further extended in this paper, a promise is defined as a tuple <x, y, p, d, q, d'> where x is the *promisor agent*, y is the *promisee agent*, p is the *proposition* which a promisor should bring about before *deadline* d, q is a *pre-condition* for realizing p by x, and d', is the *deadline* before which q should hold.

Please note that the VOSAT system considers the notion of time, and can decide when and if a deadline is reached. In this framework, as shown in Fig. 2, different states are considered for a promise. If the pre-condition of a promise is fulfilled on time, the promise's state is Conditional, but if it is not fulfilled before reaching its deadline, the promise's state is Unconditional. However, if the deadline of pre-condition is passed and
its pre-condition is not fulfilled then the promise is Dissolved. If the deadline of a promise is passed and the promised proposition is fulfilled then the state of the promise is Kept, but if it is not fulfilled by the deadline, the promise's state is Not Kept. The Invalidated promise is the one that has not been fulfilled due to some reasons beyond the promisor's control. If the promisee cancels its promise its state is Withdrawn, while if the promisor cancels it, its state is Released. For example $Pr^C$, as it appears in Fig. 3, represents a conditional promise.

<table><caption>Fig. 2. Different states of a promise</caption><thead><tr><th>Promise States</th><th>Description</th></tr></thead><tbody><tr><td>Conditional (C)</td><td>There are some conditions that need to be fulfilled first.</td></tr><tr><td>Unconditional (UC)</td><td>All conditions are fulfilled before their deadlines</td></tr><tr><td>Kept (K)</td><td>The promise is assessed to have been kept.</td></tr><tr><td>Withdrawn (W)</td><td>The promise is withdrawn by the promisor.</td></tr><tr><td>Invalidated (In)</td><td>The promise has not been fulfilled due to some reasons beyond the promisor's control.</td></tr><tr><td>Released (R)</td><td>The promise is not needed any more, so it is cancelled by the promisee.</td></tr><tr><td>Not Kept (NK)</td><td>The promise is assessed to have not been kept.</td></tr><tr><td>Dissolved (Dis)</td><td>When conditions are not fulfilled before their specified deadlines.</td></tr></tbody></table>

Furthermore, in VOSAT, the state of promises made by each partner is monitored. For this purpose, a set of rules are defined that apply to promises during their life cycle, and represent how the facts from the environment, and actions taken by the promisor potentially cause the state transition in given promises. Fig. 3 represents a set of rules, which are further explained below.

* $T, T, Agree(x, y, p, d, q, d') \Rightarrow Pr^C(x, y, p, d, q, d')$

* $Pr^C(x, y, p, d, q, d'), \neg d', Fulfill(z, q) \Rightarrow Pr^{UC}(x, y, p, d, q, d')$ where z is an arbitrary agent

* $Pr^{UC}(x, y, p, d, q, d'), \neg d, Fulfill(x, p) \Rightarrow Pr^K(x, y, p, d, q, d')$

* $Pr^{UC}(x, y, p, d, q, d'), \neg d, Withdraw(x, y, p) \Rightarrow Pr^W(x, y, p, d, q, d')$

* $Pr^{UC}(x, y, p, d, q, d'), d \wedge \neg p, nop \Rightarrow Pr^{NK}(x, y, p, d, q, d')$

* $Pr^C(x, y, p, d, q, d'), d' \wedge \neg q, nop \Rightarrow Pr^{Dis}(x, y, p, d, q, d')$

* $Pr^{UC}(x, y, p, d, q, d'), T, Fail(p) \Rightarrow Pr^{In}(x, y, p, d, q, d')$

* $Pr^{UC}(x, y, p, d, q, d'), T, Release(y, x, p) \Rightarrow Pr^R(x, y, p, d, q, d')$

<caption>Fig. 3. Rules for state transition of a promise during its life cycle [4]</caption>

The rules primarily express how agents' interactions influence the state of promises. In these definitions, we use “T” for true and “nop” for no-action operation. A rule is represented as $\rho, \phi, \alpha \Rightarrow \psi$ where $\rho, \phi, \alpha$ respectively represent a *promise-related fact*, an *environment-related fact* and an *action*, and $\psi$ represents a *promise-related fact*.
Promise-related facts describe the state of a promise as addressed above, e.g., $Pr^C(x, y, p, d, q, d')$ denotes the fact that this promise is in conditional state.

Environment-related facts describe the state of the environment e.g. if $\varphi$ is *d*, then it represents that a deadline is reached, and if $\varphi$ is *T*, it represents that the environment facts are irrelevant to this rule.

We further define a specific set of actions that influence the state the promise as shown below. For instance, the creation of a new promise is achieved through an "Agree" action.

* Agree(x, y, p, d, q, d'): x agrees with y to make the proposition p true, before the deadline d, if the proposition q is true before deadline d'.

* Withdraw(x, y, p): x informs y that he withdraws his promise to make the proposition p true in the environment.

* Release (y, x, p): y tells x that it is no longer needed to keep his promise to make the proposition p true in the environment.

* Fulfill (x, p): x fulfills its promise and thus the proposition p is now true in the environment.

* Fail(p): proposition p can no longer become true in the environment, due to an external failure. This is considered as an environmental action rather than an agent's action.

The state of our proposed behavior supervisory system makes a transition either when an agent performs an action, as mentioned above, or when a deadline is reached. Agents' actions may also cause some environment-related facts to become true, which in turn may trigger some rules applied to the life cycle of a promise. Consequently, in time both the sets of promise-related facts and the environment-related facts will be changed, and new facts are derived.

It should be noticed that for each promise, at most one of the mentioned states is true at any point in time and that some states including kept, not kept, withdrawn, invalidated and dissolved will in principle remain true in the VOSAT framework, once they are true. Not kept and withdrawn states are considered as violation states, and may be decided in a VO to remove them once the sanctions (e.g. charging some damage costs to an agent or adding the agent to some blacklists) are applied. Other states except for conditional and unconditional will remain in the promise-related facts forever.

The derived promise-related facts, such as the withdrawn and not kept states of promises, may in turn also trigger some *sanction rules*, and result in some new facts, which should then be transmitted to the VO coordinator. For instance $Pr^{NK}(x, y, p, d_1, q, d_2) \Rightarrow Add\ balcklist (x)$ is a kind of sanction rule, relating promise-related facts to environment-related facts. This sanction rule adds the promisor of a not kept promise to a specific black list, for example to be used later for making decisions by the VO coordinator, such as in the case of selecting suitable VO partners for creation of a new VO.

We have used Organization Oriented Programming Language (2OPL) [6] as the organizational setting to implement our supervisory assistance tool. Some details about our implemented tool are addressed in [4].
# 4 Agents Trustworthiness Level as a Fuzzy Norm

The violation of activity-related norms may only trigger sanction rules but also trigger some new norms, such as the trust(worthiness) level norm related to involved agents. In other words, the trustworthiness level of an agent may increase/decrease depending respectively on its fulfillment or violation of its activity-related norms. Therefore, as explained earlier, the activity-related norms are norms of level 0, while the trustworthiness level norms are norms of level 1.

## 4.1 Trust Modeling

There are a large number of approaches introduced in the literature for an agent to build a model of trustworthiness for other agents in an environment. In [7], a survey of trust models and approaches is presented and two categories of techniques, i.e. objective external evaluation agencies [8] and subjective external evaluation agencies are introduced. Our approach applies a combination of these two general approaches, i.e. a VOSAT's normative artifact collects on one hand the information related directly to an agent's behavior norm abidance containing certain pre-defined criteria, and on the other hand collects the ranking by others about each agent's collaboration, as provided by other agents when together performing joint-responsibilities. These two factors are then combined for calculating an overall trust value of each agent, applying a fuzzy comprehensive evaluation method. Our approach is flexible and different factors can also be added to it if needed for evaluating agents' trustworthiness. The fuzzy comprehensive evaluation method [9], which we introduce for trust evaluation in VOS, considers various factors (each indicated as $f_i$) that influence a certain element, and it applies fuzzy mathematical methods to evaluate the merits and demerits of that element [9]. At the first stage, the fuzzy factor set $F = \{f_1, f_2, ..., f_n\}$ and the evaluation set $E = \{e_1, e_2, ..., e_m\}$ are established. In VOSAT approach, we have considered two fuzzy factors of individual norm abidance - evaluated in the interval of [0,2], and collective norm abidance - evaluated in the interval of [0,2]; thus $f_1$ denotes individual norm abidance and $f_2$ denotes collective norm abidance. To evaluate the individual norm abidance for an agent A, the interaction/collaboration experiences of agent A with all other agents are considered, namely the number of all kept promises made by A to other agents, and the number of all violated promises. To evaluate the collective norm abidance for an agent A, all received ranking recommendations from other agents about A are aggregated.

Our proposed evaluation set for trustworthiness, i.e. {$e_1$ = high distrust, $e_2$ = medium distrust, $e_3$ = low distrust, $e_4$ = low trust, $e_5$ = medium trust, $e_6$ = high trust} is defined in the interval of [0,2].
<figure><img src="image_3.png" /><figcaption>Fig. 4. The proposed trust evaluation set</figcaption></figure>

As Fig. 4 shows, membership functions from left to right correspond to {$e_1, e_2, ..., e_6$}. At the second stage, we establish the fuzzy evaluation matrix R. So far, we have defined how to collect/calculate the value for each factor. Then according to the value of factor $f_i$, we can determine its grade of membership in each $e_j$, which is expressed as $r_{ij}$. For example, if for agent A we have {$f_1 = 1.6, f_2 = 1.2$} then the fuzzy evaluation matrix R to calculate its trust level is $R = \begin{bmatrix} 0.00010 \\ 0.00100 \end{bmatrix}$.

At the third stage, we introduce different weights for each introduced factor, i.e. $W = \{w_1, w_2, ..., w_n\}$, where $\sum_{i=1}^{n} w_i = 1$. Depending on the type of VO the importance of one factor may be more than another factor. Assuming the $W = \{0.7, 0.3\}$ relating to the factor set of {$f_1 = 1.6, f_2 = 1.2$}, the result of the comprehensive fuzzy evaluation is calculated as $B = W \cdot R = \begin{bmatrix} 0.7, 0.3 \end{bmatrix} \cdot \begin{bmatrix} 0.00010 \\ 0.00100 \end{bmatrix} = \begin{bmatrix} 0 & 0 & 0 & 0.3 & 0.7 & 0 \end{bmatrix}$.

For the defuzzification process, we directly apply the center of gravity method, as introduced in [10]. As a result of defuzzification, the result of B will transfer to a single value, denoting the overall trustworthiness of each agent. For the mentioned example, the $[0 0 0 0.3 0.7 0]$ is then defuzzified to 1.49, showing the overall trust value of agent A. If in a specific VO, the minimum accepted level of trustworthiness is "medium trust" then the violation of this agent (with the overall trust value 1.49) is zero.

In our approach we also emphasize the importance of having sufficient information about an agent for evaluating its trustworthiness level. Therefore, for fairness purposes, a confidence level should also be considered for each factor related to the agents in the VO. In other words, if the confidence level for a factor is above a predefined threshold, then it has a direct effect on the trust evaluation results. For example, if an agent's involvement in the VO is either for longer than T period of time or in more than C number of activities, his trustworthiness level norm is active and it will be monitored if any activity-related norm of his is violated. Trustworthiness level norm can also be treated differently based on the type of the VO. For example in one VO it may be sufficient to have a medium trust level for agents, while in another one it may be necessary to fulfill the high trust level for all agents.

One advantage of monitoring this fuzzy norm for trustworthiness of agents is to enable the VO coordinator with finding the best potential candidate for handling the exceptions that require re-assignment of tasks. Another advantage is to assist the
coordinator with identifying which one of the running tasks (current promises) might involve risks, through identifying which agents' trustworthiness level norm are violated.

## 5 Related Works

In relation to agents formal commitments, a number of research in the multi-agents community address the area of agents' formal contracting, the concept of leveled commitment contract [11], and reasoning about commitments and penalties [12]. The authors in [13] propose a contract fulfillment protocol based on the normative statements' lifecycle. Our approach however differs from the above approaches due to targeting the specificities of the VO environments, e.g. the aspects of continuous evolution, joint responsibilities, etc., for which we introduce and apply the concept of promises that are not necessarily bilateral.

Related to categorization of norms in agent communities, in [14] a two-level normative agent interactions is proposed for a society of agents including an institutional level, and an operational level. The institutional, constitutional, and operational levels are also addressed as a hierarchical organization of norms in [15]. Although our proposed approach has some aspects in common with these approaches, it addresses the VO dynamism as activity-related norms, defined as promises, which do not directly represent contractual obligations. Furthermore in our approach, such dynamic aspects are specified gradually during the VO operation phase.

In relation to research on the formalization of promises and norms, in [16] and [17] some definition of promises are represented by Modal logic. However this logic does not allow for reasoning about the complex life cycle of promises, which is needed in our proposed operational framework for VOs. In [18] norm conflicts and inconsistencies in Virtual Organizations of software agents are addressed. Dynamic nature of a VO results in changing the agents' normative position, and consequently conflicts may occur in an agent's norms. In their model of norm-governed agency inspired by the BDI model, agents can independently decide either to obey their norms or to violate them. However, in VOSAT framework, the internal states and operations of individual agents are not considered and only the external actions of agents are monitored against the given set of norms. Moreover, agents' actions are not limited to the two types of actions - obedience and violation- rather a number of different states are considered for promises.

According to the theorem of "duality of structure" [19], in a VO where agents repeatedly refer to the social structures (norms) to do their actions, trust is a medium of structuration. In relation to computational reputation and trust models, in [8] four different categories are classified, which are explained below. In the so called category of Agent-Oriented Solitary Approaches [20], the evaluations are calculated by the agent itself according to its own previous experiences without any exchange of information. In the category of Agent-Oriented Social Approaches [21], agents calculate the evaluations considering both their own experiences as well as the third-party information. Trust modeling in social networks is usually categorized in agent-oriented approaches [22]. In the categories of Objective External Evaluation Agencies [8] and Subjective External Evaluation Agencies [23] instead of agents, external agencies
collect the information. The former computes evaluations according to certain objective criteria, while the latter aggregates the subjective agents' evaluations that are collected by the system. In proposed approach, we combine elements from the last two approaches above, to make the trust model for VO partners.

# 6 Conclusions

This paper addresses a framework and tool called VOSAT for virtual organizations to assist with controlling the behaviors of agents involved in VOs, through monitoring their activity-related norms and imposing appropriate sanctions when agents fail and norms are violated. To perform day-to-day activities a number of individual and joint promises are made by the involved agents, which in turn define the activity-related norms in the VOs. The main focus of the paper is on the definition of the needed framework as well as the extension of the tool to define and monitor trustworthiness level of agents as a fuzzy norm. Consequently, the VOSAT enables the VO coordinator with both finding suitable candidate partners to replace a failing partners and handle such exceptions during the VO operation phase, as well as identifying the weakest points or the high risk tasks in the VO's planned operations. These in turn assist with improving the success rate of VOs. The proposed tool is prototypically implemented using the Organization Oriented Programming Language (2OPL) [6], and more details about the tool implementation are addressed in [4].

# Acknowledgments

The work on this paper is partially supported by the FP7 project GLONET.

# References

1. Afsarmanesh, H., Camarinha-Matos, L.M. : On the classification and management of Virtual organisation Breeding Environments. IJITM, vol. 8(3), 234-259 (2009)

2. Gao, W.: Study on the Construction and Benefits and Risks of Virtual Enterprise. Harbin Institute of Technology (2004)

3. Sun, X., Huang, M., Wang, X.: Tabu search based distributed risk management for virtual enterprise. In: Industrial Electronics and Applications, pp. 2366-2370. IEEE Press, Harbin (2007)

4. Shadi, M., Afsarmanesh, H. and Dastani, M. : Agent Behaviour Monitoring in Virtual Organizations. In: 22th IEEE International Conference on Enabling Technologies Infrastructure for Collaborative Enterprises (WETICE), IEEE Press, Tunisia (2013)

5. Tinnemeier, N. A., Dastani, M. M., Meyer, J. J., Torre, L.: Programming normative artifacts with declarative obligations and prohibitions. In : Web Intelligence and Intelligent Agent Technologies, IEEE/WIC/ACM International Joint Conferences, pp. 145-152. (2009)
6. Dastani, M., Grossi, D., Meyer, J.-J., Tinnemeier N. A.: Normative multi-agent programs and their logics. In: Workshop on Knowledge Representation for Agents and Multi-Agent Systems, pp. 16–31, LNCS 5605, (2009)

7. Social knowledge for e-governance (eRep), http://megatron.iiia.csic.es/eRep

8. Msanjila, S. S., Afsarmanesh, H.: Trust analysis and assessment in virtual organization breeding environments. International Journal of Production Research, vol. 46(5), 1253-1295 (2008)

9. Li, L. J., Shen, L. T. : An improved multilevel fuzzy comprehensive evaluation algorithm for security performance. The Journal of China Universities of Posts and Telecommunications, vol. 13(4), 48-53 (2006)

10. Patel, A. V., Mohan, B. M.: Some numerical aspects of center of area defuzzification method. Fuzzy Sets and Systems, vol. 132(3), 401-409 (2002)

11. Sandholm, T., Lesser, V. R.: Levelled Commitment Contracts and Strategic Breach. Games and Economic Behavior, 212-270 (2001)

12. Excelente-Toledo, C. B., Bourne, R. A., Jennings, N. R. : Reasoning about commitments and penalties for coordination between autonomous agents. In: Fifth International Conference on Autonomous Agents, ACM, pp. 131-138, (2001)

13. Sallé, M. : Electronic Contract Framework for Contractual Agents. Advances in Artificial Intelligence. In: 15th Conference of the Canadian Society for Computational Studies of Intelligence, pp. 349-353, Springer, (2002)

14. Dignum, V., Dignum, F.: Modelling agent societies: co-ordination frameworks and institutions. Progress. Artificial Intelligence: Knowledge Extraction, Multi-agent Systems, Logic Programming, and Constraint Solving, LNAI 2258, pp. 191-204 (2001)

15. Cardoso, H. L., Oliveira, E. : Virtual Enterprise Normative Framework Within Electronic Engineering societies in the agents world, pp. 898-898 (2005)

16. Burgess, M. : An Approach to Understanding Policy Based on Autonomy and Voluntary Cooperation. In: Proceeding of 16th IFIP/IEEE Distributed Systems Operations and Management, (2005)

17. Zhao, X. and Lin, Z.: Modeling belief, capability and promise for cognitive agents- A Modal Logic Approach, ICNC, pp. 825-834 (2005)

18. Vasconcelos, W.W., Kollingbaum, M.J., Norman, T.J. : Normative conflict resolution in multi-agent systems. Journal of Autonomous Agents and Multi-Agent Systems, vol. 19(2), pp. 124-152 (2009)

19. Giddens, A.: The constitution of society: Outline of the theory of structuration. John Wiley & Sons (2013)

20. Herzig, A., Lorini, E., Hubner, J.F., Ben-Naim, J., Castelfranchi, C., Demolombe, R., Longin, D., Vercouter, L. : Prolegomena for a logic of trust and reputation. In: NORMAS’08, pp. 143–157(2008)

21. Carbo, J., Molina, J. M., Davila, J.: Trust management through fuzzy reputation. International Journal of Cooperative Information Systems, vol. 12(1), 135-155 (2003)

22. Yu, B., Singh, M. P. A: social mechanism of reputation management in electronic communities. Cooperative Information Agents IV-The Future of Information Agents in Cyberspace, 154-165 (2000)

23. Josang, A., Ismail, R., Boyd, C. A.: survey of trust and reputation systems for online service provision. Decision support systems, vol. 43(2), 618–644 (2007)