# An experience feedback process for learning from collaboration experiences

Sofia Melendez<sup>b</sup>, Xingyu Sima<sup>a</sup>, Thierry Coudert<sup>a,*</sup>, Laurent Geneste<sup>a</sup>, Aymeric de Valroger<sup>b</sup>

<footnote><sup>a</sup>Laboratoire Génie de Production, University of Toulouse/INP-ENIT, Tarbes, France</footnote>

<footnote><sup>b</sup>Axsens-bte, Toulouse, France</footnote>

<footnote>*Thierry Coudert</footnote>

<footnote>Email addresses: dsmelendez@gmail.com (Sofia Melendez), xingyu.sima@axsens.com (Xingyu Sima), thierry.coudert@enit.fr (Thierry Coudert), laurent.geneste@enit.fr (Laurent Geneste), aymeric.devalroger@axsens.com (Aymeric de Valroger)</footnote>
# An experience feedback process for learning from collaboration experiences

## Abstract

In the fast evolving context of industry 4.0, companies or organizations must continuously evolve and improve. To maintain their business efficiency, they join industrial networks in which they have to collaborate. In more recent context of industry 5.0, humans are placed at the heart of the industrial processes by developing human-centric and society-centric approach. In such a context, human collaboration experiences are numerous and constitute meaningful pieces of knowledge which can be reused within a human-centric approach. This needs to i) formalize and capitalize collaboration experiences, ii) enable actors to assess collaboration and, iii) develop reusing mechanisms. This article proposes an experience feedback approach where collaboration experiences are formalized and directly assessed by actors who have collaborated. Assessment grids are proposed to guide the human evaluations. From the individual evaluations, aggregation mechanisms are proposed to compute the collaboration performance of organizations. Finally, a reusing mechanism allows to learn from prior experiences, identifying the best organizations to make collaborating for a new industrial process.

**Keywords:** Collaboration, Learning, Experience feedback, Industrial processes, Performance assessment

## 1. Introduction

The current industrial context is pushing companies to be more and more networked which leads them to consider collaboration as a key enabler of productivity, competitiveness and performance, especially in light of the new opportunities offered by industry 4.0 [1, 2, 3]. In order to grow, organizations need to jointly mobilize and connect assets and resources to achieve self-sufficiency and sustainable development using collaboration [4]. In [5], the authors suggest that collaboration is about selecting actors and resources, building a network in order to define ways to cope with strategic and operational goals. More recent context of industry 5.0 [6, 7, 8] prone a human-centric approach where actors are put at the heart of the industrial processes. This article is proposing a human-centric approach based on experience feedback which allows to learn from prior human collaborations and reuse them in current situations.

Experience feedback is a powerful human-centric method where human feedbacks are capitalized, analyzed and assessed by actors in order to be reused in similar contexts [9]. Therefore, in this article, an experience feedback method is proposed to elaborate new collaborations from prior ones. The actors who have collaborated are put at the heart of the method to formalize and assess the collaboration experiences. Mechanisms which allows to assess the collaboration of several organizations on the basis of human assessments are proposed. Finally, a method which allows to select the best organizations to make collaborating
in a new industrial process is described.

This research has been performed in collaboration with Axsens-bte, a consulting company specialized in supply chains, industrial methods, logistics and transports, industry 4.0 and change management, quality and continuous improvement. Axsens-bte shared their industrial experiences in order to feed the contribution. The research was conducted following three steps by collaborating with the actors of the Research and Development service: i) requirements and scientific gaps definition, ii) development of solutions, iii) test and experimentation of the proposals in industrial context. The requirements were identified by the Axsens-bte company for which the elaboration of new efficient collaborations between organizations (and between the actors) is a strong issue. The questions are: for a new industrial process, what are the best organizations to select for its realization? How to enable human-centric assessment of collaboration? How to capitalize and reuse collaboration experiences? All the proposals are built and tested in collaboration with the Axsens-bte company.

In section 2, after presenting the background of this research on collaboration in the context of industry 4.0 and 5.0, the literature review on collaboration modeling and assessment are presented before defining the scientific contributions of the article.

## 2. Background and literature review

### 2.1. Background

Collaboration between organizations plays a key role in the achievement of successful results in industrial processes by developing interrelations and strategic collaborations [10, 11, 12]. In order to manage these relationships, contract governance, mutual commitments and management procedures are drivers in shaping the collaboration strategy [13, 14]. From a collaboration perspective, a networked organization can be seen as a network of heterogeneous individuals who have their own characteristics and work for these organizations [15, 11]. The collaboration of organizations leads to the collaboration of their actors, creating ties that cross organizational boundaries. In this context, [16, 17] consider Collaborative Networks (CN) as a major enabler.

In the context of industry 4.0, industrial performance has to be improved in order to achieve productivity, flexibility, competitiveness, sustainability, customization and innovativeness in manufacturing [3]. These goals need collaboration between organizations [18]. In the recent context of industry 5.0 [6, 7, 8], three pillars are defined: human-centric, sustainability and resilience. The human-centric approach puts actors at the heart of the production process, shifting from technology-driven progress to a thoroughly human-centric
*and society-centric approach* [8]. Industry 4.0 needs efficient collaborations between actors and organizations in order to improve business processes performances, while industry 5.0 puts the humans at the center of these collaborations. Therefore, the need of efficient approaches which allow actors and organizations to better collaborate is linked to both contexts.

As vector for innovativeness, efficient collaboration between organizations is necessary for knowledge creation and transfer [19, 20]. Knowledge transfer can be obtained by Experience Feedback (EF) [21]. EF is a powerful approach for knowledge management. It allows to capitalize and reuse information and knowledge from the analysis of positive or negative events [22]. Its effectiveness to facilitate decision making in many sectors has been demonstrated (see for instance [23]). In previous works, EF has been successfully applied to problem-solving [9, 24], agile processes [25] and risk engineering [26]. EF is a human-centric approach where the expertise detained and used by individuals to solve problems is capitalized for learning and reusing. Therefore, in industry 4.0/5.0 contexts, approaches to learn from past efficient collaboration experiences can be used by organizations in a meaningful way.

First, it is necessary to have models of collaboration in order to understand, analyze, improve and implement collaborations between actors and between organizations. Second, these models have to integrate collaboration assessment methods to evaluate the efficiency and effectiveness of collaboration. Based on a human-centric approach, the collaboration assessment has to be done directly by the individuals who collaborate, with the aim to compute the collaboration performance of organizations. Third, an experience feedback approach is necessary in order to learn from prior collaborations.

The following literature review is decomposed into collaboration modeling (section 2.2) and collaboration assessment (section 2.3) before presenting the contributions. The references are mainly found in “Science Direct”, “Web of Science”, “Springer” and “IEEE Xplore” databases. The keywords used are “Collaboration”, “Processes”, “Collaborative engineering”, “Experiences”, “Experience feedback” and/or “Industry 4.0”. The articles describing specific collaborations between companies or academics and which contained some of these keywords have been ignored. Moreover, our research experience in the fields of collaboration, problem solving methodologies, knowledge management and experience feedback has been useful to complete this literature review.

## 2.2. *Collaboration modeling*

According to [27], collaboration arises over time as organizations interact formally and informally in order to negotiate, develop commitments, and execute these commitments. Collaboration has several definitions
but one of the most generally accepted is that it is the way that two or more entities work together toward
a shared goal [28] and irrespective of geographical separation [10].

In the literature, Collaborative Networks (or social networks) are frequently used to model and assess collaboration [29, 30, 31, 32, 33]. In such networks, nodes can be individuals or collective structures (teams, departments, divisions, subsidiaries, organizations, etc.). Collaboration in networked organizations can be seen as a network of heterogeneous individuals who have their own characteristics and work for these organizations [15, 11]. According to [10], considering organizations as networks is a useful method for promoting organizational flexibility and adaptability, particularly in the quality and sharing of information. Using graph representations, the vertices represent the actors within networks and the edges indicate the relationships between the actors. Therefore, collaboration between individuals affect collaboration between organizations and both should be taken into consideration simultaneously. In the domain of Collaborative Engineering (CE), based on the collaboration model of [34], a computational support for collaboration is provided by [35] (3C model). Collaborative systems [36, 37] are classified following Communication, Coordination and Cooperation general key concepts. Moreover, from the literature review, twenty other key concepts which are useful to model and assess collaboration have been identified (table 1). This literature review allows to clearly define key concepts which have to be integrated into our own collaboration model but also to assess collaboration. However, to the best of our knowledge, there is no collaboration models dedicated to human-centric experience feedback methods, i.e. which permits to capitalize and learn from prior collaborations.

## 2.3. Collaboration assessment

In order to define an experience feedback process for collaboration, it is necessary to assess the collaboration experiences. Collaboration assessment has been widely studied in the literature. In [28], in order to evaluate the level of collaboration of grant partners, a collaboration measure based on a collaboration scale is provided. [55] proposed a framework for collaboration assessment based on a questionnaire sent to organizations directors and where assessments are aggregated. Collaboration competency is defined by [48] as the ability of the partners to trust, communicate, and coordinate. The assessments are based on questionnaires. Many approaches are based on Collaboration Networks represented by graphs. The collaboration assessment is based on the topology of the collaboration graph. In [15], a multilevel model for inter-organizational collaboration is proposed. Indicators such clustering, efficiency, alliance strength are computed on the basis of the graph topology. In [10], the authors propose a mathematical model to analyze how individuals in an organization work together. Three quantitative intra-organization indicators (team-
<table><caption>Table 1: Collaboration key concepts</caption><thead><tr><th></th><th>Information processing</th><th>Knowledge transfer</th><th>Sharing information</th><th>Working together</th><th>Negotiation skills</th><th>Communication</th><th>Trustworthiness</th><th>Sharing ideas</th><th>Coordination</th><th>Bonds</th><th>Trust</th><th>Satisfaction</th><th>Common vision</th><th>Involvement</th><th>Common goals</th><th>Sharing values</th><th>Sharing resources</th><th>Mutual understanding</th><th>Mutual commitments</th><th>Achieving collective goals</th></tr></thead><tbody><tr><td>[4]</td><td></td><td>×</td><td>×</td><td>×</td><td>×</td><td>×</td><td></td><td></td><td>×</td><td>×</td><td>×</td><td></td><td>×</td><td>×</td><td>×</td><td>×</td><td></td><td>×</td><td>×</td><td>×</td></tr><tr><td>[20]</td><td></td><td>×</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[38]</td><td></td><td></td><td>×</td><td></td><td></td><td></td><td></td><td></td><td></td><td>×</td><td></td><td></td><td></td><td></td><td>×</td><td>×</td><td>×</td><td></td><td></td><td></td><td></td></tr><tr><td>[39]</td><td>×</td><td></td><td>×</td><td></td><td></td><td></td><td></td><td></td><td></td><td>×</td><td>×</td><td></td><td>×</td><td></td><td>×</td><td></td><td>×</td><td></td><td></td><td></td><td></td></tr><tr><td>[37]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>×</td></tr><tr><td>[40]</td><td></td><td></td><td>×</td><td>×</td><td></td><td></td><td></td><td>×</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>×</td><td></td><td></td><td>×</td></tr><tr><td>[41]</td><td></td><td></td><td></td><td>×</td><td></td><td></td><td></td><td></td><td>×</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>×</td></tr><tr><td>[42]</td><td></td><td></td><td></td><td></td><td></td><td>×</td><td></td><td></td><td>×</td><td></td><td>×</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>×</td><td></td><td></td></tr><tr><td>[43]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>×</td><td>×</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[19]</td><td></td><td></td><td></td><td>×</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>×</td><td>×</td><td></td><td></td><td>×</td><td></td><td></td></tr><tr><td>[44]</td><td></td><td></td><td></td><td>×</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>×</td><td></td><td></td><td>×</td><td>×</td><td></td><td></td><td>×</td><td></td></tr><tr><td>[45]</td><td></td><td>×</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>×</td><td></td><td></td><td></td><td></td><td></td><td>×</td><td></td></tr><tr><td>[46]</td><td>×</td><td></td><td></td><td></td><td>×</td><td>×</td><td>×</td><td>×</td><td>×</td><td>×</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[47]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>×</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>×</td><td></td><td></td></tr><tr><td>[48]</td><td></td><td></td><td></td><td></td><td></td><td>×</td><td></td><td>×</td><td>×</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[49]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>×</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>×</td><td></td><td></td></tr><tr><td>[50]</td><td></td><td></td><td></td><td>×</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>×</td><td></td></tr><tr><td>[51]</td><td></td><td></td><td></td><td>×</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>×</td><td></td><td></td><td></td><td></td><td></td><td></td><td>×</td><td></td></tr><tr><td>[52]</td><td>×</td><td>×</td><td>×</td><td></td><td>×</td><td></td><td></td><td>×</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>×</td><td>×</td><td>×</td><td>×</td></tr><tr><td>[53]</td><td>×</td><td>×</td><td>×</td><td>×</td><td>×</td><td></td><td>×</td><td></td><td></td><td></td><td></td><td>×</td><td></td><td></td><td></td><td></td><td></td><td>×</td><td>×</td><td></td><td></td></tr><tr><td>[18]</td><td>×</td><td></td><td>×</td><td></td><td></td><td></td><td>×</td><td></td><td></td><td>×</td><td>×</td><td>×</td><td>×</td><td></td><td></td><td></td><td></td><td>×</td><td>×</td><td>×</td><td></td></tr><tr><td>[54]</td><td>×</td><td>×</td><td>×</td><td>×</td><td></td><td></td><td>×</td><td>×</td><td></td><td>×</td><td>×</td><td></td><td>×</td><td>×</td><td>×</td><td></td><td></td><td>×</td><td>×</td><td>×</td><td></td></tr></tbody></table>

work scale, decision-making scale and coordination scale) are based on the network structure. In [31, 56], a collaborative network which links a social network with an activity network is proposed in order to compute topological indicators. These topological approaches are powerful to assess collaboration by characterizing the connections between individuals, clusters of individuals and the activities they perform. In [53], the collaboration performance measurement is based on the conceptual and technical interoperability of the different pairwise collaborations. Collaboration maturity models are powerful tools for collaboration assessment [57, 58, 59, 60, 52]. They are structured in four hierarchical maturity levels, from an unstructured collaboration to a formal, structured and documented collaboration.

All these approaches allow to model, assess and continuously improve collaboration within industrial processes. However, they are not based on human-centric factual feedbacks. This means that the assessments are not performed directly by the individuals who collaborate. For an organization, the evaluation of
collaboration should be based on the individual collaborations of actors who perform the activities and
how they perceive every collaboration. The collaborations should be capitalized as experiences and an
experience feedback process should allow to evaluate collaboration performances by a learning mechanism
from the different experiences.

## 2.4. *Contribution*

The main contribution developed in this article is a human-centric experience feedback process which enables to: i) capitalize collaboration experiences, ii) to assess collaboration and, iii) to learn from prior collaboration experiences and help to build new collaborations. Collaboration is observed according to two complementary points of view: collaboration between actors and collaboration between organizations. First, a conceptual collaboration model allowing to capitalize experiences of collaboration is proposed. Assessment grids are defined in order, for the actors, to express without ambiguity how they perceived the collaboration of other actors. Second, from the assessments of these elementary collaborations, an aggregation mechanism is defined in order to assess the collaboration at the organizations level. Third, a learning mechanism is defined in order to show to decision makers what are the best organizations (and their actors) to assign to a new industrial process in order to continuously improve performances.

The next sections are organized as follows. In section 3, the overall collaboration Experience feedback process is presented. In section 4, the conceptual model of a collaboration experience is described. In section 5, the method which allows to assess the collaboration by actors who collaborated is presented. Aggregation mechanisms which allow to define collaboration evaluations for the different organizations are described. In section 6, the method which allows to compute the performance of the organizations from the evaluations of the commitments satisfaction is detailed. In section 7, the experience based evaluation method is presented. The application of these models and methods is illustrated in section 8 where twenty past experiences are used in order to assess five organizations and build a Pareto-front where three of them are selected. The conclusion is given in section 9.

# 3. Collaboration Experience feedback process

The proposed experience feedback process (firstly presented in [61]) allows to capitalize standardized experiences of collaboration performed during industrial processes and to reuse them to help to define future collaborations.

The capitalization of an experience is done on the basis of the collaborations that take place during the activities of a process as shown in Figure 1. Future experience reuse needs standardization as shown in
[9, 24]. Thus, in the proposed approach, a Knowledge Base (KB) gathers elements that allow to build an experience and assess its performances before its storage into an Experience Base (EB). It contains:

* a model of an experience of collaboration: the experience model is a template that defines the structure of an experience with the entities and their relations. It should be used every time a new experience has to be defined and capitalized;

* a taxonomy of concepts: the concepts are stored into a taxonomy. They are hierarchically organized and they are used to define the values of some experience attributes in order to ensure the standardization of the collaboration experiences but also to facilitate their reuse.

For every execution of an activity of the industrial process where collaborations take place, the corresponding information is memorized into the experience using the template. At the end of the industrial process, the experience is assessed (using assessment grids), verified and stored into the Experience Base. Every time a new process has to be realized, similar experiences have to be identified and retrieved from the experience base in order to help decision makers to build new efficient collaborations.

<figure><img src="image_1.png" /><figcaption>Figure 1: Overall experience feedback process to help building new collaborations in industrial processes</figcaption></figure>

The collaboration experience model is presented in the section 4 and the method to assess experiences and to learn from them is presented in the section 5.

## 4. Collaboration experience modeling

In this section, the knowledge which is used to capitalize a collaboration experience is defined. It is stored into the knowledge base and it gathers the necessary material for an experience characterization: i) a
UML diagram that permits to define the entities and their relationships (section 4.1) and ii) the taxonomy of concepts used to guide the definition of some attributes values (section 4.2).

## 4.1. *Structure of an experience*

A collaboration experience has to be modeled during the execution of an industrial process as the different actors and organizations are collaborating. The UML diagram of figure 2 represents the different entities with their relationships. Some activities of the industrial process can be performed by several organizations in a collaborative way and some defined goals have to be reached. The goals are represented by commitments and they must satisfy requirements. Contracts are used in order to represent the participation of the different organizations. Actors are employed by organizations and participate to activities. Within some classes, for the attributes written in bold, the values will be chosen in the taxonomy presented in section 4.2.

<figure><img src="image_2.png" /><figcaption>Figure 2: UML diagram of a collaboration experience</figcaption></figure>

## 4.2. *Taxonomy of concepts for a collaboration experience*

The proposed taxonomy gives the possible values to be used for the different types (commitment, contract, organization, department, activity, requirement and role). An illustration of a partial collaboration experience taxonomy is represented in figure 3. Only general concepts are represented and each of them can be specialized. The method used to build the taxonomy is based on the “Empirical-to-conceptual” method proposed in [62]: every subset or object used by the organization is identified and grouped following its characteristics in order to formalize concepts. In order to help this knowledge formalization, the European
taxonomy CPA (Classification of Products by Activity) is used as a guideline to build such a taxonomy,
especially for the types of activities, products and services (see [63] for an application of this taxonomy).

<figure><img src="image_3.png" /><figcaption>Figure 3: Taxonomy of concepts for the standardization of collaboration experiences</figcaption></figure>

In the next section, the method used to assess a collaboration experience according to the collaboration
performances is presented (section 5).

**5. Collaboration experience assessment**

5.1. *Collaboration evaluation input*

In [36], three criteria are used to evaluate the collaboration: Coordination, Cooperation and Commu-
nication (3C model). Based on these criteria, three assessment grids (tables 2, 3 and 4) are proposed by
analyzing the twenty key concepts of table 1 and by defining the different assertions. They can be used
directly to assess the collaboration or can be adapted by the organizations.

The method proposed to assess the collaboration is based on individual evaluations performed by actors
who assess the other actors they have worked with during the different activities of the industrial process.
Every actor evaluates the other actors according to the coordination, cooperation and communication cri-
teria. Every actor has to evaluate, for every specific criterion, how she/he perceived the quality of the
collaboration provided by each other actor according to the assertions given in tables 2, 3 and 4. Consider-
ing the number of activities and actors, which can be large, an actor can give the same global evaluation to
all the other actors. The guidelines provided by the grids of tables 2, 3 and 4 and the confidence between
actors (every actor evaluates every actor) allow to guarantee the consistence of the evaluations.

Let $I_C$ be the collaboration evaluation matrix which contains the evaluations given by each actor of the
process to the other actors according to the criterion $C$ (e.g. table 5.a). Let $|a|$ be the number of actors
<table><caption>Table 2: Coordination assessment grid</caption><thead><tr><th colspan="2">PLANNING</th></tr></thead><tbody><tr><td>Commitments</td><td><ul><li>A contract or agreement exists and it is accepted by all parties</li><li>A clear mission statement/charter allows to guide the activities of the teams</li><li>The team keeps commitments and completes assignments on time</li></ul></td></tr><tr><td>Schedule</td><td><ul><li>There was a team planning</li><li>Team members have the means to see the progress toward important objectives</li><li>The team met important deadlines on time</li></ul></td></tr><tr><td>Resources</td><td><ul><li>There was someone in the team who coordinated the tasks</li><li>The team managed time effectively</li><li>There was a Knowledge transfer</li><li>The labor's division was fair, and team members presented the completed work</li></ul></td></tr><tr><td>Control</td><td><ul><li>We try to have set procedures or protocols to ensure that things are orderly and run smoothly (e.g., minimize interruptions; everyone gets the opportunity to have their say)</li><li>Team plans and schedules were effective</li></ul></td></tr><tr><th colspan="2">ORGANIZATION</th></tr><tr><td>Roles</td><td><ul><li>The team roles were distinct without ambiguity</li><li>The team members did their part of the work</li></ul></td></tr><tr><td>Responsibility</td><td><ul><li>The team responsibilities were distinct without ambiguity</li><li>Each team member had at least one backup who was able to perform their role if necessary</li></ul></td></tr><tr><td>Competences</td><td><ul><li>The team has the right members in order to be successful</li><li>Tasks are assigned to those best qualified to undertake them, whether internal or external</li></ul></td></tr></tbody></table>

<table><caption>Table 3: Cooperation assessment grid</caption><thead><tr><th colspan="2">TEAM</th></tr></thead><tbody><tr><td>Commitment</td><td><ul><li>When the situation changed, the whole team adapted quickly to new circumstances.</li><li>Team members were willing to perform other team members work tasks necessary for this</li><li>The team was willing to make changes to the work approach along the process</li><li>Each member has the ability to make compromises</li><li>Everyone on the team participates at an acceptable level</li><li>There were not missed deadlines, work unprepared or absent for team meetings</li></ul></td></tr><tr><td>Team spirit</td><td><ul><li>There was a 'We are together' attitude</li><li>Team members were willing to adjust strategies because someone else needed assistance</li><li>The team members positively tuned to teamwork along the way in the work process</li><li>Team members support each other</li><li>When a team member needed help, variable team members helped.</li><li>This team works well together</li></ul></td></tr><tr><th colspan="2">ENVIRONMENT</th></tr><tr><td>Training</td><td><ul><li>Each member of the team correctly assumes their role and responsibilities</li><li>There were real attempts to share information</li><li>The team have a common understanding of its goals</li><li>This team works well with other teams or departments in the organization</li></ul></td></tr><tr><td>Trust</td><td><ul><li>Fosters a climate of trust</li><li>The actions that affect teammates were taken without their input</li><li>Team members fully trust the other team members and don't closely monitor others</li><li>Our team feels that we are all in it together and shares responsibilities</li></ul></td></tr><tr><td>Leadership</td><td><ul><li>The leader promoted teamwork</li></ul></td></tr></tbody></table>

involved in the industrial process. The evaluations received/given by the actors are values between 0 and 1. The value 0 corresponds to the worst quality of collaboration, and the value 1 to the best quality. The row j of $I_C$ contains the evaluations given by the actor $a_j$ to the other actors who collaborated with her/him according to the collaboration criteria $C$ ($C \in \{Coordination, Cooperation, Communication\}$). The method which allows to compute the collaboration performances of the experience is proposed in section 5.2.
<table><caption>Table 4: Communication assessment grid</caption><thead><tr><th colspan="2">CONTEXT</th></tr></thead><tbody><tr><td>Channel</td><td><ul><li>There were top-management communication channels</li><li>The communication channels facilitate a prompt communication of changes to team members</li></ul></td></tr><tr><td>Guidelines</td><td><ul><li>The plan was communicated to all team members</li><li>Team members are encouraged to express different points of view</li><li>There was a common language</li><li>The team leader has a process for sharing information with all team members</li></ul></td></tr><tr><th colspan="2">MESSAGE</th></tr><tr><td>Synergy</td><td><ul><li>The team members shared all the information necessary to execute the activities successfully</li><li>People keep each other informed</li><li>Team members were afraid or did not like to ask others for help</li><li>There was effective communication between team members</li><li>There was an environment for open dialogue</li></ul></td></tr><tr><td>Clearness</td><td><ul><li>The team leader always reassured that his/her instructions were understood</li><li>When team members talk, we understand each other</li><li>There was a clear communication route between the partners (open and frequent)</li></ul></td></tr><tr><td>Correctness</td><td><ul><li>The team leader's instructions were always explicit and directed</li><li>Team members express disagreements constructively</li><li>Information passed between team members is accurate</li></ul></td></tr></tbody></table>

## 5.2. Collaboration assessment

Let $Ma^A$ be the connectivity matrix between the actors for the activity $A$. It indicates which actor $a_i$ really collaborated with actor $a_j$ when performing the activity $A$. The value 1 means that the actors have collaborated during the execution of the activity $A$, the value 0 means that they have not collaborated. The value 0 is given to the diagonal of the connectivity matrix.

$EV_C^A$, the assessment matrix between actors for the activity $A$ and the criterion $C$ (tables 5.d and 5.e), is computed using the Hadamard product ([64]) (equation 1).

$$EV_C^A = I_C \circ Ma^A \qquad (1)$$

Let $Oa$ be the connection matrix between actors and organizations (table 5.f). The value 1 in a cell of $Oa$ indicates that the actor $a_i$ is employed by the organization $O_j$. The value 0 indicates that the actor is not employed by this organization. The matrix $EV_C^A$ is multiplied by the matrix $Oa$ in order to obtain $EV_a O_C^A$, the assessment matrix between actors and organizations (equation 2). This matrix contains the evaluations that the actors received from each organization for the activity $A$ and the criterion $C$ (tables 5.g, 5.h).

$$EV_a O_C^A = EV_C^A \times Oa \qquad (2)$$

Let $EVOO_C^A$ be the assessment matrix between organizations for the activity $A$ and the criterion $C$ (tables 5.j, 5.k). It contains the collaboration evaluations given by the organizations to the other organizations.
It also contains the evaluations given by the actors of an organization to its own actors (self-evaluation). The assessment matrix between organizations $EVOO_C^A$ is obtained by multiplying $Oa^T$ (i.e. the matrix $Oa$ transposed) by the matrix $EVaO_C^A$ (equation 3).

$$EVOO_C^A = Oa^T \times EVaO_C^A \qquad (3)$$

Let $EVOO_C$ be the assessment matrix according to the criterion $C$ for the $n$ activities of the process (table 5.1). It is the sum of the matrices $EVOO_C^A$ (equation 4).

$$EVOO_C = \sum_{A \in \{A_1, A_2, \dots, A_n\}} EVOO_C^A \qquad (4)$$

In order to calculate $EV_C^{O_i}$ (table 5.n), the evaluation of the organization $O_i$ according to the criterion $C$, the sum of the values of the row corresponding to the organization $O_i$ in $EVOO_C$ is done (equation 5). $EVOO_C(i, j)$ is the evaluation of the organization $O_i$ by the organization $O_j$ and $|O|$ is the number of organizations.

$$EV_C^{O_i} = \sum_{j=1}^{|O|} EVOO_C(i, j) \qquad (5)$$

Finally, in equation 6, the indicator $EV^{O_i}$, i.e. the evaluation of the collaboration for the organization $O_i$ according to the three criteria, is obtained by aggregating the values of these criteria using a GOWA operator (Generalized Ordered Weighted Averaging - see [65]). The parameter $\lambda$ allows to control the GOWA operator. When $\lambda \to +\infty$, the GOWA operator returns the maximum and when $\lambda \to -\infty$, the operator returns the minimum value.

$$EV^{O_i} = \left( \sum_C \frac{1}{3} \times (EV_C^{O_i})^\lambda \right)^{1/\lambda} \qquad (6)$$

An illustration of a collaboration experience assessment is given in table 5. The process gathers two activities $A_1$ and $A_2$ carried out by three actors $a_1, a_2$ and $a_3$. The actor $a_3$ is employed by the organization $O_1$ and the actors $a_1$ and $a_2$ are employed by $O_2$ (table 5.f). For the activity $A_1$, the three actors $a_1, a_2$ and $a_3$ have collaborated (table 5.b). For the activity $A_2$, the actors $a_1$ and $a_2$ have collaborated (table 5.c). The illustration relates only the assessment for the cooperation criterion ("Coop").

The table shows the different matrices and results of the collaboration assessment according to the cooperation criterion. The values in brackets correspond to the maximum possible values. From the evaluations
given by the actors in the input matrix $I_{Coop}$ (table 5.a), the calculations are done. The output relates to the assessment of both organizations $O_1$ and $O_2$ according to the cooperation criterion (table 5.n).

<figure><img src="image_4.png" /><figcaption>Table 5: Illustration of an experience collaboration assessment according to the cooperation criterion</figcaption></figure>

The global evaluations obtained by the organizations are: 85% for $O_1$ and 73% for $O_2$ (table 5.n). The organization $O_2$ is the more penalized one because of the low evaluation given by the actor $a_3$ (she/he is employed by $O_1$) to the actors $a_1$ and $a_2$ (they are employed by $O_2$) in $I_{Coop}$ (third column in table 5.a; values 0.6 and 0.2). In addition, to assess every organization, it is necessary to evaluate the process performances according to the satisfaction of the requirements (next section).

## 6. Process performance assessment

Every organization which has participated to an industrial process has contributed to the satisfaction of some of the requirements. Therefore, the input matrix for performance evaluation is the matrix $I_{RO}$ (table 6.a). It contains the evaluations given by each organization $O_i$ to each requirement $r_j$. A rate between 0 and 1 is given to the requirements. The rate 1 given by an organization $O_i$ to a requirement $r_j$ means that the organization thinks that the requirement is fully satisfied. The rates 0 means either that the organization
did not participate to the fulfillment of this requirement or that the requirement is not satisfied. $|r|$ is the number of requirements and $|C|$ is the number of contracts.

The following matrices are used:

* $M_{Req-Com_k}$ is the column vector (dimension $|r|$) which connects the different requirements with the commitment $Com_k$ (e.g. tables 6.c & 6.f, value = 1 if a requirement is related to the commitment $Com_k$, 0 otherwise).

* $M_{Com_k-C}$ is the row vector (dimension $|C|$) which connects the commitment $Com_k$ with the different contracts (e.g. tables 6.b & 6.e, value = 1 if the commitment $Com_k$ is related to a contract, 0 otherwise).

* $M_{Req-C}^{Com_k}$ is the connection matrix ($|r|$ lines and $|C|$ columns) between requirements and contracts for the commitment $Com_k$: $M_{Req-C}^{Com_k} = M_{Req-Com_k} \times M_{Com_k-C}$ (e.g. tables 6.b & 6.e).

* $M_{C-O}$ is the connection matrix ($|C|$ lines and $|O|$ columns) between contracts and organizations (e.g. tables 6.d, 1 if the contract $C_i$ is related to the organization $O_j$, 0 otherwise).

* $M_{Req-O}^{Com_k}$ is the connection matrix ($|r|$ lines and $|O|$ columns) indicating which organization participated to which requirement satisfaction for the commitment $Com_k$: $M_{Req-O}^{Com_k} = M_{Req-C}^{Com_k} \times M_{C-O}$ (e.g. tables 6.g & 6.l).

To compute $Perf^{O_i}$, the overall performance indicator of the organization $O_i$, the following assessment matrices are computed:

* $A_{Req-O}^{Com_k}$ is the assessment matrix between requirements related to the commitment $Com_k$ and the organizations (e.g. tables 6.h & 6.m). It is obtained using the Hadamard product (operator $\circ$):
  $$A_{Req-O}^{Com_k} = M_{Req-O}^{Com_k} \circ I_{RO}$$

* $A_O^{Com_k}$ is the assessment row vector between the commitment $Com_k$ and the organizations: $A_O^{Com_k} = M_{Req-Com_k}^T \times A_{Req-O}^{Com_k}$ (e.g. tables 6.i & 6.n).

* $A_{C-O}^{Com_k}$ is the assessment matrix between the contracts and the organizations: $A_{C-O}^{Com_k} = M_{Com_k-C}^T \times A_O^{Com_k}$ (e.g. tables 6.k & 6.p).

* $A_{O-O}^{Com_k}$ is the assessment matrix between organizations according to the commitment $Com_k$:
  $$A_{O-O}^{Com_k} = M_{C-O}^T \times A_{C-O}^{Com_k}$$ (e.g. tables 6.q & 6.r).
* Perf is the assessment matrix which contains all the evaluations of the organizations according to the requirements satisfaction (e.g. table 6.s). Perf(i, j) is the global evaluation given by the organization O_j to the organization O_i. It is the sum of the assessment matrices for all the commitments (eq. 7).

$$ Perf = \sum_{k=1}^{|Com|} A_{O-O}^{Com_k} \qquad (7) $$

* Perf<sup>max</sup> is the assessment matrix which contains the maximum possible evaluations for all the organizations considering that all the requirements are fully satisfied (e.g. table 6.t).

* Perf<sup>%</sup> is the relative assessment matrix which contains all the relative evaluations of the organizations such that Perf<sup>%</sup>(i, j) = Perf(i, j)/Perf<sup>max</sup>(i, j) (e.g. table 6.t).

* Perf<sup>O<sub>i</sub></sup> is the evaluation obtained by the organization O<sub>i</sub> (e.g. table 6.t). It is the sum of the self-evaluation with the evaluations given by the other organizations:

$$ Perf^{O_i} = \sum_{j=1}^{|O|} Perf(i, j) \qquad (8) $$

* Perf<sup>O<sub>i</sub></sup> is the relative evaluation obtained by the organization O<sub>i</sub>. Perf<sup>O<sub>i</sub></sup> being the maximum performance of the organization O<sub>i</sub>, Perf<sup>O<sub>i</sub></sup> is given by:

$$ Perf_{\%}^{O_i} = Perf^{O_i} / Perf_{max}^{O_i} \qquad (9) $$

In the illustration of performance assessment represented in table 6, three organizations (O<sub>1</sub>, O<sub>2</sub>, O<sub>3</sub>) are involved into the satisfaction of four requirements (r1.1, r1.2, r2.1 and r2.2) with two commitments (Com<sub>1</sub> and Com<sub>2</sub>) and two contracts (C1 and C2). The maximum possible values are given between parenthesis. The input matrix I<sub>RO</sub> (table 6.a) contains the evaluations given by each organization about their perception of the requirements satisfaction.

The results (table 6.t) show that the organizations O<sub>1</sub> and O<sub>2</sub> obtain a lower performance evaluation (resp. 90% and 95 %) because of the lower evaluation given by the organization O<sub>1</sub> to the satisfaction of the requirements r1.1 and r1.2 of the commitment Com<sub>1</sub> (value 0.8 in the first column of table 6.a). The organization O<sub>3</sub> which is only involved into the satisfaction of the requirements r2.1 and r2.2 received only good evaluations (value 1 in the last column of table 6.a). Its overall evaluation is then of 100%.
<table><caption>Table 6: Illustration of performance evaluation</caption><thead><tr><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th></tr></thead><tbody><tr><td rowspan="4">I<sub>RO</sub></td><td>r1.1</td><td>O<sub>1</sub></td><td>O<sub>2</sub></td><td>O<sub>3</sub></td><td>M<sub>Com1-C</sub></td><td>C<sub>1</sub></td><td>C<sub>2</sub></td><td>M<sub>Req-Com1</sub></td><td>Com<sub>1</sub></td><td>r1.1</td><td>1</td><td></td><td></td><td></td><td></td></tr><tr><td>r1.2</td><td>0.8 (1)</td><td>1 (1)</td><td>0 (1)</td><td>Com<sub>1</sub></td><td>1</td><td>0</td><td></td><td></td><td>r1.2</td><td>1</td><td></td><td></td><td></td><td></td></tr><tr><td>r2.1</td><td>0.8 (1)</td><td>1 (1)</td><td>0 (1)</td><td>b)</td><td></td><td></td><td></td><td></td><td>r2.1</td><td>0</td><td></td><td></td><td></td><td></td></tr><tr><td>r2.2</td><td>1 (1)</td><td>1 (1)</td><td>1 (1)</td><td></td><td></td><td></td><td></td><td></td><td>r2.2</td><td>0</td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="2">M<sub>C-O</sub></td><td>C<sub>1</sub></td><td>O<sub>1</sub></td><td>O<sub>2</sub></td><td>O<sub>3</sub></td><td>M<sub>Com2-C</sub></td><td>C<sub>1</sub></td><td>C<sub>2</sub></td><td>M<sub>Req-Com2</sub></td><td>Com<sub>2</sub></td><td>r1.1</td><td>0</td><td></td><td></td><td></td><td></td></tr><tr><td>C<sub>2</sub></td><td>1</td><td>1</td><td>0</td><td>Com<sub>2</sub></td><td>0</td><td>1</td><td></td><td></td><td>r1.2</td><td>0</td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="4">M<sub>Com1</sub></td><td>r1.1</td><td>O<sub>1</sub></td><td>O<sub>2</sub></td><td>O<sub>3</sub></td><td>A<sub>Com1</sub></td><td>O<sub>1</sub></td><td>O<sub>2</sub></td><td>O<sub>3</sub></td><td></td><td>r1.1</td><td>0.8 (1)</td><td>1 (1)</td><td>0 (0)</td><td></td><td></td></tr><tr><td>r1.2</td><td>1</td><td>1</td><td>0</td><td>h)</td><td>0.8 (1)</td><td>1 (1)</td><td>0 (0)</td><td></td><td>r1.2</td><td>0.8 (1)</td><td>1 (1)</td><td>0 (0)</td><td></td><td></td></tr><tr><td>r2.1</td><td>0</td><td>0</td><td>0</td><td></td><td>0 (0)</td><td>0 (0)</td><td>0 (0)</td><td></td><td>r2.1</td><td>0 (0)</td><td>0 (0)</td><td>0 (0)</td><td></td><td></td></tr><tr><td>r2.2</td><td>0</td><td>0</td><td>0</td><td></td><td>0 (0)</td><td>0 (0)</td><td>0 (0)</td><td></td><td>r2.2</td><td>0 (0)</td><td>0 (0)</td><td>0 (0)</td><td></td><td></td></tr><tr><td rowspan="2">A<sub>O</sub><sup>Com1</sup></td><td></td><td>O<sub>1</sub></td><td>O<sub>2</sub></td><td>O<sub>3</sub></td><td>M<sub>Com1-C</sub></td><td>Com<sub>1</sub></td><td>A<sub>C-O</sub><sup>Com1</sup></td><td>O<sub>1</sub></td><td>O<sub>2</sub></td><td>O<sub>3</sub></td><td>j)</td><td>C<sub>1</sub></td><td>1</td><td></td><td></td><td></td></tr><tr><td></td><td>1.6 (2)</td><td>2 (2)</td><td>0 (0)</td><td>C<sub>2</sub></td><td>0</td><td>C<sub>2</sub></td><td>1.6 (2)</td><td>2 (2)</td><td>0 (0)</td><td></td><td>0 (0)</td><td>0 (0)</td><td>0 (0)</td><td></td><td></td></tr><tr><td rowspan="4">M<sub>Req-O</sub><sup>Com2</sup></td><td>r1.1</td><td>O<sub>1</sub></td><td>O<sub>2</sub></td><td>O<sub>3</sub></td><td>A<sub>Com2</sub></td><td>O<sub>1</sub></td><td>O<sub>2</sub></td><td>O<sub>3</sub></td><td></td><td>r1.1</td><td>0 (0)</td><td>0 (0)</td><td>0 (0)</td><td></td><td></td><td></td></tr><tr><td>r1.2</td><td>0</td><td>0</td><td>0</td><td>m)</td><td>0 (0)</td><td>0 (0)</td><td>0 (0)</td><td></td><td>r1.2</td><td>0 (0)</td><td>0 (0)</td><td>0 (0)</td><td></td><td></td><td></td></tr><tr><td>r2.1</td><td>0</td><td>0</td><td>0</td><td></td><td>0 (0)</td><td>1 (1)</td><td>1 (1)</td><td></td><td>r2.1</td><td>0 (0)</td><td>1 (1)</td><td>1 (1)</td><td></td><td></td><td></td></tr><tr><td>r2.2</td><td>0</td><td>1</td><td>1</td><td></td><td>0 (0)</td><td>1 (1)</td><td>1 (1)</td><td></td><td>r2.2</td><td>0 (0)</td><td>1 (1)</td><td>1 (1)</td><td></td><td></td><td></td></tr><tr><td rowspan="2">A<sub>O</sub><sup>Com2</sup></td><td></td><td>O<sub>1</sub></td><td>O<sub>2</sub></td><td>O<sub>3</sub></td><td>M<sub>Com2-C</sub></td><td>Com<sub>2</sub></td><td>A<sub>C-O</sub><sup>Com2</sup></td><td>O<sub>1</sub></td><td>O<sub>2</sub></td><td>O<sub>3</sub></td><td>o)</td><td>C<sub>1</sub></td><td>0</td><td></td><td></td><td></td><td></td></tr><tr><td></td><td>0 (0)</td><td>2 (2)</td><td>2 (2)</td><td>C<sub>2</sub></td><td>1</td><td>C<sub>2</sub></td><td>0 (0)</td><td>2 (2)</td><td>2 (2)</td><td></td><td>0 (0)</td><td>2 (2)</td><td>2 (2)</td><td></td><td></td></tr><tr><td rowspan="3">A<sub>O-O</sub><sup>Com1</sup></td><td></td><td>O<sub>1</sub></td><td>O<sub>2</sub></td><td>O<sub>3</sub></td><td>A<sub>O-O</sub><sup>Com2</sup></td><td>O<sub>1</sub></td><td>O<sub>2</sub></td><td>O<sub>3</sub></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td>1.6 (2)</td><td>2 (2)</td><td>0 (0)</td><td>O<sub>1</sub></td><td>0 (0)</td><td>0 (0)</td><td>0 (0)</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td>1.6 (2)</td><td>2 (2)</td><td>0 (0)</td><td>r)</td><td>0 (0)</td><td>2 (2)</td><td>2 (2)</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="3">s)</td><td></td><td>O<sub>2</sub></td><td>O<sub>3</sub></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td>1.6 (2)</td><td>4 (4)</td><td>2 (2)</td><td></td><td>0 (0)</td><td>2 (2)</td><td>2 (2)</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td>0 (0)</td><td>2 (2)</td><td>2 (2)</td><td></td><td>0 (0)</td><td>2 (2)</td><td>2 (2)</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="3">Perf</td><td></td><td>O<sub>1</sub></td><td>O<sub>2</sub></td><td>O<sub>3</sub></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td>1.6 (2)</td><td>2 (2)</td><td>0 (0)</td><td></td><td>Perf<sup>O<sub>i</sub></sup></td><td>3.6</td><td>4</td><td>90 %</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td>1.6 (2)</td><td>4 (4)</td><td>2 (2)</td><td></td><td>Perf<sup>O<sub>i</sub></sup><sub>max</sub></td><td>7.6</td><td>8</td><td>95 %</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td></td><td>0 (0)</td><td>2 (2)</td><td>2 (2)</td><td></td><td>Perf<sup>O<sub>i</sub></sup></td><td>4</td><td>4</td><td>100 %</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr></tbody></table>

## 7. Methodology for the evaluation of organizations

Based on past collaboration experiences, it is possible to evaluate several organizations according to their past performances (collaboration performance and process performance). Therefore, for a new process to carry out, the set of new commitments allows to retrieve past similar collaboration experiences from the experience base. It is then necessary to evaluate the similarity between the current experience and the
past experiences. For this purpose, the set of current commitments and the commitments corresponding to the past experience has to be compared and a similarity measure have to be computed. The similarity computation method is described in section 7.1. Then, from the set of past similar experiences, their similarity measure according to the current commitments, the evaluations of past collaborations, every organization can be evaluated. The aim is to select organizations according to their ability to collaborate efficiently in the current process. The method which allows this experience based evaluation of organizations is described in section 7.2.

## 7.1. Similar experiences retrieval

Let $Exp_j$ be a past experience stored in the experience base. Let $COM(Exp_j)$ be the set of commitments corresponding to the industrial process which was carried out during the experience $Exp_j$. Let $Current$ be the current experience for which collaborations have to be defined. Let $COM(Current)$ be the set of current commitments. The commitments of $COM(Exp_j)$ and $COM(Current)$ have to be compared in order to compute their similarity. As mentioned in section 4.2, every commitment is related to a concept in the taxonomy. Therefore, these concepts are used in order to compute the similarity using the equation (10) of Wu and Palmer [66]. The similarity between two concepts $c_1$ and $c_2$ (denoted by $Sim(c_1, c_2)$) is given by the equation (10). The expression $depth(c_i)$ is the distance (i.e. the number of arcs) between the concept Universal and the concept $c_i$. The concept $c_{com}$ is the nearest common ancestor of $c_1$ and $c_2$ in the taxonomy.

$$ Sim(c_1, c_2) = \frac{2 * depth(c_{com})}{depth(c_1) + depth(c_2)} \qquad (10) $$

In order to compute the similarity between a past experience $Exp_j$ and the current experience on the basis of the sets $COM(Exp_j)$ and $COM(Current)$, the principle is to compute the similarity of every couple of commitments avoiding to compute many times a same similarity, to aggregate them and to calculate the maximum.

Considering the following example where $COM(Exp_j) = \{c'1, c'2\}$ and $COM(Current) = \{c1, c2, c3\}$, several similarities between couples have to be computed and aggregated using an average operator. Therefore, the similarity between $Exp_j$ and $Current$ is obtained computing the maximum value of the obtained averages (table 7). In this example, the couples of commitments $(c'1, c3)$, $(c'2, c1)$ are the most similar (average similarity of 0.97 is the higher) and then, the value 0.97 is chosen as the similarity evaluation between the past experience $Exp_j$ and the current experience.
<table><caption>Table 7: Similarity computation example between a past experience Exp<sub>j</sub> and the current experience</caption><thead><tr><th>Similarities of couples of commitments</th><th>Average of similarities</th></tr></thead><tbody><tr><td>Sim(c'1, c1), Sim(c'2, c2)</td><td>0.80</td></tr><tr><td>Sim(c'1, c1), Sim(c'2, c3)</td><td>0.75</td></tr><tr><td>Sim(c'1, c2), Sim(c'2, c1)</td><td>0.65</td></tr><tr><td>Sim(c'1, c2), Sim(c'2, c3)</td><td>0.88</td></tr><tr><td>Sim(c'1, c3), Sim(c'2, c1)</td><td><b>0.97</b></td></tr><tr><td>Sim(c'1, c3), Sim(c'2, c2)</td><td>0.72</td></tr><tr><td>Sim(Exp<sub>j</sub>, Current) = Max(Averages of similarities) = 0.97</td><td></td></tr></tbody></table>

In order to retrieve only the more similar past experiences from the experience base, a threshold $\alpha$ ($0 \le \alpha \le 1$) is provided by the user. Only the past experiences which have a similarity greater or equal to $\alpha$ are retrieved and they are added to the set $L_{exp}$. This set is used for the next step, the experience based evaluation of organizations which is described in section 7.2.

## 7.2. Experience based evaluation of organizations

From the set of similar past experiences $L_{exp}$, the evaluation of every organization $O_i$ that has been involved at least one time in a past experience has to be done. Let $L_{exp}^{O_i}$ be the set of experiences involving the organization $O_i$ (such that $L_{exp}^{O_i} \subseteq L_{exp}$). Let $EV_j^{O_i}$ be the collaboration evaluation of the organization $O_i$ in the experience $j$ (computed with equation (6)). Let $Perf_j^{O_i}$ be the performance evaluation of the organization $O_i$ in the experience $j$ (computed with equation (8)).

The overall collaboration evaluation of the organization $O_i$, named $C_{O_i}$ (resp. the overall performance evaluation of the organization $O_i$, named $P_{O_i}$) is given by equation (11). It is the weighted sum of the collaboration evaluations (resp. the weighted sum of the performance evaluations) of the organization $O_i$ for all the past experiences where it was involved. The weights are the relative similarities between past experiences and the current one.

$$C_{O_i} = \frac{\sum_{Exp_j \in L_{Exp}^{O_i}} EV_j^{O_i} \times Sim(Exp_j, Current)}{\sum_{Exp_j \in L_{Exp}^{O_i}} Sim(Exp_j, Current)} \qquad P_{O_i} = \frac{\sum_{Exp_j \in L_{Exp}^{O_i}} Perf_j^{O_i} \times Sim(Exp_j, Current)}{\sum_{Exp_j \in L_{Exp}^{O_i}} Sim(Exp_j, Current)} \qquad (11)$$

For instance, two past experiences have involved four organizations and the results of their assessment is provided in the table 8.

Therefore, every organization $O_i$ is related to the couple $(C_{O_i}, P_{O_i})$ corresponding to its experience based evaluation. In order to identify the best organizations to make collaborating in the current experience in
<table><caption>Table 8: Example of experience based evaluation of four organizations</caption><thead><tr><th>E<sub>j</sub></th><th>O<sub>i</sub></th><th>Sim(E<sub>j</sub>, Current)</th><th>EV<sup>O<sub>i</sub></sup><sub>j</sub></th><th>Perf<sup>O<sub>i</sub></sup><sub>j</sub></th><th>O<sub>i</sub></th><th>C<sub>O<sub>i</sub></sub></th><th>P<sub>O<sub>i</sub></sub></th></tr></thead><tbody><tr><td rowspan="3">E<sub>1</sub></td><td>O<sub>1</sub></td><td>0.8</td><td>1.0</td><td>1.0</td><td>O<sub>1</sub></td><td>0.953</td><td>0.833</td></tr><tr><td>O<sub>3</sub></td><td>0.9</td><td>0.75</td><td>1.0</td><td>O<sub>2</sub></td><td>0.7</td><td>0.8</td></tr><tr><td>O<sub>4</sub></td><td>0.5</td><td>0.65</td><td>1.0</td><td>O<sub>3</sub></td><td>0.85</td><td>0.956</td></tr><tr><td rowspan="3">E<sub>2</sub></td><td>O<sub>1</sub></td><td>0.4</td><td>0.86</td><td>0.5</td><td>O<sub>4</sub></td><td>0.65</td><td>1.0</td></tr><tr><td>O<sub>2</sub></td><td>1.0</td><td>0.7</td><td>0.8</td><td></td><td></td><td></td></tr><tr><td>O<sub>3</sub></td><td>0.7</td><td>0.98</td><td>0.9</td><td></td><td></td><td></td></tr></tbody></table>

order to perform the industrial process and to fulfill the requirements (via the commitments), a Pareto-based method is defined in section 7.3.

### 7.3. Experience based selection of organizations

In order to select the best organizations, a multi-criteria approach is used. Two criteria are taken into account: the overall collaboration and the overall performance which have been computed on the basis of the past similar collaboration experiences. For the user in charge of the selection of organizations and actors who are going to collaborate, it is necessary to take into account simultaneously both criteria. Therefore, an efficient method which allows to ease user interactions is used: the Pareto-front. The principle is to keep on the Pareto-front only the organizations which are not Pareto-dominated by others. An organization $O_i$ is Pareto-dominated by an organization $O_j$ according to both criteria if $(1 - C_{O_i}) \le (1 - C_{O_j})$ and $(1 - P_{O_i}) \le (1 - P_{O_j})$ and there is at least one criteria which is better (i.e. $(1 - C_{O_i}) < (1 - C_{O_j})$ or $(1 - P_{O_i}) < (1 - P_{O_j})$). Regarding the example of table 8, the organization $O_2$ is Pareto-dominated by the organizations $O_1$, $O_3$ and $O_4$ (see figure 4). These three organizations are not Pareto-dominated (they belong to the Pareto-front represented in red). The user can select them in order to build the new collaboration process. From the different experiences, the experience models can be used and adapted to the current experience (organizations, actors, activities, requirements, etc.).

<figure><img src="image_5.png" /><figcaption>Figure 4: Example of Pareto-front using indicators of table 8</figcaption></figure>
## 8. Illustration of the proposal on real experiences

In order to illustrate and assess the proposed approach, twenty experiences (see table 9) have been formalized retrospectively, recorded and assessed by the Axsens-bte company using the proposed models and methods. From these experiences, organizations are evaluated according to new commitments to reach ("lean improvement plan" and "lean six sigma training"). Every experience corresponds to an actual project or process carried out by the Axsens-bte company. These experiences are related to several types of projects: PhD thesis research programs, organization of workshops, research projects, industrial problem solving processes, lean six sigma and training. The assessment has been achieved with the help of people involved in the projects (Axsens-bte consultant and managers, academic researchers, PhD student). The assessment from the external actors in the projects has been subjectively estimated since it was difficult to have an a posteriori feedback from the actors themselves. For each experience, every assessor was representing one organization and rated the other organizations (between 0 and 1) following the three criteria on the basis of the assertions provided in the tables 2, 3 and 4. They also rated the process performance. For simplicity reasons, the processes were composed of only one activity and every organization were represented by only one actor. In order to synthesize the results, only overall data are provided (columns $EV_{Coordination}^{O_i}$, $EV_{Cooperation}^{O_i}$, $EV_{Communication}^{O_i}$ and $Perf_j^{O_i}$ in table 10).

The value of the parameter $\lambda$ is 1.0 for the aggregation of the indicator $EV_j^{O_i}$ using the GOWA operator (eq. 6). Thus, the aggregation is the average of coordination, cooperation and communication assessments in table 10.

### 8.1. Similar experiences retrieval

Considering a current experience where the Axsens-bte company had to provide solutions for a lean improvement plan and lean six sigma training to one of its customer (aeronautics domain), two commitments has been defined (*Com*₁ and *Com*₂). The concept of the commitment *Com*₁ is "Lean Improvement Plan" and the concept of *Com*₂ is "Lean six sigma training". In order to define the type of each commitment, the taxonomy of concepts of the company has been used. For the twenty experiences, the similarities according to *com*₁ and *com*₂ (table 9) are computed using the method proposed in section 7.1.

In order to select only the most suitable experiences, the value 0.7 is given to the threshold $\alpha$. That implies that the experiences corresponding to PhD thesis, organization of workshops, training and research projects are not selected. Only the "Lean Six Sigma" experiences (the most similar to the current process) are selected for assessment. The five retrieved experiences are {#10, #11, #17, #19, #20} (table 10).
<table><caption>Table 9: List of collaboration experiences stored in the Axsens-bte experience base</caption><thead><tr><td>Exp<sub>j</sub></td><td>Id</td><td>Description</td><td>Similarity</td></tr></thead><tbody><tr><td>1</td><td>Thesis I</td><td>Research project that describes a conceptual framework, process, and methodology for the collaborative resolution of problems in supply chains.</td><td>0.18</td></tr><tr><td>2</td><td>IPSM 2015</td><td>Industrial Problem-Solving Methods is an annual workshop- conference organized to bring together industrial and academic partners who work together around quality management and the improvement of company performance in 2015.</td><td>0.19</td></tr><tr><td>3</td><td>IPSM 2016</td><td>idem IPSM 2015.</td><td>0.19</td></tr><tr><td>4</td><td>IPSM 2017</td><td>idem IPSM 2016.</td><td>0.19</td></tr><tr><td>5</td><td>Thesis II</td><td>Research project which describes a proposition of an agile problem-solving process driven by the reuse of experiences and knowledge.</td><td>0.18</td></tr><tr><td>6</td><td>Thesis III</td><td>Research project which describes the formalization of collaboration experiences and collaboration and performance indicators in industrial processes in order to reuse past experiences to improve current situations (this research project).</td><td>0.18</td></tr><tr><td>7</td><td>OPERA</td><td>Research project funded by the french government that aims to formalize the bidding process and support it with decision support tools that enable bidding companies to quickly propose fairer and more accurate bids in which there is greater confidence.</td><td>0.07</td></tr><tr><td>8</td><td>EASYNOV</td><td>Research project to define a methodology and a tool for solving problems in the industry 4.0.</td><td>0.08</td></tr><tr><td>9</td><td>ALIES</td><td>Research project in order to develop a problem-solving course and a prototype of Knowledge Management for a consulting group.</td><td>0.08</td></tr><tr><td>10</td><td>LSS-053</td><td>Lean Six-Sigma Project which aims to optimize an aircraft painting activity.</td><td>0.78</td></tr><tr><td>11</td><td>LSS-028</td><td>Lean Six Sigma Project which aims to optimize the aircraft molding activity.</td><td>0.78</td></tr><tr><td>12</td><td>PSP-08</td><td>Project which aims to structure the process of problem solving in a manufacturing company of aerospace finished components.</td><td>0.15</td></tr><tr><td>13</td><td>PSP-04</td><td>Project which aims to structure the process of problem solving in a surface treatment company specialized in the aeronautical sector.</td><td>0.15</td></tr><tr><td>14</td><td>PSP-05</td><td>Project which aims to structure the process of problem solving in a manufacturing company specialized in precision mechanical assemblies, machining of complex parts for the aeronautic sector, especially engine parts.</td><td>0.15</td></tr><tr><td>15</td><td>AL-02</td><td>Excel training for logistics optimization in the transport sector.</td><td>0.5</td></tr><tr><td>16</td><td>AL-01</td><td>Excel training for logistics optimization in the transport sector.</td><td>0.5</td></tr><tr><td>17</td><td>LSS-15</td><td>Lean Six-Sigma Project which aims to optimize an aircraft assembly activity.</td><td>1</td></tr><tr><td>18</td><td>QA-12</td><td>Project which carries out the quality control of a Procurement Management System.</td><td>0.63</td></tr><tr><td>19</td><td>LSS-041</td><td>Lean Six Sigma Project which aims to optimize the preparation and loading of products in a producer company of Powder Metallurgy Steels and High-Speed Steels.</td><td>0.71</td></tr><tr><td>20</td><td>LSS-033</td><td>Lean Six Sigma Project which aims to optimize the manufacturing process of prefabricated bathrooms.</td><td>1</td></tr></tbody></table>

## 8.2. Experience based evaluation of organizations and multicriteria selection

From the results obtained in table 10, the experience based evaluation of organizations is performed. The indicators $C_{O_i}$ and $P_{O_i}$ are relative and computed using the equation (11). For the five organizations, they are computed and gathered into the table 11.

Therefore, in order to select the best organizations which can be used to collaborate for the new current process, a Pareto-front is built. On figure 5, the organizations $O_5$ and $O_7$ are dominated. The Pareto-front is composed of the organizations $O_1$, $O_4$ and $O_6$ which are not Pareto-dominated. They constitute a panel of organizations which can be reused to collaborate in the current experience. Moreover, the information capitalized into the five retrieved experiences can be reused, i.e. adapted to the current experience. Contracts,
<table><caption>Table 10: Collaboration and performance assessments for every organization within every experience</caption><thead><tr><th>Exp<sub>j</sub> / Id</th><th>O<sub>i</sub></th><th>Sim</th><th>EV<sup>O<sub>i</sub></sup><sub>Coordination</sub></th><th>EV<sup>O<sub>i</sub></sup><sub>Cooperation</sub></th><th>EV<sup>O<sub>i</sub></sup><sub>Communication</sub></th><th>EV<sup>O<sub>i</sub></sup><sub>j</sub></th><th>Perf<sup>O<sub>i</sub></sup><sub>j</sub></th></tr></thead><tbody><tr><td rowspan="3">#10 / LSS-053</td><td>O<sub>1</sub></td><td rowspan="3">0.78</td><td>0.84</td><td>0.89</td><td>1</td><td>0.91</td><td>1</td></tr><tr><td>O<sub>4</sub></td><td>0.84</td><td>0.89</td><td>1</td><td>0.91</td><td>1</td></tr><tr><td>O<sub>7</sub></td><td>0.78</td><td>0.82</td><td>0.92</td><td>0.84</td><td>0.82</td></tr><tr><td rowspan="3">#11 / LSS-028</td><td>O<sub>1</sub></td><td rowspan="3">0.78</td><td>1</td><td>0.95</td><td>1</td><td>0.98</td><td>1</td></tr><tr><td>O<sub>5</sub></td><td>0.94</td><td>0.89</td><td>0.94</td><td>0.92</td><td>0.66</td></tr><tr><td>O<sub>7</sub></td><td>0.92</td><td>0.87</td><td>0.92</td><td>0.9</td><td>0.74</td></tr><tr><td rowspan="4">#17 / LSS-15</td><td>O<sub>1</sub></td><td rowspan="4">1</td><td>0.84</td><td>1</td><td>1</td><td>0.94</td><td>0.71</td></tr><tr><td>O<sub>4</sub></td><td>0.85</td><td>1</td><td>1</td><td>0.95</td><td>0.64</td></tr><tr><td>O<sub>5</sub></td><td>0.79</td><td>0.95</td><td>0.97</td><td>0.9</td><td>0.5</td></tr><tr><td>O<sub>7</sub></td><td>0.78</td><td>0.93</td><td>0.95</td><td>0.88</td><td>0.54</td></tr><tr><td rowspan="2">#19 / LSS-041</td><td>O<sub>1</sub></td><td rowspan="2">0.71</td><td>0.7</td><td>0.45</td><td>0.72</td><td>0.62</td><td>1</td></tr><tr><td>O<sub>6</sub></td><td>0.6</td><td>0.4</td><td>0.61</td><td>0.53</td><td>1</td></tr><tr><td rowspan="3">#20 / LSS-033</td><td>O<sub>1</sub></td><td rowspan="3">1</td><td>0.78</td><td>0.84</td><td>0.81</td><td>0.81</td><td>0.97</td></tr><tr><td>O<sub>6</sub></td><td>0.67</td><td>0.73</td><td>0.7</td><td>0.7</td><td>0.88</td></tr><tr><td>O<sub>7</sub></td><td>0.73</td><td>0.78</td><td>0.76</td><td>0.76</td><td>0.68</td></tr></tbody></table>

<table><caption>Table 11: Overall evaluations of organizations</caption><thead><tr><th>Organization (O<sub>i</sub>)</th><th>Collaboration (C<sub>O<sub>i</sub></sub>)</th><th>Performance (P<sub>O<sub>i</sub></sub>)</th></tr></thead><tbody><tr><td>O<sub>1</sub></td><td>0.86</td><td>0.92</td></tr><tr><td>O<sub>4</sub></td><td>0.93</td><td>0.8</td></tr><tr><td>O<sub>5</sub></td><td>0.91</td><td>0.57</td></tr><tr><td>O<sub>6</sub></td><td>0.63</td><td>0.93</td></tr><tr><td>O<sub>7</sub></td><td>0.84</td><td>0.68</td></tr></tbody></table>

organizations, activities, actors belonging to the selected organizations (O<sub>1</sub>, O<sub>4</sub>, O<sub>6</sub>) can be reused.

<figure><img src="image_6.png" /><figcaption>Figure 5: Pareto-front based on collaboration and performance assessment of every organization</figcaption></figure>

Finally, in order to build the new process and reach the two commitments *Com*<sub>1</sub> (Lean Improvement Plan) and *Com*<sub>2</sub> (Lean six sigma training), the organizations O<sub>1</sub> and O<sub>4</sub> have been reused by adapting the processes of the experiences LSS-053 and LSS-15 (figure 10). The organization O<sub>6</sub> has not been retained because of a lower overall collaboration assessment (C<sub>O<sub>6</sub></sub> = 0.63). Additionally, the stability of the
overall assessments over the time can be used to select the organizations. An organization which has de-
creasing performances is less attractive than a company with stable (and good) performances or increasing
performances.

9. Conclusion and further research

9.1. *Theoretical contributions*

The overall contribution of this paper is to provide a human-centric way to model, evaluate and record past collaborative situations, then to retrieve and reuse them to implement new collaborations. For this purpose, different more specific contributions have been integrated in order to form a coherent whole:

* A model of collaboration experience composed of: i) an UML diagram, formalizing the structure of a collaboration experience and providing a template for collaboration experience capture, ii) a taxonomy of concepts which gathering key knowledge related to the industrial collaboration domain.

* A human-centric method for assessment of collaboration experiences based on two indicators: i) perception of the collaboration experience by actors and ii) satisfaction of the requirements. Two bottom-up mechanisms enable to compute a global collaboration appreciation and a global assessment of process performance.

* A retrieve and reuse mechanism to select the most efficient organizations which can be invited to collaborate in a new process under construction. From a set of new requirements to be fulfilled during the new process, past similar experiences are retrieved from the experience base. From this set of experiences, the overall collaboration evaluation and the overall performance of every organization is assessed. Then, using a Pareto based approach, the best organizations can be selected with respect to the decision-maker preferences.

9.2. *Practical implications*

In the context of an increasing interest in human-centric production, one of the pillars of Industry 5.0 [6],
collaboration between humans and its relation with collaboration between companies still needs investigation
to be improved. Nowadays, with the fast development of IT technologies, many tools can support a running
collaboration process. However, collaboration experiences are seldom recorded and companies do not learn
from them. This is why it is important to have an organizational memory of past experiences, to be able to
learn from the past and to reuse successful experiences.
The illustration of the utilization by the Axsens-bte company of the proposed models and methods, on the basis of twenty past collaboration experiences, proved the practical feasibility of the overall framework (collaboration experience modeling and human-centric assessment, reuse and retrieve of past relevant experiences to design a new collaboration).

The building of an organizational memory requires efforts but, unlike data, formalized experiences are reusable pieces of knowledge which can be transferred to other organizations of the collaborative network, improving knowledge sharing and collective learning. Moreover, the selection of partners to collaborate is consuming resource and time, but can be supported and rationalized by using the proposed methodology.

### 9.3. Further investigations

Some limitations of the proposed approach, combined with the experimentation feedback, allowed to draw different perspectives of work and to improve the contribution described in this article:

* the retrieval of past collaboration experiences is done on the basis of a comparison between types of commitments. This could be improved by adding types of activities, types of roles and types of requirements. This would enable a more relevant experience retrieval.

* the introduction of additional expert knowledge into the experience model, such as:

i) encountered problems during collaboration and their solutions,

ii) key knowledge learned from experience (good practices, bad decisions and their impacts, critical points, etc.),

iii) knowledge on organizations maturity, evolution of actors roles and skills within the organizations.

To achieve this goal, models have to be extended to embed more expert knowledge and ontologies are an interesting way to investigate.

* the development of a tool dedicated to collaboration and based on the proposed experience feedback process is necessary for practical deployment. Such a tool could facilitate the formalization and capitalization of knowledge (structure of an experience, taxonomy of concepts and assessment grids) but also the capitalization and reuse of collaboration experiences.

* the uncertainty on the obtained values is not addressed in this paper and would require to use more complex uncertainty models relying for instance on probability theory, possibility theory or Dempster-Shafer theory.
References

[1] G. Schuh, T. Potente, C. Wesch-Potente, A. R. Weber, J.-P. Prote, Collaboration mechanisms to increase productivity in the context of industrie 4.0, Procedia CIRP 19 (2014) 51–56, 2nd CIRP Robust Manufacturing Conference (RoMac 2014).

[2] J. Herter, J. Ovtcharova, A model based visualization framework for cross discipline collaboration in industry 4.0 scenarios, Procedia CIRP 57 (2016) 398–403, factories of the Future in the digital environment - Proceedings of the 49th CIRP Conference on Manufacturing Systems.

[3] L. S. Dalenogare, G. B. Benitez, N. F. Ayala, A. G. Frank, The expected contribution of industry 4.0 technologies for industrial performance, International Journal of Production Economics 204 (2018) 383–394.

[4] N. Kwibisa, S. Majzoub, Challenges faced in inter-organizational collaboration process. a case study of region skane, Journal of Sustainable Development 11 (5) (2018) 34–56.

[5] R. Agranoff, M. McGuire, Collaborative public management: New strategies for local governments, Georgetown University Press, 2004.

[6] S. Nahavandi, Industry 5.0—a human-centric solution, Sustainability 11 (16).

[7] M. Breque, L. De Nul, A. Petridis, Industry 5.0: towards a sustainable, human-centric and resilient European industry, European Commission and Directorate-General for Research and Innovation - Publications Office, 2021.

[8] X. Xu, Y. Lu, B. Vogel-Heuser, L. Wang, Industry 4.0 and industry 5.0—inception, conception and perception, Journal of Manufacturing Systems 61 (2021) 530–535.

[9] B. K. Foguem, T. Coudert, C. Béler, L. Geneste, Knowledge formalization in experience feedback processes: An ontology-based approach, Computers in Industry 59 (7) (2008) 694–710.

[10] C. Durugbo, W. Hutabarat, A. Tiwari, J. R. Alcock, Modelling collaboration using complex networks, Information Sciences 181 (15) (2011) 3143–3161.

[11] K. Rostek, New approaches in supporting to SMEs competitiveness, in: Benchmarking Collaborative Networks, Springer, 2015, pp. 29–57.

[12] H. Thomas, T. Pollock, From i-o economics’ s-c-p paradigm through strategic groups to competence-based competition: reflections on the puzzle of competitive strategy, British Journal of Management 10 (2) (1999) 127–140.

[13] J. K. Halvey, B. M. Melby, Information technology outsourcing transactions: process, strategies, and contracts, John Wiley & Sons, 2005.

[14] A. Arrais-Castro, M. L. R. Varela, G. D. Putnik, R. A. Ribeiro, J. Machado, L. Ferreira, Collaborative framework for virtual organisation synthesis based on a dynamic multi-criteria decision model, International Journal of Computer Integrated Manufacturing (2018) 1–12.

[15] M. C. Goossen, Intraorganizational networks, interorganizational collaboration and firm innovation (2014).

[16] L. M. Camarinha-Matos, R. Fornasiero, H. Afsarmanesh, Collaborative Networks as a Core Enabler of Industry 4.0, in: L. M. Camarinha-Matos, H. Afsarmanesh, R. Fornasiero (Eds.), 18th Working Conference on Virtual Enterprises (PROVE), Vol. AICT-506 of Collaboration in a Data-Rich World, Springer International Publishing, Vicenza, Italy, 2017, pp. 3–17, part 1: Collaboration in Industry 4.0.

[17] L. dos Santos, M. Costa, J. Kothe, G. Benitez, J. Schaefer, I. Baierle, E. Nara, Industry 4.0 collaborative networks for industrial performance, Journal of Manufacturing Technology Management 32 (2) (2020) 245–265.

[18] S. Cisneros Cabrera, G. Pishchulov, N. Mehandjiev, Z. Liu, S. Kununka, An approach and decision support tool for forming industry 4.0 supply chain collaborations, Computers in Industry 125 (2021) 103391. doi:10.1016/j.compind.2020.103391.
[19] K. Blomqvist, J. Levy, Collaboration capability-a focal concept in knowledge creation and collaborative innovation in networks, International Journal of Management Concepts and Philosophy 2 (1) (2006) 31-48.

[20] E. van Burg, H. Berends, E. M. van Raaij, Framing and interorganizational knowledge transfer: A process study of collaborative innovation in the aircraft industry, Journal of Management Studies 51 (3) (2014) 349-378.

[21] L. Argote, P. Ingram, J. M. Levine, R. L. Moreland, Knowledge transfer in organizations: Learning from the experience of others, Organizational Behavior and Human Decision Processes 82 (1) (2000) 1-8.

[22] R. Bergmann, Experience Management: Foundations, Development Methodology, and Internet-based Applications, Springer-Verlag, 2002.

[23] S. Duffield, S. J. Whitty, Developing a systemic lessons learned knowledge model for organisational learning through projects, International Journal of Project Management 33 (2) (2015) 311-324.

[24] J. C. Romero Bejarano, Collaborative Problem Solving Within Supply Chains: General Framework, Process and Methodology, Toulouse, INPT, 2013.

[25] V. Llamas, Towards an agile methodology for industrial problem solving, Toulouse, INPT, 2017.

[26] R. Ayachi, D. Guillon, F. Marmier, E. Vareilles, M. Aldanondo, T. Coudert, L. Geneste, Y. Beauregard, Towards a Knowledge based Support for Risk Engineering When Elaborating Offer in Response to a Customer Demand, International Conference on Industrial Engineering and Engineering Management IEEM 2018, 2018, pp. 1056-1060.

[27] A. M. Thomson, J. L. Perry, Collaboration processes: Inside the black box, Public Administration Review 66 (2006) 20-32.

[28] B. B. Frey, J. H. Lohmeier, S. W. Lee, N. Tollefson, Measuring collaboration among grant partners, American Journal of Evaluation 27 (3) (2006) 383-392.

[29] L. M. Camarinha-Matos, H. Afsarmanesh, Collaborative networks: a new scientific discipline, Journal of intelligent manufacturing 16 (4) (2005) 439-452.

[30] L. M. Camarinha-Matos, H. Afsarmanesh, Collaborative networks: Reference modeling, Springer Science & Business Media, 2008.

[31] C. Durugbo, Modelling user participation in organisations as networks, Expert Systems with Applications 39 (10) (2012) 9230-9245.

[32] C. Durugbo, Collaborative networks: a systematic review and multi-level framework, International Journal of Production Research 54 (12) (2016) 3749-3776.

[33] S. Paruchuri, M. C. Goossen, C. Phelps, Conceptual foundations of multilevel social networks, in: The handbook of multilevel theory, measurement, and analysis, American Psychological Association, 2019, pp. 201-221.

[34] C. A. Ellis, S. J. Gibbs, G. Rein, Groupware: some issues and experiences, Communications of the ACM 34 (1) (1991) 39-58.

[35] M. A. Gerosa, M. Pimentel, H. Fuks, C. J. P. de Lucena, Development of groupware based on the 3c collaboration model and component technology, in: International Conference on Collaboration and Technology, Springer, 2006, pp. 302-309.

[36] H. Fuks, A. Raposo, M. A. Gerosa, The 3c collaboration model, IGI Global, 2008.

[37] K. Kelly, A. Schaefer, Creating a collaborative organizational culture, UNC Kenan-Flagler Business School Executive Development (2014) 2-15.

[38] J. M. Bryson, B. C. Crosby, M. M. Stone, The design and implementation of cross-sector collaborations: Propositions from the literature, Public Administration Review 66 (2006) 44-55.
[39] A. R. Jassawalla, H. C. Sashittal, An examination of collaboration in high-technology new product development processes, Journal of product innovation management 15 (3) (1998) 237-254.

[40] A. E. Ellinger, Improving marketing/logistics cross-functional collaboration in the supply chain, Industrial Marketing Management 29 (1) (2000) 85-96.

[41] M. Borsato, M. Peruzzini, Collaborative engineering, in: Concurrent Engineering in the 21st Century, Springer, 2015, pp. 165-196.

[42] K. Storbacka, T. Strandvik, C. Grönroos, Managing customer relationships for profit: the dynamics of relationship quality, International journal of service industry management 5 (5) (1994) 21-38.

[43] J. Wieselquist, C. E. Rusbult, C. A. Foster, C. R. Agnew, Commitment, pro-relationship behavior, and trust in close relationships, Journal of personality and social psychology 77 (5) (1999) 942-966.

[44] K. B. Kahn, J. T. Mentzer, Logistics and interdepartmental integration, International Journal of Physical Distribution & Logistics Management 26 (8) (1996) 6-14.

[45] A. K. Gupta, S. P. Raj, D. Wilemon, The r&d-marketing interface in high-technology firms, Journal of Product Innovation Management 2 (1) (1985) 12-24.

[46] B. B. Tyler, The complementarity of cooperative and technological competencies: a resource-based perspective, Journal of Engineering and technology management 18 (1) (2001) 1-27.

[47] J. J. Kasvi, M. Vartiainen, M. Hailikari, Managing knowledge and knowledge competences in projects and project organisations, International journal of project management 21 (8) (2003) 571-582.

[48] E. Sivadas, F. R. Dwyer, An examination of organizational factors influencing new product success in internal and alliance-based processes, Journal of marketing 64 (1) (2000) 31-49.

[49] T. Niemelä, Interfirm cooperation capability in the context of networking family firms: The role of power, Family Business Review 17 (4) (2004) 319-330.

[50] K. G. Smith, S. J. Carroll, S. J. Ashford, Intra-and interorganizational cooperation: Toward a research agenda, Academy of Management journal 38 (1) (1995) 7-23.

[51] J. Knoben, L. Oerlemans, Proximity and inter-organizational collaboration: A literature review, International Journal of Management Reviews 8 (2) (2006) 71-89.

[52] I. Boughzala, G.-J. De Vreede, Evaluating team collaboration quality: The development and field application of a collaboration maturity model, Journal of Management Information Systems 32 (3) (2015) 129-157.

[53] A. P. Neghab, A. Etienne, M. Kleiner, L. Roucoules, Performance evaluation of collaboration in the design process: Using interoperability measurement, Computers in Industry 72 (2015) 14-26.

[54] I. M. Lazarte, L. H. Thom, C. Iochpe, O. Chiotti, P. D. Villarreal, A distributed repository for managing business process models in cross-organizational collaborations, Computers in Industry 64 (3) (2013) 252-267.

[55] A. M. Thomson, J. L. Perry, T. K. Miller, Conceptualizing and measuring collaboration, Journal of Public Administration Research and Theory 19 (1) (2007) 23-56.

[56] C. Durugbo, Modelling information for collaborative networks, Production Planning & Control 26 (1) (2015) 34-52.

[57] E. Santanen, G. Kolfschoten, K. Golla, The collaboration engineering maturity model, in: System Sciences, 2006. HICSS'06. Proceedings of the 39th Annual Hawaii International Conference on, Vol. 1, IEEE, 2006, pp. 16c-16c.

[58] J. Alonso, I. M. de Soria, L. Orue-Echevarria, M. Vergara, Enterprise collaboration maturity model (ECMM): preliminary definition and future challenges, in: Enterprise Interoperability IV, Springer, 2010, pp. 429-438.
[59] A. M. Magdaleno, R. M. de Araujo, C. M. L. Werner, A roadmap to the collaboration maturity model (CollabMM) evolution, in: Proceedings of the 2011 15th International Conference on Computer Supported Cooperative Work in Design (CSCWD), IEEE, 2011, pp. 105–112.

[60] A. Van Looy, M. De Backer, G. Poels, M. Snoeck, Choosing the right business process maturity model, Information & Management 50 (7) (2013) 466–488.

[61] D. S. Meléndez-González, T. Coudert, L. Geneste, A. De Valroger, Collaboration evaluation methodology for experience capitalization in industrial processes, IFAC-PapersOnLine 52 (13) (2019) 1210–1215, 9th IFAC Conference on Manufacturing Modelling, Management and Control MIM 2019.

[62] R. C. Nickerson, U. Varshney, J. Muntermann, A method for taxonomy development and its application in information systems, European Journal of Information Systems 22 (3) (2013) 336–359.

[63] A. Stolz, B. Rodriguez-Castro, A. Radinger, M. Hepp, Pcs2owl: A generic approach for deriving web ontologies from product classification systems, in: The Semantic Web: Trends and Challenges, Springer International Publishing, Cham, 2014, pp. 644–658.

[64] R. A. Horn, C. R. Johnson, Matrix Analysis, The Trustees of the Stevens Institute of Technology, 1990.

[65] R. Yager, Generalized owa aggregation operators, Fuzzy Optimization and Decision Making 3 (1) (2004) 93–107.

[66] Z. Wu, M. Palmer, Verbs semantics and lexical selection, in: Proceedings of the 32nd annual meeting on Association for Computational Linguistics, Association for Computational Linguistics, 1994, pp. 133–138.