# Fundamentals for the Development of a Performance Analysis Approach in Collaborative Networks

Hendrik Jähn

<figure><img src="image_1.png" /></figure>

## ► To cite this version:

Hendrik Jähn. Fundamentals for the Development of a Performance Analysis Approach in Collaborative Networks. 15th Working Conference on Virtual Enterprises (PROVE), Oct 2014, Amsterdam, Netherlands. pp.521-533, 10.1007/978-3-662-44745-1_52. hal-01392158

HAL Id: hal-01392158

https://inria.hal.science/hal-01392158

Submitted on 4 Nov 2016

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Fundamentals for the Development of a Performance Analysis Approach in Collaborative Networks

Hendrik Jähn¹

¹Chemnitz University of Technology, Department of Economic Sciences, Thüringer Weg 7
09126 Chemnitz, Germany
hendrik.jaehn@wirtschaft.tu-chemnitz.de

**Abstract.** This contribution focuses the challenge of analyzing the performances of enterprises which participate in modular Virtual Enterprises. Modular in that context means "consisting of independent participants". In order to be able to draw valuable conclusions a comprehensive performance analysis approach (PAA) is required. For the development of an approach for the value-added process-related performance analysis the fulfilment of some preconditions is necessary. A comprehensive approach for the analysis of performances within networked production structures is an indispensable component of the operative management to secure sustainable success. In the following the fundamentals are distinguished and some effects are discussed. Additionally some possibilities for the extension of the approach are introduced.

**Keywords:** Virtual Enterprise, Performance Analysis, Soft-Factors, Collaborative Network Model.

## 1 Motivation and Objective

The production of goods in collaborative networks has become a widespread form of economic added value. Nevertheless both for the theory and in practice, numerous starting points arise that require an improvement of the current situation. The present article focuses on a topic, which is very relevant from the practical perspective, however, but only has found little attention from the theoretical point of view. This refers to approaches for the value-added process-related performance analysis of economic units that operate in dynamic networked production structures i.e. collaborative networks.

The scope of the term performance analysis in this contribution is broader than the understanding of performance evaluation in common. Now an appropriate approach for a value-added process-related performance analysis is developed. For a comprehensive approach for performance analysis at first relevant perspectives need to be identified and then appropriate perspective combinations have to be developed. In that context perspectives mean the starting point for research activities. This procedure is followed by a classification of selected theoretical approaches for performance management by taking into account its relevant perspective combination.
Finally the derivation of a framework for an approach to the performance analysis for the given situation is started. Considering the specific practical framework that approach can be completed. After validation and verification it is ready for practical application.

The main objective in that context is to model a framework for the development of approaches for performance analyses which are very flexible. That means these approaches are adaptable to a high degree according the scope of application. Therefore it is necessary to research fundamentals i.e. possible perspectives. The research methodology includes a literature research. Also already applied approaches for the analysis of performances in enterprises are considered. The suitability is checked and the necessary range of adaptation is determined.

# 2 Consideration of Different Perspectives

## 2.1 Classification of Perspectives

At first, reasonable prospects for a value-added process-related performance analysis are considered. This allows different possibilities concerning the focused perspectives and therefor the intended modelling. The commonest perspectives include a chronological and an organisational perspective. These perspectives are introduced and discussed in section 2.2 and 2.3 in this contribution. Further perspectives can be included in the already mentioned perspectives. For example the perspective of number of participants in included in the organisational perspective. A financial perspective could be considered as well, however finance is not focused here. Product quality and distribution channel represent sub-categories with minor impact.

## 2.2 Organizational Perspectives

The organizational perspective focuses the organizational alignment. The most common form of organization depicts the company (enterprise) itself, with large companies and corporations, medium-sized businesses (SME) or small enterprises can be distinguished. As a consequence of the increase in competition, enterprise collaborations gain growing popularity. Distinction can be made in two main categories: static networks and dynamic networks.

Static networks (also called resource pools) include all types of supply chains, which are a form of long-term business cooperation (exist value-added process-neutral) and dynamic networks, e.g. virtual enterprises that embody a short-term business cooperation (value-added process-related) form of cooperation. That means cooperation in a specific configuration is formed only to realize one value-added process. In an idealized form, consequently, companies, virtual enterprises and enterprise networks can be distinguished. As companies turn to be part of both of virtual enterprises and enterprise networks and virtual networks emerge from corporate networks, there are several different combinations which allow an assessment from an organisational perspective.
## 2.3 Chronological Perspectives

As a second dimension the chronological perspective is considered. It focuses on the time horizon of the analysis. Here, the strategic, tactical or operational performance analysis can be distinguished. This variety shows that generally only a retrospective analysis can be initiated. From the practical point of view, a subdivision into a value-added process-neutral and value-added process-related form of performance analysis is used for research.

The value-added process-neutral perspective is longer term in nature, i.e. it is a strategic performance analysis process including multiple value-added processes. This perspective is very wide-spread among the current approaches of performance measurement and management and related research works. In that case, appropriate data are determined, allowing a statement on the long-term competitiveness on the basis of services provided in the past performances. This perspective, however, can only identify locally occurring weaknesses in the provision of services to a limited degree as such weaknesses can be overlaid by strengths in the same range.

The analysis of the realized performance of a particular value-added process is focused on the value-added process-related perspective. It is to provide an operative, i.e. short-term analysis. Unlike the value-added process-neutral approach, it is possible to identify specific weaknesses that have occurred at a specific value-added process. That allows gaining a more detailed knowledge on the quality of a value-added process and its participating organizational units.

It is comprehensible that a performance analysis can be performed based solely on already existing data. Freedom of choice, however, exists on the number of value-added processes to be considered, and thus the orientation of the performance analysis. In the following, the introduced perspectives are combined to identify a relevant perspective combination for the performance analysis and to classify selected approaches presented in the scientific literature according to this spectrum.

## 2.4 Synthesis and Focus

Although further perspectives like product range, the number of participants or the distribution channel can be taken into account, the organizational and the chronological perspective are sufficient to create an appropriate index. Both perspectives and the resulting assignments can be best represented in a matrix.

As relevant organizational perspectives single enterprises, Virtual Enterprises (dynamic networks) and enterprise networks (static networks) were identified. Additionally, there exist enterprises that operate in virtual enterprises or enterprise networks. That means these enterprises are involved in a value-added process realized by intensive cooperation represented by a Virtual Enterprise or an enterprise network.

Also there are virtual enterprises (with several enterprises), which are integrated into enterprise networks. That means within a long-term network a short-term cooperation is evolved. After completing one value-added process that Virtual Enterprise splits up. Further characteristics are not relevant here and therefore are not considered.
From a chronological point of view, both the value-added process-related and the value-added process-neutral perspectives are distinguished. A further classification is not necessary.

By combining these two different perspectives the relevant perspective combinations for a performance analysis can be derived. However not every combination is of practical relevance here. In general virtual enterprises are constructs that are configured in order to perform a specific value-added process. Thus, a value-added process-neutral performance analysis in or with Virtual Enterprises as an organisational construct is generally not feasible and marked black.

<figure><img src="image_3.png" /><figcaption>Fig. 1. Overview on Perspective Combinations.</figcaption></figure>

Figure 1 illustrates possible perspective combinations and its relevancies according to the practical situation of value-adding analysis. Here “E” stands for the classic “Enterprise”, “VE” means “Virtual Enterprise” synonymously for a short-term cooperation and “EN” represents “Enterprise Network” standing for a long-term cooperation.

All feasible perspectives (1) to (9) including selected examples from the literature are discussed subsequently. In that context the matrix is to be read as follows: combination (3) means that the (performance) analysis focuses an enterprise which is part of a Virtual Enterprise (short-term cooperation). In that context a value-added process-related performance analysis is realized. In that way every combination is interpreted.

In general there is little novelty according to the identified perspectives and its combinations. However this approach allows a sophisticated (literature) research. So deficits in research can be identified and eliminated. Hence a comprehensive literature research is done. The major findings and its consequences are introduced in the next section.
## 2.5 Literature Review

The number of publications focusing on performance analysis in networked structures and related research areas is very difficult to overlook. It is almost impossible to mention all important publications in a brief literature review. For this reason, only those publications are referenced, which have a significant influence on the development of the performance analysis approach in collaborative networks.

According to figure 1 the variants (1) and (2) focus on enterprises which participate in the view of network theory to no cooperation. An performance analysis is possible both short-term (1) or long-term (2). While for option (1) hardly exist any approaches for performance analysis, case (2) includes all established approaches for performance measurement, such as the EFQM-model, the Performance Pyramid or the Balanced Scorecard.

With variant (3) the performance of enterprises is evaluated which operate within a virtual enterprise (dynamic network). This variant represents the focus of this contribution. This value-added process-specific form of performance analysis and evaluation is hardly considered yet. This deficit is to be eliminated. As an organisational unit both an enterprise and a part of an enterprise can be considered. There is no difference for the research methodology.

The evaluation of Virtual Enterprises as an independent organization form is considered with option (4). This may exclusively focus on a value-added process-related examination, since a long-term view because of the dynamic nature of this form of organization is not possible. A framework for the development of performance measurement systems for virtual enterprises can be found in [1].

The perspective combinations (5) to (9) are related to a cooperation which is realized in enterprise networks. An approach for the performance analysis of enterprises operating in enterprise networks related on a value-added basis (5) was not found. This variant is conceivable; however, has hardly any practical relevance. The value-added process-neutral evaluation of enterprises in long-term networks, however, represents some practical relevance. A few publications exist [2],[3]. This corresponds to option (6).

A further option is variant (7) where the performance of a virtual enterprise with respect to a particular value-added process in the context of a participation in an enterprise network, is evaluated [4].

Finally, there is the possibility to realize both a short-term (8), as well as a long-term (9) performance analysis of an entire enterprise network. Because enterprise networks exist value-added process-independent and thus have a high degree of practical relevance in the literature, numerous approaches, especially for the value-added process-neutral perspective [5],[6],[7],[8] exist. Also approaches for the value-added process-related, i.e. the dynamic perspective can be found, e.g. [9].

Based on the synthesis shown in Figure 1 and the associated research literature is to be concluded that the value-added process-related performance analysis of enterprises in dynamic networks is inadequately treated. A major reason is that for the majority of companies especially the long-term survival is the major objective. In order to evaluate the long-term situation of an enterprise especially established systems of Performance Measurement, such as the Balanced Scorecard, the EFQM-model or the Du Pont-System [7] are suitable.
The value-added process-related performance includes a target-actual comparison of parameters without the strategic direction of performance measurement. Thus, parameters such as warranty claims per month, error rate in the production, delivery rate, conversion times, downtime, the return on investment, employee satisfaction and corporate culture for assessment are less or not at all relevant.

# 3 Development of the Analysis Approach

## 3.1 Analysis of Requirements

Within the process of the development of an approach for the performance analysis initially an analysis of requirements is accomplished. From the gained findings the corresponding model is derived and structured.

Since the approach of performance analysis is integrated into a comprehensive model for the distribution of profits, including incentive and sanction mechanisms [10], it focuses primarily the task of determining sanctions in case of a poor performance or to calculate bonuses when an outstanding performance was delivered by an enterprise. This is realized by an appropriate implementation of calculation algorithms.

In that context, therefore, it is necessary to analyse the services provided by enterprises within the framework of a specific value-added process. At this point, it must be clarified with what conception such performance analysis can be conducted and the possible requirements in detail need to be identified. Figure 2 (right column) shows an exemplary sequence of the operative, i.e. value-added process-related phases of the intended performance analysis approach.

Conceived in three stages the value-added process-related performance analysis approach focuses on the following sequential steps in the form of modules:

* Performance Measurement (measurement phase),

* Performance Assessment (assessment phase) and

* Performance Evaluation (evaluation phase).

This modular structure enables the application-oriented exchange of individual components and individual related instruments of application.

As part of the measurement phase the corresponding figures from the considered performance parameters are recorded. By monitoring and workflow management, this can be realized computer-aided and automated to a large degree. The implementation takes place in the measurement phase. It is the premier step of the process-related performance analysis.

During the assessment phase as the core of the performance analysis approach for each performance parameter a comparison of the service provided with fixed set targets is realized. While the actual performance is determined value-added process-related, the target performance of the performance parameters is determined independently of value-added process as a comparison criterion. This is making these values the character of static variables. The determination of the reference values belongs to the value-added process-neutral part of the performance analysis. This is explained in detail later.
Finally, the evaluation phases complete the performance analysis approach. As a key challenge, the various results of the performance evaluation of the different performance parameters are summarized into an aggregated measure of performance based on relevant indicators. The choice of appropriate weights allows considering the different importance of the performance parameters.

Based on the aggregated measure of performance finally charged sanctions can be imposed and promulgated as a result of the performance evaluation. In order to realize the operational phases of the approach, strategic, i.e. basic preparatory work is required.

The key challenges in this context include defining the performance parameters, identifying the key figures, determining of the assessment function and the determination of the weights of the performance parameters. While the first three operations can be assigned to the basic stages, the determination of appropriate weights should be interpreted in preparation for the actual performance analysis, but also has a strategic character, since the weights of the parameters are not determined anew for each value-added process. Figure 2 (left column) shows an exemplary sequence of the value-added process-neutral i.e. strategic phases and its contents.

<figure><img src="image_4.png" /><figcaption>Fig. 2. Strategic and operational Phases of the Analysis Approach.</figcaption></figure>
Deriving the requirements both from a strategic, but mainly from an operative perspective the understanding of the performance analysis and the associated (strategic) processes in the context of production-economic networking is summarized as follows:

"Performance analysis in the environment of networked production structures is the determination of the degree of target achievement in terms of performance of an enterprise in a specific production processes."

The performance is determined at the end of a value-added process based on a general performance evaluation system using selected performance parameters for an enterprise which are combined to an aggregated performance figure. Based on that figure, profit share affecting sanctions (income cuts) or bonuses (additional payments) can be calculated. This allows new steering mechanisms for the collaborative network management.

## 3.2 Derivation of the Approach

In developing an approach for the performance analysis in collaborative networks focusing specific value-added processes, care was taken to ensure a high degree of flexibility in terms of applications. In that context individual aspects of established approaches are accessed in order to be integrated with equivalent qualifications in the model.

A comparison of established approaches of the performance management shows that particularly the balanced scorecard approach is suitable to determine the performance parameters to be considered in the performance analysis approach. However the actual function of the performance parameters for evaluating the performance of business enterprises and to define possible consequences is not an instrument of the Balanced Scorecard (BSC) [11], hence must be supported by other concepts and approaches.

The utility-benefit analysis [12] represents a potential concept, which is applied in a modified form. The "set of complex alternatives for acting" for the performance analysis is given by the number of different values for performance parameters of an enterprise. The "preferences of the decision maker" form in the given context of the performance analysis the general requirements, as introduced in the previous section. The "multidimensional target system" finally is equivalent to the requirements (target values), which are imposed on any enterprise. In conclusion, this means that in terms of the issue of this research work the utility-benefit analysis classifies the performance parameters characteristics of enterprises into the multicriteria objective system of performance analysis by considering the specific requirements of the network.

Thus a basis for judgment is created on which conclusively sanctions or bonuses can be defined as consequences. Based on these findings it should be noted that some aspects of the BSC and an adapted form of utility-benefit analysis can be incorporated as established approaches in the context of the value-added process-based performance analysis. The BSC is to serve as a basis for the proper identification of relevant performance parameters. That part represents one of the most challenging aspects of modelling.
The utility-benefit analysis is included in the performance evaluation and performance weighting in a modified form. Based on an aggregate measure of performance appropriate consequences in the form of sanctions for a poor performance or bonuses on exceptional performance should be quantified. The focus of this approach is the highest possible degree of automation in order to ensure a smooth implementation and application.

## 3.3 Structuring of the Approach

The following the requirements to be taken to a performance analysis approach described in section 3.1 are considered and merged with the previously derived coarse approach to performance analysis in Section 3.2. in order to form a comprehensive model.

The global approach for performance analysis in virtual enterprises is divided, as already shown in figure 2 in a strategic and an operational part. The strategic, i.e. value-added process-neutral phases including their processes are to be carried out at regular intervals, however of course not for each value-added process separately. At regular intervals hereby means, that a review should enforced after each modification of the basic conditions or periodically (e.g. annually). This is composed by decision-makers on all resource pool parties or enterprises. In contrast, there are a number of process steps, which are directly related to the performance analysis approach. These tasks are assigned to the dynamic or value-added process-related phases. All relevant steps are to be executed for every particular value-added process. The order always remains unchanged.

Among the value-added process-neutral steps are the basic phases and a preparation phase. The basic phases are represented by three steps that are directly related to each other. Primarily the relevant performance parameters to be considered in the performance analysis approach need to be identified. Under a performance parameter in this case an attribute is understood, which is of significant importance for the evaluation of a performance of an enterprise considering a value-added process. Among the most important performance parameters the price, delivery date or the quality are expected. However there are more parameters which can be included depending on the specific application field. Especially soft-factors play an important role in network management. This includes the quality of a cooperation or the cooperation climate.

After determination of relevant performance parameters by the help of the Balanced Scorecard approach, the corresponding key figures are determined. The key figure of a performance parameter enables the quantitative analysis of performance of an enterprise. Key figures are closely related with the considered performance parameters.

Based on the key figures of the performance parameters, in the following an appropriate assessment (evaluation) function needs to be derived. An assessment function can be understood as a mathematical function representing the dependence between degree of fulfilment of a performance considering the appropriate key figure and the credits granted. Credits represent the input for the utility-benefit analysis. In that context also the nominal performance of a performance parameter and its key
figure is defined. In case the nominal performance is realised the full score is awarded. In case of a performance not reaching the nominal value less credits are granted. The range of credits must be defined in advance. Usually the maximum is 10 and the minimum is 0 credits.

To reflect the different importance of each performance parameter, they are integrated in the performance analysis in a weighted form. The determination of the weights is carried out by a suitable method of decision theory, for example, the trade-off method [13]. This process is represented by the step "determination of weights", which is the (only) part of the preparation phase. Consequently, the basic stages help to create proper conditions for performance analysis approach, while in the preparatory phase the requirements for the consideration of weights are met. Basically, a performance analysis can also be performed without any weighting of the performance parameters. Then all parameters are interpreted equally important.

By working on the aforementioned steps all preparation work is done, i.e. the value-added process-neutral phases are complete. Now the structure of the performance analysis approach is detailed in that way that it is applicable.

The value-added process-related phases include the measurement phase, the assessment phase and an evaluation phase.

Central component of the measurement phase is represented by the measurement of the performance provided by an enterprise (actual performance) by considering all relevant performance parameters. The measurement of the performance is realized on the basis of the already established key figures and is supported by monitoring functionalities or by the workflow management system used the information system.

Based on this performance measurement procedure consecutively the step of performance assessment can be carried out as part of the assessment phase. This is realized by a comparison of the target and the actual values. It therefore a target value must be given representing the desired target performance (which was determined within the framework of the evaluation function) and an actual value, which results from the performance assessment, i.e. in terms of the actual performance provided considering the performance parameters. Hereby the assessment system should not be constructed in a binary way, i.e. "met target or "target not met". It should enable a finer gradation in order to provide a meaningful assessment of the performance.

The assessment of individual performance parameters respectively their key figures is realized by using the prepared assessment functions for each performance parameter.

Subsequently, the services provided by an enterprise can be evaluated. This is done in the evaluation phases. By weighting the assessments the calculation of an aggregated performance measure is possible. This aggregated performance measure represents the realized performance of an enterprise in an aggregated form with respect to a specific value-added process under consideration of all relevant performance parameters. Hereby the assessment functions and the corresponding weights of the performance parameters are included.

The steps performance assessment, weighting of performances and calculation of the performance key figure are based on the fundamental ideas of the utility-benefit analysis and result in an adapted approach for application in the context of performance analysis in virtual enterprises. Finally, from the enterprise-specific performance key figures, appropriate consequences can be derived. As applicable
consequences the aforementioned sanctions in the form of profit cuts or the payment of bonuses in case of an outstanding performance are conceivable. By combining the different phases the comprehensive approach for the value-added process-related performance analysis in virtual enterprises results.

## 3.4 Advancement of the Approach

For the practical implementation a further detailing of the individual phases is required. Future works on the performance analysis approach include a more detailed modelling on the interdependences between degree of fulfilment of a performance of an enterprise and granted profit share. It has to be clarified whether there is a linear or a non-linear coherency. That interrelation also needs to be modelled by a mathematical function. However, to ensure flexibility, different proposals for the application are made available.

A specific feature of the performance analysis-approach is the comprehensive consideration of soft-factors. The major problem when including soft-factors in a performance analysis approach is that there are no figures available originally. That means the performance of an enterprise regarding a soft-factor is expressed as a linguistic term, for example very well, well, average or bad. Now these terms need to be quantified. Social sciences offer different approaches for the quantification of soft-factors. One very promising approach in that context is the Repertory Grid-methodology [14]. The procedure for the quantification of soft-factors already has been explained in [15]. It serves as a solid base for future works.

## 3.5 Validation and Practical Relevancy

After completing the approach the validation is planned. That procedure can be realized by using real-world data. The results need to be discussed by experts. According to the specific situation the approach and its calculation schemes can be adapted until satisfactory results are available. There are not "right" or "wrong" but "realistic" and not "realistic" results. The high degree of adaptability allows applying the approach in almost every collaborative network.

The practical relevancy of the approach can be derived from the necessity of a retrospective analysis of a performance of an enterprise. In most cases the analysis of the performance of an enterprise represents the base for future cooperation.

## 4 Conclusions

The present paper primarily focused on the foundations of a value-added process-related performance analysis approach. Within the context of an analysis of various perspectives an appropriate framework for the topic was developed. Hereby the operational, i.e. the value-added process-related performance analysis of organizational units that operate in virtual enterprises is focused. In this case the order-specific production network represents the virtual enterprise. It could here be
noted that for this combination of perspectives hardly relevant approaches for
performance analysis have been developed and published.

Based on the corresponding previews and considering established performance
measurement systems as a rough concept, a comprehensive approach has been
derived. Hereby elements of the Balanced Scorecard are applied in order to determine
the relevant performance parameters. The appropriate weights are calculated by
approaches of decision theory. Finally a modified utility-benefit analysis supports the
assessment and evaluation of the information collected. This combination of different
approaches is novel and allows the processing of the topic with a high degree of
adaptability. That means that the performance analysis approach can always be
adapted to the circumstances given.

References

1. Chalmeta, R., Grangel, R.: Performance measurement systems for virtual enterprise integration. In: Int Journal of Computer Integrated Manufacturing, 18 (1), 73--84 (2005).

2. Katzy, B.R., Sydow, J., Aston, D., Helin, R.: Zur Bewertung vernetzter Unternehmen. In: zfo, 70 (2), 99--107 (2001).

3. Seifert, M., Wiesner, S., Thoben, K.D.: Prospective performance measurement in virtual organizations. In: Camarinha-Matos, L.M., Afsarmanesh, H. (Eds.) Collaborative Networks: Reference Modeling, pp. 319--326, Springer, Boston (2008).

4. Graser, F., Jansson, K., Eschenbächer, J., Westphal, I., Negretto, U.: Towards Performance Measurement in Virtual Organizations - Potentials, Needs, and Research Challenges. In: Camarinha-Matos, L.M. et al. (Eds.) Collaborative Networks and their breeding Environments, pp. 301--310, Springer, New York (2005).

5. Richert, J.: Performance Measurement in Supply Chains, Gabler, Wiesbaden (2006).

6. Kulmala, H.I., Lonnqvist, A.: Performance measurement of networks: Towards a non-financial approach. In: International Journal of Networking and Virtual Organisations, 3 (3) 299--316 (2006).

7. Erdmann, M.: Supply Chain Performance Measurement. Josef Eul, Lohmar (2003).

8. Hess, T., Wohlgemuth, O., Schlembach, H.-G.: Bewertung von Unternehmensnetzwerken. In: zfo 70 (2) 68--74 (2001).

9. Westphal, I., Thoben, K.D., Seifert, M.: Predictive Performance Measurement in Virtual Organisations. In: Camarinha-Matos, L.M. et al., (Eds.) Establishing the Foundation of Collaborative Networks, pp. 33-42, Springer, New York (2007).

10. Jähn, H., Burghardt, T.: Fundamentals for the Allocation of Financial Benefits in Virtual Enterprises. In: Camarinha-Matos, L.M. et al. (Eds.): PRO-VE 2012, IFIP AICT 380, pp. 539--547 (2012).

11. Kaplan, R.S., Norton, D.P.: The Balanced Scorecard – Measures that Drive Performance. In: Harvard Business Review. Jan.-Feb. 1992, 71--79 (1992).

12. Keeney, R.L., Raiffa, H.: Decisions with Multiple Objectives; Preferences and Value Tradeoffs, John Wiley & Sons (1976).

13. Eisenführ, F., Weber, M., Langer, T.: Rational Decision Making. Springer, Heidelb. (2010).

14. Fransella F, Bannister D.: A Manual for Repertory Grid Technique, Academic Press, London, New York, San Francisco (1977).

15. Jähn, H.: Quantitative Analysis of the soft factor “Cooperation Climate” in collaborative networks. In: Camarinha-Matos, L.M. et al. (Eds.): PRO-VE 2008, IFIP AICT 283, pp. 65--72 (2008).